Last modified: 2014-05-06 17:31:49 UTC
ADD JAR /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar; set hive.mapred.mode = nonstrict; SELECT * FROM (SELECT user_agent,x_analytics FROM webrequest_mobile WHERE year = 2014 AND content_type RLIKE('text/html') AND ip NOT RLIKE(':') ORDER BY rand()) wmrand LIMIT 5000000; FAILED: RuntimeException MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe com.cloudera.hive.serde.JSONSerDe does not exist) Mostly appears when running from a file, but not uniformly. Has only appeared while I was simultaneously running other queries from a different file - not sure if there's a link there. This is a pretty substantial blocker on some open requests from Erik, and so since the entire analytics team is putting their heads together at the mo, a fast fix would be most appreciated :).
That's very strange, as com.cloudera.hive.serde.JSONSerDe is not the correct class. We are using org.apache.hcatalog.data.JsonSerDe, as you can see in the output of show create table webrequest_mobile. Oliver, just to be sure. Are you running this query on the wmf database? wmf.webrequest_mobile table?
I'll experiment and see if I can consistently replicate it. (I don't suppose the cloudera SerDe is a dependency of the hcatalog SerDe?)
Huh; okay, debugging and experimenting after a good night's sleep, looks like it was a weird combination of issues at my end. Ignore me! :D I'll reopen if it crops up again and I _can't_ solve it.
