Last modified: 2014-02-15 00:20:39 UTC
The kibana interface doesn't tag searches with stats groups [0] and the ganglia monitoring script assumes otherwise: [PYTHON] Can't call the metric_init function in the python module [elasticsearch _monitoring]. Traceback (most recent call last): File "/usr/lib/ganglia/python_modules/elasticsearch_monitoring.py", line 511, in metric_init for group in load(url)['_all']['total']['search']['groups']: KeyError: 'groups' When queried on the logstash elasticsearch nodes the relevant section of the _stats/search request looks like this: "_all" : { "primaries" : { "search" : { "open_contexts" : 0, "query_total" : 10317, "query_time" : "1.6h", "query_time_in_millis" : 5915635, "query_current" : 0, "fetch_total" : 7731, "fetch_time" : "23.5s", "fetch_time_in_millis" : 23525, "fetch_current" : 0 } }, "total" : { "search" : { "open_contexts" : 0, "query_total" : 19499, "query_time" : "3.1h", "query_time_in_millis" : 11201005, "query_current" : 0, "fetch_total" : 14517, "fetch_time" : "42.8s", "fetch_time_in_millis" : 42819, "fetch_current" : 0 } } } [0]: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search.html#stats-groups
Nik says that it should be fine to just skip these metrics if not present in the data. Should be a pretty simple "if 'groups' in ..." check to add.
Change 113471 had a related patch set uploaded by BryanDavis: Make elasticsearch ganglia monitor compatible with logstash https://gerrit.wikimedia.org/r/113471
Change 113471 merged by Ori.livneh: Make elasticsearch ganglia monitor compatible with logstash https://gerrit.wikimedia.org/r/113471