Last modified: 2014-06-20 23:47:38 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T67482, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 65482 - Story: Event Logging writing throughput needs monitoring
Story: Event Logging writing throughput needs monitoring
Status: RESOLVED FIXED
Product: Analytics
Classification: Unclassified
EventLogging (Other open bugs)
unspecified
All All
: Unprioritized enhancement
: ---
Assigned To: Nobody - You can work on this!
u=nuria@wikimedia.org c=EventLogging ...
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-05-19 12:24 UTC by nuria
Modified: 2014-06-20 23:47 UTC (History)
9 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description nuria 2014-05-19 12:24:04 UTC
Event Logging writing volume needs monitoring. 

We had an incident last Friday where volumen of EL increased dramatically:

https://ganglia.wikimedia.org/latest/graph.php?r=month&z=xlarge&c=Miscellaneous+eqiad&h=vanadium.eqiad.wmnet&jr=&js=&v=391444071&m=eventlogging_all-events&vl=events&ti=all-events


We should have an alert in process for this. We even have a handy script that could pipe its results to the alert (thanks to ori):

https://gist.github.com/atdt/8deed4bc2d311ba0122f#file-el-status-py

Output of script is as follows:

<output>
Listening for 60 seconds...
UniversalLanguageSelector-tofu                    64.30% (72.80/sec)
PageContentSaveComplete                           9.92% (11.23/sec)
NavigationTiming                                  5.14% (5.82/sec)
TrackedPageContentSaveComplete                    3.86% (4.37/sec)

</output>
Comment 1 Gerrit Notification Bot 2014-06-04 09:35:09 UTC
Change 137280 had a related patch set uploaded by Nuria:
[WIP] monitoring: monitor eventlogging thresholds

https://gerrit.wikimedia.org/r/137280
Comment 2 Gerrit Notification Bot 2014-06-04 13:30:32 UTC
Change 137280 merged by Giuseppe Lavagetto:
Monitoring: monitor eventlogging thresholds

https://gerrit.wikimedia.org/r/137280

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links