Last modified: 2014-07-15 20:38:38 UTC
While testing how many instances of recurring reports we could spawn as part of the daily run, we discovered a lack of understanding around wikimetrics database performance and queue concurrency. We want to address that as follows: * find out why, if we run more than about 100 report instances at the same time, we get mysql errors about QueuePool limit. * if this QueuePool error is completely fixable with configuration, great * if configuration doesn't completely solve our problem, look into throttling the number of parallel tasks that we can submit to the queue from the recurrent_reports() function. Right now, it's unlimited and that's not great. Celery has lots of constructs like chords and groups that can help us batch things up better.
Change 142007 had a related patch set uploaded by Milimetric: Remove limit on recurrent, add throttling https://gerrit.wikimedia.org/r/142007
Change 142007 merged by jenkins-bot: Remove limit on recurrent, add throttling https://gerrit.wikimedia.org/r/142007
All the research and fixing has been done or split up into these two related issues: https://bugzilla.wikimedia.org/show_bug.cgi?id=67823 https://bugzilla.wikimedia.org/show_bug.cgi?id=67822
