Last modified: 2013-12-19 02:48:12 UTC
Wikimetrics does not seem to recycle either celery workers or mysql connections properly. This seems like a new problem since things seemed fine a short time ago. But that could also be because we weren't getting as much traffic as we are now.
Prioritization and scheduling of this bug is tracked on Mingle card https://wikimedia.mingle.thoughtworks.com/projects/analytics/cards/1326
The connection starving problem turned out to be due to a labs db configuration revert. This is fixed now. Closing connections is also happening more reliably all over the wikimetrics code base, but I believe this was not part of the root cause here.