Last modified: 2014-11-07 19:14:03 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T74635, the corresponding Phabricator task for complete and up-to-date bug report information.
First Last Prev Next    This bug is not in your last search results.
Bug 72635 - report table performance, cleanup, and number of items
report table performance, cleanup, and number of items
Status: RESOLVED FIXED
Product: Analytics
Classification: Unclassified
Wikimetrics (Other open bugs)
unspecified
All All
: Highest normal
: ---
Assigned To: Marcel Ruiz Forns
u=AnalyticsEng c=Wikimetrics p=13 s=2...
:
Depends on:
Blocks:
  Show dependency tree / graph
 
Reported: 2014-10-28 18:06 UTC by Dan Andreescu
Modified: 2014-11-07 19:14 UTC (History)
8 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Description Dan Andreescu 2014-10-28 18:06:51 UTC 
A Central Auth cohort creates many rows, because with the current implementation, one MetricReport node is made for each project in a cohort, which is like 800 projects for most Central Auth cohorts.

This has performance implications if we schedule these reports recurrently.  We should take the necessary steps to clean up old data, maybe not create so many records to start with, add indices, etc.
Comment 1 nuria 2014-10-30 14:38:24 UTC 
There are several ways to go about this:

#1. Purge from db anything older than 30 days that is not a recurrent reports. This can be done via a scheduler task

#2 do not write to report table from nodes that are not the report node, those records are written now but we do not use them for anything.
Comment 2 nuria 2014-10-30 14:51:34 UTC 
We estimated #2, please have in mind recurrent reports need to be working as they are today.
Comment 3 Gerrit Notification Bot 2014-11-03 14:42:26 UTC 
Change 170703 had a related patch set uploaded by Mforns:
Do not store reports that are not going to be used

https://gerrit.wikimedia.org/r/170703
Comment 4 Gerrit Notification Bot 2014-11-05 14:21:26 UTC 
Change 170703 merged by Milimetric:
Do not store reports that are not going to be used

https://gerrit.wikimedia.org/r/170703
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.

Navigation
Links