Last modified: 2013-10-22 19:36:51 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T51858, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 49858 - Remove deleted wiki's entries from global tables
Remove deleted wiki's entries from global tables
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
General/Unknown (Other open bugs)
wmf-deployment
All All
: Normal normal (vote)
: ---
Assigned To: Nobody - You can work on this!
https://commons.wikimedia.org/wiki/Fi...
:
Depends on: 54894
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-20 06:33 UTC by Raimond Spekking
Modified: 2013-10-22 19:36 UTC (History)
5 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Raimond Spekking 2013-06-20 06:33:59 UTC
de-labs.wikimedia.org and en-labs.wikimedia.org were closed in the past. Subdomains were removed from DNS but there are still entries in the GlobaleUsage table, see URL. These entries are misleading.

As far as I see there is no maintenance script to fix this error.

I suggest either

a) Remove these entries per manual SQL magic

and/or

b) Write a maintenance script to do this
Comment 1 Alex Monk 2013-06-22 17:32:30 UTC
Deleted wiki names from https://git.wikimedia.org/blob/operations%2Fmediawiki-config.git/HEAD/deleted.dblist

MariaDB [commonswiki_p]> SELECT COUNT(*) FROM globalimagelinks WHERE gil_wiki IN ('bawiktionary', 'chwikimedia', 'closed_zh_twwiki', 'de_labswikimedia', 'dkwikibooks', 'dkwiktionary', 'en_labswikimedia', 'flaggedrevs_labswikimedia', 'liquidthreads_labswikimedia', 'nomcomwiki', 'ptwikimedia', 'readerfeedback_labswikimedia', 'ru_sibwiki', 'strategyappswiki', 'tlhwiki', 'tlhwiktionary', 'tokiponawiki', 'tokiponawikibooks', 'tokiponawikiquote', 'tokiponawikisource', 'tokiponawiktionary');
+----------+
| COUNT(*) |
+----------+
|   191913 |
+----------+
1 row in set (0.53 sec)
Comment 2 Alex Monk 2013-06-22 20:35:27 UTC
Expanding the scope of this bug a bit to include CentralAuth's localuser and localnames tables. According to TS user scfc_de, localuser has 3865 entries from deleted wikis and localnames has 4128.
Comment 3 db [inactive,noenotif] 2013-06-23 18:25:43 UTC
A script should be written for deleting wikis (a script for adding wikis exists), where this maintainance work can be added.
Comment 4 MZMcBride 2013-06-23 18:31:23 UTC
(In reply to comment #0)
> a) Remove these entries per manual SQL magic
> 
> and/or
> 
> b) Write a maintenance script to do this

Only option B seems reasonable. :-)
Comment 5 MZMcBride 2013-06-23 18:34:23 UTC
(In reply to comment #2)
> Expanding the scope of this bug a bit to include CentralAuth's localuser and
> localnames tables. According to TS user scfc_de, localuser has 3865 entries
> from deleted wikis and localnames has 4128.

Hmmm. Are you thinking about one maintenance script with an array of tables to check or are you thinking about individual maintenance scripts (i.e., one script for CentralAuth, one script for GlobalUsage, etc.)?

Either approach could work, but depending on the path forward, it may make sense to split this bug out into smaller, more discrete action items.
Comment 6 Sam Reed (reedy) 2013-10-03 20:23:36 UTC
GlobalUseage has been dealt with in bug 54894, simple script created and ran on the cluster.

Other than CentralAuth, do we have anything else that needs cleaning up?
Comment 7 Gerrit Notification Bot 2013-10-12 22:04:30 UTC
Change 89484 had a related patch set uploaded by Reedy:
Remove deleted wikis entries from CentralAuth tables

https://gerrit.wikimedia.org/r/89484
Comment 8 Gerrit Notification Bot 2013-10-16 17:48:47 UTC
Change 89484 merged by jenkins-bot:
Remove deleted wikis entries from CentralAuth tables

https://gerrit.wikimedia.org/r/89484
Comment 9 Sam Reed (reedy) 2013-10-22 19:36:51 UTC
reedy@tin:/a/common/php-1.22wmf22$ mwscript extensions/WikimediaMaintenance/removeDeletedWikis.php --wiki=commonswiki
bawiktionary:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
chwikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
39
39 localnames rows deleted
localuser:
25
25 localuser rows deleted
closed_zh_twwiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
de_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
500
634
634 localnames rows deleted
localuser:
500
594
594 localuser rows deleted
dkwikibooks:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
2
2 localnames rows deleted
localuser:
1
1 localuser rows deleted
dkwiktionary:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
5
5 localnames rows deleted
localuser:
4
4 localuser rows deleted
en_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
500
1000
1500
1962
1962 localnames rows deleted
localuser:
500
1000
1500
1785
1785 localuser rows deleted
flaggedrevs_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
500
894
894 localnames rows deleted
localuser:
500
856
856 localuser rows deleted
liquidthreads_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
3
3 localuser rows deleted
nomcomwiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
ptwikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
200
200 localnames rows deleted
localuser:
195
195 localuser rows deleted
readerfeedback_labswikimedia:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
392
392 localnames rows deleted
localuser:
360
360 localuser rows deleted
ru_sibwiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
3
3 localuser rows deleted
strategyappswiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tlhwiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
13
13 localuser rows deleted
tlhwiktionary:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
31
31 localuser rows deleted
tokiponawiki:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tokiponawikibooks:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tokiponawikiquote:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tokiponawikisource:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
tokiponawiktionary:
globalimagelinks:
0
0 globalimagelinks rows deleted
localnames:
0
0 localnames rows deleted
localuser:
0
0 localuser rows deleted
Done.
reedy@tin:/a/common/php-1.22wmf22$

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links