Last modified: 2014-10-16 12:59:54 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T32571, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 30571 - GlobalUsage not working with a simple one-database with multiple wikis setup (REL1.18)
GlobalUsage not working with a simple one-database with multiple wikis setup ...
Status: RESOLVED WORKSFORME
Product: MediaWiki extensions
Classification: Unclassified
GlobalUsage (Other open bugs)
REL1_18-branch
All All
: Low critical (vote)
: ---
Assigned To: Nobody - You can work on this!
: testme
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-08-25 17:17 UTC by Gregor Hagedorn
Modified: 2014-10-16 12:59 UTC (History)
4 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Gregor Hagedorn 2011-08-25 17:17:01 UTC
Running REL1.18 (http://www.species-id.net/openmedia/Special:Version). 

GlobalUsage is installed, the refresh logged many updates, directly checking the table confirms 38716 in openmedia.globalimageusage, but GlobalUsage does not find them.

----

Test example:
http://species-id.net/openmedia/File:Descurainia_sophia_0_Bl%C3%BCte_IMG_3492_Wohlers.jpg

used on:

http://offene-naturfuehrer.de/web/Sisymbrium_sophia_%E2%80%93_Gemeine_Besenrauke_%28JKI-Pflanzenportraits%29

Existing record in openmedia.globalimageusage table:

gil_wiki 	gil_page 	gil_page_namespace_id 	gil_page_namespace 	gil_page_title 	gil_to
onwiki 	9589 	0 	  	Descurainia_sophia_–_Gemeine_Besenrauke_(JKI-Pflan... 	Descurainia_sophia_0_Blüte_IMG_3492_Wohlers.jpg

However, 

http://species-id.net/o/index.php?title=Special%3AGlobalUsage&target=Descurainia_sophia_0_Blüte_IMG_3492_Wohlers.jpg

returns:

"File:Descurainia sophia 0 Blüte IMG 3492 Wohlers.jpg is not used on other wikis." Same for all other files tested so far.

----

PS: does GlobalUsage allow to be used like on commons.wikimedia.org, to display a usage at the bottom of each file page? Please mention on http://www.mediawiki.org/wiki/GlobalUsage (both if yes, only if configured, or no).
Comment 1 Bryan Tong Minh 2011-08-31 07:23:43 UTC
Does this only happen on files with a special character like ü in the name, or on all files?
Comment 2 Gregor Hagedorn 2011-08-31 08:44:07 UTC
Happens with all files. 

http://species-id.net/o/index.php?title=Special%3AGlobalUsage&target=Chenopodium_album_1a_im_Stoppelfeld_IMG_2597_Wohlers.JPG

----

Note: I assumed that all wikis would write their own images into a single global usage table. However, although 

cd /var/www/v-species/o;    sudo php ./maintenance/sql.php ./extensions/GlobalUsage/GlobalUsage.sql --conf ./LocalSettings.php

was executed only on the shared repository, all wiki checked seem to contain a 
globalimagelinks table, and this table is filled with the records for the local images of each wiki.

Since GlobalUsage.sql was definitely not executed for all wikis, either refresh does this, or a general update.php script (which was executed, due to version changes) may do this?

-----

The config for all wikis is:
require_once( "$IP/extensions/GlobalUsage/GlobalUsage.php" );
$wgGlobalUsageDatabase = "openmedia";


The documentation on mediawiki says: "In LocalSettings, set $wgGlobalUsageDatabase to the identifier of the wiki where the GlobalUsage data is located (usually the database name). It should be something understandable to wfGetDB. Example: $wgGlobalUsageDatabase = 'commonswiki';"

I interpret this that this should be the sql database name, i.e. the name in $wgDBname. Is this perhaps incorrect?
Comment 3 Bryan Tong Minh 2011-08-31 13:48:34 UTC
(In reply to comment #2)

> Note: I assumed that all wikis would write their own images into a single
> global usage table. However, although 
> 
> cd /var/www/v-species/o;    sudo php ./maintenance/sql.php
> ./extensions/GlobalUsage/GlobalUsage.sql --conf ./LocalSettings.php
> 
That is correct.

> was executed only on the shared repository, all wiki checked seem to contain a 
> globalimagelinks table, and this table is filled with the records for the local
> images of each wiki.
> 
> Since GlobalUsage.sql was definitely not executed for all wikis, either refresh
> does this, or a general update.php script (which was executed, due to version
> changes) may do this?
> 
> -----
This is indeed due to update.php and probably because of an incomplete configuration.

> 
> The config for all wikis is:
> require_once( "$IP/extensions/GlobalUsage/GlobalUsage.php" );
> $wgGlobalUsageDatabase = "openmedia";
> 
> 
> The documentation on mediawiki says: "In LocalSettings, set
> $wgGlobalUsageDatabase to the identifier of the wiki where the GlobalUsage data
> is located (usually the database name). It should be something understandable
> to wfGetDB. Example: $wgGlobalUsageDatabase = 'commonswiki';"
> 
> I interpret this that this should be the sql database name, i.e. the name in
> $wgDBname. Is this perhaps incorrect?
That is indeed incorrect. It should be the name given in a load balancer configuration [1]. I don't know myself how the load balancer works exactly or how to configure it, but perhaps GlobalUsage should be fixed to also work with a given database name.


[1] http://www.mediawiki.org/wiki/Manual:$wgLBFactoryConf
Comment 4 Gregor Hagedorn 2011-08-31 14:41:53 UTC
Question: So the extension GlobalUsage, at present cannot be used if you have a wiki farm with a single database server and no database load balancer? 

I habe tentatively added to the extension page:

#: At present, this cannot be NOT a database name of the wiki, but must be a name given in a load balancer configuration: http://www.mediawiki.org/wiki/Manual:$wgLBFactoryConf -- it might be that on a setup without a multi-master-database Loadbalancer, the extension is not usable.

I have changed the bug title - can you make it work for $wgDBname relatively easy? I assume that most mediawikis work without $wgLBFactoryConf being set. It seems consistent that if $wgLBFactoryConf is not set, all wikis are databases on the same database host, so all other settings are simply re-used, except for the database one. This would work for us. For others, an additional variable $wgGlobalUsageDatabasePrefix would fix it (one database, tables have different prefixes).
Comment 5 Bryan Tong Minh 2011-08-31 16:07:14 UTC
You can use LBFactory_Multi even if you use a single server, multiple databases setup. Unfortunately I don't have time currently to look into in more detail.
Comment 6 Gregor Hagedorn 2011-08-31 20:41:52 UTC
(In reply to comment #5)
> You can use LBFactory_Multi even if you use a single server, multiple databases
> setup. 

Not really without an example. I tried to set it up by comparison with the WMF settings, but I am not sure how it should really be working:

$wgLBFactoryConf = array(
'class' => 'LBFactory_Multi',
'sectionsByDB' => array(
 'enwiki' => 's1',
 'ispiwiki' => 's1',
 'k2n_handbook' => 's1',
 'k2n_it' => 's1',
 'k2n_sl' => 's1',
 'k2n_wiki' => 's1',
 'liaswiki' => 's1',
 'malesiana_wiki' => 's1',
 'metawiki' => 's1',
 'nnvmwiki' => 's1',
 'onwiki' => 's1',
 'openmedia' => 's1',
 'orowiki' => 's1',
 'plantnet_terms_en' => 's1',
 'plantnet_terms_fr' => 's1',
 'plantnet_uses_en' => 's1',
 'plantnet_uses_fr' => 's1',
 'plazi_wiki' => 's1',
 'pmedwiki' => 's1',
 'rosawiki' => 's1',
 'specialmedia' => 's1',
 'specieswiki' => 's1',
 'species_fr' => 's1',
 'studienstiftungswiki' => 's1',
 'testwiki' => 's1',
 'testwiki2' => 's1',
 'wbwiki' => 's1',
 'zsm_entomology' => 's1'
),
'sectionLoads' => array('s1' => array( 'db1' => 100,),),
'serverTemplate' => array(
    'dbname'      => $wgDBname,
    'user'        => $wgDBuser,
    'password'    => $wgDBpassword,
    'type'        => 'mysql',
    'flags'       => DBO_DEFAULT,
    'max lag'     => 30,
),
'groupLoadsBySection' => array(),
'groupLoadsByDB' => array(),
'hostsByName' => array('db1'  => 'localhost',),
'externalLoads' => array(),
'masterTemplateOverrides' => array(),
'externalTemplateOverrides' => array(),
'templateOverridesByCluster' => array(),
'readOnlyBySection' => array(),
);

On the positive side: after deleting all tables, and reinitializing I note:

1. maintenance/update.php will still create the table globalimagelinks in any wiki, not just in the central repository.

2. However, the refreshGlobalimagelinks commands, issued for all wikis, now write links only in the repository-globalimagelinks table.

3. The images, present in the shared repo openmedia, used in offene-naturführer.de, still report that they are not used anywhere. phpmyadmin shows that records are present in the globalimagelink table, however.

Thus the original bug still seems to exist.
Comment 7 Andre Klapper 2012-11-06 18:25:32 UTC
Gregor: Do you still face this problem?

(In reply to comment #0)
> Running REL1.18 (http://www.species-id.net/openmedia/Special:Version). 
> http://species-id.net/o/index.php?title=Special%3AGlobalUsage&target=Descurainia_sophia_0_Blüte_IMG_3492_Wohlers.jpg

"You have requested an invalid special page."

Also, that wiki runs 1.18.1. Upgrading to 1.18.5 recommended.
Comment 8 Bawolff (Brian Wolff) 2012-11-30 20:46:33 UTC
(In reply to comment #7)
> Gregor: Do you still face this problem?

Well global usage hasn't changed, so this still exists as an issue.

Arguably though it wouldn't be totally unreasonable to wontfix this and tell people to just set up $wgLBConfFactory. But it is also not unreasonable to make this work without extra effort. Hence lowering priority to low (perhaps should be lowest?)

-----

For reference this is the config you would need if you had 3 wikis, with database names wikidb1, wikidb2, and wikidb3, each with a single master and no slaves. Below is what you would put in LocalSettings.php for wikidb1 (only thing that would change is $wgDBname parameter). If you use prefixes some of the varibles for db names take the form "dbname-prefix", but i can't remember which ones do that and which have separate variables off the top of my head.


$wgGlobalUsageDatabase = 'db name where the global usage db is';
$wgDBname = 'wikidb1';
$wgDBuser = 'your db username'; //must be same for all 3 wikis in this simplified setup
$wgDBpassword = 'pass'; // Same pass must be usable for all 3 wikis
                        // In order to get cross db access to work
$wgLBFactoryConf = array(

'class' => 'LBFactory_Multi',

// This assumes all databses are on the same server
// If the databases were on different servers, you'd
// have each database host be a different s.
'sectionsByDB' => array(
    'wikidb1' => 's1', // Assumes db name for first wiki is wikidb1
    'wikidb2' => 's1', // and so on.
    'wikidb3' => 's1',
),

'sectionLoads' => array(
    's1' => array(
        'localhost'  => 0, // All on section s1, which has single master, at localhost.
    ),
),


'serverTemplate' => array(
    'dbname'      => $wgDBname,
    'user'          => $wgDBuser,
    'password'      => $wgDBpassword,
    'type'          => 'mysql',
    'flags'          => DBO_DEFAULT,
    'max lag'      => 30,
),
);
Comment 9 Gregor Hagedorn 2012-11-30 22:13:54 UTC
Thanks for asking back and apologies for the delay in answering. Yes, we never could fix this under 1.18, and we did try using the wgLBConfFactory. But we are presently testing the move to 1.20 and retest global usage with the new code.  I will update this bug as soon as I can.
Comment 10 Andre Klapper 2014-05-16 22:39:26 UTC
(In reply to Gregor Hagedorn from comment #9)
> Thanks for asking back and apologies for the delay in answering. Yes, we
> never could fix this under 1.18, and we did try using the wgLBConfFactory.
> But we are presently testing the move to 1.20 and retest global usage with
> the new code.  I will update this bug as soon as I can.

Gregor: How did this work out? And is this still a problem in a supported MediaWiki version?
Comment 11 Andre Klapper 2014-07-04 11:45:49 UTC
(In reply to Gregor Hagedorn from comment #9)
> Thanks for asking back and apologies for the delay in answering. Yes, we
> never could fix this under 1.18, and we did try using the wgLBConfFactory.
> But we are presently testing the move to 1.20 and retest global usage with
> the new code.  I will update this bug as soon as I can.

Gregor: How did this work out? And is this still a problem in a supported MediaWiki version?
Comment 12 Andre Klapper 2014-08-05 22:28:20 UTC
Gregor: Could you answer comment 11, please?
Comment 13 Andre Klapper 2014-10-16 12:59:54 UTC
Unfortunately closing this report as no further information has been provided.

Gregor: Please feel free to reopen this report if you can provide the information asked for in comment 10 - comment 12 and if this still happens. Thanks!

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links