Last modified: 2012-01-24 23:21:02 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T35767, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 33767 - add to wfGetDB an ability to change server
add to wfGetDB an ability to change server
Status: RESOLVED WORKSFORME
Product: MediaWiki
Classification: Unclassified
Database (Other open bugs)
1.18.x
All All
: Unprioritized normal (vote)
: ---
Assigned To: Nobody - You can work on this!
: crosswiki
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-01-17 07:01 UTC by Pavel Malakhov
Modified: 2012-01-24 23:21 UTC (History)
4 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Pavel Malakhov 2012-01-17 07:01:25 UTC
Currently (MW 1.18.1) wfGetDB() is able to access to any wiki on the server by setting/changing wikiID. For example:
$wikiID = $wiki->db['name'] ."-". $wiki->prefix;
$groups = array();        //fake array for wfGetDB
$dbr = wfGetDB( DB_SLAVE, $groups, $wikiID );

But there is no way how we can use wfGetDB() to access the DB on other servers.

I need this feature in WikiFarm  extention (http://www.mediawiki.org/wiki/Extension:WikiFarm) in order to manager several wikis on several servers.
Comment 1 Bawolff (Brian Wolff) 2012-01-18 13:54:07 UTC
I was under impression this already worked provided LoadBalancer was set up the right way (Admittedly the documentation there is lacking)
Comment 2 Pavel Malakhov 2012-01-19 06:21:07 UTC
(In reply to comment #1)
> I was under impression this already worked provided LoadBalancer was set up the
> right way (Admittedly the documentation there is lacking)

LoadBalancer (as I undersand) aimed to split load on one wiki, i.e. all servers in $wgDBservers have the same wiki on them. I need to work with different wikis. I still can use wfGetDB() as long as my wikis on one server, but it becomes useless if I'd like to manage wiki from another server.
Comment 3 Bawolff (Brian Wolff) 2012-01-19 06:29:42 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > I was under impression this already worked provided LoadBalancer was set up the
> > right way (Admittedly the documentation there is lacking)
> 
> LoadBalancer (as I undersand) aimed to split load on one wiki, i.e. all servers
> in $wgDBservers have the same wiki on them. I need to work with different
> wikis. I still can use wfGetDB() as long as my wikis on one server, but it
> becomes useless if I'd like to manage wiki from another server.

Not necessarily. See for example [[mw:Extension:GlobalUsage]] which uses wfGetDB to load data from a foreign wiki on (potentially) a different server. [See for example wikimedia's rather complex config - http://noc.wikimedia.org/conf/highlight.php?file=db.php GlobalUsage is in use, but commons and en wikipedia sit on different db servers from my understanding]

marking worksforme - but please feel free to re-open if you feel there's still stuff to do here.
Comment 4 Pavel Malakhov 2012-01-19 07:45:09 UTC
1) Read Extension:GlobalUsage description and looked through the code => it uses LoadBalancer => does not how to fit it in my needs. No documentaion on my case.

2) looked through db.php... Please point me to the line I should pay attention to. If it is the $wgLBFactoryConf settings then... yes, it is rather complex :)

Anyway, if the solusion is to use LoadBalancer then it is too complacated one, when we need just to ask a foreing wiki some simple sql question (f.e. get a page).
It would be nice to use something like this:

$wikiID[17] = array(
    'server'   => $wgDBserver,
    'name'     => $wgDBname,
    'user'     => $wgDBuser,
    'password' => $wgDBpassword,
    'prefix'   => 'main__',
);

$dbr = wfGetDB( DB_SLAVE, array(), $wikiID[17] );

Or somehow have an ability to pass all the connection parameters to wfGetDB().
Comment 5 Aaron Schulz 2012-01-19 08:01:30 UTC
See http://noc.wikimedia.org/conf/highlight.php?file=db.php. We have DBs on different servers.

You can also use DatabaseBase::factory() directly, unless you want load balancing for each connection and each DB has a master/slave setup.
Comment 6 Daniel Friesen 2012-01-19 08:08:10 UTC
All wfGetDB does is make a call to wfGetLB which returns the primary load
balancer for a wiki and then makes a getConnection call on it.

If all you're doing is trying to access a 3rd party database without a load
balancer then you should just use DatabaseBase::newFromType (previously
::factory) directly.
Comment 7 Aaron Schulz 2012-01-19 08:12:01 UTC
(In reply to comment #6)
> All wfGetDB does is make a call to wfGetLB which returns the primary load
> balancer for a wiki and then makes a getConnection call on it.
> 
> If all you're doing is trying to access a 3rd party database without a load
> balancer then you should just use DatabaseBase::newFromType (previously
> ::factory) directly.

Note that newFromType() is marked as deprecated.
Comment 8 Bawolff (Brian Wolff) 2012-01-19 08:16:16 UTC
Note, for an example of code that does that, see includes/filerepo/ForeignDBRepo.php
Comment 9 Pavel Malakhov 2012-01-19 10:14:15 UTC
1) to Aaron Schulz:
I can see in db.php that you use several servers, but I don't understand how I
can use it... Is there a simple example how to set several servers (with different wikis) and connect to them in write mode?

2) to Daniel Friesen:
Can you provide an URL for more info on using DatabaseBase::newFromType ?
I could not find a useful page on mediawiki.org

3) to Bawolff
I looked through the code. It seems like this approach can only read data, but I need to write it as well. Is it possible?
Comment 11 Pavel Malakhov 2012-01-24 22:57:28 UTC
Excuse me for annoying you. But this is not a bug report, but a feature request for wfGetDB().
So it is fair to close it (as I understand) in one of the following cases:
1. the feature is implemented
2. there is another way to perform the same task (and the existed way is simple and comfortable just the same)
3. there is no way to implement the requested feature
None of above is achieved yet, so forgive me for reopening the request. It is still urgent.


1) As to item "1": your suggestions and links helped me to understand (by investigating the code) that wfGetDB() is binded to LoadBalancer. Using LoadBalancer for connection to foreign databases is
a) rather complex
b) most likely lead to errors when connect to wiki, which uses LoadBalancer (with several servers) itself


2) We can use class DatabaseMysql:
$db = new DatabaseMysql();
if ( $db->Open($wiki->db['server'], $wiki->db['user'], $wiki->db['password'], $wiki->db['name']) ) {...

But when try to work with DB I stumble upon the automated 'correction' of DatabaseBase::tableName() which "adds a table prefix if only given a table name with no quotes". In real life that means that the code
$lastmod = $db->selectField( 'recentchanges', 'MAX(rc_timestamp)', false, __METHOD__ );

becomes incorrect for any DB where table prefix is different from local wiki tables. The code should be modified to:
$table_recentchanges = '`'. $wiki->prefix .'recentchanges`';
$lastmod = $db->selectField( $table_recentchanges, 'MAX(rc_timestamp)', false, __METHOD__ );

That turns the code to rather complecated one when you start to build SQL statements (and I could not make it work in some cases yet)


3) As to first item: the optional parameter "server" could be added to wfGetDB() and treated by MediaWiki engine with or without LoadBalancer by checking if it (the server) exists in the LoadBalancer servers array.
Or/And 
As to second item: the option "tablePrefix" could be added to DatabaseBase::Open and to constructor of DatabaseMysql (particularly) class
Comment 12 Bawolff (Brian Wolff) 2012-01-24 23:21:02 UTC
As said previously

>1. the feature is implemented

It is, provided that $wgLBFactoryConf is configured correctly so it can map wiki-id to database. (The fact $wgLBFactoryConf is poorly documented, and somewhat complex is a separate issue. If the $wgLBFactoryConf setting format is unduly complicated given the audiance of your extension, you could potentially even write a class that extends LBFactory which just wraps around the existing complicated class, similar to what LBFactory_Simple does)


>2. there is another way to perform the same task (and the existed way is simple
>and comfortable just the same)

Database::factory (Not new DatabaseMysql). Assuming $dbType, $dbServer, etc is set to something appropriate:

$dbw = DatabaseBase::factory( $dbType,
                                array(
                                        'host' => $dbServer,
                                        'user'   => $dbUser,
                                        'password' => $dbPassword,
                                        'dbname' => $dbName,
                                        'flags' => $dbFlags,
                                        'tablePrefix' => $dbtablePrefix
                                )
                        );


----
Adding another parameter to wfGetDB probably isn't going to happening

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links