Last modified: 2013-01-29 21:47:54 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T45340, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 43340 - Design new on-disk layout for MediaWiki install on tin/eqiad Apaches
Design new on-disk layout for MediaWiki install on tin/eqiad Apaches
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
General/Unknown (Other open bugs)
unspecified
All All
: High enhancement (vote)
: ---
Assigned To: Sam Reed (reedy)
:
Depends on:
Blocks: 43338 43339
  Show dependency treegraph
 
Reported: 2012-12-22 01:15 UTC by Rob Lanphier
Modified: 2013-01-29 21:47 UTC (History)
1 user (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Rob Lanphier 2012-12-22 01:15:37 UTC
One thing that will be different in from pmtpa to eqiad are file paths.  This task is about making sure the configuration files and other places can deal with different paths in different data centers.
Comment 1 Brad Jorsch 2012-12-23 17:44:00 UTC
First, let's list "the configuration files and other places".

As far as the MediaWiki config files go, "can deal with" is already done: I52a4b0ed added $wmfDatacenter which is filled in from /etc/wikimedia-site, which is being created by puppet as "pmtpa" currently and should be "eqiad" in the eqiad configuration. Anything needing different paths can switch based on this variable just like it does now with $wmfRealm (and formerly did with $cluster) if it needs something different for production vs labs. And I would hope the rest of MediaWiki goes by what's in the config files or uses paths relative to __DIR__.

There may be some maintenance scripts that hard-code paths rather than using relative paths or reading the config files; if nothing else, those too can be modified to read /etc/wikimedia-site and switch based on the value. They could even use MWRealm.php or MWRealm.sh out of operations/mediawiki-config, if those can be located.

And then there's Apache, Varnish, and so on that I would assume are dealt with directly by puppet. And I expect puppet already can easily deal with different paths in different configurations.

Anything else?
Comment 2 Rob Lanphier 2013-01-03 19:09:49 UTC
Sam and Tim will be working together on a layout on fenari, building out a symlink farm, and migrating us to the new layout.  This will hopefully happen before the switchover.
Comment 3 Sam Reed (reedy) 2013-01-07 12:49:48 UTC
[23:55:49] <TimStarling>	 Reedy: still awake?
[23:56:06] <Reedy>	 Yup
[23:56:26] <Reedy>	 Just had one of my internet connections die, just been trying to work out why
[23:59:57] <TimStarling>	 Reedy: so I see 366 references to /usr/local in the apache configuration
[00:01:04] <TimStarling>	 plus about 339 symlinks in mediawiki-config
[00:01:04] <Reedy>	 TimStarling: Is that the primary path being used?
[00:01:19] <TimStarling>	 that is the primary path in the apache configuration
[00:01:29] <TimStarling>	 in mediawiki-config I am including links to /apache since that is more common
[00:01:49] <Reedy>	 yay, consistency
[00:02:00] <TimStarling>	 not my fault
[00:02:09] <Reedy>	 wasn't blaming you
[00:02:19] <TimStarling>	 blame brion ;)
[00:02:22] <Reedy>	 heh
[00:02:35] <Reedy>	 Who do we have to <s>thank</s>blame for /h/w/c/p and stuff?
[00:02:47] <TimStarling>	 jeronim
[00:03:07] <TimStarling>	 do you know him?
[00:03:20] <TimStarling>	 I think /apache was either brion or lee
[00:03:20] <Reedy>	 don't think I do..
[00:03:36] <TimStarling>	 but /h/w was jeronim
[00:04:09] <TimStarling>	 he's australian, he did some sysadmin work for us in maybe 2004-2005
[00:05:39] <TimStarling>	 anyway, Ryan's proposed layout has various git-deploy managed repos in /srv/deployment
[00:07:31] <Ryan_Lane>	 if it's far too much of a pain in the ass, I can make the deployment system deploy to different destination locations
[00:07:37] <TimStarling>	 maybe we should try to avoid updating these 700 /usr/local/apache references more than once
[00:08:04] <TimStarling>	 we could change scap to push out to a new location, say /srv/old
[00:08:21] <TimStarling>	 then replace the old /usr/local/apache with a symlink farm
[00:08:28] * Ryan_Lane  nods
[00:08:58] <TimStarling>	 then once git-deploy is stable, update the apache configuration and document root symlinks so that we can remove /usr/local/apache
[00:23:09] <TimStarling>	 Reedy: mediawiki-config needs to be in a separate directory to the MW checkouts
[00:23:28] <TimStarling>	 let's say /srv/old/mediawiki-config
[00:23:43] <TimStarling>	 also, can we change the php- prefix now?
[00:24:06] <TimStarling>	 maybe just remove it? 1.21wmf6 seems like a good directory name to me
[00:24:40] <TimStarling>	 the php- prefix predates the name "mediawiki", it annoys me that it was kept through hetdeploy
[00:25:08] <Reedy>	 can't see why not
[00:25:09] <TimStarling>	 i.e. /srv/old/1.21wmf6
[00:26:45] <TimStarling>	 so the idea is to commit relevant changes to mediawiki-config, puppet, etc. to allow these directory structure changes
[00:27:34] <TimStarling>	 then swap the existing /usr/local/apache with a symlink farm going to /srv/old
[00:28:18] <TimStarling>	 before the swap, /usr/local/apache will have the old code in it, referring to the old directory structure
[00:28:59] <TimStarling>	 and people will have to be somehow temporarily prevented from doing a git pull on the old code location
[00:29:07] <TimStarling>	 maybe it will have to be in a separate branch of mediawiki-config
[00:29:17] <TimStarling>	 to make git-pull harmless

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links