Last modified: 2014-07-14 15:15:07 UTC
When a tool attempts to connect to any of the tools.wmflabs.org/* addresses you get timeout issues
That's an openstack limitation; its networking layer is actually unable to route from the inside to floating IPs. You can, on the other hand, connect to the /internal/ IP of the proxy to the same effect: tools-webproxy
That just seems like a hack, and adds an additional level of coding (detecting when functioning from within labs and from external) I know petan said he had a workaround to fix this.
Can't we work around OpenStack's limitations with an entry in /etc/hosts? In general, however, IMHO we should *strongly* discourage tools from accessing the webserver internally. This not only wouldn't make use of the load-balancing grid, but put focused stress on the webserver, it also replaces the simplicity of an exec() call with the complexity of remote IPC, i. e. the network being available and not timing out, the webserver functioning as designed, the sub-job signalling its exit status properly back to the master, etc.
That all depends on how its used, If say I wanted to use the output of someone elses tool, I have three options, some kind of internal only hack (using internal host names and what not) This only works from within labs, so if I wanted to share a function between something on labs and something on my local machine I would need to create a kludgey hack to see if it was being executed from within labs or not, and a fallback method for when its not being used on labs. OR I could just use one sane process thats not fucked up and not more complex than quantum mechanics and just use the tools.wmflabs.org url. One example is there is a tool on the toolserver that checks if an ip is part of tor or not. I incorporated the results of that tool into a more complex and more detailed report about an IP address. If that tool moves to labs I can no longer use it.
Whilst clearly ugly, stuffing the proxy address in /etc/hosts should provide a suitable workaround; at least until we move to a new networking layer for OpenStack.
"Fixed" (that is, tools.wmflabs.org now resolves to the web proxy's internal address).
I would assume since the move to eqiad, it no longer functions correctly. Currently unable to access tools.wmflabs urls
Change 123149 had a related patch set uploaded by Tim Landscheidt: Tools: Alias tools.wmflabs.org to internal webproxy https://gerrit.wikimedia.org/r/123149
*** Bug 63787 has been marked as a duplicate of this bug. ***
To me and my bot this is a blocker - so please continue and merge.
You can easily work around that by using tools-webproxy as Coren wrote in comment #1.
(In reply to Tim Landscheidt from comment #11) > You can easily work around that by using tools-webproxy as Coren wrote in > comment #1. I don't understand how this should solve the issue? Could you please explain? I assume - as Betacommand pointed out in comment #2 - this works only if pywikibot code would be changed? Right?
(In reply to DrTrigon from comment #12) > I don't understand how this should solve the issue? Could you please explain? Connect to tools-webproxy (i.e. using the internal IP) instead of tools.wmflabs.org (the external IP). > I assume - as Betacommand pointed out in comment #2 - this works only if > pywikibot code would be changed? Right? How is this related to pywikibot?
subster.py takes user-defined urls and retrieves their content - how can I tell pywikibot to genrally use tools-webproxy instead of tools.wmflabs.org? According to [1] it works if I run: export HTTP_PROXY=tools-webproxy first, but then pages not on tools.wmflabs.org do not work anymore. [1] http://ehc.ac/p/pywikipediabot/mailman/message/11020503/ How to solve?
tools-webproxy is not a proxy server you should use, but it's the internal address for tools.wmflabs.org. However, this does show there is a clear need for tools.wmflabs.org to be accessible *on that address* from the inside: tools such as weblinkchecker and subster take URLs that are used on a wiki, and do something with that URL.
+1 If OpenStack has (DNS)-related limits here, maybe a hosts-entry can fix this issue.At least on Grid-/Exec-nodes and both Bastions.
So when will there be a fix (resp. merge of the proposed on above)? As mentioned this a blocker for me and I cannot continue with testing, migrating from TS etc.
*** Bug 64701 has been marked as a duplicate of this bug. ***
Change 123149 abandoned by coren: Tools: Alias tools.wmflabs.org to internal webproxy Reason: This is moot (and was not the right way to do it -- /etc/host is generated programmatically from the maintain-replicas script) https://gerrit.wikimedia.org/r/123149
There is now a local alias on all instances to point the public name at the private IP (though not done with the proposed patch).
(In reply to Gerrit Notification Bot from comment #19) > Change 123149 abandoned by coren: > Tools: Alias tools.wmflabs.org to internal webproxy > Reason: > This is moot (and was not the right way to do it -- /etc/host is generated > programmatically from the maintain-replicas script) > https://gerrit.wikimedia.org/r/123149 No, it doesn't looking at <http://git.wikimedia.org/blob/operations%2Fsoftware.git/7a1eac5e4e9a6bc16b0ec052acfe068c8db79472/maintain-replicas%2Fmaintain-replicas.pl>, and no other script in that repository creates an alias for tools.wmflabs.org either. /etc/hosts is unmaintained at the moment.
Sorry, unmerged forked of it; bringing it in back to git is on my todo. Same idea. :-)
(In reply to Marc A. Pelletier from comment #22) > Sorry, unmerged forked of it; bringing it in back to git is on my todo. > Same idea. :-) Then let's leave this bug open until it's merged in case you get run over by a bus tomorrow :-).
What Tim said :)
The actual problem being fixed; bumping down.
Change 146000 had a related patch set uploaded by Tim Landscheidt: Tools: Add IP mapping for tools.wmflabs.org https://gerrit.wikimedia.org/r/146000
Change 146000 merged by coren: Tools: Add IP mapping for tools.wmflabs.org https://gerrit.wikimedia.org/r/146000
And finally :)
