Last modified: 2013-11-12 23:44:56 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T58781, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 56781 - Parsoid stacktrace on beta cluster
Parsoid stacktrace on beta cluster
Status: RESOLVED FIXED
Product: Parsoid
Classification: Unclassified
Web API (Other open bugs)
unspecified
All All
: Immediate blocker
: ---
Assigned To: Gabriel Wicke
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-08 15:30 UTC by Chris McMahon
Modified: 2013-11-12 23:44 UTC (History)
11 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Chris McMahon 2013-11-08 15:30:32 UTC
http://en.wikipedia.beta.wmflabs.org/wiki/Talk:Flow_QA seen for both anon and logged in user: 


[42c45aa0] /wiki/Talk:Flow_QA Exception from line 62 of /data/project/apache/common-local/php-master/extensions/Flow/includes/ParsoidUtils.php: Failed contacting parsoid

Backtrace:

#0 /data/project/apache/common-local/php-master/extensions/Flow/includes/ParsoidUtils.php(21): Flow\ParsoidUtils::parsoid(string, string, string)
#1 /data/project/apache/common-local/php-master/extensions/Flow/includes/Model/AbstractRevision.php(315): Flow\ParsoidUtils::convert(string, string, string)
#2 /data/project/apache/common-local/php-master/extensions/Flow/includes/Model/AbstractRevision.php(290): Flow\Model\AbstractRevision->getConvertedContent(string)
#3 /data/project/apache/common-local/php-master/extensions/Flow/templates/header.html.php(19): Flow\Model\AbstractRevision->getContent(User, string)
#4 /data/project/apache/common-local/php-master/extensions/Flow/includes/Templating.php(76): include(string)
#5 /data/project/apache/common-local/php-master/extensions/Flow/includes/Templating.php(50): Flow\Templating->_render(string, array)
#6 /data/project/apache/common-local/php-master/extensions/Flow/includes/Block/Header.php(116): Flow\Templating->render(string, array)
#7 /data/project/apache/common-local/php-master/extensions/Flow/includes/View.php(65): Flow\Block\HeaderBlock->render(Flow\Templating, array)
#8 /data/project/apache/common-local/php-master/extensions/Flow/Hooks.php(155): Flow\View->show(Flow\WorkflowLoader, string)
#9 [internal function]: FlowHooks::onPerformAction(OutputPage, Article, Title, User, WebRequest, MediaWiki)
#10 /data/project/apache/common-local/php-master/includes/Hooks.php(199): call_user_func_array(string, array)
#11 /data/project/apache/common-local/php-master/includes/GlobalFunctions.php(4032): Hooks::run(string, array)
#12 /data/project/apache/common-local/php-master/includes/Wiki.php(423): wfRunHooks(string, array)
#13 /data/project/apache/common-local/php-master/includes/Wiki.php(305): MediaWiki->performAction(Article, Title)
#14 /data/project/apache/common-local/php-master/includes/Wiki.php(596): MediaWiki->performRequest()
#15 /data/project/apache/common-local/php-master/includes/Wiki.php(460): MediaWiki->main()
#16 /data/project/apache/common-local/php-master/index.php(49): MediaWiki->run()
#17 /data/project/apache/common-local/w/index.php(3): require(string)
#18 {main}
Comment 1 Oliver Keyes 2013-11-08 17:03:17 UTC
For obvious reasons, moving to blocker/immediate - we can't really test any written code (or consider sandbox releases) without a working sandbox.
Comment 2 spage 2013-11-08 19:09:22 UTC
The WMF core features team tracks this bug on Mingle card https://mingle.corp.wikimedia.org/projects/flow/cards/429, but people from the community are welcome to contribute here and in Gerrit.
Comment 3 spage 2013-11-08 20:56:14 UTC
The WMF core features team tracks this bug on Mingle card https://mingle.corp.wikimedia.org/projects/flow/cards/434, but people from the community are welcome to contribute here and in Gerrit.
Comment 4 Antoine "hashar" Musso (WMF) 2013-11-11 09:00:14 UTC
Note the error:

 Failed contacting parsoid

So it seems the parsoid daemon on beta cluster was/is dead.
Comment 5 Oliver Keyes 2013-11-11 10:45:22 UTC
Well, plausibly, but given that we got it working again on ee-flow...

Erik/S, can you chip in and explain how you un-screwed the situation?
Comment 6 Antoine "hashar" Musso (WMF) 2013-11-11 13:57:48 UTC
I have upgraded varnish on all caches including deployment-parsoidcache3.

When looking at that cache backend varnish (varnishncsa) I see requests passed to  to http://parsoid//enwiki/Main_Page , the 'parsoid' host is not resolvable so that is unlikely to work.
Comment 7 Antoine "hashar" Musso (WMF) 2013-11-11 14:08:17 UTC
Disregard my comment about 'parsoid' host not being found. The queries do hit the deployment-parsoid2 parsoid backend, using tcpdump I get:

POST //enwiki/Main_Page HTTP/1.0
Accept: */*
Accept-Encoding: deflate, gzip
Referer: http://en.wikipedia.beta.wmflabs.org/wiki/Talk:Flow_QA
User-Agent: MediaWiki/1.23alpha
Content-Length: 144
Content-Type: multipart/form-data; boundary=----------------------------3cae3ac95a02
X-Varnish: 553410961
host: parsoid
X-Forwarded-For: 10.4.0.187, 10.4.0.187, 127.0.0.1, 127.0.0.1
X-Varnish: 1458843141


It replies back with a stacktrace:

HTTP/1.1 500 Internal Server Error
X-Powered-By: Express
Content-Type: text/x-mediawiki; charset=UTF-8
Content-Length: 747
Date: Mon, 11 Nov 2013 14:05:06 GMT
Connection: close

TypeError: Cannot call method 'match' of undefined
    at Object.DOMUtils.parseHTML (/srv/deployment/parsoid/Parsoid/js/lib/mediawiki.DOMUtils.js:1714:12)
    at app.post.cb (/srv/deployment/parsoid/Parsoid/js/api/ParserService.js:677:13)
    at getParserServiceEnv (/srv/deployment/parsoid/Parsoid/js/api/ParserService.js:385:4)
    at MWParserEnvironment.getParserEnv (/srv/deployment/parsoid/Parsoid/js/lib/mediawiki.parser.environment.js:281:3)
    at setupWikiConfig (/srv/deployment/parsoid/Parsoid/js/lib/mediawiki.parser.environment.js:318:3)
    at ApiRequest._processListeners.processSome (/srv/deployment/parsoid/Parsoid/js/lib/mediawiki.ApiRequest.js:110:17)
    at process.startup.processNextTick.process._tickCallback (node.js:244:9)

So that seems to be an issue in Parsoid backend.
Comment 8 Antoine "hashar" Musso (WMF) 2013-11-11 14:17:35 UTC
I have changed Parsoid version from 4 months old cdbfdbb to 986c1e7 (the one in 1.23wmf3).

Deleted the node modules in /var/lib/parsoid/Parsoid/js/node_modules and reran npm install .

Stopped parsoid entirely using the init.d script and restarted it.


It seems to be working again :-)
Comment 9 Chris McMahon 2013-11-12 15:55:47 UTC
Antoine, can Parsoid be updated from master along with all the other core+extensions in place already?
Comment 10 Antoine "hashar" Musso (WMF) 2013-11-12 20:50:57 UTC
(In reply to comment #9)
> Antoine, can Parsoid be updated from master along with all the other
> core+extensions in place already?

There is some discussion on bug 56622 to have Parsoid support heterogeneous deployment on beta labs.  Currently the Parsoid instance is updated manually (git pull and restart of node service).
Comment 11 Gabriel Wicke 2013-11-12 23:44:56 UTC
Could https://gerrit.wikimedia.org/r/#/c/93939/ be deployed easily?

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links