Last modified: 2014-08-20 19:35:42 UTC
It doesn't seem to be related to any memory / time limits. The output of the script is just truncated. It works when run under php. (php below is aliased to hhvm) php extensions/Translate/scripts/createMessageIndex.php --memory-limit=max [e18f17bf] [no req] Exception from line 113 of /www/dev.translatewiki.net/w/extensions/Translate/ffs/PythonSingleFFS.php: Failed to decode python file /resources/projects/pywikibot/add_text.py Backtrace: #0 /www/dev.translatewiki.net/w/extensions/Translate/ffs/PythonSingleFFS.php(86): PythonSingleFFS->parseFile() #1 /www/dev.translatewiki.net/w/extensions/Translate/messagegroups/FileBasedMessageGroup.php(52): PythonSingleFFS->read(string) #2 /www/dev.translatewiki.net/w/extensions/Translate/messagegroups/MessageGroupBase.php(81): FileBasedMessageGroup->load(string) #3 /www/dev.translatewiki.net/w/extensions/Translate/messagegroups/MessageGroupBase.php(183): MessageGroupBase->getDefinitions() #4 /www/dev.translatewiki.net/w/extensions/Translate/utils/MessageIndex.php(186): MessageGroupBase->getKeys() #5 /www/dev.translatewiki.net/w/extensions/Translate/utils/MessageIndex.php(119): MessageIndex->checkAndAdd(array, FileBasedMessageGroup) #6 /www/dev.translatewiki.net/w/extensions/Translate/scripts/createMessageIndex.php(31): MessageIndex->rebuild() #7 /www/dev.translatewiki.net/w/maintenance/doMaintenance.php(109): CreateMessageIndex->execute() #8 /www/dev.translatewiki.net/w/extensions/Translate/scripts/createMessageIndex.php(36): include(string) #9 {main} The relevant code: $command = wfEscapeShellArg( "import json; execfile(\"$filename\"); " . "print json.dumps(msg)" ); $json = wfShellExec( "python -c $command" ); $parsed = FormatJson::decode( $json, true ); if ( !is_array( $parsed ) ) { throw new MWException( "Failed to decode python file $filename" ); }
Thanks for the report. Could you var_dump() the value of $json after the wfShellExec() invocation and the value of $parsed after the FormatJson::decode() invocation?
Created attachment 15922 [details] Output returned by wfShellExec I added the following line after the decode call: var_dump( 'JSON', $json, 'PARSED', $parsed ); die(); The result is in the attachment. Interesting thing is that the length of the truncated output is 8192 bytes.
It's a general wfShellExec() problem.
Change 147635 had a related patch set uploaded by MaxSem: Fix race condition in wfShellExec() https://gerrit.wikimedia.org/r/147635
Change 147635 merged by jenkins-bot: Fix race condition in wfShellExec() https://gerrit.wikimedia.org/r/147635
Change 148419 had a related patch set uploaded by MaxSem: Fix race condition in wfShellExec() https://gerrit.wikimedia.org/r/148419
Change 148419 merged by jenkins-bot: Fix race condition in wfShellExec() https://gerrit.wikimedia.org/r/148419