Last modified: 2014-01-08 16:25:40 UTC
From bug 56622: In https://gerrit.wikimedia.org/r/#/c/93939/ I made the beta autoupdater to run `npm install` under common/php-master/extensions/Parsoid/js Regardless, the parsoid instance is deployment-parsoid2 and runs its own copy of the code under /srv/deployment/parsoid . So it is still being updated manually by whoever has access on it.
Change 96270 had a related patch set uploaded by Hashar: rename misc::parsoid to role::parsoid::production https://gerrit.wikimedia.org/r/96270
I am working on this, basically: * adding deployment-parsoid2 as a jenkins slave * clearing up misc::parsoid to be usable on beta * create a job to update the job whenever a Parsoid change is merged
Change 96270 merged by Faidon Liambotis: rename misc::parsoid to role::parsoid::production https://gerrit.wikimedia.org/r/96270
+James +Gabriel, so they know what is going on. The plan is in Comment #2, aka use Jenkins to update Parsoid whenever a change is merged for it.
Completing browser tests for ULS/ MobileFrontend first then will come back to get Parsoid auto deployed. Lowering priority (still high though).
Parsoid is more out of date: (10:30:10 AM) chrismcmahon: gwicke: hashar is it true that the parsoid in beta labs is very old? https://gerrit.wikimedia.org/r/97652 (10:31:22 AM) gwicke: chrismcmahon: at least 2 weeks from what I have seen (10:31:59 AM) gwicke: there is no git checkout afaik, so my guess is based on headers that were removed then (10:32:59 AM) gwicke: before November 7th
Change 98007 had a related patch set uploaded by Hashar: (WIP) beta: autoupdate should restart parsoid (WIP) https://gerrit.wikimedia.org/r/98007
Change 98014 had a related patch set uploaded by Hashar: parsoid: role class for beta and factor out common code https://gerrit.wikimedia.org/r/98014
Change 98014 merged by Faidon Liambotis: parsoid: role class for beta and factor out common code https://gerrit.wikimedia.org/r/98014
Change 98685 had a related patch set uploaded by Hashar: beta: let jenkins-deploy restart Parsoid https://gerrit.wikimedia.org/r/98685
Change 98685 merged by ArielGlenn: beta: let jenkins-deploy restart Parsoid https://gerrit.wikimedia.org/r/98685
The jenkins-deploy user on deployment-bastion is now able to connect to the deployment-parsoid2 instance over ssh. Additionally it is able to run the parsoid init script: jenkins-deploy@deployment-bastion:~$ ssh deployment-parsoid2 jenkins-deploy@deployment-parsoid2:~$ sudo -u parsoid /etc/init.d/parsoid status * parsoid is running Now got to integrate those commands in the beta autoupdater script.
Change 98798 had a related patch set uploaded by Hashar: beta: parsoid init script needs root https://gerrit.wikimedia.org/r/98798
Change 98798 abandoned by Hashar: beta: parsoid init script needs root https://gerrit.wikimedia.org/r/98798
Change 98798 restored by Hashar: beta: parsoid init script needs root Reason: hmm it really needs root :( https://gerrit.wikimedia.org/r/98798
Change 98798 merged by ArielGlenn: beta: parsoid init script needs root https://gerrit.wikimedia.org/r/98798
Change 98007 merged by ArielGlenn: beta: autoupdate should restart parsoid https://gerrit.wikimedia.org/r/98007
Should be fine now. The Jenkins job beta-code-update runs wmf-beta-autoupdater.py which now support sshing to deployment-parsoid2 to restart the server whenever the git repository has changed somehow. https://integration.wikimedia.org/ci/job/beta-code-update/ Whenever a change is merged in Parsoid repository, I will have a look at the job and validate it is indeed working properly.
The script could not connect to deployment-parsoid2 :( 00:02:52.098 INFO:__main__:Waiting for parsoid to launch... 00:02:52.098 Host key verification failed. 00:02:57.015 INFO:__main__:Checking parsoid is running... 00:02:57.036 Host key verification failed. Caused by the script being run as mwdeploy user :/
Change 99656 had a related patch set uploaded by Hashar: parsoid: startup script now has cleared out FDs https://gerrit.wikimedia.org/r/99656
Change 99659 had a related patch set uploaded by Hashar: beta: properly connect to parsoid instance https://gerrit.wikimedia.org/r/99659
Change 99656 abandoned by Hashar: parsoid: startup script now has cleared out FDs Reason: will redo the patch using upstart https://gerrit.wikimedia.org/r/99656
Change 99656 restored by Hashar: parsoid: startup script now has cleared out FDs https://gerrit.wikimedia.org/r/99656
Change 99656 merged by ArielGlenn: beta: manage parsoid using upstart https://gerrit.wikimedia.org/r/99656
I have reenabled puppet on deployment-parsoid2.pmtpa.wmflabs and forced run puppet. Parsoid is now managed via upstart on that instance.
Change 99659 merged by ArielGlenn: beta: properly connect to parsoid instance https://gerrit.wikimedia.org/r/99659
Change 104933 had a related patch set uploaded by Hashar: beta-parsoid-update job https://gerrit.wikimedia.org/r/104933
Change 104967 had a related patch set uploaded by Hashar: update Parsoid service on beta cluster https://gerrit.wikimedia.org/r/104967
Change 104968 had a related patch set uploaded by Hashar: dashboard: add beta-parsoid-update https://gerrit.wikimedia.org/r/104968
Change 104967 merged by jenkins-bot: update Parsoid service on beta cluster https://gerrit.wikimedia.org/r/104967
Change 104933 merged by jenkins-bot: beta-parsoid-update job https://gerrit.wikimedia.org/r/104933
Change 104968 merged by Hashar: dashboard: add beta-parsoid-update https://gerrit.wikimedia.org/r/104968
Change 104971 had a related patch set uploaded by Hashar: update parsoid on beta when deploy is updated https://gerrit.wikimedia.org/r/104971
With the changes above, the Jenkins job beta-parsoid-update will be run whenever a change is merged in the master branches of mediawiki/services/parsoid and mediawiki/services/parsoid/deploy The job will checkout the repositories then rsync them locally under /srv/deployment/parsoid/ deleting any local hacks. The localsettings.js configuration is managed by puppet and is in /srv/deployment/parsoid/ The upstart script uses /srv/deployment/parsoid as a base path and it is restarted after the repositories have been copied under /srv/deployment/parsoid. I guess that should fix the bug. Will have to monitor it is working properly whenever a change is merged in one of the repositories.
Change 104971 merged by jenkins-bot: update parsoid on beta when deploy is updated https://gerrit.wikimedia.org/r/104971
Change 104978 had a related patch set uploaded by Hashar: beta: remove old parsoid updater https://gerrit.wikimedia.org/r/104978
There was a minor issue which was that repositories would not be checked out to the tip of branch in Gerrit but to the tip of the branch in Zuul repository. Which could mean an unwanted patchset would be deployed. Tracked in bug 59224 and solved by https://gerrit.wikimedia.org/r/#/c/105161/ . The job has been running a few time now and is working properly as much as I can see. So I am marking this bug resolved.
Change 104978 merged by Alexandros Kosiaris: beta: remove old parsoid updater https://gerrit.wikimedia.org/r/104978
