Last modified: 2013-12-06 06:19:35 UTC
I have setup up browsertests in Wikimedia Jenkins setup (bug 54388). The first run console using phantomjs browser dies miserably with some "Unable to pick a platform for the provided browser (RuntimeError)" errors. Full output: https://integration.wikimedia.org/ci/job/qa-browsertests-run/14/console 48 scenarios (25 failed, 23 skipped)
Blocks bug 53697 "trigger browser tests from Gerrit" since it would be nice to have tests passing :-]
As far as I understand it, console says phantomjs is not installed. For example: https://integration.wikimedia.org/ci/job/qa-browsertests-run/lastCompletedBuild/consoleFull ... 03:47:27 Unable to find phantomjs executable. (Selenium::WebDriver::Error::WebDriverError) ..
I forgot to install phantomjs during my crazy night. Next issue: unable to connect to phantomjs @ http://127.0.0.1:8910 after 20 seconds (Selenium::WebDriver::Error::WebDriverError That is supposed to be the port of the phantomjs Selenium driver vendor/ruby/1.9.1/gems/selenium-webdriver-2.35.0 \ ... /lib/selenium/webdriver/phantomjs/service.rb 12: DEFAULT_PORT = 8910
The timeout is caused by {{bug|45868}}, that is, instances cant not access the public IP address. One has to rewrite outgoing packets using iptables rules as a workaround: iptables -t nat -I OUTPUT --dest 208.80.153.219 -j DNAT --to-dest 10.4.1.133 iptables -t nat -I OUTPUT --dest 208.80.153.242 -j DNAT --to-dest 10.4.0.211 iptables -t nat -I OUTPUT --dest 208.80.153.243 -j DNAT --to-dest 10.4.0.51 iptables -t nat -I OUTPUT --dest 208.80.153.244 -j DNAT --to-dest 10.4.0.48 iptables -t nat -I OUTPUT --dest 208.80.153.243 -j DNAT --to-dest 10.4.1.82 No more timeout: https://integration.wikimedia.org/ci/job/qa-browsertests-run/20/console
We know have some issues such as: timed out after 5 seconds, waiting for {:text=>"Yes", :tag_name=>"span"} to become present (Watir::Wait::TimeoutError) Error Message => 'Can't find variable: mw' Željko told me it is because the pages do not look the same under phantomJS, thus the tests would need to be fixed :-)
See related thread at QA mailing list: http://lists.wikimedia.org/pipermail/qa/2013-November/000723.html
Following a pair session with Željko, the tests can be flagged with a tag which we can then exclude when running tests. For ULS, I have introduced the tag @specific-settings which let us skip any tests that are not going to pass on a fresh wiki installation: https://gerrit.wikimedia.org/r/#/c/97487/
Antoine, is this fixed with this commit? https://gerrit.wikimedia.org/r/#/c/97487/
https://gerrit.wikimedia.org/r/#/c/97487/ was for ULS. This bug is for qa/browsertest. The idea was that whenever a change is proposed (or merged?) for qa/browsertests, we would have run the whole test suites using phantomJS on the WMF Jenkins. Lot of tests being migrated to extensions I don't think it make sense anymore. We might want to repurpose qa/browsertest as smoke tests targeting production which would be quite useful for monitoring. So either wont fix or resolve later on :-D
In that case, resolved wontfix. We can open new bugs for specific repositories if needed.
