Last modified: 2014-10-15 22:30:41 UTC
Browser tests explicitly load pages and click buttons that load pages. Neither seems to detect outright page load failures. E.g. https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/263/testReport/%28root%29/Actions%20menu%20Permalink/Topic_Actions_menu_Permalink/ failed with status 500 response and a printed exception, but the test instead reported Sauce Labs job URL: http://saucelabs.com/jobs/b15aad5529da47ea979bb0196f17c320 timed out after 5 seconds, waiting for {:css=>".flow-topic h2", :index=>0, :tag_name=>"h2"} to become present (Watir::Wait::TimeoutError) Like "page has no ResourceLoader errors" (implemented) and "browser tests should assert there are no pink error boxes on the page" (bug 61304), most tests should have an easy way to indicate that an error in a page load should fail outright with something like "unexpected HTTP response (500)". However, note the comment in http://stackoverflow.com/a/5629462/451712 : "There's a bit of a philosophical debate on whether watir or watir-webdriver should provide HTTP return code information." so this may be hard to fix. Since beta and production show errors slightly differently, and some HTTP response errors have friendlier HTML content like "Wikipedia does not have an article with this exact name", I'm not sure how to easily and generically detect from page content that there's been a breakdown, even though it's obvious to a human being :)
It's nothing to do with watir, the Selenium maintainers have been explicit from the very beginning that Selenium will never read HTTP response codes. If you care about those, you'll need to run your HTTP through a proxy of some sort and do the analysis there and not in the browser.
marking WONTFIX, see https://code.google.com/p/selenium/issues/detail?id=141 and http://lmgtfy.com/?q=selenium+webdriver+http+response+code
