Last modified: 2014-10-20 12:02:33 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T74009, the corresponding Phabricator task for complete and up-to-date bug report information.
First Last Prev Next    This bug is not in your last search results.
Bug 72009 - Fedora 20 ships python3 httplib2 0.7.7 with unpatched SSL cert file
Fedora 20 ships python3 httplib2 0.7.7 with unpatched SSL cert file
Status: NEW
Product: Pywikibot
Classification: Unclassified
network (Other open bugs)
core-(2.0)
PC Linux
: Unprioritized normal
: ---
Assigned To: Pywikipedia bugs
: upstream
Depends on: 72247
Blocks:
  Show dependency tree / graph
 
Reported: 2014-10-13 23:35 UTC by John Mark Vandenberg
Modified: 2014-10-20 12:02 UTC (History)
0 users

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Description John Mark Vandenberg 2014-10-13 23:35:41 UTC 
IIRC, there is a problem with certificates in httplib2 pre-0.9 (bug 65189).  FC20 ships 0.7.7 on python2 and python3

$ rpm -qi python-httplib2
Name        : python-httplib2
Version     : 0.7.7
Release     : 3.fc20
...
$ rpm -qi python3-httplib2
Name        : python3-httplib2
Version     : 0.7.7
Release     : 3.fc20
...

On py2, the 0.7.7 package works correctly.
On py3, the following occurs running site_tests

On py2, all tests pass using 0.7.7.  On python3, tests fail badly (using en.wp as default site).

$ python3 pwb.py tests/site_tests.py
.../pywikibot/family.py:879: DeprecationWarning: imp.load_source() is deprecated; use importlib.machinery.SourceFileLoader(name, pathname).load_module() instead
  myfamily = imp.load_source(fam, config.family_files[fam])
ERROR: Traceback (most recent call last):
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 992, in _conn_request
    response = conn.getresponse()
  File "/usr/lib64/python3.3/http/client.py", line 1135, in getresponse
    raise ResponseNotReady(self.__state)
http.client.ResponseNotReady: Idle

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../pywikibot/data/api.py", line 517, in submit
    headers=headers, body=body)
  File ".../pywikibot/tools.py", line 367, in wrapper
    return method(*__args, **__kw)
  File ".../pywikibot/comms/http.py", line 258, in request
    raise request.data
  File ".../pywikibot/comms/threadedhttp.py", line 219, in request
    max_redirects, connection_type
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 1272, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 1026, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 999, in _conn_request
    conn.connect()
  File "/usr/lib64/python3.3/http/client.py", line 1202, in connect
    server_hostname=server_hostname)
  File "/usr/lib64/python3.3/ssl.py", line 245, in wrap_socket
    _context=self)
  File "/usr/lib64/python3.3/ssl.py", line 345, in __init__
    raise x
  File "/usr/lib64/python3.3/ssl.py", line 341, in __init__
    self.do_handshake()
  File "/usr/lib64/python3.3/ssl.py", line 548, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:547)

WARNING: Waiting 5 seconds before retrying.
ERROR: Traceback (most recent call last):
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 992, in _conn_request
    response = conn.getresponse()
  File "/usr/lib64/python3.3/http/client.py", line 1135, in getresponse
    raise ResponseNotReady(self.__state)
http.client.ResponseNotReady: Idle

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../pywikibot/data/api.py", line 517, in submit
    headers=headers, body=body)
  File ".../pywikibot/tools.py", line 367, in wrapper
    return method(*__args, **__kw)
  File ".../pywikibot/comms/http.py", line 258, in request
    raise request.data
  File ".../pywikibot/comms/threadedhttp.py", line 219, in request
    max_redirects, connection_type
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 1272, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 1026, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 999, in _conn_request
    conn.connect()
  File "/usr/lib64/python3.3/http/client.py", line 1202, in connect
    server_hostname=server_hostname)
  File "/usr/lib64/python3.3/ssl.py", line 245, in wrap_socket
    _context=self)
  File "/usr/lib64/python3.3/ssl.py", line 345, in __init__
    raise x
  File "/usr/lib64/python3.3/ssl.py", line 341, in __init__
    self.do_handshake()
  File "/usr/lib64/python3.3/ssl.py", line 548, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:547)

WARNING: Waiting 10 seconds before retrying.
^CTraceback (most recent call last):
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 992, in _conn_request
    response = conn.getresponse()
  File "/usr/lib64/python3.3/http/client.py", line 1135, in getresponse
    raise ResponseNotReady(self.__state)
http.client.ResponseNotReady: Idle

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../pywikibot/data/api.py", line 517, in submit
    headers=headers, body=body)
  File ".../pywikibot/tools.py", line 367, in wrapper
    return method(*__args, **__kw)
  File ".../pywikibot/comms/http.py", line 258, in request
    raise request.data
  File ".../pywikibot/comms/threadedhttp.py", line 219, in request
    max_redirects, connection_type
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 1272, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 1026, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/lib/python3.3/site-packages/httplib2/__init__.py", line 999, in _conn_request
    conn.connect()
  File "/usr/lib64/python3.3/http/client.py", line 1202, in connect
    server_hostname=server_hostname)
  File "/usr/lib64/python3.3/ssl.py", line 245, in wrap_socket
    _context=self)
  File "/usr/lib64/python3.3/ssl.py", line 345, in __init__
    raise x
  File "/usr/lib64/python3.3/ssl.py", line 341, in __init__
    self.do_handshake()
  File "/usr/lib64/python3.3/ssl.py", line 548, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:547)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "pwb.py", line 164, in <module>
    run_python_file(fn, argv, argvu)
  File "pwb.py", line 69, in run_python_file
    exec(compile(source, filename, "exec"), main_mod.__dict__)
  File "tests/site_tests.py", line 1481, in <module>
    unittest.main()
  File "/usr/lib64/python3.3/unittest/main.py", line 125, in __init__
    self.runTests()
  File "/usr/lib64/python3.3/unittest/main.py", line 265, in runTests
    self.result = testRunner.run(self.test)
  File "/usr/lib64/python3.3/unittest/runner.py", line 168, in run
    test(result)
  File "/usr/lib64/python3.3/unittest/suite.py", line 67, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib64/python3.3/unittest/suite.py", line 105, in run
    test(result)
  File "/usr/lib64/python3.3/unittest/suite.py", line 67, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib64/python3.3/unittest/suite.py", line 105, in run
    test(result)
  File "/usr/lib64/python3.3/unittest/case.py", line 529, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib64/python3.3/unittest/case.py", line 477, in run
    self._executeTestPart(testMethod, outcome, isTest=True)
  File "/usr/lib64/python3.3/unittest/case.py", line 422, in _executeTestPart
    function()
  File "tests/site_tests.py", line 1065, in testRandompages
    rn = list(mysite.randompages(total=10))
  File ".../pywikibot/site.py", line 3574, in randompages
    g_content=content)
  File ".../pywikibot/site.py", line 1357, in _generator
    gen = gen_class(type_arg, site=self, **args)
  File ".../pywikibot/data/api.py", line 1100, in __init__
    QueryGenerator.__init__(self, generator=generator, **kwargs)
  File ".../pywikibot/data/api.py", line 819, in __init__
    self.update_limit()  # sets self.prefix
  File ".../pywikibot/data/api.py", line 914, in update_limit
    for param in self._modules[mod].get("parameters", []):
  File ".../pywikibot/data/api.py", line 865, in _modules
    data = paramreq.submit()
  File ".../pywikibot/data/api.py", line 764, in submit
    self._data = super(CachedRequest, self).submit()
  File ".../pywikibot/data/api.py", line 535, in submit
    self.wait()
  File ".../pywikibot/data/api.py", line 651, in wait
    time.sleep(self.retry_wait)
KeyboardInterrupt
<class 'KeyboardInterrupt'>
CRITICAL: Waiting for 1 network thread(s) to finish. Press ctrl-c to abort
Comment 1 John Mark Vandenberg 2014-10-19 15:35:50 UTC 
In addition, setting Family.ignore_certificate_error does not work in py3 (bug 72236 - upstream)
Comment 2 John Mark Vandenberg 2014-10-20 12:02:33 UTC 
The reason for the problem with py3 is Fedora patches httplib2-0.7.7 to use the system certs, but only for py2.  Therefore python3-httplib2-0.7.7 is using the certs included in the distributed 0.7.7 tarball, which is mostly junk data nowadays.

fwiw, it looks like FC21 is likely to be based on py34 and include httplib2-0.9

Our typical workarounds for httplib2 work well; either install 0.9 using pip, or recursively clone the pywikibot repo, so it can use 'externals/httplib2' - both tested and work correctly.
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.

Navigation
Links