Last modified: 2013-05-07 18:38:05 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T49639, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 47639 - Jenkins: Job mediawiki-core-qunit fails sometimes due to database being read-only
Jenkins: Job mediawiki-core-qunit fails sometimes due to database being read-...
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
Continuous integration (Other open bugs)
wmf-deployment
All All
: Highest critical (vote)
: ---
Assigned To: Krinkle
:
: 47855 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-04-25 13:11 UTC by Krinkle
Modified: 2013-05-07 18:38 UTC (History)
5 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Comment 1 Krinkle 2013-04-30 15:06:11 UTC
*** Bug 47855 has been marked as a duplicate of this bug. ***
Comment 2 Krinkle 2013-04-30 15:07:09 UTC
Raising priority. Happening on more branches.

Something is causing it to be read-only. Needs to be fixed soon. Blocking smooth development.
Comment 3 Krinkle 2013-04-30 15:08:42 UTC
I've ruled out relation to QUnit, PhantomJS or Grunt as those are the same across all branches.

The error from PhantomJS (about .start not being called) is bogus. The problem is that the page doesn't contain QUnit at all (or MediaWiki for that matter) since it is an HTTP 500 response:



23:16:50 HTTP/1.1 500 MediaWiki exception
23:16:50 Date: Mon, 29 Apr 2013 23:16:51 GMT
23:16:50 Server: Apache/2.2.22 (Ubuntu)
23:16:50 X-Powered-By: PHP/5.3.10-1ubuntu3.5+wmf1
23:16:50 X-Content-Type-Options: nosniff
23:16:50 Status: 500 MediaWiki exception
23:16:50 Vary: Accept-Encoding
23:16:50 Content-Length: 392
23:16:50 Connection: close
23:16:50 Content-Type: text/html; charset=utf-8
23:16:50 
23:16:50 <!doctype html>
23:16:50 <html><head><title>Database error</title></head><body>
23:16:50 A database error has occurred.  Did you forget to run
maintenance/update.php after upgrading?  See:
https://www.mediawiki.org/wiki/Manual:Upgrading#Run_the_update_script<br />
23:16:50 Query: DELETE FROM msg_resource<br />
23:16:50 Function: MessageBlobStore::clear<br />
23:16:50 Error: 8 attempt to write a readonly database<br />
23:16:50 </body></html>
Comment 4 Antoine "hashar" Musso (WMF) 2013-04-30 16:09:30 UTC
So I guess that is because several queries hits MessageBlobStore::clear() and possibly over writing queries. SQLite only accepts one connection so multiples HTTP queries doing writes to the database would cause the 'readonly issue'.

I am not sure whether we can solve that.
Comment 5 Krinkle 2013-04-30 18:33:12 UTC
The multiple connections seemed plausible at first, though not on second thought.

Lots of people use sqlite for their MediaWiki installs. MediaWiki and/or php-sqlite can handle that.

Also, it is not happening on master builds, only on older branches.
Comment 6 Antoine "hashar" Musso (WMF) 2013-04-30 18:36:50 UTC
Following a discussion with Timo:

- this happen constently with REL1_20
- no issue with REL1_21

Maybe it can be bisected to find out the commit that fixed the issue.
Comment 7 Andre Klapper 2013-05-02 12:10:43 UTC
(In reply to comment #2 by Krinkle)
> Happening on more branches.

(In reply to comment #6 by Hashar)
> - this happen constently with REL1_20
> - no issue with REL1_21

Asking for clarification: So this only happens with REL1_20, not with git master or REL1_19 (if there are builds)?
Comment 8 Krinkle 2013-05-02 15:15:43 UTC
(In reply to comment #7)
> (In reply to comment #2 by Krinkle)
> > Happening on more branches.
> 
> (In reply to comment #6 by Hashar)
> > - this happen constently with REL1_20
> > - no issue with REL1_21
> 
> Asking for clarification: So this only happens with REL1_20, not with git
> master or REL1_19 (if there are builds)?

I'm not sure about REL1_19, but assuming this is related to a (yet to be identified) problem with the Database interface in MediaWiki that was fixed in 1.21, I'd guess that this will fail for REL1_19 as well just like it fails for REL1_20.

REL1_21 and master work fine.
Comment 9 Krinkle 2013-05-02 15:16:17 UTC
If that is indeed problem, we need to identify the fix (we already fixed it, apparently) and backport to REL1_20/REL1_19.
Comment 10 Antoine "hashar" Musso (WMF) 2013-05-06 19:04:20 UTC
Resetting assignee, I am not working on this sorry.
Comment 11 Krinkle 2013-05-06 20:33:37 UTC
Assigning to myself.

Using reverse git-bisect (good=bad, bad=good) has brought me to 1da33a49a0a95ad7066926ca1a1a771411074946 as the point where the problem was fixed.
Comment 12 Gerrit Notification Bot 2013-05-06 20:34:04 UTC
Related URL: https://gerrit.wikimedia.org/r/62519 (Gerrit Change I9441d897d45bca189c17b8bcca4ea7b3469af8b0)
Comment 13 Krinkle 2013-05-06 20:45:03 UTC
Looking at this change tells me something else however.

Though it is (beyond a doubt) the commit that makes the readonly warning disappear, it does just that.

The readonly problem is real, and not caused by something in MediaWiki 1.20, instead it is happening on 1.21 and master as well, but MediaWiki is no longer throwing on it.

Let's backport the change and then fix the chmod of the sqlite file as well.
Comment 14 Gerrit Notification Bot 2013-05-07 16:46:03 UTC
Related URL: https://gerrit.wikimedia.org/r/62639 (Gerrit Change Id3862690f035d026b579403fb2b4d2c2dfb62617)
Comment 15 Krinkle 2013-05-07 18:38:05 UTC
Fixed.

* Backported Idc83a0fe042806263f to fix the mediawiki fatal caused by
  ResourceLoader previously not accounting for read-only mode.

* Submitted Id3862690f035d026b to fix the fact the db is not writable by Apache.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links