Last modified: 2014-10-09 02:20:18 UTC
While testing the backport of Gerrit change #163976 ("Add Echo job to keep user notifications in reasonable volume") on testwiki, I got exceptions from Special:Notifications and from the Echo flyout. These continue after the backport was reverted. From a test account I added a section and a mention to https://test.wikipedia.org/wiki/User_talk:SPage_%28WMF%29 , and now SPage (WMF) gets this exception clicking the Echo [NN] badge or visiting Special:Notifications. I assume it isn't happening on other wikis. Page source includes "wgPoweredByHHVM":true;, but beta labs is also running HHVM. Unexpected non-MediaWiki exception encountered, of type "Exception" [08fd482a] /wiki/Special:Notifications Exception from line of : DateTimeZone::__construct(): Unknown or bad timezone (+00:00) Backtrace: #0 /srv/mediawiki/php-1.25wmf1/extensions/Echo/model/Event.php(225): DateTime->__wakeup() #1 /srv/mediawiki/php-1.25wmf1/extensions/Echo/model/Event.php(285): EchoEvent->loadFromRow(stdClass) #2 /srv/mediawiki/php-1.25wmf1/extensions/Echo/model/Notification.php(184): EchoEvent::newFromRow(stdClass) #3 /srv/mediawiki/php-1.25wmf1/extensions/Echo/includes/mapper/NotificationMapper.php(174): EchoNotification::newFromRow(stdClass) #4 /srv/mediawiki/php-1.25wmf1/extensions/Echo/special/SpecialNotifications.php(51): EchoNotificationMapper->fetchByUser(User, integer, NULL, array) #5 /srv/mediawiki/php-1.25wmf1/includes/specialpage/SpecialPage.php(363): SpecialNotifications->execute(NULL) #6 /srv/mediawiki/php-1.25wmf1/includes/specialpage/SpecialPageFactory.php(562): SpecialPage->run(NULL) #7 /srv/mediawiki/php-1.25wmf1/includes/MediaWiki.php(275): SpecialPageFactory::executePath(Title, RequestContext) #8 /srv/mediawiki/php-1.25wmf1/includes/MediaWiki.php(584): MediaWiki->performRequest() #9 /srv/mediawiki/php-1.25wmf1/includes/MediaWiki.php(435): MediaWiki->main() #10 /srv/mediawiki/php-1.25wmf1/index.php(46): MediaWiki->run() #11 /srv/mediawiki/w/index.php(3): include(string) #12 {main}
It could also be that my only notifications on testwiki are very very old.
Change 163979 had a related patch set uploaded by Spage: Don't fatal if a notification fails to initialize https://gerrit.wikimedia.org/r/163979
Change 163979 merged by jenkins-bot: Don't fatal if a notification fails to initialize https://gerrit.wikimedia.org/r/163979
Gerrit change #163979 logs the problem to the 'Echo' debug log. We still need to figure out why the problem occurs.
With the change to log the exceptions, the Echo debug log when I visit contains three lines 2014-10-09 02:15:02 mw1017 testwiki: EchoNotificationMapper::fetchByUser: Failed initializing event: 2325 2014-10-09 02:15:02 mw1017 testwiki: EchoNotificationMapper::fetchByUser: Failed initializing event: 2091 2014-10-09 02:15:02 mw1017 testwiki: EchoNotificationMapper::fetchByUser: Failed initializing event: 1895 while the exception.log contains three exceptions in DateTimeZone::__construct(), the same as above. ErikB comments "the problem with spage Special:Notifications is that an unserializable string was written to the echo db. I'm not sure the exact bug number but thats probably related to a recent production problem where the igbinary serialization format was mis-configured onto a server causing other production serialization problems as well."
