Last modified: 2014-01-03 15:53:00 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T56690, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 54690 - Tools-exec instances are not running the same software installs
Tools-exec instances are not running the same software installs
Status: RESOLVED FIXED
Product: Wikimedia Labs
Classification: Unclassified
tools (Other open bugs)
unspecified
All All
: Highest blocker
: ---
Assigned To: Marc A. Pelletier
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-09-27 11:18 UTC by Dirk Beetstra
Modified: 2014-01-03 15:53 UTC (History)
4 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Dirk Beetstra 2013-09-27 11:18:18 UTC
COIBot fails to run on tools-exec-08 because of missing perl-modules (the first it encounters is WWW::Mechanize, but likely the others are missing as well).  It has always been running fine on earlier instances, in fact, manually running it on tools-exec-01 does not result in any errors.

Can the tools-exec-## instances please be made all exactly the same so that in case a bot restarts on another instance it does run smoothly?

Thanks.

Dirk Beetstra
Comment 1 Yuvi Panda 2013-09-30 06:56:13 UTC
Can you tell me what modules exactly you require so I can add them to puppet?
Comment 2 Dirk Beetstra 2013-09-30 14:03:41 UTC
I will try to make a list (though, with dependencies .. ).

However, I am not the only one running perl-bots, and I presume there are others.  Also, does this have to be repeated when tools-exec-09 is spawned?  Is it not easier to have them all the same as the 01 or 02 (COIBot is running on 02 at the moment ..).  Also, I presume that for others running bots in other frameworks, similar situations exist.
Comment 3 Yuvi Panda 2013-09-30 14:08:45 UTC
The problem currently seems to be that some packages were manually apt-get installed on the exec hosts, thus causing problems when new exec nodes are spawned. The 'canonical' way to add packages is to make a patch to the operations/puppet.git repository, listing the name of the package - thus it gets installed on every exec node, present and future. 

Solution to this problem is to make sure that all such installs go through puppet, and not through 'ad-hoc' apt-gets.
Comment 4 Marc A. Pelletier 2013-10-10 13:11:19 UTC
Until we get a list of missing modules, there is no way to know what to update.

(Hint: dependencies are tracked automatically, we only need the list of directly used modules).

Closing as WFM; feel free to reopen with a list of modules.
Comment 5 Dirk Beetstra 2013-12-29 16:53:23 UTC
Finally have some time to work on this - these are the modules my bots use:

BSD::Resource;
Data::Dumper;
Date::Parse qw(str2time);
Date::Parse;
DBI;
DBI; 
Encode qw(encode_utf8);
Encode;
HTML::Entities;
HTTP::Status;
List::Util qw(shuffle);
LWP::UserAgent;
POE qw (Component::Client::TCP);
POE qw(Component::IRC Component::IRC::Plugin::BotAddressed Component::Client::TCP);
POE qw(Component::Server::TCP);
POE::Component::IRC::Plugin::BotAddressed;
POE::Component::IRC::Plugin::Connector;
POE::Component::IRC::State;
POE::Component::Server::TCP;
POE;
Socket;
strict;
URI::Escape qw( uri_escape_utf8 );
URI::Escape qw(uri_escape_utf8);
URI::Escape;
utf8;
warnings qw(all);
warnings;
XML::Simple;
Comment 6 Tim Landscheidt 2013-12-29 18:52:10 UTC
All those modules are installed on development and execution hosts.  What do you miss?
Comment 7 Tim Landscheidt 2013-12-29 18:59:00 UTC
(In reply to comment #6)
> All those modules are installed on development and execution hosts.  What do
> you miss?

I'm sorry, I should have waited for the query to finish.  BSD::Resource isn't installed on tools-exec-07, tools-exec-08 and tools-exec-09.
Comment 8 Dirk Beetstra 2013-12-29 19:02:04 UTC
Heh.  I only know that the bots tend to crash on some exec-instances, especially the 07, 08, 09 - this bug report / list is mainly to make sure that all are now the same.
Comment 9 Gerrit Notification Bot 2013-12-29 19:09:36 UTC
Change 104401 had a related patch set uploaded by Tim Landscheidt:
Tools: Install requested packages

https://gerrit.wikimedia.org/r/104401
Comment 10 Gerrit Notification Bot 2013-12-29 19:12:07 UTC
Change 104401 merged by coren:
Tools: Install requested packages

https://gerrit.wikimedia.org/r/104401

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links