Last modified: 2014-09-26 18:02:01 UTC
review/jdlrobson/mantleview x ~/Sites/w/extensions/MobileFrontend $ vagrant provision ==> default: Running provisioner: lsb_check... ==> default: Running provisioner: puppet... Running Puppet with site.pp... Warning: Could not retrieve fact ipaddress Error: Cannot allocate memory - fork(2) on node mediawiki-vagrant Error: Cannot allocate memory - fork(2) on node mediawiki-vagrant The following SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed! FACTER_fqdn='mediawiki-vagrant.dev' FACTER_forwarded_port='8080' FACTER_shared_apt_cache='/vagrant/cache/apt/' FACTER_environment='vagrant' FACTER_port_fragment=':8080' FACTER_share_owner='502' FACTER_share_group='20' FACTER_provider_name='virtualbox' puppet apply --templatedir /vagrant/puppet/templates --verbose --config_version /vagrant/puppet/extra/config-version --fileserverconfig /vagrant/puppet/extra/fileserver.conf --logdest /vagrant/logs/puppet/puppet.20f0fbdd9.log --logdest console --write-catalog-summary --modulepath '/tmp/vagrant-puppet-1/modules-0:/etc/puppet/modules' --hiera_config=/tmp/vagrant-puppet-1/hiera.yaml --manifestdir /tmp/vagrant-puppet-1/manifests --detailed-exitcodes /tmp/vagrant-puppet-1/manifests/site.pp || [ $? -eq 2 ] Stdout from the command: Stderr from the command: Warning: Could not retrieve fact ipaddress Error: Cannot allocate memory - fork(2) on node mediawiki-vagrant Error: Cannot allocate memory - fork(2) on node mediawiki-vagrant
(In reply to Jon from comment #0) > Warning: Could not retrieve fact ipaddress > Error: Cannot allocate memory - fork(2) on node mediawiki-vagrant > Error: Cannot allocate memory - fork(2) on node mediawiki-vagrant Any idea what's eating all the ram in your VM? Redis full of unprocessed jobs perhaps?
I'm using Flow, MobileFrontend and VisualEditor. I notice this seems to correlate when I have a Flow page with over 20 posts and I continuously keep hitting. Nothing looked suspect. Is there any particular commands you would like me to run?
(In reply to Jon from comment #2) > Nothing looked suspect. Is there any particular commands you would like me > to run? # Top 20 processes by resident memory size $ ps -e -orss=,args= | sort -b -k1,1nr | head -20 # Free/used ram $ free -m # Job queue status $ mwscript showJobs.php --wiki=wiki --group
Jon, poke. Any progress in reproducing and determining which application is eating your instance RAM?
I've had this happen several times, not because of applications eating ram but because of https://bugzilla.wikimedia.org/show_bug.cgi?id=70304 I have to reload vagrant from the appropriate directory and it starts up with my configured 4G instead of 1G
I'll also add the easiest way to trigger the error is to try and load the hhvm debugger, hhvm -m debug --debug-host ::1 always fails for me with not enough memory if it booted with 1G due to reading the wrong settings.yml
It hasn't happened to me in a while (maybe correlated to less Flow coding?)... soon as it does I'll be back here.
I know that some of the OOM issues during provisioning were happening during Ruby gem compilation (usually ffi). It probably wasn't the root cause—more likely the compounding effect of less base VM RAM and more memory required by the base MW dependencies (HHVM)—but it's worth noting that there have been far fewer reported cases since we switched to using system packages. [1][2] [1] https://gerrit.wikimedia.org/r/#/c/158564/ [2] https://gerrit.wikimedia.org/r/#/c/156207/
