Last modified: 2014-09-11 19:03:42 UTC
I added role "browsertests", but every time I run `vagrant provision` it fails. It seems to be trying to install an "ffi" (foreign function interface) module. Fri Sep 06 20:26:53 +0000 2013 /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns (notice): Using rake (10.1.0) Fri Sep 06 20:26:53 +0000 2013 /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns (notice): Using builder (3.2.2) Fri Sep 06 20:26:53 +0000 2013 /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns (notice): Installing ffi (1.9.0) with native extensions /usr/lib/ruby/1.9.1/rubygems/installer.rb:552:in `rescue in block in build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError) Fri Sep 06 20:26:53 +0000 2013 /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns (notice): Fri Sep 06 20:26:53 +0000 2013 /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns (notice): /usr/bin/ruby1.9.1 extconf.rb Fri Sep 06 20:26:53 +0000 2013 /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns (notice): Fri Sep 06 20:26:53 +0000 2013 /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns (notice): Gem files will remain installed in /home/vagrant/.gem/ruby/1.9.1/gems/ffi-1.9.0 for inspection. Fri Sep 06 20:26:53 +0000 2013 /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns (notice): Results logged to /home/vagrant/.gem/ruby/1.9.1/gems/ffi-1.9.0/ext/ffi_c/gem_make.out gem_make.out has only: /usr/bin/ruby1.9.1 extconf.rb
I was not able to reproduce this in a fresh VM
I ran into this problem before. If I recall correctly, it's an out-of-memory error that bundler is unhelpfully catching and re-throwing as an uninformative generic exception. The memory requirements of building the ffi gem were the original impetus for bumping the VM's memory allocation to 768 Mb, but since we've done that I have not run into this problem. Have you modified the VM's memory allocation setting at all? If you have not modified the VM, can you reproduce the output of "vboxmanage showvminfo"? It should be invoked with the machine's UUID as the final command-line argument. The UUID can be found in .vagrant/machines/default/virtualbox/id, which should exist under your Vagrant repo root.
[mass-moving from Tools>MediaWiki-Vagrant to separate product. See bug 54041. Filter bugmail on this comment.]
Apparently having two different people attempt to verify your report is not quite enough to warrant a follow-up. Closing as invalid.
Thanks for the updates. I did not modify the VM's memory size. Maybe the difference between me and Chris is my roles include eventlogging, gettingstarted (which loads redis) and visualeditor+parsoid. I increased the VM to 1024, `vagrant halt` `vagrant up`! In the vagrant VM, on both successful and unsuccessful runs, puppet spends many minutes running /usr/bin/ruby /usr/bin/bundle install --path /home/vagrant/.gem with no output. /home/vagrant/.gem/ruby/1.9.1/gems contains a bunch of old gems from May14, and the new revisions of gems from this run. drwxr-xr-x 4 vagrant vagrant 4096 May 14 04:28 builder-3.2.0 drwxr-xr-x 6 vagrant vagrant 4096 Sep 6 22:14 builder-3.2.2 drwxr-xr-x 4 vagrant vagrant 4096 May 14 04:35 childprocess-0.3.9 drwxr-xr-x 6 vagrant vagrant 4096 May 14 04:35 chunky_png-1.2.8 drwxr-xr-x 10 vagrant vagrant 4096 May 14 04:44 cucumber-1.3.1 drwxr-xr-x 6 vagrant vagrant 4096 May 14 04:45 data_magic-0.14 drwxr-xr-x 7 vagrant vagrant 4096 May 14 04:35 diff-lcs-1.2.4 drwxr-xr-x 4 vagrant vagrant 4096 May 14 04:44 faker-1.1.2 drwxr-xr-x 7 vagrant vagrant 4096 May 14 04:29 ffi-1.8.1 drwxr-xr-x 7 vagrant vagrant 4096 Sep 16 03:50 ffi-1.9.0 <--- the problem drwxr-xr-x 10 vagrant vagrant 4096 May 14 04:35 gherkin-2.12.0 drwxr-xr-x 5 vagrant vagrant 4096 May 14 04:44 i18n-0.6.4 drwxr-xr-x 4 vagrant vagrant 4096 May 14 04:35 multi_json-1.7.3 drwxr-xr-x 4 vagrant vagrant 4096 May 14 04:45 net-http-persistent-2.8 drwxr-xr-x 4 vagrant vagrant 4096 May 14 04:45 page_navigation-0.9 drwxr-xr-x 5 vagrant vagrant 4096 May 14 04:45 page-object-0.8.10 drwxr-xr-x 4 vagrant vagrant 4096 May 14 04:45 parallel-0.6.4 drwxr-xr-x 5 vagrant vagrant 4096 May 14 04:45 parallel_tests-0.12.3 drwxr-xr-x 6 vagrant vagrant 4096 May 14 04:28 rake-10.0.4 drwxr-xr-x 6 vagrant vagrant 4096 Sep 6 22:14 rake-10.1.0
Created attachment 13289 [details] output of `vboxmanage showvminfo ...` before I increased VM memory
(In reply to comment #5) > I increased the VM to 1024, `vagrant halt` `vagrant up`! And -- did it work? I guess it did, because you later speak of "successful and unsuccessful runs". I don't think it's viable to keep increasing the default RAM allocation just so that this one Gem can compile, so the way to resolve this is to document the fact that compiling the ffi Gem is a memory-hungry task and that if it fails one should try freeing some memory in the VM or increasing the RAM allocation.
(In reply to comment #5) > I increased the VM to 1024, `vagrant halt` `vagrant up`! ... that should have continued "and the vagrant provision complete successfully, and I can run tests!" Yay, thanks for the memory suggestion. (In reply to comment #6) > the way to resolve this is to document... I added something, https://www.mediawiki.org/w/index.php?title=MediaWiki-Vagrant&diff=783886&oldid=781408 > ... one should try freeing some memory in the VM You mean by temporarily disabling other roles until vagrant provisions browsertests successfully? I didn't try stopping apache/mysql/parsoid/redis in the vagrant VM, because I assumed `vagrant provision` would have restarted them before running the ruby gem bundler. I guess the opaque exception reporting is an upstream issue.
I have the same problem. I will attach output of "vagrant enable-role browsertests" and "VBoxManage showvminfo". How do I increase VM memory size? (Looks like it is 768MB now.) In Vagrantfile[1] or "Oracle VM VirtualBox Manager > VM > right click > Settings > System > Base Memory"? 1: https://github.com/wikimedia/mediawiki-vagrant/blob/master/Vagrantfile#L65
Created attachment 13383 [details] vagrant provision
Created attachment 13384 [details] VBoxManage showvminfo
I have tried this. #1 $ vagrant halt #2 changed vb.customize ['modifyvm', :id, '--memory', '768'] in Vagrantfile to vb.customize ['modifyvm', :id, '--memory', '1024'] #3 $ vagrant up $ vagrant provision but both "VBoxManage showvminfo" and "Oracle VM VirtualBox Manager" still say "Memory size: 768MB".
Found it. Mediawiki-Vagrant[1] says to edit Vagrantfile-extra.rb. Now VM memory size is 1024. $ vagrant halt edit Vagrantfile-extra.rb $ vagrant up $ VBoxManage showvminfo ... ... Memory size: 1024MB ... 1: https://www.mediawiki.org/wiki/Mediawiki-Vagrant
I managed to get tests running after increasing memory size to 1024. Is it possible to increase memory size just when gems are installing, and then decrease it back to 512?
Is this still an active problem? The browsertests role has been removed, but there are now several extension specific roles which install gems via bundler to provide testing support.
Can we do `apt-get install ruby-ffi` instead of compiling ffi with bunder?
The ruby-ffi package is for Ruby 1.9. Perhaps we should try switching back to a shared directory for bundler gems. That would at least mitigate the failures by only performing ffi compilation (and other native gem compilation) once.
After profiling the `bundle install` process, I'm seeing around 128M as the peark memory usage. That seems like a reasonable amount of free memory to expect on the VM considering some of the other memory intensive processes that need to run (xvfb/firefox in browser testing, for example, uses around 192M).
Forgot to tag the commit in https://gerrit.wikimedia.org/r/#/c/158564/ but the patch should have fixed this and other bugs related to botched ffi compilation. A good handful of the most problematic gems are now installed from packages.
