Wikimedia Bugzilla is closed!

It'd be extremely valuable to have Jenkins tests for Vagrant.

Note that this request is different from bugs 45499 & 48407, which are about utilizing a known-good Vagrant configuration as part of the testing infrastructure for other Wikimedia software projects.

When using Vagrant to test other systems, we'd want to use a fully pre-built and pre-configured image. For testing Vagrant itself, however, we want specifically test the ability of the Vagrant setup to successfully provision MediaWiki & related software on top of an unmodified, vanilla image. Ideally we'd test optional roles as well as the default MediaWiki role.

Testing Vagrant is going to be resource-hungry. We can load the vanilla image and apt packages from an internal host, but we should still expect the job to take some time and eat up a lot of disk, CPU, and RAM.

So my question is: should we use our internal Jenkins instance, or a third party, like Cloudbees? I'd prefer to use our own Jenkins instance, but I wouldn't want to negatively affect the performance of CI for patches in other projects.