Last modified: 2014-03-31 09:16:17 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T54354, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 52354 - Run Minion testing instance for security testing
Run Minion testing instance for security testing
Status: RESOLVED WONTFIX
Product: Wikimedia Labs
Classification: Unclassified
deployment-prep (beta) (Other open bugs)
unspecified
All All
: Normal enhancement
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-31 21:26 UTC by Tyler Romeo
Modified: 2014-03-31 09:16 UTC (History)
11 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Tyler Romeo 2013-07-31 21:26:48 UTC
Mozilla recently deployed an open-source security testing framework:

http://blog.mozilla.org/security/2013/07/30/introducing-minion/
https://github.com/mozilla/minion

It's an automated security testing framework for use in testing web
applications. I managed to get it working on my Vagrant instance.

Here's a brief summary of what I learned:
* It uses a MongoDB backend with Python and Flask as a front-end
* There are plugins that implement certain tests (e.g., nmap, skipfish)
* Plans are combinations of plugins, basically a test plan
* Sites are added into groups, and are then assigned plans
* Finally, you run plans on the frontend and they're run by a celery job queue

From the looks of it, I don't think this would be particularly useful for individual developers, because many of the tests require a full TLS setup and whatnot.

What might be useful is to have a security instance running MediaWiki with a similar setup to the actual en-wiki, and then have Minion running on an instance and have it run the tests that way. Unfortunately, I don't know how we would manage users (since it doesn't have LDAP integration) or when we would run these tests (I'd imagine there wouldn't be a need to run them on every change).
Comment 1 Antoine "hashar" Musso (WMF) 2013-08-01 08:07:53 UTC
If you are interested in this, you could setup a labs instance somewhere and point it to the beta cluster installation.  That would potentially be a nice playground since you get a setup similar to the production cluster and can hack out useful logs in beta :-]

We could get the instance in the deployment-prep project itself and make you an admin there to let you setup minion.  There is a few culprit to handle such as instances not being able to access the external NAT public IP, but that can be worked around.
Comment 2 Tyler Romeo 2013-08-02 16:19:15 UTC
That would be a great idea. How would I go about requesting such an instance?
Comment 3 Chris Steipp 2013-08-02 17:46:13 UTC
You could also request a new project. Then both projects just need to adjust their security groups so the projects can talk to each other, but the instances would be managed entirely by you, or other admins in the testing project, and not get in the way of beta.

Otherwise an admin in the deployment prep group can setup the instances and give you access, if that seems like a better route.
Comment 4 Tyler Romeo 2013-08-04 21:28:01 UTC
(In reply to comment #3)
> You could also request a new project. Then both projects just need to adjust
> their security groups so the projects can talk to each other, but the
> instances
> would be managed entirely by you, or other admins in the testing project, and
> not get in the way of beta.
> 
> Otherwise an admin in the deployment prep group can setup the instances and
> give you access, if that seems like a better route.

Unfortunately I don't have much experience with labs, so I actually don't know how to go about doing either of these things. :/
Comment 5 Antoine "hashar" Musso (WMF) 2013-08-05 08:21:34 UTC
(In reply to comment #1)
> There is a few culprit to handle such as instances not being able
> to access the external NAT public IP, but that can be worked around.

I was referring there to bug 45868 which in ops land.



@tyler just get yourself an instance inside deployment-prep project, this way anyone will be able to access it :-)

If accessing the HTTP caches public IP from the instance, you will have to play with iptables to rewrite the instance outgoing packets so they use the internal instance address instead of the public one:

 iptables -t nat -I OUTPUT --dest <public IP of cache> \
   -j DNAT --to-dest <corresponding instance IP address>
Comment 6 Antoine "hashar" Musso (WMF) 2014-03-31 09:16:17 UTC
Closing bug for now since there was no progress since last summer.  Feel free to reopen when there is interest again.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links