Last modified: 2014-10-30 13:58:07 UTC
EventLogging currently doesn't allow flagging specific events (or events originating from a specific client/IP address/IP range) as "test events". As a result, the production logs are filled with spurious events originating from hosts such as "localhost", "MyWiki.local" etc. This is problematic because data consumers often query/count events unaware that they need to filter out these events. Whitelisting production hosts on the data consumer end is not a practical option. Engineers are resorting to hacks like [1] to address this problem, but a test flag doesn't belong in a data model. Instead, we need to support a test mode that the client can enable which will flag any incoming event as test. If the event passes validation, it should be stored in a different log (it could be a <schema_name>_<schema_revid>_test table or a table in a separate db) to allow data QA and troubleshooting. [1] see "testing" field in https://meta.wikimedia.org/wiki/Schema:MobileWebWikiGrokResponse
This is due to the fact that EL does not have a server testing environment in which you can try your data setup. Adding a test flag to the production schemas is really not a solution to the riot cause, we should rather invest time in creating a testing env for EL server side.
(In reply to nuria from comment #1) > This is due to the fact that EL does not have a server testing environment > in which you can try your data setup. Vagrant can run EventLogging. (At least it did when I had to do debugging around EventLogging a few months back.) This was a nice and solid test environment for me.
>Vagrant can run EventLogging. Can run the client code, yes. But it has not "storage" for events thus you do not known if they validate and whether they are going into the DB. And this, more often than than not, is the cause of people trying EL in "test mode".
(In reply to nuria from comment #3) > >Vagrant can run EventLogging. > > Can run the client code, yes. And it can also run the relevant server code. EventLogging comes with a dedicated “devserver”. See: https://git.wikimedia.org/blob/mediawiki%2Fextensions%2FEventLogging.git/0454a1d077f36bc1b785f1dda42502d00f2508ee/server%2Fbin%2Feventlogging-devserver > But it has not "storage" for events thus you > do not known if they validate [...] The above devserver does validation. And it even does better than plain “validates/does not validate”. The devserver even tells you about validation errors. For me, this proved really helpful. > And > this, more often than than not, is the cause of people trying EL in "test > mode". Vagrant (and it's shipped EventLogging's devserver) already offers a complete setup to test events and validate them.
You are right, my bad. Should have looked at code instead of relying on memory. Now, Validation point being made I see little value in adding a test flag and I am of the opinion that we should not do it.
(In reply to nuria from comment #5) > [...] I see little value in adding a test flag > and I am of the opinion that we should not do it. Same here. (Hence, being bold and RESOLVED/WONTFIX. Please REOPEN, if you think this needs more discussion)
