Last modified: 2014-09-26 06:32:09 UTC
Tim described it better than anyone else: HHVM has a mode called RepoAuthoritative, which requires you to pre-compile all the bytecode before you run the program. It's said to give 30% faster execution or so. During pre-compilation for RepoAuthoritative, an attribute called AttrUnique is set on functions and classes that have only one implementation for a given name. So when there is only one MW version in pre-compilation, AttrUnique should be set on most things, but if there is more than version, it should be set on almost nothing. If you don't use RepoAuthoritative, it should also be set on nothing. AttrUnique is used by the IR and MC emitters, apparently for optimisation. My suspicion is that having multiple versions of MW will not completely stop code from being JITed, rather it will just give you a slowdown of up to 30%, equivalent to not using RepoAuthoritative. I think a good benchmark would be to parse some large page that has anything Lua-related stripped out of it (since LuaSandbox still has bugs in it). Then get benchmarks with: * RepoAuthoritative off * RepoAuthoritative on with one MW version precompiled * RepoAuthoritative on with two MW versions precompiled
Created attachment 15805 [details] Test results with HipHop VM 3.2.0-dev (rel) I have created a test harness on osmium in /home/bd808/projects/hhvm/bug-67168 that can compile hhbc caches and run tests using maintenance/benchmarks/benchmarkParse.php. A typical test invocation is done as `/clean.sh && ./build.sh && ./test.sh 2>&1 | tee results/test-$(date +%Y%m%dT%H%M).log`. Results for a typical run: | MIN | MEDIAN | AVG | MAX ---------------+--------+--------+--------+------- No RA, No JIT | 1.0700 | 1.0890 | 1.1139 | 1.5070 No RA, JIT | 0.9210 | 0.9860 | 0.9937 | 1.0810 ---------------+--------+--------+--------+------- RA, 1v, No JIT | 1.0960 | 1.1240 | 1.1285 | 1.1470 RA, 2v, No JIT | 1.1210 | 1.1440 | 1.1601 | 1.3480 ---------------+--------+--------+--------+------- RA, 1v, JIT | 1.0010 | 1.0240 | 1.0433 | 1.1600 RA, 2v, JIT | 0.9790 | 1.0400 | 1.0464 | 1.1070 Preliminary conclusions: * JIT is faster than no JIT. This seems to be the case even in this contrived test where the HHVM interpreter is not performing a long running task and allowed to build up a large optimized bytecode system. * Using RepoAuthoritative seems to cause a performance penalty when running a one-off job. This seems likely to be related to the overhead of reading the sqlite3 database to find the initial bytecode to process. This may have implications for the execution of jobs and maintenance scripts via hhvm . * The performance difference between having one and two branches in the HHVM bytecode cache for RepoAuthoritative execution seems to be nominal at least in tests which only exercise a single branch. The difference may be related to the relative size of the sqlite3 databases (55M vs 60M).
Would it make sense to exercise it with mediawiki core PHPUnit tests? Assuming they still pass under HHVM and they will not run against a production database. That exercise a bit more code than benchmarkParse.php and might better highlight differences.
Using <https://gist.github.com/atdt/1a401372a2306cf666f0> on osmium, I see: Backend response time for enwikivoyage/wiki/Main_Page, in ms. mean median min max mw1020 264.87 255.00 226.00 346.00 osmium(-jit) 162.40 165.00 138.00 174.00 osmium(+jit) 109.73 110.00 96.00 120.00 40% difference between JIT and no JIT.
> My suspicion is that having multiple versions of MW will not > completely stop code from being JITed, rather it will just give you a > slowdown of up to 30%, equivalent to not using RepoAuthoritative. This is mostly correct. You will miss out on a lot of the potential optimizations by having non-unique classes, but a few big ones will be left. Most importantly, the server won't have to stat every file included by each request to see if it needs to be re-parsed (this won't affect CLI mode, obviously). The benefit from this depends heavily on the structure of your PHP code and how many files are required by typical requests. For short-lived processes like in comment 1, the overhead you mentioned (among others) is a big problem for the JIT. We want to make this a smarter default in the near future, but for now it's commonly a net win to turn off the JIT for scripts that run a large footprint of code a small number of times. I think you'll see a much bigger difference between RA and non-RA with more complex benchmarks, especially since I suspect a lot of the time being measured here is process startup, bytecode processing, and time spent in the JIT itself.
When we get back to the point where we can run multiple HHVM based servers in beta I think it would be interesting to configure one to use repo authoritative mode and one without and compare the performance. This may be less than optimal in beta however due to the high number of deployments there per day. It may be an easier experiment to conduct in the limited release production HHVM cluster.
The current plan is to go for RepoAuthoritative ASAP, and in the interim to clear the bytecode cache in a cron job and to have scap restart HHVM after syncing code.
