Is this thing on?
GoFaster: hiding the mochitest results table
August 9, 2011Posted by on
I’m sure anyone who has ever submitted a patch to a Mozilla tree is familiar with this drill:
- hg push
- check TBPL, wait
- check TBPL again, wait some more
- go to Starbucks for a caramel macchiato, install a new OS on your laptop, review all the patches in your queue, plan next winter’s tropical vacation, check TBPL, and….
- wait some more
Recently, the total end-to-end time from submit to all-tests-done has been around 6-8 hours, depending on load. That’s too long, and RelEng and the A-Team think we can do something about it. For the past couple of months we’ve been working on the GoFaster project; our goal is to get that turnaround time down to 2 hours. We have a list of tasks, and recently one of these landed with some significant improvements.
Cameron McCormack wrote a patch which hides the mochitest results table when MOZ_HIDE_RESULTS_TABLE=1 (see: bug 479352). The initial version of this patch caused frequent hangs during mochitest-1/5. We didn’t discover the reason behind this, but I updated the patch to hide the result table in a different way, and the hang vanished. I pushed this change to mozilla-central, and Cameron made a table displaying before and after durations for all the test runs.
The results? That one change saves about 13 hours of machine time per checkin. The entire suite of unit tests which prior to that change took about 40 machine-hours to run now takes 27. Wow!
What kind of improvement in the end-to-end time does that translate into? I’m not sure. Sam Liu, an A-Team intern, has been working on a dashboard to help track this, but it’s currently using canned (stale) data. RelEng is working on exposing live data to be consumed by the dashboard, and when that’s ready we should be able to easily track the effect of changes like this in the overall time.