JGriffin's Blog

Is this thing on?

GoFaster: hiding the mochitest results table

I’m sure anyone who has ever submitted a patch to a Mozilla tree is familiar with this drill:

  1. hg push
  2. check TBPL, wait
  3. check TBPL again, wait some more
  4. go to Starbucks for a caramel macchiato, install a new OS on your laptop, review all the patches in your queue, plan next winter’s tropical vacation, check TBPL, and….
  5. wait some more

Recently, the total end-to-end time from submit to all-tests-done has been around 6-8 hours, depending on load.  That’s too long, and RelEng and the A-Team think we can do something about it.  For the past couple of months we’ve been working on the GoFaster project; our goal is to get that turnaround time down to 2 hours.  We have a list of tasks, and recently one of these landed with some significant improvements.

Cameron McCormack wrote a patch which hides the mochitest results table when MOZ_HIDE_RESULTS_TABLE=1 (see: bug 479352).  The initial version of this patch caused frequent hangs during mochitest-1/5.  We didn’t discover the reason behind this, but  I updated the patch to hide the result table in a different way, and the hang vanished.  I pushed this change to mozilla-central, and Cameron made a table displaying before and after durations for all the test runs.

The results?  That one change saves about 13 hours of machine time per checkin.  The entire suite of unit tests which prior to that change took about 40 machine-hours to run now takes 27.  Wow!

What kind of improvement in the end-to-end time does that translate into?  I’m not sure.  Sam Liu, an A-Team intern, has been working on a dashboard to help track this, but it’s currently using canned (stale) data.  RelEng is working on exposing live data to be consumed by the dashboard, and when that’s ready we should be able to easily track the effect of changes like this in the overall time.

Meanwhile, check out the project’s wiki page or attend one of our meetings.  If you have thoughts on ways we can improve our total turnaround time, we’d love to hear from you.

One response to “GoFaster: hiding the mochitest results table

  1. Pingback: GoFaster: deeper data analysis « JGriffin's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: