JGriffin's Blog

Is this thing on?

GrafxBot Results Update

Thanks to the many thousands of you who have downloaded GrafxBot and submitted test results to Mozilla!  In case you’re curious about what we’ve done with all that data, here are some statistics:

  • According to AMO, GrafxBot has been downloaded about 4700 times.
  • We have nearly 30,000 sets of test results that were submitted to our database, for a total of 3.7 million tests.
  • The test results span 282 unique video cards on Windows, 30 on Mac, and 171 on Linux.
  • The failure rate (which is somewhat subjective given the manual pass/fail mechanism) averages around 0.4% on Windows, 0.5% on Mac, and 1.1% on Linux.
  • Test data has resulted in a total of 37 bugs being filed.

Aside from the raw test results, many of you have submitted useful comments.  Some have noted that fonts look bad when Firefox is accelerated; others have described scrolling or other issues.  Not all of these problems can be detected by GrafxBot, so if you notice problems like these when browsing, I encourage you to file a bug report in Bugzilla, under Core -> Graphics.  If you submit a bug report, please include the details of your graphics hardware, and include a screenshot if possible.

GrafxBot continues to be updated along with Firefox betas, so I encourage interested folks to continue running GrafxBot each beta release.  Thanks for all your help in making Firefox 4 the fastest ever!

8 responses to “GrafxBot Results Update

  1. Snap November 24, 2010 at 2:03 pm

    I’m glad to see people volunteer in such experience.

    In the last test I’ve run (it was using Firefox 3.6, maybe I should not run grafbox on it ?), I have got a pass ratio of 65%, this is much bigger than the ~1% average failure rate, isn’t it ?

    After Grafbox has finished the tests, it often asks me to compare 2 images, and press success/fail. Did you tried to automate this process ? I think about images diff tools or saving the pictures (using canvas ?) then comparing the files.

    • jagriffin November 24, 2010 at 2:36 pm

      There’s no need to run it on 3.6; the purpose of GrafxBot is to help test hardware acceleration, and hardware acceleration isn’t in 3.6 in any form.

      I have tried to automate the pass/fail process. Currently, GrafxBot compares canvases, and if they’re different, it asks the user to judge. The reason we do this, instead of automatically failing, is that some differences will be just a couple of pixels (due to pixel rounding issues and the like), which won’t be visible to the user, and we don’t consider this a real fail. It’s possible we could apply a Gaussian blur to both images, and compare the results, in order to weed out problems like this, but then it’s possible we’d miss out on real, visually apparent failures. So there’s no great solution. 😦

      I have had a number of suggestions on how to improve the UI for this, however, and I hope to implement them at some point.

  2. ssam January 11, 2011 at 8:55 am

    does grafxbot also do profiling? it would be nice to know how much of a speedup there is with acceleration.

    Also on linux is it worth checking if there is a compositing window manager (compiz) running?

    • jagriffin January 11, 2011 at 9:47 am

      GrafxBot does a limited amount of profiling on Windows, but not on Mac or Linux, since on those platforms, it is impossible to dynamically toggle hardware acceleration without a browser restart.

  3. Virtual_ManPL January 11, 2011 at 8:59 am

    How about update add-on to Firefox 4.0.* ?
    Nice will be also creating version for 64bit Firefox on Win7 64bit.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: