As my Twitter followers will no doubt know by now, I joined Mozilla two weeks ago. I’m working on the Firefox Sync (née Weave Sync) project which allows Firefox users to synchronize personal browsing data securely with the cloud and thus with other devices.

When I joined, the team was working hard on getting the 1.3 release out, which meant I wasn’t really able to help out. So I spent most of my time so far improving the tests (see bugs 566575, 557590, 557596, 557588 if you’re interested). This not only benefits the project in the long run of course, it’s also an excellent way of getting to know the code.

As I’ve written in my last blog post, the lack of a decent code coverage tool for JavaScript is bothering me a bit when writing unit tests in JS and this time was no different. Fortunately, the xpcshell test harness that we’re using runs with much different privileges than what your average webapp’s JavaScript runs with. That means we get access to things like the JavaScript debugger service. That, after all, has features like breakpoint and stepping, so surely one should be able to (ab)use it somehow to record code coverage, right?

It turns out, the debugger service actually has what’s called an interrupt hook that is called for every expression the interpreter encounters. Normally you would use this in stepping mode to step through individual expressions, but it may just as well be used to do code coverage. Mind you, it’s not fast. I believe invoking the debugger already turns of the JIT, and with the interrupt hook in place, tests run about an order of magnitude slower. Ugh.

But it worked well enough for a proof of concept. So I hacked together a small patch for the test harness to dump the coverage data it collects as a JSON file. This is then picked up by a small analytics tool written in Python which generates a simple HTML report like this one. When I started writing tests we were at 49% line-by-line coverage, now we’re at 58%. Of course these numbers have to be taken cum grano salis since line-by-line coverage is just one metric. It doesn’t say anything about the quality of your tests. For all you know, some piece of code may be calling a whole bunch of code, thus making it appear covered, but you haven’t actually tested the relevant behaviour.

Anyway, have a look at the bug in the issue tracker if you’re interested in the patch and the report generator. Feedback is, of course, most welcome. It would also be fun to apply this or a similar technique to Firebug (it looks like somebody has tried in the past) so you could get code coverage on your webapp’s JavaScript without having to resort to crude code rewriting tools.

Follow

Get every new post delivered to your Inbox.