This past week I’ve spent some time here and there improving my Weave Sync client for Google Chrome. It all started out as a proof of concept, so initially I only hacked together enough to get it to show some lights on the screen. I decided that if I were to take this any further, I should man up and write some tests for it, because refactoring code or adding features without knowing for sure your stuff still works freaks me out.

I guess nowadays you don’t have to defend the whole test malarkey anymore. I’ve never really minded writing tests as I my code grows anyways. I’ve done TDD on a couple of projects before so I’m used to writing tests before or, when not in TDD mode, shortly after writing the code. When fixing a bug, for instance, I tend to always write the test first. This week, though, I was paying the price of graduating a toy project into something a bit more serious. Writing nothing but tests for a while is not that much fun.

QUnit, JSpec

I decided to go with QUnit as the testing framework. I’ve never really liked JUnit-based frameworks anyway, and especially in JavaScript the particular flavour of object-orientendness provided by JsUnit felt out of place. QUnit is from the jQuery folks — it’s used to test jQuery itself. I can highly recommend it: it’s a neat little and nimble library, much like jQuery itself.

The only other contender for me was JSpec, a JavaScript interpretation of RSpec. Their idea is basically to make your tests look like real language. It’s a bit like the doctest concept that exists in the Python world. Doctests combine documentation and unit test code which is great when you want to write well-documented tests that can also serve as an API example document to developers. RSpec and clones go further by merging the natural language and unit test into one. I find this approach intriguing and I want to try it out some time, but for now I decided against it. That said, JSpec also provides a JavaScript API which reads quite nicely. So maybe that’s a good stepping stone…

A mock XMLHttpRequest implementation

The Weave client is obviously all about talking to a server, so I needed a way to mock this XMLHttpRequests. JsUnit has a mock implementation of it but it’s pretty much useless. JSpec has a better one, but it’s all tied into JSpec and it doesn’t support Progress Events handlers, something that Mozilla and WebKit based browsers do nowadays. Because frankly the readystatechange event is just silly.

So I decided to roll out my own mock XMLHttpRequest implementation, MockHttpRequest. It tries to follow the XMLHttpRequest and Progress Events specs where possible and practical. Of course it itself is 100% tested. Try it, fork it, improve it, critique it and let me know when you do! 🙂

Code coverage

When you’re writing tests for your code, it’s always a good idea to track code coverage. That way you can make sure your tests hit every code path. This is especially, but not only, useful when you’re writing tests after the fact.

Sadly it seems there aren’t that many tools for that out there. There’s an abandoned Firefox extension that hooks into spidermonkey (Mozilla’s JavaScript interpreter) and an abandoned Firebug extension that hooks into Firebug’s profiling functionality. Hooking into an interpreter or profiler makes a lot of sense since you don’t have to change the code in question. Hopefully one of these projects will be revived one day or a similar one will come along. If somebody wants to pick this up, I could perhaps be bothered to help. 🙂

For now your best shot for JavaScript code coverage seems to be JSCoverage. It takes your existing code tree, makes a copy while inserting code to keep track of code coverage (essentially a counter that’s increased on every line of code). It’s crude but effective, so for now I’m happy to get a feel for how complete my tests are.

Advertisements

Tabs, tabs, tabs

May 1, 2010

Choosing the web browser as the application for most of your daily tasks means that it better well be usable. In my case that primarily means dealing with lots of tabs. So it happens that much of the time I’m spending hacking Firefox is on improving the tab story.

Lazy tab loading FTW

A few months ago I was fed up with the long (re)start times of a tab encumbered Firefox and came up with BarTab, a Firefox add-on that changes tab loading behaviour so that they’re loaded only when they’re accessed. As I wrote in a blog post then, it was initially intended for tab addicts like myself, but thanks to lots of feedback from all kinds of users — tab addicts and novices alike — BarTab has continually improved to serve related use cases as well.

Firefox after a restart. The three tabs on the left haven't been loaded yet, hence their dim appearance.

I’m really overwhelmed by BarTab’s popularity. It’s been reviewed by various online journals such as LifeHacker, scrutinized for its memory and CPU cycle conserving powers, featured on Rock Your Firefox and suggested for integration into Firefox proper. That’s a steep career for an add-on that’s barely three months old!

Making bookmarks obsolete

On the other hand, three months was enough time to take a step back, observe my own interaction with the browser and analyze the user feedback. I think there’s some real usability potential in lazy tab loading.

Novice users typically don’t keep a lot of open tabs. What many of them do is bookmark regularly visited sites. This behaviour is encouraged by the bookmark toolbar being visible by default in Firefox. However, with bookmark-related UI elements in the location bar, a toolbar, the sidebar, and the menubar, I’m sure you’ll agree that the usability of the current implementation could definitely be improved. Or in the words of a Reddit user:

“It’s like someone taped a bunch of cats together.”

I would even go as far as challenging the whole concept of bookmarks. I think they require too much user interaction. Tabs, on the other hand, are easy to open, close and discover. They’re a reasonably simple concept and one that’s part of the browser already anyway. When the brower supports lazy-loading tabs via a BarTab-like mechanism, tabs can essentially serve as bookmark substitutes. You simply remember a site by keeping its tab open. The difference to bookmarks is that you don’t need to do anything to “bookmark” them. In fact, you have to do one thing less: close them. Memory wastage won’t even be an issue with smart unloading. BarTab already supports this, albeit in a crude manner.

Improving tab discoverability

Eradicating bookmarks in favour of tabs will, at least to a certain degree, make everybody a “tab addict.” This has a few scalability implications. There’s of course the challenge of making the tab mechanism itself scale to hundreds, possibly thousands of tabs. That certainly is a tricky one to say the least, but I’ll leave that aside for now. Another real issue is the tab UI.

First there’s the tab arrangement. There are very good reasons for having a vertical arrangement to the side of the browsing window. Most computers have widescreen displays these days, so vertical screen real estate is more valuable than horizontal one. More to the point, a vertical arrangement provides more space for more tabs, so it scales better to a higher number of tabs. When I maximize Firefox on my laptop, I can see about 14 tabs in a horizontal and 40 in a vertical arrangement. That’s a factor of about 3 or half an order of magnitude more.

About 14 tabs are visible in the horizontal tab bar.

With a vertical tab arrangement one can overlook about 40 tabs.

Of course not everybody needs to or wants to see that many tabs. The novice users just have a few sites they visit regularly. In this case, one could still make good use of all that vertical space, for instance by showing thumbnails for each site. This scales well even to tens of tabs and helps users visually recognize their sites.

But even tab addicts might be interested just seeing a selection of all tabs available. The current tool of choice here is Tree Style Tab. It not only displays your tabs vertically, it also allows you to arrange them in arbitrary trees. In addition to that it has about a million other features. Don’t get me wrong, it’s a fabulous and brilliant add-on, but I think it’s a bit too complicated. And the complication comes at a price: Yes, thanks to BarTab restarting Firefox with about 100 tabs is quite fast — until you enable Tree Style Tab and have it restore your tree hierarchy.

This is why I’ve started work on Vertical Tabs (code on GitHub). Its proposed feature set is simple and much constrained: arrange tabs vertically in such a way that looks native on all platforms and allow grouping of tabs. In a second iteration there might be room for some additional group features such as previews and automatic unloading of tabs (via BarTab) when groups are collapsed. But all in all I’d like to keep it simple, fast and open to as many types of users as possible.

Tabs arranged with Tree Style Tab. A bit too much structure, methinks.

Tabs arranged in groups with Vertical Tabs.

Outlook to history

To sum up, a simple experiment with tabs has shown that the browser UI and is concepts can yet be improved. Merging tabs with bookmarks to a unified concept, both in terms of the UI as well as the underlying mechanics, is without doubt a challenge. But conceptually I think it makes a lot of sense.

What’s more, this way tabs become invariably linked to the browsing history. They essentially represent an open window or link to a certain item in that history. A loaded tab is an active link, an unloaded tab is a inactive link that just happens to be represented in the UI as a tab.

This, no doubt, should make us think about how to represent the browsing history as a whole in the UI. Of course the browsing history is riddled with information that you’re normally not interested in, so you need some sort of filter. This is what the tab bar could eventually become: a filered view on your browsing history.

Right now we’re seeing stuff happening in the webbrowser that starts to go beyond the traditional web application. I’m talking about things that make either the browser itself or the interaction between different web applications a richer experience.

On the browser side, the basket of technology commonly referred to as HTML 5 falls into this category. It makes the browser a much more powerful client, thus rebalancing a lot of weight between web server and client to an IMHO much more natural equilibrium.

On the web application side we are seeing are starting to see lots of interesting application mashups, thanks to technology like OAuth and cross-origin AJAX hacks. Facebook’s recent innovation, the social graph and the ability to embed it into other applications, is a powerful example.

As many have noted, there are huge problems with this and they all have to do with security and privacy. Virtually none of the technologies coming forward under the HTML 5 umbrella help with security. Yes there are some attempts to fix one of the most backwards security policies in the browser. But these are mere sticking plasters over oozing flesh wounds.

Weaving a richer web

As I’ve written here before, I think we need a browser designed for the cloud. Back then I was mostly speaking in terms of usability and explicitly ignored the privacy issue. It think it’s time to come back to that now. With HTML 5, we’re giving the browser many ways to be a richer client. But in terms of user data, we’re still treating it as a dumb terminal.

Weave, a Mozilla Labs project, has a different vision. Its broad idea is to create a richer web experience while still having users control their data. Concretely, the Weave Sync service allows you to synchronize your tabs, history, bookmarks, settings and passwords between different browsers. And here’s the thing: the data is all encrypted, on the client. All that Mozilla is storing on their storage nodes (which you don’t have to use, by the way) is a bunch of encrypted JSON.

Sure, you may be saying, that’s great for making Firefox a richer browser, but how does that help the general web?

Well, it turns out that doing RSA and AES cryptography in JavaScript isn’t such a far fetched idea at all. With some inspiration from an abandoned project, I was able to hack together a (very rough) version of Weave Sync for Google Chrome. Since it’s entirely written in JavaScript, it actually works in any browser.

See for yourself. (To try it out, you need to create a Weave account with some tab data in it, e.g. by installing the Weave Sync add-on in your Firefox and syncing your tabs).

Sure, you may be thinking, encryption is great for dealing with personal data, but it would be impossible in a social web. What if you wanted to share your bookmarks with other people?

Well, is it really that impossible? Let’s look at what Weave does. In a nutshell, it encrypts your private data with a symmetric “bulk key.” This bulk key is stored along with data on the server, but in encrypted form: encrypted with your public key. That means to get to your data you’ll need your private key to decrypt the bulk key which in turn can then decrypt your private data.

If I now wanted to share my bookmarks with you, I could simply give you my bulk key by encrypting it with your public key. Job done. You can see my data (and only the data I’ve encrypted with the particular bulk key that I’m sharing with you), but nobody else can. Not even Mozilla.

I know, sharing bookmarks is so 1998. But it’s essentially the same thing as the Like button (or the LUUV button). Or your address book. Or your whole social graph. Point is, we no longer need the server to do the heavy lifting for us because the browser environment is getting richer — be it the HTML templating, session storage or even cryptography. The server can become a dull data storage that we can scale the heck out of and, more crucially, potentially ditch for different one if you like. While all the data is in the client’s hands and leaves it only in encrypted form.

This is the kind of cloud I can definitely get on board with.

Today I had one of those it-came-to-me-as-I-was-under-the-shower moments. For some reason, a conversation from a couple of months ago popped back into my mind. Somebody had asked me about associative arrays in JavaScript. To which I simply replied something like, well, they’re just there, built into the language. Just use an object! You could technically even use an array (since JS arrays aren’t really arrays but hash tables with string keys whose strings happen to convert to integers), but that’s not a good idea for various reasons.

This morning I realized that my answer wasn’t entirely complete. Objects work fine as associative arrays a.k.a. hash tables a.k.a. dictionaries if your keys are strings or can be uniquely converted to strings (in which case you could write toString() methods for all your objects you want to use as keys.) Python is a bit less restrictive with its dictionaries and requires that your object be hashable. Immutable built-in types like strings and tuples are hashable, so as long as you can find a one-to-one unique mapping to those (and implement that in __hash__()), you’re good. The advantage of this system is that string representation and hashing aren’t mixed into one interface.

The good news is that thanks to the concept of object identity (present in both JavaScript and Python), you can actually write an associative array that accepts arbitrary keys:

function dict () {
    var keys = [];
    var values = [];

    return {
        get: function (key) {
            return values[keys.indexOf(key)]
        },

        set: function (key, value) {
            var i = keys.indexOf(key);
            if (i === -1) {
                i = keys.length;
            }
            keys[i] = key;
            values[i] = value;
        },

        del: function (key) {
            var i = keys.indexOf(key);
            keys.splice(i, 1);
            values.splice(i, 1);
        },

        keys: function () {
            return keys.slice();
        },

        values: function () {
            return values.slice();
        },
    };
}

I’m not going to bother with a Python implementation because it would look almost identical (modulo syntax). Also, if you find yourself wanting to use something like this in Python, you’re probably doing something wrong. Python’s dictionary implementation is insanely fast and the range of immutable types (strings, tuples, frozensets, etc.) should be sufficient.

And come to think of it, so far I haven’t felt the need for something like this in JavaScript either. But it came to me under the shower, so I had to write it down.

A few months ago I wrote a post about JavaScript titled Curly braces are not the problem wherein I pointed out one of JavaScript’s biggest weakness, the new operator and how to spell an object constructor as well as methods on the corresponding prototype. Some commentators mistook that post for critique of the prototype model itself. It was far from it, I think the prototype model is great, just the spelling was awful. Consider this:

function MyObject() {
    /* constructor here */
}
MyObject.prototype = {
    aMethod: function () {
        /* method here */
    }
};

which is alright until you now want to inherit from this and add methods:

function YourObject() {
    /* constructor here */
}
YourObject.prototype = new MyObject();
YourObject.prototype.anotherMethod = function () {
    /* another method here */
};

There are several problems with this. First of all because YourObject inherits from MyObject, it has to be spelled differently. Secondly, we can’t reuse the constructor, at least not without resorting to func.apply() tricks. Thirdly, we have to know what to pass to the constructor of MyObject at definition time.

It turns out, Doug Crockford not only agrees with me on this but also has come up with a better way. Back in January I thought that we needed more syntax to fix this, but it turns out we need less (by which I mean ditching the new statement). In Vol. III of his excellent Crockford on JavaScript lectures, he defines a constructor maker:

function new_constructor (extend, initializer, methods) {
    var prototype = Object.create(extend && extend.prototype);

    if (methods) {
        methods.keys().forEach(function (key) {
            prototype[key] = methods[key];
        });
    }

    var func = function () {
        var that = Object.create(prototype);
        if (typeof initializer === 'function') {
            initializer.apply(that, arguments);
        }
        return that;
    };

    func.prototype = prototype;
    prototype.constructor = func;
    return func;
}

I’ll let you work out the details of this yourself and instead just show you how you would define the equivalent of the two cases above:

var new_my_object = new_constructor(Object, function () {
    /* constructor here */
}, {
    aMethod: function () {
        /* method here */
    }
});

var new_your_object = new_constructor(my_object, function () {
    /* constructor here */
}, {
    anotherMethod: function () {
        /* method here */
    }
})

See how symmetrical both forms are now? And if both object constructors really were were to share the same initializer, I could easily define that as a separate function and reuse it.

Btw, if you do any sort of web development, I highly recommend you watch the Crockford on JavaScript talks. They’re not only entertaining but are an excellent lesson in history of all the technology that makes up the web.

The Firefox UX team is working on fixing the download manager. I haven’t actually seen what they’re planning yet, but Limi invited me to tell him what I thought. So here it goes:

The separate window sucks. Google Chrome has a tab for the download history which definitely is an improvement over the separate window, and it allows you to search downloads.

There are two cases that should IMHO be handled differently:

  1. downloads that result in the opening of a program (e.g. PDF)
  2. downloads that just drop a file in a directory

For a) you don’t really need that many visual cues. Add an entry somewhere to the download history so you can find it later, but that’s all. The user doesn’t need to or want to do much else. The file opens in another program, case closed. The b) case, however, often concerns long-running downloads, and they definitely require more user action when they’re done. Here the UI should be as non-intrusive as possible, allowing the user to continue working while still giving feedback on the download’s progress.

Personally I use the Download Statusbar extension. It gets the workflow for b) pretty much right IMHO: There’s non-intrusive visual feedback at the bottom of the screen, so I can continue doing something else. Once the download is done I can come back to it without having to check a separate window or tab. Once I double click on the file to open it, it disappears from the download bar. So it’s gone from my “todo list” of downloads to process (it’s still in the download history, though). This is definitely better than having a manual Clear button.

A really nice feature, particularly for the a) case, would be if the download history was actually put to use. Here’s something that happens to me a lot: I open a scientific paper (PDF) from the web in Preview.app. At some later point I read another paper and it refers to the first one. The hyperlink in the references takes me to some page where, unaware that I downloaded it already, I end up getting a PDF I have on disk already. It would be very cool if Firefox recognised this and, instead of dropping another PDF turd in my directory, just opened the original download. Of course you’d need to check Last-Modified headers etc., and perhaps there could be a simple enough option to force a re-download.

To summarise, I would do this:

  • The download history (née Download Manager) is in a tab, searchable, and there’s no Clear button (much like your page history it could be cleared from Preferences).
  • Downloads that open in a program get an entry in the history but no visual cues. If people want to revisit the file, they could simply go to the page they got it from and Firefox would search in its own download history for them. Of course, they can always manually search the download history as well.
  • Progress for downloads that are long-running and/or don’t open in a program is shown in a non-intrusive way (e.g. some bar on the bottom). Double clicking or having the file exposed in Finder/Explorer will clear them off this bar, but of course not from download history.

This is part 3 in a loosely coupled series of blog posts about improving the browser for the cloud, particularly Firefox. In the first post of the series I made my case for why we need web browsers that are designed for today’s cloud apps. In parts 2a and 2b I have been exploring tabs, particularly how to deal with lots of them. And how a simple change in tab behaviour can allow me to blend the previously separate concepts of bookmarks and tabs (cf. my BarTap extension.)

In this post I’d like to muse a bit about creating tighter integration between the browser and webapps. We’re using a lot of webapps these days and a lot of them we never quit. Hence people came up with the idea of “app tabs” which are tabs in your browser that are always there. There’s an extension for this, and more importantly, app tabs will be an integral part of the future Firefox UI.

Part of the idea is that app tabs don’t need any chrome, which is geek-speek for the browser UI. Think back/forward buttons, URL bar, etc. In a way that’s true because the webapps displayed in these app tabs are typically self-contained and don’t need navigational UI elements. But this got me thinking: Perhaps there are other UI elements that a browser could provide to make the webapp experience easier? As outlined in the first post of the series, I find it a bit worrying that webapps employ so many different UI paradigms, thereby evading the most humane feature a UI can have: consistency.

But as I also said in that post, closed environments like a mobile platform (e.g. the iPhone) have shown that apps like Facebook and WordPress can actually be made to have a similar look and feel. Why not do the same thing, possibly on a smaller scale, in the browser?

Due to a transcontinental flight I had some otherwise unproductive time earlier this week, so I hacked together a small extension called Vanadium that does just that, albeit in a very very primitive way. It basically hides the navigation chrome for app tabs and instead displays the “vanadium”, a toolbar that represents actions for the webapp. Here is Vanadium in action using Gmail:

Using Vanadium for Gmail. All of the toolbar buttons are hooked up, including the search. The Home button takes you to the inbox.

I’ve also integrated it with Twitter, though of course the Reply button doesn’t make that much sense there:

Vanadium hooked up to Twitter. Not all of the toolbar buttons make sense.

All in all it was a fun experiment (certainly a nice way to spend a flight) and I think it’s worth pursuing the idea further. Of course not all toolbar buttons make sense in all webapps. And the way the Vanadium toolbar works together with the webapp would definitely have to more sophisticated. I think the most promising item would be the search bar so I’m thinking I’ll be investigating a cross-webapp search bar next.

What do you guys think?