Status report for the Holiday Season

December 18, 2010

It’s been more than a month since my last status report. Here’s what happened since then:

Firefox 4.0b8, Firefox Mobile 4.0b3, Sync 1.6

We’re getting ready to ship Firefox 4.0b8 and Firefox Mobile
4.0b3. Along with those beta releases we will ship Sync 1.6 as an add-on for Firefox 3.5/3.6. It will match the version of Sync built-in to the betas so that you can sync between Firefox 3.5/3.6 and the betas releases. Here’s what’s new in these releases and what we’ve been working on these past weeks:

Easy setup

Up until now, setting up Sync on a new computer has been a bit of a chore. With the wizard redesign by Alex Faaborg we reduced the number of necessary steps, but you still have to type your account name, password and Sync Key — particularly annoying on mobile devices.

We looked at how other systems cope with this problem in terms of usability. Bluetooth is a good example where you pair another device with an existing one while it’s often hard or even impossible to type on one of them. Obviously for Sync we need something that works on all platforms where Firefox is supported, ranging from desktop computers to mobile phones. Moreover, it needs to be secure so that Sync’s mission of protecting the user’s privacy isn’t compromised in any way.

The solution was PAKE (password-authenticated key agreement), a cryptographic mechanism for two parties to agree upon a strong key based on a weak shared secret, e.g. a Bluetooth-like PIN. In our case it allows a desktop computer that’s already connected to Sync to securely transfer the credentials to, say, a mobile phone. The mobile phone displays a random PIN that simply has to be entered on the desktop computer. Then both devices will go through the PAKE algorithm (J-PAKE in our case) to agree upon a strong key, communicating through a simple server via HTTPS. Once it’s verified on both sides, the desktop will send the credentials to the mobile phone. This way, you can set up a mobile phone for Sync with zero typing on the phone itself. Of course, this also works for setting up another desktop or laptop computer.

This project included design and implementation work at all levels: designing and implementing the server API and communication protocol with my colleagues, making a prototype implementation of the J-PAKE algorithm, refactoring it to be based on the J-PAKE implementation Brian Smith created for NSS, and implementing the easy setup UI. It was challenging, sometimes outside my comfort zone, but possibly the most fun I’ve had in years.

Simpler crypto

Then and now

Up until now, Sync would protect your personal data as follows:

  • Your Sync Key (a.k.a. secret phrase) is turned into a 256 bit AES key using PBKDF2.
  • That key is used to encrypt/decrypt the private key of an RSA key pair that’s stored on the server.
  • The RSA key pair is used to encrypt/decrypt 256 bit AES bulk keys, one per collection, that are used to encrypt/decrypt your private data.

The original vision was that indirection introduced by the RSA key pair as well as the simple PKI we had on the server would allow for easy sharing mechanisms. Alas these weren’t and aren’t a priority — rightly so IMHO — and the complexity of the setup had some serious impact on client and server performance. It also had some shortcomings security-wise.

We now have a much simpler setup that still guarantees the security and privacy of your personal data (better in some places, in fact).

  • The Sync Key is now always a 128 bit random key, user generated Sync Keys are a thing of the past. This means you will no longer have to come up with a secure secret phrase of some kind, let alone understand what it means for a secret phrase to be cryptographically secure (fact is, most users don’t). When shown in the UI we encode the Sync Key in a modified base32 alphabet, making it 26 characters long. Fortunately you will rarely have to type those, thanks to the new easy setup mechanism.
  • A 256 bit AES key and a 256 bit HMAC key are derived from the Sync Key using an HKDF.
  • This key bundle is used to encrypt/decrypt a special keys record on the server. It contains the 256 bit AES and HMAC bulk keys that are used to encrypt/decrypt your personal data. By default there’s only one pair of AES and HMAC keys.

What gives?

So instead of storing at least half a dozen individual records on the server for the various keys, we only have one server record to fetch at the beginning, and we have plans to even cache that locally in the password manager. With 256 bit keys everywhere this is much more viable now.

On average the simplified crypto setup reduces the number of request per sync by ca. 15%. Our ops guys love us for this. On the client it makes the first sync per browser session much faster because there are fewer requests to the server and no RSA crypto is involved — particularly important on mobile devices. It also makes the first sync ever much faster because no RSA key pair has to be generated. Moreover, we can now optimize our crypto code for the simpler setup much better. Initial investigations have shown that a ~25% speed up and ~50% improvement in memory usage might be possible.

I can’t take all the credit for this. Our resident cryptographer Brian Smith helped us design much of the system and, following the proud Mozilla tradition of giving newbies projects that end up touching every single piece of code in the system, our new colleague Richard Newman implemented nearly all of the code.

Tighter integration with Places

Sync interacts a lot with Places, Firefox’s history and bookmark database. Now that Sync is being integrated into Firefox proper, we can couple them more tightly and gain better performance as a result.

For instance, Places will assign the GUIDs as used by Sync to history records and bookmarks as they are created, so that by the time Sync needs to deal with them, they will already be there. Having Places take over the disk I/O here means it will be not on the main thread and it will be part of operations that Places has to do anyway, so it’s less disk I/O overall.

We’re also considering adding more asynchronous APIs to Places, specifically with Sync’s usage in mind.

Coming up

Simplified crypto and easy setup were the last big changes we had lined up for Firefox 4. We still have a lot of work to do, but it’s mostly polish and paper cuts from here on.

There’s also increasing interest from third party developers in building clients for other browsers. Marcus Wolschon, for instance, has written a Firefox Sync client for the popular Androind browser Dolphin HD. Unfortunately, our API and developer docs are currently scattered across many pages in different locations of the Mozilla wiki. As we get closer to Firefox 4, I would like to spend some time collecting and updating that info and putting it in its own section on MDC/MDN.


39 Responses to “Status report for the Holiday Season”

  1. Jack Says:

    Easy setup?i think not easy.Chrome sync is better

    • philikon Says:

      How is it better? Have you tried the easy setup at all? I welcome comments and critique, but trolling isn’t helpful.

  2. Mads Says:

    I tried Sync for the first time a few days ago, and that key is next to impossible to remember. As a user I’m baffled as to why I need 2 passwords, one of which is completely impossible to remember.

    • philikon Says:

      The Sync Key is not a password, it’s an encryption key that’s used to encrypt your personal data (the setup wizard mentions that, btw ;)). Mozilla cares about your privacy and does not want anybody but you to be able to read your data.

      You don’t have to remember the Sync Key as long as you always have a machine connected to Sync. Because then you can either look it up on that machine or just use the new easy setup mechanism to transfer the credentials automatically.

  3. Jack Says:

    Yeah,you’re right.Mads.

  4. Jack Says:

    I don’t like Sync key.

    • philikon Says:

      The Sync Key ensures that your data is protected when it’s stored in the cloud. Mozilla does not want to be in the position of being able to read users’ data. It’s an unusual concept, but protecting the user’s data is part of the Mozilla way.

  5. Kadir Says:

    Wow, wishing for Chrome’s process, where you hand over a list of all your visited sites to Goole, when you can have Firefox’ encrypted sync? Just wow.

  6. Sync’s great. This looks like it might make it a lot easier for users. I of course welcome performance improvements too. I just wish I had a phone that can use Firefox Mobile and Sync too!

    I wonder if the account manager work is progressing, this is something that’s quite crucial for me as I often have multiple identities on websites, such as Gmail.

    • philikon Says:

      Thanks, Jon. What kind of phone do you have?

      As for identity management, this is a project we’ll be taking on in 2011. Stay tuned!

      • I only have an older Symbian phone (Nokia N82), I’m waiting to see what happens with MeeGo before I upgrade.

        Sync makes better security possible for me. For example, I quite recently started using randomly generated passwords, there’s no way I’d practically be able to use that without Firefox Sync, and the idea of extending that to mobile browsing, is a real win. Your emphasis on security of data is very important, and to be applauded.

        I look forward to continue monitoring yours, and the project’s progress.

  7. James Says:

    This sounds great! Especially looking forward to the Places integration and asynchronous processing. Also many thanks for sticking with strong local encryption.

  8. Erunno Says:

    Sync is one of the features which adds so much value to Firefox that it’s currently next to impossible to switch to any other browser. Like some of the other people also baffled about people wishing for Chrome’s solution as either they are not aware that all of their sync data is readable by Google or they simply don’t care (sadly I suspect it’s the latter).

    Also looking for to Account manager. The recent Lifehacker leak shows how important it is to use different passwords for each site and Firefox enabling people to have as many cryptographically secure passwords as needed will be a huge step forward.

  9. Anonymous Says:

    I had to think about the new sync key mechanism for a while, but I like it better than having to come up with a passphrase.

    This raises another issue, though: when will the mobile version of Firefox start having the option of a master password, or something equivalent? I don’t want to sync all my passwords to my phone unless I can set a master password on that phone. (I don’t do that on my laptop because my laptop requires a password to get in, though ideally I’d set up whole-drive encryption to go with that. But given that I don’t have that option on my phone, a Firefox master password would at least protect my data.)

  10. […] For existing users: You can now use the easy setup mechanism to add new devices if you already use Firefox Sync (requires the latest Firefox 4 Beta and is not available for the Firefox Sync add-on). Note that the original setup method still works if you prefer entering a username, password and Sync Key. If you are interested in all the technical details, be sure to read Sync engineer Philipp’s blog post. […]

  11. w1ngnut Says:

    This is quite a good work, congrats! Just one question: do you expect to sync extensions as well?

    And also as our friend @Erunno pointed out, Account Manager would be a cool feature to have…


  12. Ben Poliakoff Says:

    Fantastic work! Just went through the new streamlined setup with Firefox Home on the iPhone, huge improvement.

    Can’t wait to ditch the iPhone for an Android so I can run Firefox Mobile.

  13. 901 Says:

    I’m concern about privacy so I understand the reasoning for the sync key and I’m quite happy with it.

    However I had an idea some time ago. I’m not good at cryptography and security so probably I’m just missing something.

    Why not having just one password:

    First you derive a smaller pass which will be used to log in.

    The server allows to download the user data.

    The whole password is used to decrypt/encrypt the data, always in the local computer.

    I know that this process put Mozilla in a privileged position to crack the whole password since they have the derived key. I also know that security is decreased as a result of using just one password and the derived key will be very easy to crack if the whole one is already weak.

    Performance problems also arise, probably traffic increase, longer synchronization and other things.

    It is just that I don’t know if such a solution has been considered.

    • philikon Says:

      Thanks for your comment. This approach has indeed been considered. But as you mention, this puts Mozilla in a privileged position to dictionary-attack the passphrase (which, since it’d be user-generated, would in all likelihood have much less entropy than the Sync Key does now.)

  14. This is all neat but it is a bit frustrating that it seems like all this fiddling with the sync key setup keeps breaking existing setups.

    Already a couple of weeks back I ran into some bug or other which meant I had to come up with a new sync key and enter it on all my systems. Now I just updated my Windows install to FF4b8 and my laptop to plugin v1.6 and oh, wahay, they’re both claiming the sync key is incorrect and they can’t sync. To add to the fun, the ‘Change Sync Key’ button is greyed out on the laptop. So, how exactly am I supposed to fix this? Does the fact that the first system I set up in this sync group – my desktop – is several thousand miles away make it impossible?

    It’d be nice if these changes could be done in a way which is less disruptive to current users :/

    • philikon Says:

      This was the last time we fiddled with the Sync Key for quite a while. Promise! 🙂

      The fact that 4.0b8 and Sync 1.6 disagree on the Sync Key worries me. This isn’t supposed to happen. If you continue to experience problems, please get in touch with us on the mailing lists or the bug tracker. Thanks!

      As for the “Change Sync Key” button: I believe you’re in the wrong dialog. You want the “Update Sync Key” dialog (click Connect, then see the Wrong Sync Key notice, then click Update). Admittedly that UI needs an overhaul which we’ve scheduled for 4.0b9.

      • That ‘update’ button didn’t show up the first two or three times I went into the Preferences, but now I tried it twice again and it popped up on the second try.

        It says my sync key has changed and asks me to enter the new one. I didn’t change my sync key. I don’t know what ‘the new one’ is. Unless updating the install on the Windows partition from FF4b7 to FF4b8 somehow changed the sync key and now the Linux partition has the wrong one. I did nothing besides update the Windows partition’s Firefox from 4b7 to 4b8 (I can’t recall if it was then able to connect to Sync, now) and update the Linux partition’s Sync plugin to 1.6 when I was prompted. I didn’t make *any* manual change to the sync key at all. The Linux partition’s Firefox install was definitely connecting to Sync correctly a few days ago.

      • I just tried it a couple more times, and each time, hitting ‘Connect’, then clicking on ‘Error signing in’, then ‘Preferences’, took me to the Preferences dialog but didn’t show the red ‘wrong sync key’ bar and the ‘update’ link. I’ve only seen that once. I don’t know why it fails erratically like that.

      • well, that’s annoying: I just went to check exactly how Windows is behaving and something – either installing FF4b8 (v. unlikely), updating Microsoft Security Essentials (much more likely) or mounting the Windows partition in Linux (possible) has utterly nerfed Windows. It won’t boot any more. Just keeps dying at blue screens. fun!

        at this point I’m not in a position to file any useful bugs, but the update certainly didn’t work smoothly :/

  15. Panagiotis Kapadelis Says:

    Hi! Is it compatible with Windows Mobile devices (e.g. HTC Touch HD)?

  16. Panagiotis Kapadelis Says:

    Thanks philikon. I hope to release a version in the future. Merry Christmas!

  17. philikon Says:

    Adam, as I’ve explain in this blog post, the Sync Key has been updated for everybody in the 1.5 -> 1.6 and 4.0b7 -> 4.0b8 upgrade. Something probably has gone wrong for you, but the comment section of this blog post probably isn’t the best way to assist you with your problems. I suggest you head over to the support forums at In any case you can look up the Sync Key on your machines that are syncing correctly.

    • as i said, with the windows partition nerfed, I can’t file a bug that would be of any use to anyone.

      (displaying the sync key within the browser, which most people will leave running all the time, seems a potentially bad design, btw; wouldn’t it be safer to require you to re-input the account password before it’ll display the sync key?)

  18. John Says:

    I’ve been using Weave since version 0.6 and was very excited at the prospect of setting up my own Sync server. It was a service that none of the other sync-ing services could offer. That being said, it has been a chore from the very beginning to, and it seems to me that the Weave/Sync team is completely over-engineering and heading the wrong direction. The new forced Sync key + PAKE is a horrible, horrible idea in terms of usuability that only an engineer could have thought was a good solution. This entire process is completely over-engineered and from a UX standpoint fails on so many levels. Please hire a few professional UX personnel onto the Sync team. Mozilla in general seems to be failing pretty badly on this end and simply copying Opera all the time isn’t helping.

    I’ve been completely fed up with the horrible design of Weave/Sync, syncs that break every 0.1 release and the frequency of releases is entirely unnecessary. The original Weave team was very bad at project management and this continues to this day. It’s funny there’s little to no punishment for the bad practices at non-profits. But I guess with no metrics or sales goals, ideas like PAKE will get accepted.

    I’ve been trying my best to stick with Weave/Sync, but as it stands, the core team behind these technologies are spending too much time with fellow engineers and too little time with people who actually use the technology. Having my own Weave server was one of the few reasons I’ve been sticking with Firefox, but it looks like a move to Chrome seems wise.

    • philikon Says:

      I appreciate critique, but yours isn’t very helpful. Chrome won’t allow you to use your own server while Firefox Sync still does, so I can only assume you’re trolling when you’re “threatening” to move to Chrome. Calling us oh-so very bad at project management, UX and engineering isn’t constructive, either, but hey, who knows, simply shouting at people might help fix the problem! Perhaps you’ll indulge us with your solutions to the problems I carefully outlined in my blog post? And tell me, when where exactly has Mozilla copied Opera?

  19. […] při oznamování vydání Firefoxu 4.0 Beta 8 jsme zmiňovali snadnější propojování zařízení v rámci Firefox Sync. V dialogu Možnosti si tak můžete povšimnout nové podoby konfigurace […]

  20. […] 2011-01-11: J-PAKE integrated into Firefox 4 ( beta 8 ). More details here. […]

  21. Dave Cottlehuber Says:

    Hey I just wanted to thank you, Alex Faaborg, Brian Smith and Richard Newman for all your work on sync. This is for me the killer feature for firefox, and it definitely helps make the web a better place.

    The new UI + simpler setup will work better for new users. Sync is a complex problem that people expect to “just work”. My only suggestion is to put back the little icon in the addons bar that tells the world I am safely sync’d.

    Some people are grumbling about the need to change sync keys during the upgrade – it’s not a beta release for nothing 🙂 toughen up folks.


  22. John Wetsell Says:

    Every time I go to Preferences -> Sync and click “Add a device” it opens a web browser window and not the box to add a device. 😦 FF 3.6.15 Sync 1.7 Am I doing something wrong?

    • philikon Says:

      Yes and no. 🙂

      We did not implement the easy setup screens for Firefox 3.5/3.6 because it required changes to the platform. It’s Firefox 4.0 only.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: