What Zope can learn from Mozilla

September 17, 2008

Mark Ramm’s keynote at DjangoCon, A TurboGears guy on what Django could learn from Zope, has made the rounds in the Python blogosphere this week. His main point is that other projects will always have more talented people than your project. That’s why it’s always a good idea to be open, share technology and work with others. The real point, however, is that you can learn from each other’s mistakes (and he specifically advises Django to not repeat Zope’s mistakes).

This brings me to an interview with Scott Collins, published by Ars Technica in July 2004 (sic!). When asked about the mistakes Mozilla had made in the past, one of the things he mentions seems awfully familiar:

We made a version of COM, called XPCOM, a fundamental piece of every component of every part of the software. We took COM all the way down to the lowest levels of the system, and I think that XPCOM is a fantastic useful tool. We had great and important places to use it, but we used it too deeply, we allowed it to influence our memory model across the board, and there continue to be costs associated with that which I don’t think we need to be paying. I feel bad that I let it go that deep when I continually fought against it, but I am one of the perpetrators — I wrote some of the machinery that made it easy to use COM, and that allowed people to go deeper and deeper, which we knew was a mistake.

I’ve been saying this for years, and people tend to think that I’m damning COM unconditionally, and I’m not. COM is very important to us, and it’s a foundation of our scriptability layer, and scriptability is fundamental to how our application works. But I don’t think every object should be a COM object. We are finally moving away from using COM for everything, and maybe a synergy with Mono will help us get to the right level of differentiation between objects. This is a deep engineering thing, but I believe that fundamentally that we took the COM metaphor too deep and we paid a price for that: we could have more performance sooner earlier on if we had recognized and stuck to limits in our use of COM.

Most of this statement would hold true for Zope if you replaced “COM” with “Component Architecture”, even the bit about the speed. I’m not saying that the CA is slow, but it’s making us write stuff in complicated ways which seems to end up slowing down everything as a whole. Not to mention developer speed and developer learning speed. We had to build a whole framework on top of Zope just to hide the complexity of the CA for beginners.

Mozilla has learnt its lesson and refactored its code, especially the Gecko rendering engine. I wouldn’t be surprised if the speed and memory improvements in Firefox versions 2 and 3 are in part due to reducing the overuse of XPCOM. If you now think Zope hasn’t learnt its lesson yet, it’s not quite true. The Repoze guys have taken great ideas from Zope and re-implemented them in a much simpler manner that doesn’t need the Component Architecture and works well with others (by [ab]using WSGI) and makes them scale incredibly well. Examples:

  • repoze.tm2, a WSGI middleware that allows application-wide transactions (an alternative to having the transaction handling mangled into the object publisher)
  • repoze.who, a WSGI middleware for authentication (an alternative to Zope’s IAuthentication utility)
  • repoze.urispace, an implementation of W3 URISpace which describe a method to define arbitrary application policy based on URL paths (an alternative to Zope 2’s acquisition and Zope 3’s local components)
  • repoze.errorlog, repoze.profile, repoze.debug, etc., which are various WSGI middlewares that take responsibility out of the object publisher

This isn’t to say that the Component Architecture isn’t useful or elegant because it is. But after having spent nearly two years helping create a framework that tries to hide the complexity of CA (over)use, I think it’s time to change gears a bit. The Repoze guys have pioneered it, now it’s time that Zope itself embraced it. If I were to pick, the publisher, its publication machinery and the traversal code would be the first to undergo this treatment, and Martijn seems to have those on his list as well. But Zope is bigger than just the two of us.


11 Responses to “What Zope can learn from Mozilla”

  1. Tim Says:

    You probably didn’t mean it, but here it sounds like repoze.bfg doesn’t use the component architecture at all. bfg views are using the component registry though. Just to be clear.

  2. philikon Says:

    You’re right, repoze.bfg uses the Component Architecture for a specific usecase, looking up views for models. This is still a very useful application of the CA.

    Note that I actually didn’t mention bfg at all in the text because it’s not much of interest for Zope. Bfg is a minimalistic web framework while Zope and Grok want to be mega-frameworks.

  3. i think philiKON was referring to the middleware parts and not bfg… bfg sort of violates the repoze “share with other people and let other people share with us” repoze mission statement (in our defense we did need some very small core on which to build apps ourselves).. but the actual repoze focus is pretty much still the same.. make libraries that all python programmers can use without buying in to the CA.

  4. Martijn Faassen Says:

    Note also that my intended “neo traverser” doesn’t actually forgo the use of the component architecture completely. I think it’s still very useful to have an adapter lookup for each traversal step for instance. The code that does all this can however hopefully be a lot simpler to comprehend, and not be quite the scattered ravioli code that the current Zope 3 publisher is.


  5. chrism Says:

    I’ll venture this suggestion.. the bfg “router” (aka publisher) allows for traversal customization via a single interface indirection (ITraverser). It would be very simple to plug an alternate traversal policy into it. It’s also not at a point where it can’t change (unlike the z3 publisher), so even if it didn’t quite do what grok needed, it could be made to do so.

  6. Alec Munro Says:

    I would be very interested to see where this goes, and help out. I’ve spent a lot of time on both sides of the CA fence. I appreciate it’s elegance conceptually, but I’ve often been bitten by implementation details.

  7. David Pratt Says:

    Hi Phillip. I agree with much of what you have to say here about zope. If you are serious about this direction, it may be better to lend support to repoze or set up a repoze track in zope’s repository for this purpose. It is possible Zope’s future may be best a library of packages than as a framework itself.

    Consistent with your post, I also felt some urgency to change course with my own work as a result. Zope is becoming non competitive with other frameworks comparatively on resource utilization. This is a key motivator for me. Much of this is due to the large number of dependences of zope which can only be sorted but will require some considerable time.

    I see the publisher, global registry and dependencies as the major (but not impossible) changes to zope. In fact, it would create a new zope if undertaken. Keeping backwards compatibility would be a real concern for pure zope consumers. Given zope3 is also readily consumed by zope2, I can see any real changes being a real mess for the Plone folks (that already straddle two incarnations of zope). Reducing dependencies helps zope and other frameworks consume zope packages. At some point, with the framework notion, you have to start over again. This is what Chris has done with repoze.bfg in an innovative way.

    At the same time, repoze.bfg does not throw out the baby with the bath water with CA. You can fully use Component Architecture of zope which is important to me. It also takes advantage of martian (the important development of grok) as an alternative means for registering views. I am using repoze for a project and what I like about it is the following:

    1. Component Architecture: Full use of CA
    2. Publisher: provides alternative routes traversal mechanism
    3. Reworked Global Registry: Multiple apps are able to operate within a single process. You can forget about sites concept in zope to provide full isolation between apps or sites.
    4. Development Speed: Development speed is fairly fast and an alternative decorator available to register views.
    5. Simplicity: It offers development simplicity and decreased volume of everything decreases. Its like breathing fresh air.
    6. Footprint: Without the dependency issues associated with zope. A app will start in the 20MB range or less (less than half footprint of a basic zope install)

    I guess the biggest question is really what zope should be in the future. Personally, I think you would need a zope4 to pull existing zope components together into another notion of a framework. If this were done it would look alot like repoze.bfg.

  8. Martijn Faassen Says:

    Chris, Tres gave me the understanding that repoze.bfg doesn’t use an adapter lookup during traversal at all. Sounds like I need to look at the code. 🙂

  9. Robin Says:

    Mono is mentioned in the quote. What does Mozilla have to do with Mono?

  10. Tres Seaver Says:

    The CA is used once, at the start of the request, to look up the traversal policy: the default
    implementation for “graph-like” sites is the __getitem__-only one I described to you. See:

    http://svn.repoze.org/repoze.bfg/trunk/repoze/bfg/traversal.py (in ModelGraphTraverser’s

    for the implementation, and:

    http://svn.repoze.org/repoze.bfg/trunk/repoze/bfg/router.py (in Router.__call__)

    for the point at which it is looked up.

  11. […] match for Alex Limi’s proclaimed faster and lighter Plone 4, next to ditching Archetypes, reducing the Component Architecture overdose and going more […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: