Thursday, September 2, 2004

Complexity is in the Eye of the Beholder

I've got a few things in the pipeline, which is why it's been quiet around here. More on that later. In the meantime...

Ian has an interesting post here about complexity in APIs. Since he doesn't have a comment facility, I can feel justified in using it as an excuse to post here instead. :)

The part I found interesting was his analogy with cars, wherein he compares automatic and manual transmissions.

So the big problem is that in attempting to provide a simple abstraction, automatic transmission reduces the drivability of a car. I've found that the only way to get the best out of a car with automatic transmission is to learn what its behaviours are so you can anticipate the bad choices that it will make, and adapt your driving style to make it do the right thing. The irony is that this means you not only need to understand the underlying reality of the characteristics of the engine and gear ratios, you also need to understand how the abstraction placed on top of this works in order to be able to work around it.

Seen in this light, the manual transmission looks a whole lot simpler by comparison. It might have more levels, pedals and selectable options, but at least you know exactly what each control will do. There is no need to try and outsmart the abstraction simply to get it to do what you want.

The part that's interesting, of course, is that this is all completely subjective, and based on a rather large number of assumptions. Note, for example, “at least you know exactly what each control will do”. Living on Earth as I do, I've seen plenty of counterexamples to this on the roads every day. I'm sure we all know drivers who still can't operate a clutch properly, for all they've been driving one for years. And frankly, I think I'd rather the cell-phone-jabbering-morons of the world have that extra hand free. But for drivers like Ian, more precise control (probably) means more safety.

All of this is a way to say that the same interface can simultaneously be simpler and more complex, correct and incorrect, appropriate and inappropriate. It depends on the user. Which is why car companies give us a choice of manual or automatic, and why some software programs offer “expert” modes or other customizations.

It's not that I don't think Ian gets this - I'm sure he does - but after reading his piece I was left with the impression that manual transmission is superior, when in fact that just begs the definition of “superior”, which is inherently subjective.


  1. Manual transmission IS superior, it's just that some people are philistines. ;-)

  2. Heh.

    I used to want to only drive a manual. These days, living in DC, where I spent a lot of my highway time changing speeds between "medium" and "slow", I think an automatic might not be so bad.

  3. Nowadays it's possible to make automatic transmissions that drive better than you do. The analogy still holds up: one of the reasons for providing high level abstractions is because if you can say what you want instead of how to do it, the implementor has much more freedom to make it "Just work better" for you behind the scenes.

    While the abstraction is still leaking as much as primitive manual transmissions, Ian is right; experts are better off with a low level interface. Non-experts, who couldn't leverage the low-level interface are just as well off with the higher abstraction, especially as it gives them "free upgrades".

    Modern day F1 drivers use automatics.

  4. Since I landed on your blog, I'll add a self-promoting comment on this topic. My thesis project is aimed at this exact problem. Everyone wants a simple API, but a simple API is often too inefficient. My solution is to build a compiler that can be dynamically extended with new optimization routines. Someone could write a library with a simple API and a corresponding optimizer. When a programmer uses the simple API, the dynamic optimizer will analyze *how* they are using it and generate more efficient code that uses a more complex API. The simple API is really a domain-specific language that is optimized by an extensible compiler.

    If I weren't so freakishly lazily, I would have been done 3 years ago.

  5. "Modern day F1 drivers use automatics."

    On a point of information, no they don't.

    Right now, F1 cars have gearboxes where the gear selection is up to the driver, but the mechanical process of changing gears is automated. So they have a couple of paddles on the steering wheel, one to change up a gear and one to change down a gear. These are typically referred to as 'sequentials' and not 'automatics' - an 'automatic' gearbox is one that decides which ratio to use for you.

    The sequential model arguably offers the best of both worlds - it automates what can usefully be automated, but still lets the driver choose the ratio. The fact is you can't actually build an automatic gearbox that does better than a race driver in most scenarios for the reasons I mentioned in my blog - the gearbox doesn't know what's coming up, because unlike the driver, it can't see where it's going. And even if you programmed the gearbox so it knew the exact layout of the race track, it still can't know about things like safety cars, back markers, overtaking manoeuvres, and all the other transient stuff that can necessitate a change from the gear shift patterns that would get you round an empty circuit as fast as possible.

    So the driver still gets to select the ratio, but everything else is automated. So the abstraction is 'you have 7 gear ratios'. This model enables them to change gear incredibly quickly - it turns out that details such as the internal layout of the gearbox are not useful to the driver so it's OK to abstract these away. And for the most part, it's OK to abstract away the clutch too. (They do actually have the option to operate the clutch manually. That's because although this 'there are only gear ratios' abstraction is optimal for when you're moving, it's not so good for moving away from a standstill at the start of the race.)

    There was a time when F1 gearboxes could shift up automatically - if you are simply accelerating flat out in a straight line, then it is true that an automatic gearbox can pick the optimal shift point more accurately than a human can. But that's a special case that happens to be amenable to automation; these gearboxes are referred to as 'semi automatic' because they only automate gearchanges under certain circumstances. (And the FIA chose to outlaw these for the 2004 season, which is why F1 cars no longer have this feature.)