Ludwig Wendzich

A lot of the secret sauce behind Natural User Interfaces is the understanding of the difference between abstract and direct manipulation of content. I’ve been talking about this a lot over the past couple of weeks and was delighted to see that Bill Buxton gave a half hour talk entitled “Doing What Comes Naturally” (Buxton starts at 2:13:40) at the end of MIX10’s second keynote.

Buxton started off with a very clear description of what Natural User Interfaces (NUIs), or what I’ve referred to as Natural Human Interfaces, are by using the example of a saxophone. He used a device modelled after a saxophone to simulate the sounds of a saxophone. This is a NUI, a direct manipulation. Compare this to a computer program where certain keys on a keyboard translate to the holes on the saxophone and you understand the difference between a NUI and an abstract UI. Buxton went further to say that what make the saxophone a NUI is not because it’s a representation of a real saxophone but that it’s a device that he already knows how to use because it respects the skills he’s already attained from learning how to play the sax.

He went on to demonstrate how this NUI could be used to simulate a flute and finally a guitar. Although different instruments, they all speak the same language and a note on one instrument translates to the same note on another instrument (albeit they are achieved through different interactions.) What made the saxophone a NUI was that Buxton didn’t need to learn how to use it.

Natural User Interfaces are interfaces that respect the skills acquired by users.

A guitar or flute would not have been a NUI for Buxton because he didn’t have the skills to use them, he could play a saxophone so a saxophone was.

If you found a PC 2000 years in the future and all trace of human kind had been lost. What would a good physical anthropologist infer about human beings today? And they’d say you have no legs, you have no mouth, you have an eye that can do fully saturated colours you have an ear that can do 200Hz bandwidth and you have one claw with about 80 fingers.

The gap between the tools we currently use to interface with computers and the way we interact with our world is clear. A computer is not built for US, we have been forced to learn how to speak the computer’s language. The most important part about technology is not the hardware, not the software it’s the wetware, the human being.

The important part of this equation is not what is the latest and greatest input technology. The important technology is to focus on is the Human Being, how we work, how we naturally interface. This is why I referred to these interfaces as Natural Human Interfaces, the Human part is important, it’s the most important. The input technology changes swiftly and constantly, but human beings change very slowly and so you will benefit so much more in the long term from understand human beings and designing for them and not a particular technology.

It’s been said that Generation X grew up with the internet and that Generation Y is growing up with mobile. With all this focus on mobile, there must be some exciting things happening the software space, right?

Let’s first take a look at the PC space: Windows is ubiquitous and suffers from version bloat, but it’s there and almost every PC maker has access to it. Apple’s Macintosh may have a smaller market share but their focus on high-end products means they make most of the industry profits. Apple’s strict control of their operating system (Human Interface Guidelines etc) and the fact that it’s so easy to use (there’s only version and anything for Mac will just work on it) means that it’s often held as the gold standard in operating systems. Linux; it exists, but not really.

In the mobile space: Apple holds the same place — lower market share but high-end high-quality products means high profits and customer satisfaction; the gold standard. What’s different is that the Linux offer (Google’s Android is based on a Linux kernel) and the offering from Redmond has switched places. There are many versions of Linux on PC but no-one cares because it’s not really good enough for average consumers to use. There are many versions of Windows Mobile too, but no-one except die-hard WinMo fans care; it’s just not good enough. Android OS is the new Windows (in PC land) of the mobile space.

There are so many versions of Android out right now: 1.5, 1.6, 2.0 and then you get the layers that manufacturers add like Sense and TouchFLO. The thing is, Android is easy for manufacturers to use with their phones so they use it (just like Windows.) The experience may not be that of the iPhone, but it’s good enough for out-of-the-box use. Most people aren’t going to bother updating their phone unless you encourage them to do so or unless they buy a new handset, so just like Windows (PC) most people won’t upgrade. This is ok for manufacturers but is it good for developers? And consumers?

Just like Apple’s Mac OS X users, Apple’s Mobile OS X users are actively encouraged to upgrade to the latest and greatest version of the OS. And it’s made very simple thanks to iTunes. This means developers know that they have a standard to develop to and they know that most people who have ever bought a device will be able to install their application. Unlike with Android where if I develop for Android 2.0, I’ll be missing out on almost all of the G1, HTC Magic etc users who aren’t tech-savvy enough to go to HTC’s (of whichever manufacturer) website and download the latest ROM. This means developers’ market is effectively segmented into different OSes (1.5, 1.6, 2.0, Sense, TouchFLO etc) which means multiple versions of the same app are required to get the whole Android market. This is bad for consumers because their options are now severely limited — unless they are savvy enough to upgrade their ROM or until they buy the latest and greatest handset.

The problem with Android is the problem with Windows. It’s not just an Android app (or a Windows App) like it’s just an iPhone/iPod Touch app (or a Mac App.) This app is an Android 2.0 (Windows XP) app and therefore will not run on a Android 1.5 (Windows 95) device. It’s developer orientated (version numbers) instead of consumer orientated (platform.) Most people know they have a Mac, or a Windows machine. They know they have an iPhone, or a Blackberry. They know they have a Magic but probably don’t know it’s Android 1.6. They know they have a Nokia, not an S60 device. The problem with Android is that it’s not consumer-centric in the way the Google Chrome is. Users don’t care about version numbers, which is why Chrome keeps you up to date and which is why the iPhone keeps you up-to-date.

I’m not an iPhone, Android or WinMo developer. I’m not a Windows, Mac or Linux developer. I’m just a consumer, and this has been my experience with the iPhone, iPhone 3GS, HTC Magic, with Mac OS X Leopard — Snow Leopard, Windows 95—7, and a myriad number of Linux distros that didn’t stay installed very long.

My dad, who purchased the HTC Magic, seems to think that it will get better as time goes on. I hope so but with the introduction of hundreds of new devices in 2010 — some without QWERTY keyboards, some without touch-screens and some not even mobile phones — I have a hard time believing that it will.

Android’s accessibility and interoperability will turn it into the Windows of the mobile world. Popular but sub-par.