Ludwig Wendzich

I’ve been blowing a lot of smoke about this Natural User Interface (NUI) idea lately but that’s because I think it’s very important to the development of UI that is useable to anybody and, at the same time, a joy to use.

This is evident in two approaches to device design, Android/Windows Phone 7 Series Devices and the iPhone/iPad.

Hardware buttons are required for these devices (Android has 4 and Windows Phone 7 Series has 3) which is the equivalent of the mouse and keyboard on a personal computer (abstract interaction.) The iPhone doesn’t, instead it provides a viewport on which is displayed all the UI that you interact with using some form of direct manipulation.

To be clear, the iPhone isn’t a complete NUI experience. The iPad isn’t either, it still requires abstract concepts like clicking but the iPad is a much more complete NUI than the iPhone and the iPhone much more complete than most other devices.

On the iPad you open an album using the pinch gesture which “throws” the photos onto the screen. This gesture mimics the spreading out of the photos and the gesture’s result is what is expected when we spread out physical photos. On the iPhone the idea of photo albums is still very abstract, a collection of files in “folders” displayed as a list.

On the iPhone, a strike-through gesture (known as “swipe to delete”) is used to delete items from a list. This action mimics the crossing out of an item from a physical list and the result is a request for confirmation of deletion. Other devices require you to use arrow keys (or touch) to select an item, then use other buttons (be they hardware or virtual) to display a menu from which we select the “delete” command.

The contrast between these simple, natural, gestures and the abstract actions required when using hardware to abstract interaction (be they a “Menu” button on a phone or a mouse pointer on a personal computer) show the extent to which Natural User Interface improves user experience and reduces the need to learn how to abstract reality into computer concepts which means we can easily pick up these devices and already know how to use them.

Natural User Interface seems to be a technology that both Microsoft and Apple are heavily investing in. Microsoft Research has done a lot of public work in this arena and in terms of consumer products coming out we’re seeing Project Natal from Microsoft and the iPhone/iPad from Apple. The future of User Interaction Design is an exciting one!

It’s been said that Generation X grew up with the internet and that Generation Y is growing up with mobile. With all this focus on mobile, there must be some exciting things happening the software space, right?

Let’s first take a look at the PC space: Windows is ubiquitous and suffers from version bloat, but it’s there and almost every PC maker has access to it. Apple’s Macintosh may have a smaller market share but their focus on high-end products means they make most of the industry profits. Apple’s strict control of their operating system (Human Interface Guidelines etc) and the fact that it’s so easy to use (there’s only version and anything for Mac will just work on it) means that it’s often held as the gold standard in operating systems. Linux; it exists, but not really.

In the mobile space: Apple holds the same place — lower market share but high-end high-quality products means high profits and customer satisfaction; the gold standard. What’s different is that the Linux offer (Google’s Android is based on a Linux kernel) and the offering from Redmond has switched places. There are many versions of Linux on PC but no-one cares because it’s not really good enough for average consumers to use. There are many versions of Windows Mobile too, but no-one except die-hard WinMo fans care; it’s just not good enough. Android OS is the new Windows (in PC land) of the mobile space.

There are so many versions of Android out right now: 1.5, 1.6, 2.0 and then you get the layers that manufacturers add like Sense and TouchFLO. The thing is, Android is easy for manufacturers to use with their phones so they use it (just like Windows.) The experience may not be that of the iPhone, but it’s good enough for out-of-the-box use. Most people aren’t going to bother updating their phone unless you encourage them to do so or unless they buy a new handset, so just like Windows (PC) most people won’t upgrade. This is ok for manufacturers but is it good for developers? And consumers?

Just like Apple’s Mac OS X users, Apple’s Mobile OS X users are actively encouraged to upgrade to the latest and greatest version of the OS. And it’s made very simple thanks to iTunes. This means developers know that they have a standard to develop to and they know that most people who have ever bought a device will be able to install their application. Unlike with Android where if I develop for Android 2.0, I’ll be missing out on almost all of the G1, HTC Magic etc users who aren’t tech-savvy enough to go to HTC’s (of whichever manufacturer) website and download the latest ROM. This means developers’ market is effectively segmented into different OSes (1.5, 1.6, 2.0, Sense, TouchFLO etc) which means multiple versions of the same app are required to get the whole Android market. This is bad for consumers because their options are now severely limited — unless they are savvy enough to upgrade their ROM or until they buy the latest and greatest handset.

The problem with Android is the problem with Windows. It’s not just an Android app (or a Windows App) like it’s just an iPhone/iPod Touch app (or a Mac App.) This app is an Android 2.0 (Windows XP) app and therefore will not run on a Android 1.5 (Windows 95) device. It’s developer orientated (version numbers) instead of consumer orientated (platform.) Most people know they have a Mac, or a Windows machine. They know they have an iPhone, or a Blackberry. They know they have a Magic but probably don’t know it’s Android 1.6. They know they have a Nokia, not an S60 device. The problem with Android is that it’s not consumer-centric in the way the Google Chrome is. Users don’t care about version numbers, which is why Chrome keeps you up to date and which is why the iPhone keeps you up-to-date.

I’m not an iPhone, Android or WinMo developer. I’m not a Windows, Mac or Linux developer. I’m just a consumer, and this has been my experience with the iPhone, iPhone 3GS, HTC Magic, with Mac OS X Leopard — Snow Leopard, Windows 95—7, and a myriad number of Linux distros that didn’t stay installed very long.

My dad, who purchased the HTC Magic, seems to think that it will get better as time goes on. I hope so but with the introduction of hundreds of new devices in 2010 — some without QWERTY keyboards, some without touch-screens and some not even mobile phones — I have a hard time believing that it will.

Android’s accessibility and interoperability will turn it into the Windows of the mobile world. Popular but sub-par.