To understand the magic of multi-touch systems you need only watch a child finger painting. It approaches the page with gusto, hands outstretched and daubed with paint, drawing naturally as inspiration dictates.
And Microsoft's Michael Kordahi (pictured) says Australia's executives respond similarly to multi-touch technology - those computer systems which encourage users to operate them with simple hand and finger movements known as "gestures", for example, Apple's iPhone.
Sitting in Microsoft's executive briefing centre in North Ryde about 15 minutes north of Sydney is a Surface table that looks much like the cocktail games devices that were popular in pinball parlours about 25 years ago. But the similarity ends there because Microsoft's Surface is packed with sensors and cameras that detect up to 50 inputs such as finger presses or contact with objects.
Kordahi demonstrates a prototype e-commerce and discovery system developed for travel publisher and retailer Lonely Planet in which a couple plan an overseas trip. They pull out destination information activated by a travel guide stored in a database presented on a mock hardwood desktop background including maps, photos and videos, resizing and flicking them around the 76cm display, discussing possibilities as they go.
These assets are curated and stored in a "passport", a piece of cardboard with a unique identifier that allows the travellers to revisit their selections at their destination or en route.
All this is done in a very natural way, replicating how they might interact with physical media such as print brochures and photographs, Kordahi says.
Multi-touch is not so much a technology; it's a set of behaviours that aims to replicate how people interact in the real world using multiple inputs. Earlier touch systems were distinguished by their inability to detect and operate with more than one input, many relying on styluses for activation.
Many are familiar with the iPhone's multi-touch system that enables basic icon selection and manipulation of content, such as swiping between pages or resizing images with a pinch of two fingers (invented by Myron Krueger in 1983). Apple has patented a dictionary of gestures that it added to its Mac operating system and included in its new Magic mouse late last year. The rumoured Apple touchpad to be released this year will most likely lean heavily on this gestural vocabulary.
And Microsoft has added gestural user interactions to its Windows 7 operating system and Silverlight multimedia web browser technology in anticipation of its adoption in a new breed of tablet devices. For instance, the couple in the study above could use the unique identifier on their passport to access their itinerary on a Windows 7 desktop even if a Surface table wasn't available.
Although Apple is credited with popularising multi-touch with the mainstream user, it was research on pressure-sensitive devices at the University of Toronto in 1982-85 that laid the groundwork. To put that into perspective, the Apple Macintosh was released in 1984.
Bill Buxton, a former University of Toronto researcher cum Microsoft researcher and a father of multi-touch, says Apple was aware of the early research, especially the pinch gesture, when it bought multi-touch developer Fingerworks in 2005 to jump start its ambitions in the field.
Buxton says that although "touch and gesture have an important role and great unfulfilled potential for delivering much improved user experience and capability in the future", they have their limitations. He says many developers "jumping on to the touch bandwagon" to add such gestural interactions to their products don't understand the inherent limitations of the metaphor.
"There is absolutely a chance of ‘more is less'," Buxton says. "This is something that we have already seen in pen-based systems where there are some basic gestures and shorthand symbols that are easy, intuitive and well suited to the task.
"However, very rapidly, the obviousness and suitability is strained and things become arbitrary, forced and inconsistent."
He says that proofs of concept, "flashy demos" and simple applications obscure the fissures in the user experience of complex, real-world scenarios.
"It is very unlikely that rich, deep systems can, will, or should rely on any single modality of interaction - no matter how trendy. Touch and multi-touch are great for some things, and a disaster for many others.
"I see little that would convince me that many of those jumping onto the touch bandwagon understand this adequately to bring us to the next level."
Despite his misgivings that the touch technology he has worked on for the past 25 years may become a square peg in a round hole, Buxton is enthused that it's becoming mainstream through the efforts of researchers such as Perceptive Pixel founder Jeff Han, whose work has graced the newscasts of US broadcasters such as CNN and Fox News.
"Jeff Han has not only done great work, he has been very consistent in terms of helping people appreciate the depth of the history field," Buxton says. "That reflects good scholarship and integrity - not to mention really creative work."
He's critical of any reference to multi-touch as being natural. "Most of what we do in the world is learned."
He says there is a "good reason that Picasso painted with a brush rather than finger painted".
"And writing a letter with our finger is generally inferior to doing so using a pen. Likewise, picking things up, or manipulating them, is kind of hard with a pen."
He expects future software to incorporate the best of mouse, keyboard, pens or styluses and touch or gestures. "I have two hands, 10 fingers, I have a lot of manual skills and dexterity, and I resent not being able to take advantage of them."
Another company which has worked with touch systems since the early 1980s is Japanese graphics tablet maker Wacom.
Wacom's global product manager for consumer products, Jens Krueger, says gestural interfaces are hampered by "too many parallel efforts".
"The industry has to go one step back and not try to drive a specific technology or way of using a specific technology," Krueger says. "To the user's eye, the need is simplicity of use and access: how do you interact with all the thousands of things that the computer enables you to do?"
Krueger advocates open sourcing the gestural dictionary to avoid a situation that stifles innovation.
"I would be an advocate of a more open source in this regard. If you do a handshake that's a natural gesture in a lot of cultures, would it be helpful if you patented it or licensed that?
The industry needs to go through a learning curve to think beyond features and performance and about how can I make my user interface simple to use, make it work in a way that anyone can use it."
The CRN view
The success of the iPhone and the anticipation of Apple's rumoured touch tablet computing device has entrenched multi-touch and gestural interfaces in users' minds. And Microsoft's embedding of similar technologies in Windows 7 opens them to wider audiences.
But moving beyond simple gestures and across platforms that may have competing and contradictory user interfaces may become a trial for many users. This confusion has been the history of operating systems and is set to repeat in a multi-touch world.
Look for devices from recognised makers with pedigrees in deep multi-touch research and ensure that for vertical applications such as graphic design or visualisation that touch isn't just a gimmick but solves an identified need.
-1.jpg&w=100&c=1&s=0)
.png&w=100&c=1&s=0)
