Bill Buxton: Perfect Storm of Opportunity for Touch Computing

January 7, 2010

Will PCs with touch screens soon be as de rigueur in America’s kitchens as the ShamWow? General Mills hopes so. The food company has made available a free PC application called the Betty Crocker Kitchen Assistant that provides not only recipes, but also cooking tips, personalized recipe suggestions based on what ingredients users have on hand, programmable kitchen timers, and a highly intuitive recipe search interface. And it’s all optimized to work with the features of Windows 7, like touch.

The Betty Crocker Kitchen Assistant is a PC menu application optimized for touch, so busy cooks with a little flour on their hands can find recipes, cooking tips and more.

“We believe that most families will soon have a PC on their kitchen counter,” says Mike Bettison, Web site manager, General Mills. “A touch screen takes up less precious counter-top real estate than a keyboard and mouse, and lets you check a recipe without putting down the egg beater.”

The Betty Crocker Kitchen Assistant is just one example of a new wave of consumer applications that take advantage of the touch technology built into the Windows 7 operating system. So in addition to using the mouse and keyboard, people can tap, flick and pinch their way through Web pages, files, menus and a wide variety of applications.

In his keynote presentation at the 2010 International Consumer Electronics Show (CES) in Las Vegas, Microsoft CEO Steve Ballmer talked about the growing momentum touch is gaining and showed some new touch PCs and applications just hitting the market. The Sony Vaio L, for example, is a PC built for HD entertainment that includes a 24-inch touch screen. Graphic.ly, a digital content provider, is launching an application of the same name that offers an extensive online library of classic and new comic books and graphic novels and uses touch to add interactivity and enrich the viewing experience.

CES attendees stopping by the Microsoft booth can try out many other touch applications and touch-enabled games. One example is ArtRage 3 Studio from Ambient Design, a painting app that allows users to paint with digital oils, sketch with digital pencils, sprinkle digital glitter and more. ArtRage also lets users upload photos and then embellish or alter them using their fingers.

Touch Catches On

Touch screens have been a part of life for years — on phones, bank machines, credit card readers and airport check-in kiosks. But the technology has yet to be widely available for PC users. That may be about to change.

Bill Buxton, a principal researcher at Microsoft, says several factors have converged to create a “perfect storm” of opportunity for touch computing.

Why the sudden popularity? Bill Buxton, a principal researcher at Microsoft, says three factors have converged to create a “perfect storm” of opportunity: rising consumer expectations of what should be possible, the increasing affordability of consumer electronics, and advances in technology that have made gesture recognition reliable enough for the mass consumer market.

The engine driving the touch capabilities in Windows 7 is Windows Touch, software that provides the ability to manipulate items on a screen using gestures and more than one finger at a time — up to 100 touch points, in fact. Windows Touch recognizes about half a dozen different gestures, including single and double taps, dragging, rotating, flicking, zooming in and out, and a special tap that equates to a right-click on a mouse.

To make the gestures reliable, developers tested them with consumers and used the results to fine-tune the programming. “We collected thousands of samples from hundreds of people, and then mined that data looking for problems and optimization opportunities,” says Ian LeGrow, a group program manager on the Windows team.

To optimize Windows 7 for touch, developers designed even the most basic elements of the operating system, such as the Start Menu and window borders, to be finger-friendly. The buttons are larger than usual, and specific behaviors are tuned for touch input. For example, all the icons are bigger so they can be easily selected with your finger. Jump Lists can be accessed with a simple drag up from the taskbar, and when opened with touch, the shortcuts in the Jump Lists are drawn with extra vertical spacing to make them easier to select.

How Touch Is Changing the Market

The market for touch PCs is heating up. About 10 percent of new PC models in 2010 will support touch technology, says Amy Leong, research director at Gartner Research. “The touch market is still in its infancy,” Leong says. “In the next few years, we are going to see touch technologies being tested and deployed for many applications. More than 6 million PCs with touch screens will ship in 2010, nearly four times more than in 2008.”

Microsoft has been working closely with its partner PC manufacturers to make sure Windows 7-based PCs represent a nearly flawless symbiosis of software and hardware.

PC makers that have or will shortly have touch-ready Windows 7-based computers include Acer, Asus, Dell, Fujitsu, Gateway, HP, Lenovo, Sony, and Toshiba. A wide range in prices for the touch PCs coming to market this year will put touch within just about everyone’s reach.

And PC shoppers will have plenty of options in terms of features, because manufacturers can customize the basic touch technology embedded in Windows 7. For example, HP chose to include an on-screen Twitter “button” on the desktop of its TouchSmart 600t PC, as well as some basic handwriting recognition — sans stylus.

Just as Windows helped unleash a wave of new software for graphical user interfaces, Windows Touch is inspiring applications that make all manner of tasks feel more natural and efficient. Since the Windows 7 launch in October, hundreds of new touch-optimized applications have been released in a wide variety of fields. Examples include SpaceClaim Engineer 2009+ , a modeling tool that artists and engineers can use to create 3-D designs for digital prototyping, analysis and production, and Mindjet MindManager 8 for Windows, a personal productivity solution that visually connects ideas, information and people to help save time, improve processes and drive innovation.

Customers are beginning to appreciate the value of touch screens. “Touch was one of the things that really sold us on Windows 7,” says Bob Mayer, president of Essential Apparel, an online retailer of lingerie, outerwear and sports gear. “When we’re filling orders, there’s a lot of folding and other handling required to make sure the clothing looks nice when it’s shipped to the customer. Having a touch screen instead of a mouse and keyboard not only improves efficiency, but it also actually results in nicer-looking orders because the packer doesn’t have to put things down and interrupt the packing process to update order status.”

The Future of Touch

Just how widespread is touch technology likely to become?

Buxton says no user interface should be viewed as “one size fits all.” “The mouse and keyboard are great for creating text, for example, but for composing music, not so much,” he says.

LeGrow agrees. “In many scenarios — though certainly not all — it’s faster and easier to use direct manipulation of items on the screen.” Web browsing is particularly well-suited to touch control, he adds.

“The hallmark of a well-designed interface is that it gets out of your way,” says Jennifer Fraser, user experience designer at software developer Corel. Corel’s Digital Studio 2010 program organizes, edits and shares photos and videos — all with a Windows Touch interface. “You don’t want people to focus on the interface, you want them to focus on what they’re trying to do or create. Touch removes a layer of technology between people and their content and replaces it with a direct connection.”

For the long term, researchers at Microsoft say the key is to bring together touch, machine vision and speech synthesis and recognition into an integrated system that works as an intelligent assistant for the user — that’s the promise of a natural user interface (NUI).