Hand Signals

What works better than remote controls, mice and joysticks, and works without batteries, wires or a user manual?

It's called the hand. Please click on graphic at left to enlarge.

Hand signals to the PC is the way technology is going.

In the coming months, the likes of Microsoft, Hitachi and major PC makers will begin selling devices that will allow people to flip channels on the TV or move documents on a computer monitor with simple hand gestures. The technology, one of the most significant changes to human-device interfaces since the mouse appeared next to computers in the early 1980s, was being shown in private sessions during the immense Consumer Electronics Show here last week. Past attempts at similar technology have proved clunky and disappointing. In contrast, the latest crop of gesture-powered devices arrives with a refreshing surprise: they actually work.

“Everything is finally moving in the right direction,” said Vincent John Vincent, the co-founder of GestureTek, a company that makes software for gesture devices.

Manipulating the screen with the flick of the wrist will remind many people of the 2002 film “Minority Report” in which Tom Cruise moves images and documents around on futuristic computer screens with a few sweeping gestures. The real-life technology will call for similar flair and some subtlety. Stand in front of a TV armed with a gesture technology camera, and you can turn on the set with a soft punch into the air. Flipping through channels requires a twist of the hand, and raising the volume occurs with an upward pat. If there is a photo on the screen, you can enlarge it by holding your hands in the air and spreading them apart and shrink it by bringing your hands back together as you would do with your fingers on a cellphone touch screen.

The gesture revolution will go mainstream later this year when Microsoft releases a new video game system known at this time as Project Natal. The gaming system is Microsoft’s attempt to one-up Nintendo’s Wii.

Where the Wii requires hypersensitive hand-held controllers to translate body motions into on-screen action, Microsoft’s Natal will require nothing more than the human body. Microsoft has demonstrated games like dodge ball where people can jump, hurl balls at opponents and dart out of the way of incoming balls using natural motions. Other games have people contorting to fit through different shapes and performing skateboard tricks.

Just as Microsoft’s gaming system hits the market, so should TVs from Hitachi in Japan that will let people turn on their screens, scan through channels and change the volume on their sets with simple hand motions. Laptops and other computers should also arrive later this year with built-in cameras that can pick up similar gestures. Such technology could make today’s touch-screen tools obsolete as people use gestures to control, for instance, the playback or fast-forward of a DVD.

To bring these gesture functions to life, device makers needed to conquer what amounts to one of computer science’s grand challenges. Electronics had to see the world around them in fine detail through tiny digital cameras. Such a task meant giving a TV, for example, a way to identify people sitting on a couch and to recognize a certain hand wave as a command and not a scratching of the nose.

Little things like the sun, room lights and people’s annoying habit of doing the unexpected stood as just some of the obstacles companies had to overcome.

GestureTek, with offices in Silicon Valley and Ottawa, has spent a quarter-century trying to perfect its technology and has enjoyed some success. It helps TV weather people, museums and hotels create huge interactive displays.

This past work, however, has relied on limited, standard cameras that perceive the world in two dimensions. The major breakthrough with the latest gesture technology comes through the use of cameras that see the world in three dimensions, adding that crucial layer of depth perception that helps a computer or TV recognize when someone tilts their hand forward or nods their head.

Canesta, based in Sunnyvale, Calif., has spent 11 years developing chips to power these types of 3-D cameras. In the early days, its products were much larger than an entire desktop computer. Today, the chip takes up less space than a fingernail. “We always had this grand vision of being able to control electronics devices from a distance,” said Cyrus Bamji, the chief technology officer at Canesta. Competition in the gesture field has turned fierce as a result of the sudden interest in the technology. In particular, Canesta and PrimeSense, a Tel Aviv start-up, have fought to supply the 3-D chips in Microsoft’s Natal gaming system.

At last week’s Consumer Electronics Show in Las Vegas, executives and engineers from Canesta and GestureTek were encamped in suites at the Hilton near the main conference show floor as they shuttled executives from Asian electronics makers in and out of their rooms for secretive meetings.

Similarly, PrimeSense held invitation-only sessions at its tiny, walled-off booth and forbade any photos or videos of its products.

In one demonstration, a camera using the PrimeSense chip could distinguish among multiple people sitting on a couch and even tell the difference between a person’s jacket, shirt and under-shirt. And with such technology it’s impossible, try as you might, to lose your remote control.