We know what you’re thinking
IMAGINE browsing a website when a saucy ad for lingerie catches your eye. You don’t click on it, merely smile and go to another page. Yet it follows you, putting up more racy pictures, perhaps even the offer of a discount. Finally, irked by its persistence, you frown. “Sorry for taking up your time,” says the ad, and promptly desists from further pestering. Creepy. But making online ads that not only know you are looking at them but also respond to your emotions will soon be possible, thanks to the power of image-processing software and the ubiquity of tiny cameras in computers and mobile devices.
Uses for this technology would not, of course, be confined to advertising. There is ample scope to deploy it in areas like security, computer gaming, education and health care. But admen are among the first to embrace the idea in earnest. That is because it helps answer, at least online, clients’ perennial carp: that they know half the money they spend on advertising is wasted, but they don’t know which half.
Advertising firms already film how people react to ads, usually in an artificial setting. The participants’ faces are studied for positive or negative feelings. A lot of research, some of it controversial, has been done into ways of categorising the emotions behind facial expressions. In the 1970s Paul Ekman, an American psychologist, developed a comprehensive coding system which is still widely used.
Some consumer-research companies also employ goggle-mounted cameras to track eye movements so they can be sure what their subjects are looking at. This can help determine which ads attract the most attention and where they might be placed for the best effect on a web page.
This work is now moving online. Higher-quality cameras and smarter computer-vision software mean that volunteers can work from home and no longer need to wear clunky headgear. Instead, their eyes can be tracked using a single webcam.
One of the companies doing such work, Realeyes, which is based in London, has been developing a system that combines eye-spying webcams with emotional analysis. Mihkel Jäätma, who founded the company in 2007, says that his system is able to gauge a person’s mood by plotting the position of facial features, such as eyebrows, mouth and nostrils, and employing clever algorithms to interpret changes in their alignment—as when eyebrows are raised in surprise, say. Add eye-movement tracking, hinting at which display ads were overlooked and which were studied for any period of time, and the approach offers precisely the sort of quantitative data brand managers yearn for.
At present the system is being used on purpose-built websites with, for instance, online research groups testing the effect of various display ads. The next step is to make interactive ads. Because they can spot the visual attention given to them, as well as the emotional state of the viewer, these ads could tailor their responses.
As similar gimmicks become widespread, privacy concerns will invariably mount. People would need to give consent to their webcams being used in this way, Mr Jäätma admits. One way to persuade internet users to grant access to their images would be to offer them discounts on goods or subscriptions to websites.
Realeyes is also working with Kaplan, an educational-services company, on a project in Hungary which is using the system to measure how children respond to virtual games that teach them English. The hope is that by performing the same emotion-reading trick that marketers use, the type of tasks and the characters that appear in them can be made more engaging.
The technology would make computer games more engaging, too. Sony, for one, thinks that reading players’ emotions with webcams would let software pick up on their subconscious behaviour and change the game in ways that would enhance the experience. The company claims that in the future it will be possible to have something like a detective game in which the camera can read players’ faces and measure their heart rates in order to have a stab at deciding which ones are lying.
In fact, webcams that monitor a person’s heart rate are soon to appear. Instead of sticking sensors onto the skin, Philips has developed a vital-signs camera system which the Dutch company says can measure heart and respiration rates extremely accurately. To calculate the heart rate the camera detects tiny changes in the colour of the skin. These changes, imperceptible to the human eye, occur as the heart pumps blood through the body. The person’s breathing rate is measured by detecting the rise and fall of his chest. The firm will soon launch an app for Apple’s iPad 2 which will allow people to measure their own heart and breathing rates using the two webcams in that device.
Philips is developing the technology as a contactless system to keep a virtual eye on hospital patients, such as newborn babies, who might find conventional monitors distressing. The company is also eyeing anxious parents who always want to know what their tots are up to, as well as anxious coaches and their athletes. Advertising firms will, no doubt, be just as keen to measure heartbeats, especially for ads designed to get pulses racing. Those who find it all smacks of Big Brother can turn their webcams off. If you are playing online poker, that is probably a wise idea.
