Computer Learns to Read Human Micro-Expressions

Not far from now:

A suspect sits in a police interrogation room. All the trimmings are there, a two-way mirror, the single bulb lighting fixture, steaming coffee on a steel table, the cops with rolled up sleeves and tired yet stern expressions, a video camera. These cops are positive their suspect is lying, but he isn’t breaking under pressure. However, the officers have another tool in their arsenal. Embedded in their video camera array is a micro-expression analysis system capable of picking up the slightest facial cues.

While the above scenario is entirely fictional, the technology mentioned is already here.

“In high-stake situations…an ME (micro-expression) fleeting across the face could give away a criminal pretending to be innocent, as the face is telling a different story than his statements,”write researchers from the Univ. of Oulu.

Since the 1960s, psychologists have studied micro-expressions and their ability to relay true meaning even when clouded by false statements. Paul Ekman, a co-discoverer of the phenomenon, developed micro-expression training tools to help people hone their micro-expression recognition abilities.

“Meanwhile, in computer vision, many research groups have accumulated experience in analyzing ordinary (facial expressions),” write the researchers. “Algorithms have been reported to achieve (facial recognition) performance of above 90% on frontal view.”

The researchers turned their attention to micro-expressions, and subsequently created a machine learning algorithm capable of picking them out. But first, the researchers needed to create a database f micro-expressions.  

One-hundred sixty-four micro-expressions were recorded from 16 individuals, who watched a variety of video clips meant to induce strong emotional reactions. The subjects were asked to hide their true feelings during the duration of video watching. This ensured the only facial movements picked up were micro-expressions. Reactions were recorded on a high-speed camera at 100 fps. The facial expressions were categorized as positive, negative or surprise.   

According to MIT Technology Review, “The team linked the emotions on display to the emotional content of the videos, giving them a gold-standard database with which to train their machine-learning algorithm.”

Still, the researchers faced a challenge due to micro-expressions being too low in intensity for recognition. To increase recognition, the team created an algorithm that magnified expressions. The machine-learning algorithm was then tested against 15 human subjects.

The mean accuracy for the human test subjects was 72.11%, while the computer’s mean accuracy was 81.69%.

“Our method is the first system that has ever been tested on a hard spontaneous ME dataset, containing natural MEs,” the researchers write. “It outperforms humans at ME recognition by a significant margin, and performs comparably to humans at the combined ME spotting and recognition task.”

The researchers suggest the technology can be used in lie detection, law enforcement and psychotherapy.