A computerized system developed at MIT can tell the difference between smiles of joy and smiles of frustration.
Do you smile when you’re frustrated? Most people think they don’t — but they actually do, a new study from MIT has found. What’s more, it turns out that computers programmed with the latest information from this research do a better job of differentiating smiles of delight and frustration than human observers do.
The research could pave the way for computers that better assess the emotional states of their users and respond accordingly. It could also help train those who have difficulty interpreting expressions, such as people with autism, to more accurately gauge the expressions they see.
“The goal is to help people with face-to-face communication,” says Ehsan Hoque, a graduate student in the Affective Computing Group of MIT’s Media Lab who is lead author of a paper just published in the IEEE Transactions on Affective Computing. Hoque’s co-authors are Rosalind Picard, a professor of media arts and sciences, and Media Lab graduate student Daniel McDuff.
In experiments conducted at the Media Lab, people were first asked to act out expressions of delight or frustration, as webcams recorded their expressions. Then, they were either asked to fill out an online form designed to cause frustration or invited to watch a video designed to elicit a delighted response — also while being recorded.
When asked to feign frustration, Hoque says, 90 percent of subjects did not smile. But when presented with a task that caused genuine frustration — filling out a detailed online form, only to then find the information deleted after pressing the “submit” button — 90 percent of them did smile, he says. Still images showed little difference between these frustrated smiles and the delighted smiles elicited by a video of a cute baby, but video analysis showed that the progression of the two kinds of smiles was quite different: Often, the happy smiles built up gradually, while frustrated smiles appeared quickly but faded fast.
