Nothing but an Illusion? The Perception of Extreme Facial Expressions

Background: The distinction between positive and negative facial expressions is assumed to be clear and robust. Nevertheless, recent research with intense real-life faces of winners and losers has shown that viewers are often unable to reliably differentiate the positivity or negativity (valence) of such expressions without the body context.

Aim: What do viewers believe they rely on while rating the valence of intense ambiguous expressions? Is it the face or the body? Does the allocation of our attentional resources reflect this belief?

Method: We designed an experiment in which after rating the valence (how positive or negative the emotional reaction is) of an intense image (non-diagnostic winning faces presented with the bodies), participants ranked their reliance on different facial and body features. Half of the participants completed the ratings from their memory, while the other half, were presented again with the image.

In a consequent study, we analyzed the attentional eye scanning patterns during valence ratings of contextualized intense winning and losing facial expressions.

Results: Participants reported relying more on facial than on body cues, even though the facial expression was objectively non-diagnostic (as determined by independent rating studies). We refer to this phenomenon as Illusory Facial Affect: the perceptual attribution of clear positive or negative affect to an inherently ambiguous face while disregarding the objective diagnostic source of the affect in the body. Importantly, this illusion was not merely an artifact of memory as the effect still held with images which were ranked while still presented on the screen.

Using eye tracking, we further showed that viewers allocated their attentional resources accordingly to this belief, they fixated more on the face than on the body, while rating the valence of the image. Interestingly, when the face was diagnostic (loser), it received fewer fixations than when it was not diagnostic (winner).

Discussion and Conclusion: Viewers tend to overestimate the importance of facial expressions in emotion perception, while underestimating the importance of the body context. They also allocate more attentional resources to the non-diagnostic face than the body, even though they extract information from the body.