Facial expressions provide an important behavioral
measure for the study of emotion, cognitive processes,
and social interaction. The Facial Action Coding System
(Ekman & Friesen, 1978) is an objective method for
quantifying facial movement in terms of component actions.
We applied computer image analysis to the problem of automatically
detecting facial actions in sequences of images. Three
approaches were compared: holistic spatial analysis, explicit
measurement of features such as wrinkles, and estimation
of motion flow fields. The three methods were combined
in a hybrid system that classified six upper facial actions
with 91% accuracy. The hybrid system outperformed human
nonexperts on this task and performed as well as highly
trained experts. An automated system would make facial
expression measurement more widely accessible as a research
tool in behavioral science and investigations of the neural
substrates of emotion.