Mapping and manipulating facial expression

Barry-John Theobald, Iain Matthews, Michael Mangini, Jeffrey R. Spies, Timothy R. Brick, Jeffrey F. Cohn, Steven M. Boker

Research output: Contribution to journalArticlepeer-review

34 Citations (Scopus)


Nonverbal visual cues accompany speech to supplement the meaning of spoken words, signify emotional state, indicate position in discourse, and provide back-channel feedback. This visual information includes head movements, facial expressions and body gestures. In this article we describe techniques for manipulating both verbal and nonverbal facial gestures in video sequences of people engaged in conversation. We are developing a system for use in psychological experiments, where the effects of manipulating individual components of nonverbal visual behavior during live face-to-face conversation can be studied. In particular, the techniques we describe operate in real-time at video frame-rate and the manipulation can be applied so both participants in a conversation are kept blind to the experimental conditions.
Original languageEnglish
Pages (from-to)369-386
Number of pages18
JournalLanguage and Speech
Issue number2-3
Publication statusPublished - 2009

Cite this