Paul Ekman - US grants

This is a renewal application for an ADAMHA Research Scientist Award. I propose to pursue research on three related topics concerned with facial expression and emotion. (1) I will explore the relationship between facial expression and autonomic nervous system (ANS) activity in a series of six experiments. We have evidence that voluntary facial expressions generates emotion- specific ANS activity, and that this and another task produces patterns of ANS activity which distinguish among four negative emotions. I seek to replicate these findings in another culture, in older populations and in patients with facial paralysis, in order to illuminate the underlying mechanisms. (2) I will further explore the behavioral characteristics which distinguish emotional from nonemotional expressions, in particular determining the particular features which distinguish an expression when an emotion is experienced, from expressions intended to mask or falsify feelings not experienced. (3) I will continue cross cultural studies of universals and cultural differences in the signal value of facial expressions.

Both an imagery task in which past emotional experiences are relived and a directed facial action task in which subjects are requested to produce specific patterns of facial muscular actions will be used to explore (1) emotion-specific ANS activity and (2) how voluntary facial movements can produce ANS activity. One set of studies will determine whether ANS activity generated by voluntary facial movement requires visual feedback on one's own or antoher's expressions and the cognitive labels that may be instigated, or proprioceptive feedback from the facial muscles. Another set of experiments will determine whether emotion-specific ANS activity is preserved when emotion intensity varies, two emotions occur simultaneously, and two emotions occur sequentially. Evidence of emotion-specific ANS patterning would be relevant to emotion-related illnesses, including the affect-flooded psychological disorders such as depression, and the physical disorders that involve ANS organ systems and have been linked to emotion (e.g., hypertension--anger). The findings could be useful in diagnosing such disorders, understanding their etiology and maintenance, and tracking therapeutic improvement.

We will repeat experiments we have performed in the United States in a non-Western culture in Indonesia, among the Minangkabau in Sumatra. Work in another culture is necessary, we believe, to help settle the debate about how to interpret our two recent findings on the expression and physiology of emotion: (1) there are different patterns of autonomic nervous system (ANS) activity for anger, fear, disgust and sadness; (2) voluntary performance of the facial expressions that universally signal these emotions generates emotion-specific ANS activity. We expect to replicate our findings, thereby supporting our theory which proposes that these phenomena are biologically based. We also expect to uncover specific differences which distinguish the Minangkabau from the American subjects, thereby beginning to delineate cultural influences on these biological components in emotion.

Objectives: (1) to clarify and extend our finding of differences in autonomic nervous system (ANS) that distinguish among four negative emotions (fear, anger, disgust and sadness) and between these negative emotions and the emotions of happiness and surprise. (2) to determine whether such patterns of ANS activity are emotion-specific regardless of how the emotion was brought forth, or vary with the emotion elicitor. (3) to learn how the intensity of an emotion and blends of emotion are registered in ANS activity. (4) to show that it is possible to have differentiated emotional experience (in ANS and subjective report) with no facial activity, by examining patients with Mobius syndrome. (5) to illuminate what mechanisms are involved in voluntary facial actions generating differentiated ANS activity and subjective emotional experience. (6) to examine ANS activity where two emotions are called forth simultaneously in a blend. (7) to examine individual differences in expressive and autonomic activity. Health relevance: Study of emotion blends may show that it is when emotions occur simultaneously that people become confused about how they feel. Learning what happens to physiology when emotions with quite different physiological patterns occur simultaneously may be relevant to understanding when emotions can have adverse health consequences. Determining whether voluntary efforts to generate emotion can terminate or attenuate involuntary emotions will begin to suggest whether our techniques for self-generation of emotion have any possible therapeutic value.

The objective of this project is to use an established Method of measuring facial activity to determine if negative emotional expressions are related to the occurrence of Transient Myocardial Ischemia (TMI). TMI has been found to be prevelant among patients with coronary atherosclerosis, and a significant predictor of fatal and non-fatal cardiac events. The facial measures have been used previously to differentiate individuals classified as Type A or Type B, and to identify expressions of specific negative emotions found to coincide with changes in cardiac activity. This work will be done in the context of an already funded study where it is possible to examine the relationships between facial expressions of emotion and behavioral interventions designed to reduce the frequency, duration and intensity of TMI episodes. Facial behavior from audio/video records being collected during standard structured interviews will be measured using Ekman and Friesen's Facial Action Coding System. There are three aims: Aim 1: To examine the relationship between expressive behavior and TMI during mental stress tasks. Our purpose is to specify what type of mental stress -- which emotional reaction to stress as evident in the face -- is related to the occurrence of ischemic events. Aim 2: To determine whether the efficacy of the intervention programs designed to reduce TMI is related to the expressive behavior shown prior to intervention, and whether the different interventions influence different aspects of emotional behavior. Aim 3: To explore the relationship between our facial expression measures and the other psychometric measures being collected, in particular to determine how the facial measures of hostility, anger, disgust, and contempt relate to scores on the Cook-Medley Hostility Inventory.

This workshop is and NSF-initiated planning workshop to determine research problems in the area of facial expression understanding that deserve future funding. The main goals are (a) to identify the most important areas of research on how to extract information from facial activity relevant to a person s emotional, cognitive and physical state; (b) to enhance communication between man and machine through the development of emerging computer vision capture techniques for facial processing and categorization of facial expression; (c) to consider how we can facilitate the training of new investigators in the relevant fields. The workshop will produce a set of recommendations to the investigative community and to NSF to help guide funding opportunities. There will be publications in standard journals of a report.The workshop participants represent a balance of researchers in the areas of facial expression understanding, computer vision and human computer interaction. Participants include researchers with NSF grants and researchers in government and industrial laboratories. The workshop will consist of plenary sessions and breakout groups focused on several research areas. The steering committee includes Dr. Paul Ekman of the University of California at San Francisco, Dr. Thomas Huang of the University of Illinois at Champaign-Urbana and Dr. Terrence Sejnowski of the Salk Institute.