Contact

Kathrin Kaulard

 

Picture of Kaulard, Kathrin

Kathrin Kaulard

Position: Guest Scientist  Unit: Alumni Bülthoff

Facial expressions form one of the most important and powerful communication systems of human social interaction. To date, research has mostly focused on the static, emotional aspect of facial expression processing, using only a limited set of “generic” or “universal” expression photographs, such as a happy or sad face. That facial expressions carry communicative aspects beyond emotion and that they change over time in daily life, however, has so far been largely neglected.

My PhD thesis will first create a database for natural facial expressions that allows research to investigate both the emotional and the almost unexplored conversational aspect of facial expressions. Moreover, the database also allows investigating the importance of temporal information for the recognition process. Using this database, I will examine the general categorization of everyday facial expressions and the roles of temporal and semantic information using classic psychophysical methods. Finally, neuroimaging studies are planed to investigate the brain areas that allow such a categorization.

What are the properties underlying similarity judgments of facial expressions?

 

Introduction

In our everyday interaction with the world, facial expressions are frequently used for both expressing emotions (emotional expressions) and conveying intentions (conversational expressions) [1,2]. To date, research has mostly focused on the emotional aspect of expressions, although only very few facial expressions reflect emotional content [3,4]. The perception of emotional facial expressions has often been examined by means of their visual similarity. However, the perceptual and cognitive properties (e.g. physical aspects or action tendencies) driving the similarity judgments facial expressions are largely unknown.

 

Goals

Here we attempt to map perceptual and cognitive properties of facial expressions onto their corresponding visual similarity judgments to examine the features underlying similarity judgments of facial expressions. Furthermore, we are interested in whether this mapping is different for emotional and conversational facial expressions.

 

Methods

We assessed the perceptual and cognitive properties of facial expressions, and visual similarity of facial expressions in two separate experiments. Perceptual and cognitive properties were investigated by using 27 questions addressing the emotional (taken from [5]) and conversational content of expressions using semantic differentials. The visual similarity was determined by obtaining ratings of perceived similarity of sequentially presented expression pairs. Both experiments used the same set of (conversional and emotional) facial expressions videos taken from [6]. We reasoned that if a certain cognitive property is driving visual similarity ratings, it should be a good predictor of the visual similarity rating.

 

Initial results

The mapping of cognitive properties onto visual similarity was done using multiple regression with the semantic-differential ratings as predictors. The best model for emotional expressions explained 75% of the variation in similarity ratings and consisted of the two emotional questions: “How much are the expectations of the person met?” and “How much is the person under control?”. The same model explained significantly less variation for conversational expressions (38%). Using all questions as predictors explained 72% of the variation for conversational expressions.

 

Initial conclusion

This study demonstrates a relationship between cognitive properties of facial expressions and their visual perception and sheds light onto which cognitive properties might underlie visual similarity ratings. Emotional questions regarding self-control and expectations of the actor allow precise prediction of the variation in similarity ratings of emotional expressions. These two properties, however, explain much less of the variation in the similarity ratings of conversational expressions. For these expressions, the underlying cognitive properties when rating the visual similarity seem to be more complex. Our results suggest that different perceptual and cognitive properties underlie similarity judgments about emotional and conversational expressions. This study is part of our research line on the detailed investigation of emotional and conversational properties of facial expressions [7] and was done in collaboration with Stephan de la Rosa, Johannes Schultz and Christian Wallraven.

 

References

1.   Darwin C (1965/1872), The expression of emotion in man and animals, University of Chicago Press, Chicago.

2.   Nusseck M, Cunningham DW, Wallraven C, Bülthoff HH (2008) The contribution of different facial regions to the recognition of conversational expressions, Journal of Vision, 8, 1-23.

3.   Ekman P (1979) About brows: emotional and conversational signals, in von Cranach M, Foppa K, Lepenies W, and Ploog D, editiors, Human ethology: Claims and limits of a new discipline, 169-202, Cambridge University Press, Cambridge.

4.   Reilly J and Seibert L (2009) Language and emotion, in Davidson RJ, Scherer KR, and Goldsmith HH, editors, Handbook of Affective Science, 535-559, Oxford University Press, New York.

5.   Fontaine JRJ, Scherer KR, Roesch EB, Ellsworth PC (2007) The world of emotions is not two-dimensional, Psychological Science, 18(12), 1050-1057.

6.   Kaulard K, Cunningham DW, Bülthoff HH, Wallraven C (2011), The MPI facial expression database – a validated database of emotional and conversational facial expressions, submitted.

7.   Kaulard K, Wallraven C, de la Rosa S, Bülthoff HH (2010), Cognitive categories of emotional and conversational facial expressions are influenced by dynamic information, Perception, 39 (ECVP Abstract Supplement) 157

 

 

Figure 1: Snapshots of each of the 12 facial expression videos. Emotional expressions are given in the upper row and the conversational expressions are shown in the lower row. In the experiments, all expressions were shown by each of the 6 models

 

Snapshots of each of the 12 facial expression videos

 

 

Figure 2: Results of best subset regression for the emotional expressions using the emotional questions (Q) as predictors. Color-coded are the number of predictors for different models: light green suggests a model consisting only of one predictor (Q1) that explains 69% of the variation in similarity ratings; dark green suggests a model consisting of 8 predictors explaining 80% of the variation.

 


since 2008: PhD student supervised by Prof. Dr. Christian Wallraven, Dr. Stephan de la Rosa and Prof. Dr. Heinrich H. Bülthoff on "The visual representation of emotional and conversational facial expressions"

 

2007 - 2008: Diploma thesis at the MPI for Biological Cybernetics on "Visual Perception of dynamic facial expressions - Implementation and validation of a database for conversational facial expressions" supervised by Dr. Christian Wallraven

Preferences: 
References per page: Year: Medium:

  
Show abstracts

Books (1):

Kaulard K: Visual Perception of Emotional and Conversational Facial Expressions, 224, Logos Verlag, Berlin, Germany, (2015). ISBN: 978-3-8325-3969-6, Series: MPI Series in Biological Cybernetics ; 41

Articles (5):

Perdikis D, Volhard J, Müller V, Kaulard K, Brick TR, Wallraven C and Lindenberger U (July-2017) Brain synchronization during perception of facial emotional expressions with natural and unnatural dynamics PLoS ONE 12(7) 1-23.
Kaulard K, Cunningham DW, Bülthoff HH and Wallraven C (March-2012) The MPI Facial Expression Database: A Validated Database of Emotional and Conversational Facial Expressions PLoS One 7(3) 1-18.
Wallraven C, Kaulard K, Kürner C and Pepperell R (April-2008) In the eye of the beholder: The perception of indeterminate art Leonardo 41(2) 116-117.
Tatler BW, Wade NJ and Kaulard K (December-2007) Examining art: dissociating pattern and perceptual influences on oculomotor behaviour Spatial Vision 21(1) 165-184.
Jendrusch G, Bolsinger A, Janda S, Bach M, Kaulard K, Lingelbach B and Heck H (2006) Optimierung der Klassifizierung im Blinden- und Sehbehindertensport BISp-Jahrbuch Forschungsförderung 2005/06 83-88.

Conference papers (3):

Pastra K, Wallraven C, Schultze M, Vatakis A and Kaulard K (May-2010) The POETICON Corpus: Capturing Language Use and Sensorimotor Experience in Everyday Interaction, Seventh International Conference on Language Resources and Evaluation (LREC 2010), ELRA, Paris, France, 3031-3036.
pdf
Wallraven C, Kaulard K, Kürner C, Pepperell R and Bülthoff HH (July-2007) Psychophysics for perception of (in)determinate art, 4th Symposium on Applied Perception in Graphics and Visualization (APGV 2007), ACM Press, New York, NY, USA, 115-122.
pdf
Wallraven C, Kaulard K, Kürner C, Pepperell R and Bülthoff HH (June-2007) In the Eye of the Beholder: Perception of Indeterminate Art In: Computational Aesthetics 2007, , Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging (CAe '07), Eurographics Association, Aire-la-Ville, Switzerland, 121-128.
pdf

Posters (11):

Kaulard K, Schultz JW, Bülthoff H and de la Rosa S (August-2013): How we evaluate what we see - the interplay between the perceptual and conceptual structure of facial expressions, 36th European Conference on Visual Perception (ECVP 2013), Bremen, Germany, Perception, 42(ECVP Abstract Supplement) 192.
Schultz JW, Bülthoff H and Kaulard K (August-2013): Signs of predictive coding in dynamic facial expression processing, 36th European Conference on Visual Perception (ECVP 2013), Bremen, Germany, Perception, 42(ECVP Abstract Supplement) 55.
Kaulard K, Schultz J, Wallraven C, Bülthoff HH and de la Rosa S (September-2012): Inverting natural facial expressions puzzles you, 35th European Conference on Visual Perception, Alghero, Italy, Perception, 41(ECVP Abstract Supplement) 103.
Perdikis D, Müller V, Kaulard K, Wallraven C and Lindenberg U (May-2012): EEG brain dynamics during processing of static and dynamic facial emotional expression, 1st Conference of the European Society for Cognitive and Affective Neuroscience (ESCAN 2012), Marseille, France.
Kaulard K, Fernandez Cruz AL, Bülthoff HH and Schultz J (September-2011): Uncovering the principles that allow a distinction of conversational facial expressions, 11th Annual Meeting of the Vision Sciences Society (VSS 2011), Naples, FL, USA, Journal of Vision, 11(11) 605.
Kaulard K, de la Rosa S, Schultz J, Fernandez Cruz AL, Bülthoff HH and Wallraven C (September-2011): What are the properties underlying similarity judgments of facial expressions?, 34th European Conference on Visual Perception, Toulouse, France, Perception, 40(ECVP Abstract Supplement) 115.
Kaulard K, Wallraven C, Cunningham DW and Bülthoff HH (May-2010): Laying the foundations for an in-depth investigation of the whole space of facial expressions, 10th Annual Meeting of the Vision Sciences Society (VSS 2010), Naples, FL, USA, Journal of Vision, 10(7) 606.
Kaulard K, Wallraven C, Cunningham DW and Bülthoff HH (August-2009): Going beyond universal expressions: investigating the visual perception of dynamic facial expressions, 32nd European Conference on Visual Perception, Regensburg, Germany, Perception, 38(ECVP Abstract Supplement) 83.
Jendrusch G, Janda S, Kaulard K, Bolsinger A, Bach M, Lingelbach B and Platen P (May-2007): Classification for visually impaired athletes: An interim report, Medicine and Science in Sports and Exercise, 39(5 Supplement) 265.
Janda S, Jendrusch G, Platen P, Bolsinger A, Bach M, Kaulard K and Lingelbach B (July-2006): Classification for visually impaired athletes: An interim report, Abstracts of the 11th Annual Congress of the European College of Sport Science (ECSS Lausanne 2006), 11 267-268.
Jendrusch G, Bolsinger A, Janda S, Zrenner E, Bach M, Kaulard K and Lingelbach B (May-2006): Classification for visually impaired: A stocktaking report and solutions for the future, Abstracts of the VISTA 2006 Conference, 2006 20-21.

Theses (2):

Kaulard K: Visual perception of emotional and conversational facial expressions, Eberhard-Karls-Universität Tübingen, (December-2014). PhD thesis
Kaulard K: Visual Perception of dynamic facial expressions: Implementation and validation of a database for conversational facial expressions, Hochschule Aalen, Germany, (February-2008). Diplom thesis

Talks (6):

Schultz J, Kaulard K, Pilz P, Dobs K, Bülthoff I, Fernandez-Cruz A, Brockhaus B, Gardner J and Bülthoff HH (March-27-2017) Abstract Talk: Neural processing of facial motion cues about identity and expression, 59th Conference of Experimental Psychologists (TeaP 2017), Dresden, Germany 32-33.
Schultz J, Fernandez Cruz AL, de la Rosa S, Bülthoff HH and Kaulard K (September-2012) Abstract Talk: How are facial expressions represented in the human brain?, 35th European Conference on Visual Perception, Alghero, Italy, Perception, 41(ECVP Abstract Supplement) 38.
Kaulard K, Wallraven C, de la Rosa S and Bülthoff HH (October-2010) Abstract Talk: Cognitive categories of emotional and conversational facial expressions are influenced by dynamic information, 11th Conference of Junior Neuroscientists of Tübingen (NeNa 2010), Heiligkreuztal, Germany 16.
Kaulard K, Wallraven C, de la Rosa S and Bülthoff HH (August-2010) Abstract Talk: Cognitive categories of emotional and conversational facial expressions are influenced by dynamic information, 33rd European Conference on Visual Perception, Lausanne, Switzerland, Perception, 39(ECVP Abstract Supplement) 157.
Kaulard K, Wallraven C, Cunningham DW and Bülthoff HH (November-2009) Abstract Talk: Laying the foundations for an in-depth investigation of the whole space of facial expressions, 10th Conference of Junior Neuroscientists of Tübingen (NeNa 2009), Ellwangen, Germany 11.
Kaulard K and Wallraven C (January-2009): Visual Perception of dynamic facial expressions, International Trade Fair for Trends in Optics (OPTI 2009), München, Germany.

Export as:
BibTeX, XML, Pubman, Edoc, RTF
Last updated: Monday, 22.05.2017