Meta-communication is an integral part of natural conversations among humans. Speakers add (indirect) cues to the intended meaning of their utterances. One particularly interesting phenomenon of meta-communication is (pragmatic) modification, when humans apply non-verbal signals such as gestures and body language in order to classify the semantic content of their utterances for the listener. By doing this, speakers want to communicate their viewpoints, convictions, knowledge, attitudes, among others. Listeners perceive those signals in addition to the semantics and integrate everything into a coherent message. Which signals are perceived as more meaningful and, therefore, are integrated with more weight than others depends on the articulation accuracy of the speaker and the perception sensitivity of the listener.
This research is primarily dedicated to the under-researched area of (pragmatic) modifying functions in gestures. We (Freigang & Kopp, 2015 and 2016 and in prep.) define a gesture to carry a modification if the meaning of the whole utterance is changed from the pure semantics or from the proposition of the verbally or gesturally signified content. The modification acts upon and carries meaning beyond mere propositional content and is applied analogue to this content. We identified three major functions of modification: a focusing, an epistemic and an attitudinal function. Read more ...
Freigang, F. and Kopp, S. (in prep.). Modal Pragmatic Functions of Gesture - Exploring the Dimensions of Function and Form.
Freigang, F., & Klett, S., & Kopp, S. (in press). Pragmatic multimodality: Effects of nonverbal cues of focus and certainty in a virtual human. The Seventeenth International Conference on Intelligent Virtual Agents (IVA 2017), Lecture Notes in Computer Science. Springer International Publishing. [pdf] [suppl. material]
Freigang, F., & Kopp, S. (2016). This is what's important - using speech and gesture to create focus in multimodal utterance. In D. Traum, W. Swartout, P. Khooshabeh, S. Kopp, S. Scherer & A. Leuski (Eds.), Lecture Notes in Computer Science: Vol. 10011. Intelligent Virtual Agents (pp. 96-109). Springer International Publishing. [pdf] [suppl. material]
Freigang, F., & Kopp, S. (2015). Analysing the Modifying Functions of Gesture in Multimodal Utterances. In G. Ferré & M. Tutton (Eds.)
Menke, P., Freigang, F., Kronenberg, T., Klett, S. & Bergmann, K. (2015). First Steps Towards a Tool Chain for Automatic Processing of Multimodal Corpora.
Freigang, F., Priesters, M. A., Nishio, R. & Bergmann, K. (2014). Sharing Multimodal Data: A Novel Metadata Session Profile for Multimodal Corpora. In J.Olijk (Ed.)
Freigang, F., & Bergmann, K. (2013). Towards Metadata Descriptions for Multimodal Corpora of Natural Communication Data. In J. Edlund, D. Heylen & P. Paggio (Eds.)
Feld, M., Momtazi, S., Freigang, F., Klakow, D., & Müller, C. (2012). Mobile Texting: Can Post-ASR Correction Solve the Issues? An Experimental Study on Gain vs. Costs. In
Also visit my Bielefeld University PUB profile.
An analysis of modal (pragmatic) functions of gesture. Oral presentation at the 15th International Pragmatics Conference (IPrA). Belfast, July 16-21, 2017. [pdf]
How gestures tune meaning of multimodal utterances: Analyses of modifying functions. Poster presentation at the 7th Conference of the International Society for Gesture Studies (ISGS 7). Paris, July 18-22, 2016. [pdf]
Brushing up or aside? - Analyzing the modifying functions of gesture in different multimodal utterance contexts. Oral presentation at the Workshop Mapping Multimodal Dialogue (MaMuD 3), Lille, France, November 19-20, 2015. [pdf]
Beyond propositional content: Modifying functions in gesture. Oral presentation at the Workshop Mapping Multimodal Dialogue (MaMuD 2), Leuven, Belgium, November 21-22, 2014. [pdf]
Your Data at the Center of Attention: A Metadata Session Profile for Multimodal Corpora. Poster presentation at the CLARIN Annual Conference in Soesterberg, The Netherlands, October 23-25, 2014.
Beyond propositional content: Modifying functions in gesture. Oral presentation at the KogWis PhD-Symposium. Tübingen, Germany, September 28, 2014.
The tone we don't hear - Analysing the modifying functions of gestures. Oral presentation at the 6th Conference of the International Society for Gesture Studies (ISGS 6). San Diago, CA, USA, July 8-11, 2014. [pdf]
Exploring the speech-gesture semantic continuum. Oral presentation at the 1st European Symposium on Multimodal Communication. Valletta, Malta, October 17-18, 2013. [pdf]
Sommer term 2014: Neuropsychologische Mechanismen sozialer Interaktion (with Sebastian Kahl).
Sommer term 2016: Introduction to Gesture Studies (with Matthias Priesters).
Are you interested in writing your Bachelor's or Master's thesis with me?
Thesis topics in the area of natural communication and interaction, non-verbal behaviour and gesture studies in particular. Various methods are possible: empirical studies, corpus annotation, motion capture, etc. Contact me with your research idea!
I am holding a three-year scholarship from the Cluster of Excellence Cognitive Interaction Technology 'CITEC' (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG), 2014-2017.
I received a conference stipend: 6th Conference of the International Society for Gesture Studies (ISGS 6), San Diago, CA, USA, July 2014.
|Phone||+49 521 106 12174|
|Room||CITEC 2.041 [map]|
Research Group 'Social Cognitive Systems'
CITEC - Cognitive Interaction Technology