I investigate human multimodal communication. In particular, I am interested in how communicative intent and the pragmatics of conversation influence the interplay between bodily signals and speech, as well as how social cues and context may modulate the perception and communicativeness of co-speech gestures and other bodily signals and their integration with speech.
My recent projects use paradigms that embed multimodal language production and comprehension in more situated contexts with a special focus on the interplay between different communicative modalities, including speech, gesture, facial signals and eye gaze. In addition to investigating how these modalities interact in the process of semantic comprehension, I explore the role of the visual modalities in pragmatic communication, including turn-taking and social action formation and recognition.
I have recently been awarded an ERC Consolidator grant. This new project entitled “Communication in Action (CoAct): Towards a model of contextualised action and language processing” will commence in September 2018. The project investigates the role of visual communicative signals in situated, face-to-face language production and comprehension using a variety of novel techniques (including Virtual Reality and motion capture).
I am also interested in the cognitive representations underlying the production of gestures, the development of gestural communication (in childhood and ageing, as well as in evolutionary terms), and the use of gesture in the context of impaired (e.g. Parkinson’s) and verbally challenging (e.g. pain sensation) communication.
Neurobiology of Language Department, MPI, Nijmegen (NL)
Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen(NL)
School of Psychological Sciences, University of Manchester (UK)