Regarding action and perception we are interested in the perception of one’s own action consequences (SFB 135 A3), the neural processing of actions in a language context (G3, FI_3) as well as the production-perception mechanisms regarding social cognitive functioning (GUI; STR 1146/4-1).
The goal of SFB subproject A3 (PACT) is to investigate the effects of predictive mechanisms on multisensory consequences of one’s own actions. The investigations focus on three different aspects: 1) supramodal vs. unimodal effects of action-sensory feedback matching processing, 2) the neural correlates of supramodal predictive mechanisms and their consequences and 3) the investigation of predictive mechanisms for tool-use actions and the perception of related multisensory consequences. Based on these findings related dysfunctions in patients with schizophrenia and hallucinations as well as ego-disturbances are investigated in a new DFG project (PACT-SZ; STR 1146/9-1).
Regarding the perception and comprehension of others people’s actions, we focus mainly on multisensory integration processes relevant for the integration of speech and gesture. Across a large number of experiments we found that activity in the left posterior temporal lobe is related to integration of concrete speech and gesture information (iconic co-verbal gestures) (Green et al., 2009; Straube, Green, Bromberger, & Kircher, 2011a). In a different set of experiments we found that the processing of abstract speech and gesture information (metaphoric co-verbal gestures) leads to additional activity in the left inferior frontal gyrus (Kircher et al., 2009; Straube, Green, et al., 2011a).
In patients with schizophrenia we found that the area in the left posterior superior/medial temporal gyrus was activated to the same extent as in healthy control subjects when iconic co-verbal gestures were contrasted to unimodal control conditions (speech-alone, gesture-alone) (Straube, Green, Sass, Kirner-Veselinovic, & Kircher, 2013). However, for the integration of metaphoric gestures we found aberrant processing in the patient group. For this type of co-verbal gestures it is important to build an abstract relation between concrete visual and abstract verbal information. For building this relation, additional online integration or unification processes in distantly located regions of the brain seem to be relevant (Kircher et al., 2009; Straube, Green, Bromberger, & Kircher, 2011b; Straube et al., 2013) and are likely to be disturbed in patients.
Thus, finally we could show that patients demonstrate a specific functional dysconnectivity of the left superior temporal sulcus and the inferior frontal gyrus bilaterally during the processing of metaphoric gestures (Straube, Green, Sass, & Kircher, 2013). This dysconnectivity might be the basis of dysfunctional integration of gesture information into an abstract sentence context and thus an underlying reason for interpersonal communication problems in patients. In a collaboration project with Prof. Sammer, Giessen, founded by the von-Behring-Röntgen-Stiftung (59-0002), we investigated multisensory integration processes for speech and gesture are using combined EEG and fMRI measurements.
In addition to basic integration processes, we are interested comprehension as well as social evaluation processes of multimodal semantic information. In DFG project (KI 588/6-1; PI: Kircher & Straube) we focused on the comprehension, recognition and interpretation of verbal and non-verbal information in patients with schizophrenia. In a starting grant funded by the Röhn-Klinikum AG (FI_3) we extend these investigations to patients with depression and bipolar disorder. First promising results suggest similarities and differences between groups. A future goal is to investigate rather syndrome specific dysfunctions in social communicative dysfunctions across clinical groups.
Translational Neuroimaging Marburg 