The neural correlates of communicative processes in patients with schizophrenia and the effects of a specific speech-gesture intervention

Funding: Von Behring-Röntgen-Stiftung

Funding period: 2017 – 2020

Principal investigator: Prof. Dr. Benjamin Staube

Postdoc: Dr. Lydia Riedl

PhD students: Dr. Annika Nonnenmann (Alumna), Momoko Choudhury, Anne Stütterlin, Maxi Haslach

Research assistants: Thomas Hartmann

OSF Homepage of G+: G+ OSF Repository

Description of research projects:

Schizophrenia is one of the most frequent mental disorders with symptoms like hallucinations and ego-disturbances, affecting approximately 1% of the population. In many cases, these so-called positive symptoms can be successfully treated by medication. Moreover, there are severe thought and language impairments manifesting in communicative dysfunctions and therefore in social isolation that cannot be treated medically. One of the underlying communicative dysfunctions in patients with schizophrenia is the inability to integrate gestural information: Patients tend to misinterpret information from two modalities (auditory and visually). Several previous studies have shown reduced skills, especially in interpreting gestures accompanying figurative speech.

With the G+ project, we have established a standardized eight-hour multimodal speech-gesture training (MSG training) program, training gesture skills. We investigate the neural plasticity effects of the training on social-cognitive abilities in patients with schizophrenia. Throughout the MSG training, we use perceptual tasks as well as gesture production to improve the patient’s integration skills. Furthermore, we focus on cognitive aspects of gesture to facilitate social-cognitive functioning in everyday life. We also offer information about gestures and their benefits for communication. With the help of functional Magnetic Resonance Imaging (fMRI) and behavioral measures, we seek to investigate the implications for future therapy, in order to improve treatment methods for patients with schizophrenia.

Research highlights:

  1. Pre-MSG-training, patients show reduced quality of life as compared to controls, as predicted.
  2. Quality of life in patients improves during the training period.
  3. These quality of life improvements correlate with neural activation changes in the middle temporal gyrus for the processing of abstract multimodal content.
  4. Pre-post changes in MSG training performance, self-reported measures and ratings of relatives confirm the MSG-related changes.


  1. Riedl, L., Nagels, A., Sammer, G., Choudhury, M., Nonnenmann, A., Sütterlin, A., Feise, C., Haslach, M., Bitsch, F., & Straube, B. (accepted). Multimodal Speech-Gesture Training in Patients With Schizophrenia Spectrum Disorder: Effects on Quality of Life and Neural Processing. Schizophrenia Research IF: 4.939
  2. Riedl, L., Nagels, A., Sammer, G., Choudhury, M., Nonnenmann, A., Sütterlin, A., … & Straube, B. (2021). Multimodal Speech-gesture Training in Patients with Schizophrenia Spectrum Disorder: Effects on Quality of Life and Neural Processing. PsyArXiv, 6 Sept. 2021. Web. 10.31234/osf.io/a8wn4
  3. Riedl, L., Nagels, A., Sammer, G., & Straube, B. (2020). A multimodal speech-gesture training intervention for patients with schizophrenia and its neural underpinnings–the study protocol of a randomized controlled pilot trial. Frontiers in psychiatry, 11, 110. https://doi.org/10.3389/fpsyt.2020.00110 IF: 4.157