He, Y., Lüll, S., Muralikrishnan, R., Straube, B., & Nagels, A. (accepted). Gesture’s body orientation modulates the N400 for visual sentences primed by gestures. Human Brain Mapping
… we carried out an EEG experiment presenting to participants (n=21) videos of frontal and lateral hand gestures of five-seconds (e.g., raising a hand), followed by visually presented sentences that are either congruent or incongruent with the hand gesture (e.g., ‘the mountain is high/low, said John’). All participants underwent a semantic-probe task, judging whether a target word is related or unrelated to the gesture-speech event. EEG results suggest that, during the perception phase of hand-gestures, while both frontal and lateral gestures elicited power decrease in both the alpha (8-12Hz) and the beta (16-24Hz) bands, lateral gestures elicited reduced power decrease in the beta band when compared with frontal gestures. For sentence comprehension, at the critical word whose meaning is congruent/incongruent with the gesture, frontal gestures elicited an N400 effect for gesture-sentence incongruency. More importantly, this incongruency effect was significantly reduced for lateral gestures. The findings suggest that body orientation plays a crucial role in gesture perception, and that its inferred social-communicative intention influences gesture-sentence semantic processing in an interactive manner.
Translational Neuroimaging Marburg 