Guest Speaker: Hannah Small (Graduate student at Johns Hopkins University)

Talk Title: An investigation into simultaneous visual and linguistic processing during a naturalistic movie

Abstract: Social cognition depends on integrating information from both vision and language. Prior work studying vision and language separately has yielded substantial insights into how the brain processes each of these signals. In this work, we combine these insights to investigate how the rich social visual and verbal semantic signals that occur simultaneously in natural settings are processed. To study the processing of these simultaneous signals in individual participants, we localize visual social interaction perception and language regions and investigate how these regions process a naturalistic movie (Sherlock) using a voxel-wise encoding model. We find that both visual and language features contribute to predicting fMRI responses in both social interaction and language voxels, but to different extents. Our results suggest that both regions process visual and language information, but perhaps for different purposes.