Selective Enhancement of Object Representations through Multisensory Integration.

Details

Serval ID
serval:BIB_CFD173010BA6
Type
Article: article from journal or magazin.
Collection
Publications
Institution
Title
Selective Enhancement of Object Representations through Multisensory Integration.
Journal
The Journal of neuroscience
Author(s)
Tovar D.A., Murray M.M., Wallace M.T.
ISSN
1529-2401 (Electronic)
ISSN-L
0270-6474
Publication state
Published
Issued date
15/07/2020
Peer-reviewed
Oui
Volume
40
Number
29
Pages
5604-5615
Language
english
Notes
Publication types: Journal Article ; Research Support, N.I.H., Extramural ; Research Support, Non-U.S. Gov't
Publication Status: ppublish
Abstract
Objects are the fundamental building blocks of how we create a representation of the external world. One major distinction among objects is between those that are animate versus those that are inanimate. In addition, many objects are specified by more than a single sense, yet the nature by which multisensory objects are represented by the brain remains poorly understood. Using representational similarity analysis of male and female human EEG signals, we show enhanced encoding of audiovisual objects when compared with their corresponding visual and auditory objects. Surprisingly, we discovered that the often-found processing advantages for animate objects were not evident under multisensory conditions. This was due to a greater neural enhancement of inanimate objects-which are more weakly encoded under unisensory conditions. Further analysis showed that the selective enhancement of inanimate audiovisual objects corresponded with an increase in shared representations across brain areas, suggesting that the enhancement was mediated by multisensory integration. Moreover, a distance-to-bound analysis provided critical links between neural findings and behavior. Improvements in neural decoding at the individual exemplar level for audiovisual inanimate objects predicted reaction time differences between multisensory and unisensory presentations during a Go/No-Go animate categorization task. Links between neural activity and behavioral measures were most evident at intervals of 100-200 ms and 350-500 ms after stimulus presentation, corresponding to time periods associated with sensory evidence accumulation and decision-making, respectively. Collectively, these findings provide key insights into a fundamental process the brain uses to maximize the information it captures across sensory systems to perform object recognition.SIGNIFICANCE STATEMENT Our world is filled with ever-changing sensory information that we are able to seamlessly transform into a coherent and meaningful perceptual experience. We accomplish this feat by combining different stimulus features into objects. However, despite the fact that these features span multiple senses, little is known about how the brain combines the various forms of sensory information into object representations. Here, we used EEG and machine learning to study how the brain processes auditory, visual, and audiovisual objects. Surprisingly, we found that nonliving (i.e., inanimate) objects, which are more difficult to process with one sense alone, benefited the most from engaging multiple senses.
Keywords
Acoustic Stimulation, Adult, Auditory Perception/physiology, Brain/physiology, Electroencephalography, Female, Humans, Male, Photic Stimulation, Recognition, Psychology/physiology, Visual Perception/physiology, Young Adult, EEG, decoding, multisensory integration, object recognition, representational similarity analysis
Pubmed
Web of science
Open Access
Yes
Funding(s)
Swiss National Science Foundation / Projects / 320030_169206
Create date
10/06/2020 20:33
Last modification date
20/01/2021 7:26
Usage data