From Sensation to Conception: Theoretical Perspectives on Multisensory Perception and Cross-Modal Transfer
If a person is trained to recognize or categorize objects or events using one sensory modality, the person can often recognize or categorize those same (or similar) objects and events via a novel modality, an instance of cross-modal transfer of knowledge. How is this accomplished? The Multisensory Hypothesis states that people extract the intrinsic, modality-independent properties of objects and events, and represent these properties in multisensory representations. These representations mediate the transfer of knowledge across modality-specific representations. In this talk, I'll present three studies, using experimental and computational methodologies, of the Multisensory Hypothesis. The first study examines visual-haptic transfer of object shape knowledge, the second study examines visual-auditory transfer of sequence category knowledge, and the final study examines a novel latent variable model of multisensory perception.