Title: Putting together the Puzzle of Multisensory Perception: From Illusions to Applications
Abstract: We use all our senses to construct a reliable and robust percept representing the world with which we interact. The view we take in my research group is that in many aspects of behaviour, motor actions (grasping, locomotion, navigation) and multisensory processing (vision, touch, audition) are inseparably linked and therefore they have to be studied in a closed action/perception loop. We believe that human perception and action is tailored to the statistics of the natural environment and when the environment changes our perceptions will follow these changes through the process of adaptation minimizing potential costs during interaction. In neural processing such statistics will be represented in form of probability distributions. We follow Hermann von Helmholtz in our belief that human perception is a problem of inference, for which the sensory data are often not sufficient to uniquely determine the percept. Thus, prior knowledge has to be used to constrain the process of inference from ambiguous sensory signals. A principled way to describe the combination of prior knowledge with sensory data in a probabilistic way is provided by the Bayesian Framework. Therefore, we regularly use this Bayesian Framework to construct “ideal observer” models-models that use the available information in the most optimal way, provided some task and cost function. These models can then be used as a benchmark against which human performance can be tested. We apply these models to demonstrate that many apparent illusions in the perceptual and motor domain are actually instances of optimal behavior. Recently we started to apply these kind of models also to describe brain plasticity in visually deprived children who regained sight. Methodologically, we use quantitative psychophysical and neuropsychological paradigms together with Virtual Reality techniques. Quantitative psychophysical methods are important to best determine the relevant perceptual parameters minimizing uncertainty and unknowns. Virtual Reality is important because it provides us with a tool to precisely control the perceptual situation that are investigated, while at the same time it allows for a degree of interaction, which is necessary for studying the action/perception loop. Often, however, today’s Virtual Reality techniques and Human-Computer Interaction devices are not sufficiently developed to be readily used in the study of human perception and action. Therefore, some of our work concentrates on the development of novel human-machine interfaces able to be applied to neuroscience studies.