|
|
 |
|
|
|
Marc Ernst, Dr.
Group Leader |
|
|
|
|
|
|
|
MPI for Biological Cybernetics
NWG Ernst
Spemannstraße 41
72076 Tübingen
|
| Telephone: | |
+49-7071-601-644 |
| Telefax: | |
+49-7071-601-616 |
| Room: | |
02 A 01 |
| e-mail: | |
marc.ernst@tuebingen.mpg.de |
|
|
|
|
|
|
|
| |
Multisensory Perception and Action Lab
My main research interest is the study of human perception with a focus on multimodal integration and visual-haptic interaction. For this, I use quantitative computational/statistical models together with psychophysical and neuropsychological (e.g. fMRI) methods. At present, I am leading the Independent Junior Research Group on Multisensory Perception and Action at the Max-Planck-Institute for Biological Cybernetics and being the local principle investigator for the European Projects TOUCH-HapSys, ImmerSence and CyberWalk.
Scientific area of interest: For perceiving the environment our brain uses multiple sources of sensory information derived from several different modalities, including vision, touch and audition. The question how information derived from these different sensory modalities converges in the brain in order to form a coherent and robust percept is central to understanding the process of perception. Combination and integration of multiple sources of sensory information is the key to robust perception, because no information processing system, neither technical nor biological, is powerful enough to "perceive and act" accurately under all possible conditions. (For a recent review on that topic see: Merging the senses into a robust percept, M.O. Ernst & H.H.Bülthoff, TICS, 2004)
Recent milestones in my work towards understanding the integration of sensory information: A desirable goal for the perceptual system is to maximize the reliability of the various perceptual estimates. From a statistical viewpoint the optimal strategy for achieving this goal is to integrate all available sensory information. This may be done using a “maximum-likelihood-estimation” (MLE) strategy. Then the combined percept will be a weighted average across the individual estimates with weights that are proportional to their reliabilities (i.e., inversely related to the individual estimates’ variances).
1) In a recent study we could show that humans actually integrate visual and haptic information in such a statistically optimal fashion (Ernst & Banks, Nature, 2002). Others have now demonstrated that this finding is true not only for the integration across vision and touch, but also for the integration of information across and within other modalities, such as audition or vision. This suggests that maximum-likelihood-estimation is an effective and widely used strategy exploited by the perceptual system.
2) By integrating sensory information the brain may or may not loose access to the individual input signals feeding into the integrated percept. The degree to which the original information is still accessible defines the strength of coupling between the integrated signals. Depending on the signals used we found different strengths of coupling between the signals: e.g., we found strong coupling for signals derived from within the visual modality (stereo and texture signals to slant) and weaker coupling for visual and haptic signals to size (Hillis, Ernst, Banks, & Landy, Science, 2002). As suggested by one of our recent learning studies, the strength of coupling, which can be modelled using Bayesian statistics, is likely to depend on the natural statistical co-occurrence between the individual signals (Jäkel & Ernst, in preparation).
3) Important precondition for integrating signals is to know which signals derived from the different modalities belong together and how reliable these are. Recently we could show that touch can teach the visual modality how to interpret its signals and their reliabilities. More specifically, we could show that by exploiting touch we can alter visual perception of slant (Ernst, Banks & Bülthoff, Nature Neuroscience, 2000). This finding contributes to a very old debate postulating that we only perceive the world because of our interactions with the environment. Similarly, in one of our latest studies we could show that experience can change the so-called “light-from-above” prior. Prior knowledge is essential for the interpretation of sensory signals during perception. Consequently, with the prior change we introduced a change in the perception of shape (Adams, Graf & Ernst, Nature Neuroscience, in press).
4) Integration is only sensible if the information sources carry redundant information. If the information sources are complementary, different combination strategies have to be exploited. Complementation of cross-modal information was demonstrated in a recent study investigating visual-haptic shape perception (Newell, Ernst, Tjan, & Bülthoff, Psychological Science, 2001).
Scientific Umbrella: In order to study human perception we regularly exploit Human-Computer-Interaction devices (HCI) and virtual reality techniques (VR). Studying human perception and developing or improving human-computer interfaces can be a positive feedback loop. Following are three examples from my current work:
1) TOUCH-HapSys is a European Project in the 5th Framework IST Program that on the one side focuses on understanding the psychophysical basis of human haptic perception and on the other side develops a new generation of high-fidelity haptic display technology. From the six partners in the consortium our part focuses on a better understanding of touch perception. One extension to this project we are currently discussing is an fMRI compatible haptic interface. Such a device would open very exciting possibilities to conduct visual-haptic integration experiments in the fMRI scanner. This would also pave the way to neurophysiological plausible integration models.
2) 3D Visualization with correct focus cues: Conventional 3D digital displays present stimuli at one focal distance and this could result in distortions of perceived depth at simulated distances other than the actual distance to the display. This is because there are incorrect focus cues present in the display (e.g., accommodation, vergence and blur). Within this project, which is originated by Prof. Martin Banks (UC Berkeley), we are assessing the perceptual basis for this effect. Based on these results a technical solution will then be developed.
3) CyberWalk is a European project in the planning stage (pre-proposal accepted) that will develop new technology for enabling omni-directional locomotion in virtual environments. For this, together with our partners we will develop a walking environment based on an omni-directional 2D-treadmill. Such an environment will allow us to study human multimodal perception for locomotion in naturalistic environments.
|
|
|
|
|
|
|
|
