Bernhard Riecke - Current projects

Spatial Updating in real and virtual environments

Impressions from the spatial updating experiments

Motion Platform and projection screen	3D model of the Motion-Lab
Pointer and blinders ('HMD simulation glasses')	Head Mounted Display, position tracker and headphones

The experiments were performed using the motion platform and equipment depicted above. 

Introduction

How do we find our way around in everyday life? In real world situations, it typically takes a considerable amount of time to get completely lost. In most Virtual Reality (VR) applications, however, users are quickly lost after only a few simulated turns. This happens even though many recent VR applications are already quite compelling and look convincing at first glance. So what is missing in those simulated spaces? Why is spatial orientation there not as easy as in the real world? In other words, what sensory information is essential for accurate, effortless, and robust spatial orientation? How are the different information sources combined and processed? In this thesis, these and related questions were approached by performing a series of spatial orientation experiments in various VR setups as well as in the real world. Modeling of the underlying spatial orientation processes finally led to a comprehensive framework based on logical propositions, which was applied to both our experiments and selected experiments from the literature. Using VR allowed us to disentangle the different information sources, sensory modalities, as well as possible spatial orientation processes and strategies. It further offered the precise control, repeatability, and flexibility of stimuli and experimental conditions, which is difficult to achieve in real world experiments.

Spatial orientation experiments motivating the spatial updating research

A first series of experiments (part II of my thesis) investigated the usability of purely visual cues, with particular focus on optic flow, for basic navigation and spatial orientation tasks. According to the prevailing opinion in the literature, those cues should not be sufficient: Proprioceptive and especially vestibular cues are supposedly prerequisites even for simple navigation and spatial orientation tasks if they involve rotations of the observer. Furthermore, visual cues alone are often considered insufficient for good spatial orientation, especially when useful reference points (landmarks) are missing. To test this notion, we conducted a set of experiments in virtual environments where only visual cues were provided. Participants had to execute simulated turns, reproduce distances or perform triangle completion tasks. Most experiments were performed in a simulated 3D field of blobs, thus restricting navigation strategies to path integration based on optic flow. For our experimental setup (half-cylindrical 180 x50 projection screen), optic flow information alone proved to be sufficient for untrained participants to perform turns and reproduce distances with negligible systematic errors, irrespective of movement velocity. Path integration by optic flow was sufficient for homing by triangle completion, but homing distances were biased towards the mean response. Additional landmarks that were only temporarily available did not improve homing performance. Navigation by stable, reliable landmarks, however, led to almost perfect homing performance. Compared to similar experiments using virtual environments (Kearns et al., 2002; Péruch et al., 1997) or blind locomotion (Loomis et al., 1993; Klatzky et al., 1990), we did not find any distance undershoot or strong regression towards mean turn responses. Using a Virtual Reality setup with a half-cylindrical 180 projection screen allowed us to demonstrate that visual path integration without any vestibular or kinesthetic cues can indeed be sufficient for elementary navigation tasks like rotations, translations, and homing via triangle completion.

Spatial updating experiments

Nevertheless, we did observe some systematic errors that could not be convincingly explained by the literature or by the experiments themselves. A detailed analysis of participants' behavior suggested that general cognitive abilities and mental spatial reasoning abilities in particular might have been the determining factor. Positive correlations between navigation performance and mental spatial abilities test scores corroborated this hypothesis. In comparable real world situations, however, no higher cognitive processes seem to be needed (even animals as simple as ants can perform comparable homing tasks). Instead, we seem to know automatically and effortlessly where relevant objects in our immediate surround are when moving about without having to think much about it. Hence, we hypothesized that this "automatic spatial updating" of self-to-surround relations during egomotion was not functioning properly in our and many other VR studies. So what was missing in the simulations? The literature suggests that vestibular cues from physical motions are indispensable for automatic spatial updating. Furthermore, visual cues alone should be insufficient, especially when ego-rotations are involved.

To test these hypotheses, we established a rapid pointing paradigm and performed a second series of experiments that investigated the influence and interaction of visual and vestibular stimulus parameters for spatial updating in real and virtual environments (part III of my thesis). After real and/or visually simulated ego-turns, participants were asked to accurately and quickly point towards different previously-learned target objects that were currently not visible. The rapid egocentric response ensured that participants could not solve the task cognitively.

Unpredicted by the literature, visual cues alone proved sufficient for excellent automatic spatial updating performance even without any vestibular motion cues. Furthermore, participants were virtually unable to ignore or suppress the visual stimulus even when explicitly asked to do so. This indicates that the visual cues alone were even sufficient to evoke reflex-like "obligatory spatial updating". Comparing performance in the real environment and a photorealistic virtual replica revealed similar performance as long as the field of view was the same. That is, a simulated view onto a consistent, landmark-rich environment was as powerful in turning our mental spatial representation (even against our own conscious will) as a corresponding view onto the real world. This highlights the power and flexibility of using highly photorealistic virtual environments for investigating human spatial orientation and spatial cognition. It furthermore validates our VR-based experimental paradigm, and suggests the transferability of results obtained in this VR setup to comparable real world tasks. From a number of additional parameters investigated, only the field of view and the availability of landmarks had a consistent influence on spatial updating performance. Unexpectedly, motion parameters did not show any clear influence, which might be interpreted as a dominant influence of static visual (display) information over dynamic (motion) information.

Theoretical modelling of spatial orientation processes

Modeling spatial orientation processes in a comprehensive framework based on logical propositions (part IV of my thesis) allowed for a deeper understanding of the underlying mechanisms in both our experiments and experiments from the literature. Furthermore, the logical structure of the framework suggests novel ways of quantifying spatial updating and "spatial presence" (which can be seen as the consistent feeling of being in a specific spatial context, and intuitively knowing where one is with respect to the immediate surround). In particular, it allows the disambiguation between two complementary types of automatic spatial updating found in our experiments: On the one hand, the well-known "continuous spatial updating" induced by continuous movement information. On the other hand, a novel type of discontinuous, teleport-like "instantaneous spatial updating" that allowed participants to quickly adopt the reference frame of a new location without any explicit motion cues, just by presenting a novel view from a different viewpoint. Last but not least, the framework suggested novel experiments and experimental paradigms, was used to generate new hypotheses and testable predictions, and already stimulated the scientific discussion in the presence research community.

In addition to assessing spatial cognition, the logical framework proved helpful in tackling the human-computer-interface issue. Several critical simulation and display parameters required for quick and effortless spatial orientation were pinpointed: First of all, any application that does not enable automatic spatial updating is bound to decrease quick and effortless spatial orientation performance and hence unnecessarily increase cognitive load. In addition, most current VR-displays do not allow for effective ego-motion simulation and/or tend to produce rather large artifacts in ego-motion perception. This is especially true for head-mounted displays. Hence, the importance of designing effective VR displays can hardly be overestimated. Furthermore, the simulated objects should be salient enough, non-repetitive, and constitute one coherent scene that can be updated as a whole. Maybe most critical, the physical reference frame of the VR display and the surround should become "transparent", i.e, vanish perceptually or at least be clearly dominated by the simulated (i.e., intended) spatial reference frame. Failure to do so will lead immersion and spatial presence to decrease, resulting in impaired spatial updating, which in turn prevents quick and effortless spatial orientation. Thus, by gaining a deeper understanding of how the different sensory cues are integrated in the human brain (spatial cognition aspect) we also approach human factors issues. This highlights the truly interdisciplinary nature of this research area and opens up potential applications.

Bibliography

The spatial updating project page lists more literature on the topic. See also my publication list or the publication list of the POEMS project for questions, comments, or further information, feel free to contact me.