John J. Rieser - US grants

When people move from place to place they often need to keep up to date on the resulting changes in their perspectives, that is, on the distances and directions relating them to features of the surroundings. Adults can keep up to date on such changes without spotting them visually when, for example, the features are occluded from view or the place is dark or the observer is blind. The present experiments are about the precision of infants' and young children's perceptions of self-movement and their abilities to act on them when searching for objects in the dark and in other situations where they are occluded from view. The experiments are focused on the period in life when children make the transition from infancy to early childhood, and on the role played by locomotion and perception in that transition. We devised a behavioral test and adapted psychophysical methods to assess individual levels of sensitivity to threshold changes in self- to-object directions for 1 to 5 year olds after they move to new points of observation. The proposal is to estimate individual children's threshold levels of sensitivity as a function of variations in the self-movements and the environmental reference information in a series of cross sectional experiments, and as a function of experimentally controlled variations in the children's locomotor experiences in a 20 month long longitudinal observation. Locomotor search games are used to estimate children's sensitivity. The self-movement variables are the movement's geometry (rotations, translations), distance, and complexity. The reference information variables are the presence or absence of visual information, auditory information, and leg information from walking versus riding a trolly. The organismic variables include variations in walking experience induced through the random assignment of children to groups using mechanical walkers and not using them in their homes.

This is a standard award to one of three institutions collaborating as partners on Thompson's project (0121084). No current system allows a person to naturally walk through a large-scale virtual environment. The availability of such a locomotion interface would have impacts on a broad range of applications, including education and training, design and prototyping, physical fitness, and rehabilitation; for some of these applications natural walking provides a level of realism not obtainable if movement through the simulated world is controlled by devices such as a joystick, while for others realistic walking is a fundamental requirement. Prototypes have been built for a variety of computer-controlled devices on which a person can walk, but there has been little investigation of the utility of such devices as interfaces to a virtual world and almost no study at all of the interactions of visual and biomechanical perceptual cues in such devices. This project addresses key open questions, the answers to which are needed if locomotion interfaces are to offer effective interaction between users and computer simulations. An effective locomotion interface must provide users with accurate visual and biomechanical sensations of walking; thus, a key objective of this work is to determine how to synergistically combine visual information generated by computer graphics with biomechanical information generated by devices that simulate walking on real surfaces. Thompson and his collaborators will investigates methods that allow more accurate walking in a locomotion interface while accurately conveying a sense of the spaces being walked through. Specific issues to be considered include how to facilitate the perception of speed and distance traveled, how to provide a compelling sense of turning when actual walking along a curved path is not possible, how to give a user the sense that he/she is walking over a sloped surface, and more generally how to give a user a clear sense of the scale and structure of the spaces being walked through. The team's findings on these issues will be relevant across the spectrum of possible approaches to locomotion interfaces.

DESCRIPTION (provided by applicant): This workshop's aim is to integrate recent findings about the spatial functioning of persons with severely impaired vision from the brain sciences and cognitive sciences, and to consider the implications for education and rehabilitative engineering. Central findings from brain sciences are focused on animal models and human imaging studies of plasticity and development that show the occipital cortex is recruited in tactile and auditory processing to a greater degree for people with early-onset blindness than for blindfolded-sighted people. Findings from cognitive sciences are focused on sensory substitution, individual differences, and developmental differences in spatial functioning. There has been little effort cutting across these areas of research, nor to explore how the findings may apply to learning, education, and rehabilitation of people with severe visual impairment. The proposal is to support a three day scientific workshop at Vanderbilt University. The participants would be 22 established investigators and 14 junior investigators from the brain sciences, cognitive sciences, rehabilitative engineering, and rehabilitation to present recent findings, analyze issues that cut across the different disciplines, propose applications to rehabilitation practice, and identify research needed to implement the applications.

This interdisciplinary project investigates human cognition of spaces to improve virtual environments, both from a user and an author's perspective. The objectives are to (1) improve virtual environments so that better learning can occur in them, and (2) develop authoring methods for virtual environments informed by the cognitive demands that people have when learning spaces. This research project should advance the design and authoring of virtual environments by leveraging human cognitive capabilities. The programs seeks to develop a system to increase the user's sense of presence and sensitivity to the environmental scale of virtual environments. It further seeks to develop locomotion interfaces to assist exploring large virtual environments from within small physical ones. A goal is to employ human-centered representations for locomotion in virtual environments and to develop methods for skill acquisition in virtual environments. This research proposal advances the scientific understanding of human cognition and learning as well. The research proposes studies that will be informative about the broad role that environmental geometry and self-representation play in perception, orientation, and navigation, while controlling factors that are extremely difficult, if not impossible, to control in the real world. A rigorous evaluation program for all components of the project is planned.

The importance of this proposal is that virtual environments provide people with opportunities to experience places and situations remote from their actual physical surroundings. Virtual environments allow the simulation of real-world events in a controllable and re-usable environment. They potentially allow people to learn about an environment which, for reasons of time, distance, expense, and safety, would not otherwise be available. Virtual environments could have a huge impact in education, entertainment, medicine, architecture, and training, but they are not widely used because of their expense and delicacy. The research program in this proposal should significantly improve the quality of learning in virtual environments, to reduce the time and cost of authoring virtual environments, and to overcome likely impediments to their widespread use. Moreover, this proposal builds a scientific program to develop a better understanding of the cognitive capabilities of humans in immersive virtual environments, and does so in a way that will inform the design process for such environments and our understanding of how humans reason about space.

Proposal #: CNS 08-21640
PI(s): Bodenheimer, Robert E.
Adams, Julie A.; McNamara, Timothy P.; Rieser, John J.; Sarkar, Nilanjan
Institution: Vanderbilt University
Nashville, TN 37235-7749
Title: MRI/Acq.: Instruments for Interaction, Learning, and Perception in Virtual Environments
Project Proposed:
This project, acquiring a high-fidelity instrument designed to facilitate and assess perception, interaction, and learning in immersive environments, pursues an ambitious research agenda dealing with people, their interactions with virtual environments, and the design factors underlying successful environments. The work aims to build a program to develop a better understanding of the cognitive capabilities of humans in immersive virtual environments, to inform the design process of such environments and to understand how humans reason about space. The instrument will be shared among diverse and interdisciplinary groups collaborating in the area of virtual environments, including Computer Science and Engineering (graphics, animation, artificial intelligence, human factors, robotic, etc.) and the Psychological Science (cognitive psychology, child development, rehabilitation engineering, brain sciences, etc.) The component parts of the instrument (comprising optical motion capture equipment, a head-mounted display with binocular eye-tracking, and high-performance wireless data gloves) allow the measurement, tracking, rendering, and animation of subjects in virtual environments (from their overall position, to their posture, to the actions of their hands and fingers) coupled with the measurement of their gaze. The project ranges from low-level research in how people experience virtual environments to user evaluations involving high-level interface and simulation design. Children with autism will also be studied.
Broader Impacts: This project improves the quality of learning in virtual environments, reducing the time and cost of authoring and overcoming likely impediments to their widespread use. The instrument enables courses in robotics currently infeasible with real robots and provides experience for students. The work builds a scientific program to develop a better understanding of the cognitive capabilities of humans in immersive virtual environments and may be applied to understanding the development of children?s abilities to reason about space and to coordinate perceptual-motor skills as they develop. Moreover, it may help to treat autism spectral disorder.

Technology that can create compelling immersive virtual environments is now available on the general market. However, this technology has limitations. Important frontiers for virtual environments need high quality tracking equipment. One of these frontiers is the ability to build characters that move accurately in a virtual environment. A second frontier is the ability to explore large virtual environments using methods that seem natural. Our goal is to tailor these methods to the individual user. This research proposal will equip a lab with instrumentation that will make fundamental advances on these two problems. It will also train graduate students and provide research opportunities for a number of undergraduates.

This research will equip a laboratory with a high quality motion capture system that will allow the pursuit of novel scientific questions involving the perceptual fidelity of virtual environments, examine theoretical questions involving users and their relationship to their self-avatars, and determine how individual differences in users can be effectively utilized to provide better locomotion and navigation in virtual worlds. In particular, this equipment will enable research in how to design high fidelity virtual environments, and enable the understanding of the components of fidelity that facilitate learning and transfer of training, for which self-avatars are a critical component. Likewise, the equipment will enable significant progress in locomotion methods for improved navigation and wayfinding in large virtual environments by allowing examination of how spatial information sources are used by individuals as they move through virtual worlds.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.