Robert E. Bodenheimer - US grants

The overall objectives of this project are to (1) create more effective pedagogical agents by incorporating novel methods of figure animation into them, and (2) rigorously assess the performance of these agents in K-12 classrooms. The research is divided into two components. The first component will develop and integrate a system for stylistically variable low-level motion specification, both full-bodied and facial, into the high-level cognitive components of a pedagogical agent. The particular pedagogical agent that will serve as the research infrastructure is called a ``teachable agent.'' The second component of the research will deploy the system in K-12 classrooms as each stage is developed, and conduct controlled multi-method studies to assess the influence of the animated agent on learning-related outcomes. The significance of the proposed work is its potential to improve the education of children, particularly in the domain of science. Additionally, controlled studies on the value of animated pedagogical agents in the area of K-12 education are rare. These studies should further the scientific understanding of the components
of compelling animation, and result in a broader understanding of the proper role of graphics and animation in learning systems. An interdisciplinary course in computer science and education is being co-developed for the web and will be co-taught with faculty from the field of education. Finally, by introducing education as an important application of graphics and animation, the plan is to broaden the appeal of computer science to those who are traditionally under-represented.

This interdisciplinary project investigates human cognition of spaces to improve virtual environments, both from a user and an author's perspective. The objectives are to (1) improve virtual environments so that better learning can occur in them, and (2) develop authoring methods for virtual environments informed by the cognitive demands that people have when learning spaces. This research project should advance the design and authoring of virtual environments by leveraging human cognitive capabilities. The programs seeks to develop a system to increase the user's sense of presence and sensitivity to the environmental scale of virtual environments. It further seeks to develop locomotion interfaces to assist exploring large virtual environments from within small physical ones. A goal is to employ human-centered representations for locomotion in virtual environments and to develop methods for skill acquisition in virtual environments. This research proposal advances the scientific understanding of human cognition and learning as well. The research proposes studies that will be informative about the broad role that environmental geometry and self-representation play in perception, orientation, and navigation, while controlling factors that are extremely difficult, if not impossible, to control in the real world. A rigorous evaluation program for all components of the project is planned.

The importance of this proposal is that virtual environments provide people with opportunities to experience places and situations remote from their actual physical surroundings. Virtual environments allow the simulation of real-world events in a controllable and re-usable environment. They potentially allow people to learn about an environment which, for reasons of time, distance, expense, and safety, would not otherwise be available. Virtual environments could have a huge impact in education, entertainment, medicine, architecture, and training, but they are not widely used because of their expense and delicacy. The research program in this proposal should significantly improve the quality of learning in virtual environments, to reduce the time and cost of authoring virtual environments, and to overcome likely impediments to their widespread use. Moreover, this proposal builds a scientific program to develop a better understanding of the cognitive capabilities of humans in immersive virtual environments, and does so in a way that will inform the design process for such environments and our understanding of how humans reason about space.

Proposal #: CNS 08-21640
PI(s): Bodenheimer, Robert E.
Adams, Julie A.; McNamara, Timothy P.; Rieser, John J.; Sarkar, Nilanjan
Institution: Vanderbilt University
Nashville, TN 37235-7749
Title: MRI/Acq.: Instruments for Interaction, Learning, and Perception in Virtual Environments
Project Proposed:
This project, acquiring a high-fidelity instrument designed to facilitate and assess perception, interaction, and learning in immersive environments, pursues an ambitious research agenda dealing with people, their interactions with virtual environments, and the design factors underlying successful environments. The work aims to build a program to develop a better understanding of the cognitive capabilities of humans in immersive virtual environments, to inform the design process of such environments and to understand how humans reason about space. The instrument will be shared among diverse and interdisciplinary groups collaborating in the area of virtual environments, including Computer Science and Engineering (graphics, animation, artificial intelligence, human factors, robotic, etc.) and the Psychological Science (cognitive psychology, child development, rehabilitation engineering, brain sciences, etc.) The component parts of the instrument (comprising optical motion capture equipment, a head-mounted display with binocular eye-tracking, and high-performance wireless data gloves) allow the measurement, tracking, rendering, and animation of subjects in virtual environments (from their overall position, to their posture, to the actions of their hands and fingers) coupled with the measurement of their gaze. The project ranges from low-level research in how people experience virtual environments to user evaluations involving high-level interface and simulation design. Children with autism will also be studied.
Broader Impacts: This project improves the quality of learning in virtual environments, reducing the time and cost of authoring and overcoming likely impediments to their widespread use. The instrument enables courses in robotics currently infeasible with real robots and provides experience for students. The work builds a scientific program to develop a better understanding of the cognitive capabilities of humans in immersive virtual environments and may be applied to understanding the development of children?s abilities to reason about space and to coordinate perceptual-motor skills as they develop. Moreover, it may help to treat autism spectral disorder.

This program aims to revitalize undergraduate computing education through the development of a computational science minor targeted to undergraduate majors in science and engineering. These majors represent a broad community of learners for whom computation is an increasingly critical tool. Modern scientific and engineering applications of significant complexity require high-performance computing solutions, and scientists and engineers require computational thinking competencies to achieve such solutions.

The project introduces concurrent, parallel, and distributed computing concepts, techniques, and patterns early in the curriculum. The advent of multi-core processors at the commodity level, necessitated by the efforts to prolong Moore's law, have made understanding these topics a critical learning outcome. The overall effect of this project will thus be to teach computational thinking competencies, modern software design methods, high-performance computing, and scientific computing to a broad community of learners sorely in need of them. By renovating the curriculum with non-computer science majors in mind, computer science majors will also benefit significantly because concurrent, parallel, and distributed computational methods will be infused into the curriculum earlier than they are normally encountered. The computer science curriculum will also be revitalized by introducing real-world examples from science and engineering that have computational interest.

The demographics of science and engineering are different enough from traditional computer science that underrepresented groups in computing will receive significant exposure to core ideas of computational thinking and computing. The diffusion of computational techniques throughout a variety of disciplines will also change the way computational thinking and computation are perceived and taught within the core computer science discipline. A rigorous evaluation plan throughout all phases of the project will measure quantitatively the changes in the preparation of undergraduates for scientific computing by the proposed computational sciences minor, leading to a greater likelihood of being adopted or adapted by other institutions. By improving the computational skills of scientists and engineers, at Vanderbilt and elsewhere, the project will achieve the broader impact of improving science education in the United States, making students far better prepared for the work force and advanced graduate training.

The objective of this research is to enable more effective design and use of virtual worlds. The pervasiveness of visually-oriented online and interactive digital media allows people to represent themselves increasingly through surrogates in virtual worlds. These digital personae are called "avatars," and when they closely represent the user, "self-avatars." Self-avatars enable forms of learning, interaction, and skill development that can increase a user's effectiveness in a virtual world. This project will explore how self-avatars play a significant role through three key components of perception and action: the relationship between action and the perception of space and objects, active acquisition of spatial memory, and the planning and execution of actions themselves.

This research will consider three properties of self-avatars themselves, each likely to have an effect across a broad range of situations: (1) the virtual perspective from which the avatar is seen, (2) the nature of the coupling between user size and motion and avatar size and motion, and (3) the naturalness of the interface system by which the user controls the avatar. The work builds on a growing body of knowledge about the role of body ownership in perceptual and cognitive tasks. This framework provides a theory in which to ground the research, a body of empirical knowledge about perception and action in the real world, and established methodologies that can be used for assessing the results of the research. The ability to utilize work from cognitive and perceptual science to solve a problem in computer graphics and user interaction is a major strength of the research.

Virtual environments are important in many domains, including architecture, education, medicine, simulation, training, and visualization. The core impact of this research is to enable self-avatars to enhance user experience in virtual environments, which are a major category of computer simulations. A broad impact of this project is that enhancing the user experience will lead to more capable applications of virtual environments in the aforementioned domains. This research will also have utility in entertainment systems, the dominant environments for avatars. It advances discovery and understanding while training students in cross-disciplinary research methods in an innovative intellectual environment. The interdisciplinary nature of the research and its consequent applications, together with the close integration of two research groups, will aid in bringing new students to computer science, beyond the students traditionally attracted to that field.

The objective of this research is to enable more effective design and use of virtual worlds. Virtual worlds are important in many domains, including architecture, education, medicine, simulation, and training. However, when compared to the real world, virtual worlds are hard to move through effectively, and pose challenges to effective navigation. If virtual worlds are going to be widely deployed - particularly for applications in education, training, and simulation - then these problems must be solved. This work will generate essential discoveries improving the process of wayfinding (orienting and navigating from place to place) and locomoting through immersive virtual worlds. It thus provides a critical and synergistic complement to the recent advent of low-cost commodity-level virtual reality equipment.

This research is multi-disciplinary and employs methods from computer science, cognitive science, and geographical information science in accomplishing these objectives. A transformation of wayfinding and navigation for large immersive virtual worlds can be accomplished by studying locomotion modes in conjunction with the spatial characteristics of virtual worlds and individual differences and abilities of the users of the virtual environments. In this work, virtual worlds are described and analyzed in terms of their connectivity, visual access, and integration using formal measures summarized as space syntax. Likewise, individuals traveling through virtual worlds may navigate and reason about space quite differently, and these differences can be quantified and measured. The goal is to develop locomotion modes that take into account both characteristics described by space syntax and individual attributes of users. Truly effective design and use of virtual worlds depend on an understanding of how an individual's abilities relate to the characteristics of the virtual world and the mechanisms for moving about in them. This interdisciplinary approach examines wayfinding and navigation in a multi-factor way, combining a focus on locomotion modes, a focus on spatial syntax (characteristics) of the virtual world, and a focus on the abilities and differences of individual users. In addition to improving the design and use of virtual worlds, this work will impact multiple disciplines: it not only advances computer graphics and virtual reality, but also informs the fields of cognitive science and geographical information science.

Technology that can create compelling immersive virtual environments is now available on the general market. However, this technology has limitations. Important frontiers for virtual environments need high quality tracking equipment. One of these frontiers is the ability to build characters that move accurately in a virtual environment. A second frontier is the ability to explore large virtual environments using methods that seem natural. Our goal is to tailor these methods to the individual user. This research proposal will equip a lab with instrumentation that will make fundamental advances on these two problems. It will also train graduate students and provide research opportunities for a number of undergraduates.

This research will equip a laboratory with a high quality motion capture system that will allow the pursuit of novel scientific questions involving the perceptual fidelity of virtual environments, examine theoretical questions involving users and their relationship to their self-avatars, and determine how individual differences in users can be effectively utilized to provide better locomotion and navigation in virtual worlds. In particular, this equipment will enable research in how to design high fidelity virtual environments, and enable the understanding of the components of fidelity that facilitate learning and transfer of training, for which self-avatars are a critical component. Likewise, the equipment will enable significant progress in locomotion methods for improved navigation and wayfinding in large virtual environments by allowing examination of how spatial information sources are used by individuals as they move through virtual worlds.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.