Barbara Tversky - US grants

9815553
Montello
This award supports the PI, Daniel Montello, co-PI Mary Hegarthy, and a graduate student from the University of California at Santa Barbara as well as co-PI Barbara Tversky and a graduate student from Stanford University in a collaboration with Christian Freksa of the Department of Informatics and Cognitive Sciences at the University of Hamburg, Germany. The research focus is the description and explanation of mental processes and structures underlying behaviors such as navigation, spatial learning, and spatial language. The field of study is interdisciplinary and will involve geographers, psychologists, computer scientists, and linguists. The U.S. and German groups bring complementary capabilities to the enterprise. The German side specializes in formal and computational modeling and the U.S. group, in empirical human-subjects methods and theory. Each side's contribution will enhance the breadth of research of the other side, and will result in interactions between researchers in the several disciplines mentioned.

Conveying molecular-level chemistry processes to students is notoriously difficult. Dynamic, molecular-level animations are used increasingly in chemistry and biology to help students visualize chemical structures and processes. However, animations are often too complex to be accurately perceived, especially by novices, who may not even know where to look or how to interpret what they are seeing. Animated graphics may even convey or sustain misconceptions about chemical processes. The goals of this project are to reveal how students perceive and interpret various kinds of molecular animations and to develop design principles for creating and presenting effective static graphics and dynamic visualizations in chemistry.

This proposal posits that effective animated and static visualizations can be designed. To this end, the proposal applies techniques from cognitive psychology to reveal: a) mental models novices and experts have about key chemical processes, and b) graphic tools that will convey those concepts adequately. The research has 5 steps, which will be carried out sequentially during the three-year period: 1. In-depth talk-aloud sessions with experts and novices to reveal the mental models; 2. Analysis of attempts by novices and experts to convey key concepts graphically and in language; 3. Extraction of design principles from 1 & 2; 4. Development of animated and static visualization prototypes in which those design principles have been applied; 5. Testing the new visualizations on new students.

To carry out such a wide-ranging project requires expertise in cognitive psychology, in chemistry education, in chemistry, and in the development of animations and visualizations. This project involves collaboration among the disciplines of chemistry, molecular biology, education, and cognitive science to develop research-based design principles for effective molecular animations. The project will provide useful guidelines for the effective design of dynamic molecular visualizations and illustrate them with concrete examples. It will also generate research tools that can be used to study the characteristics of animations and simulations in other scientific fields. Results of the study will allow the developers of all types of molecular visualizations to make informed design decisions and will give instructors a sound pedagogical basis for effective use of molecular animations and simulations.

To survive and flourish, people must interact with their environment in an organized fashion. To do so, they need to learn, imagine, and perform an assortment of transformations on and in the world. Primary among these are manipulation of objects and navigation in space. This project integrates research in computer science and cognitive science to develop and evaluate augmented reality tools to create effective dynamic explanations that enhance manipulation and navigation, in conjunction with identification and visualization. Augmented reality refers to user interfaces in which virtual material is integrated with and overlaid on the user?s experience of the real world; for example, by using tracked head-worn and hand-held displays. Dynamic explanations are task-appropriate sequences of actions, presented interactively, with appropriate added information. The tools will be created in collaboration with subject matter experts for exploratory use in indoor and outdoor real world domains: navigating and identifying landmarks in a wooded park area, assembling a piece of furniture, and navigating and visualizing for planning the site of a new urban campus. Cognitive science research will determine the best ways to convey explanations and information to people. Computer science research will address the design and implementation of systems that embody the best candidate approaches for identifying objects and locations, specifying actions, and adding non-visible information. In situ experiments will be used to assess and refine the systems.

Manipulation, navigation, identification, and visualization are representative of important things that people do every day, ranging from fixing broken equipment to reaching a desired destination in an unfamiliar environment. The ways in which we perform these tasks could potentially be improved significantly through augmented reality systems designed using the principles to be developed by this project. Both the cognitive principles and the augmented reality tools will have broad applicability. The systems developed will inform the design of future systems that can aid the general public, for educational and recreational ends, as well as systems that can assist people with auditory, visual, or physical impairments.

This is a project to understand and improve how augmented reality can facilitate task performance by superimposing information in users' visual fields to help identify objects and features, direct actions, and provide spatial-temporal overviews and other perspectives. Augmented reality is a rapidly developing information display technology, using either head-mounted displays or handhelds incorporating cameras, to add information to the visual field of the user. The project will develop and test the best ways to facilitate performance with multiple people, perspectives, platforms, and tasks to convey the what, where, when, and how of action. The research will be conducted in the laboratory and in the field, including the historic El Barrio section of East Harlem. Project software is expected to significantly improve the ability of residents and visitors, alone and in groups, to learn about, plan visits to, and explore the neighborhood. In their personal and professional lives, people need to perform an enormous range of complex tasks, including navigation, maintenance, and assembly. Technology can make these tasks easier, and the use of augmented reality to interactively assist users by overlaying crucial missing information directly on a user's view is especially promising.

Each of the tasks studied in this research entails an organized sequence of actions with respect to features or objects in the world. The features or objects may be hard to find or even be occluded. The actions may be complex. Work is increasingly collaborative, requiring coordination with others who have different perspectives. Each action can depend on previous and subsequent actions. The research will integrate work in cognitive science and computer science to develop principled approaches for using the transformative technology of augmented reality to assist people in navigation, maintenance, assembly, and related tasks. It will expand our abilities in understanding how people represent, transform, and communicate space and the actions in it; in designing instructions; in developing systems to assist users in assembly, maintenance, and navigation; and in designing stationary, mobile, and wearable user interfaces that use graphics, multimedia, and augmented reality, both indoors and outdoors. The scientific results will enlighten the study of the communication of objects, actions, and data, instructional design, and user interface design. The techniques and systems developed will inform the design of future systems that can aid the general public for educational and recreational ends, as well as systems that can assist people with auditory, visual, or physical impairments. Navigation, maintenance, and assembly are representative of many important daily tasks. Software created by the project will be adaptable to different situations and displays, and made available to the public as open source. The project will train students, and will be carried out in part through collaborative interdisciplinary projects in courses in computer science and cognitive science.