Robert Jacobs - US grants

This project is purchasing a network of six graphics workstations that, together with existing Macintosh and DEC computers, serves as the basis for new course developments in the area of computational chemistry within the undergraduate curriculum in the chemistry department and the Pharmacology Program of the biological sciences department. The new computers are being used in courses in Freshman chemistry, organic and advanced organic chemistry, biochemistry, polymer chemistry, computational chemistry, and upper-division pharmacology courses and are being used to support undergraduate research projects in the academic year and the summers. The capabilities of this new instrumentation makes it possible to communicate to students the reality and excitement of modern developments in chemistry and to involve these students in exercises and projects that will give them "hands-on" access to these computational techniques as an integral part of our undergraduate curriculum. Thus far, experimental courses in this area have been very enthusiastically accepted by students and give students a set of skills that serve them in graduate and professional schools and careers in the bulk chemical and pharmaceutical industries.

Both humans and non-human primates show remarkable learning abilities. However, these abilities are often limited to certain domains, developmental periods, or behavioral contexts. For example, nearly all humans acquire one or more complex linguistic systems-that is, languages -- but not all humans acquire complex musical systems. Similarly, non-human primates are exceptionally adept at learning to forage for and categorize different types of food, but are severely limited in acquiring complex communication systems. Also, both humans and non-human primates appear to learn best in several domains during early periods of development. Thus, learning is nearly always characterized by specializations, rather than by general-purpose mechanisms. Understanding the constraints on learning will contribute to basic research, by accounting for domain- and species-specializations, and to applied research, by refining our understanding of which domains, ages, and contexts are optimal for human
learners.

The goal of the present research project is to explore the ability of human adults, children, infants, and non-human primates (Tamarins) to learn rapid sequential events. A prime example of a rapid sequential event is language, in which sounds are combined to form words, and words are combined to form sentences. Recent findings have demonstrated that human adults and infants can rapidly extract and remember very detailed 'statistics' of linguistic input, such as the frequency and probability that one syllable will follow another. In our proposed research, we will employ miniature artificial 'languages' which simulate some of the structural properties of natural languages, but which can be built with equivalent structures across different domains (speech sequences, tone sequences, visual sequences, motor sequences). At issue is the facility humans and non-human primates show for the extraction of statistical structure from these different learning materials. Are they equally sensitive to the distributions of elements and higher order structure in the materials? Do learning abilities differ across learners of different ages and species, and across different structures and domains?

In addition to behavioral experiments with humans and non-human primates, a series of computational studies will allow us to investigate the formal properties of learning mechanisms, in order to ask what architectural and neural differences might underlie such differences in learning abilities. What kinds of computational architectures can learn the types of regularities and patterns that human infants learn? Is the inability of adults or non-human primates to learn some types of complex sequential events due to the absence of a learning device specialized for that domain, or could small differences in computation and/or memory lead to large differences in learning outcome?

DESCRIPTION (Adapted from applicant's abstract): The human visual system obtains information about object depth from a large number of distinct cues. A full understanding of visual depth perception requires an understanding of how information provided by these cues is combined by our visual systems. Although nearly all theories of visual depth perception use the concept of cue reliability, we lack a good understanding of what this concept means, of how observers can measure cue reliability, and of what observers can do once they have measured it. The proposed research program places much importance on the need to understand observers' estimates of cue reliabilities, and on the need to understand how observers use these reliabilities during visual reliability judgments, and the roles that these factors play in experience-dependent adaptation of visual depth perception. Two types of experience-dependent adaptation are considered. Cue combination learning refers to the adaptation of the integration process that combines depth estimates based on individual cues into a single, composite depth estimate. Cue recalibration refers to the adaptation of depth interpretations of individual visual cues, such as adaptation of depth-from-motion estimates or adaptation of depth-from-texture estimates. The research program hypothesizes that observers regard a depth cue as reliable when: (i) depth estimates based on that cue are less variable than estimates based on other cues; or when (ii) depth estimates based on that cue are positively correlated with estimates based on other cues. It also hypothesizes that observers adapt their depth perception strategies so as to: (i) rely more heavily on reliable cues during cue integration; and to (ii) recalibrate depth judgments based on unreliable cues so that they more closely match those based on reliable cues.

The proposed research program consists of experimental and computational studies of human visual learning. The project focuses on the information processing mechanisms mediating the perceptual learning that underlies expertise in a variety of STEM fields, such as biology, astronomy, and geoscience. In particular, the investigators attempt to take advantage of insights from the field of Machine Learning (e.g., its formalisms for conceptualizing the properties of different learning environments, its powerful sets of statistical learning algorithms for each environment, and its numerous mathematical and empirical findings about the advantages and disadvantages of these algorithms). The studies look at learning performance on lower-level and higher-level discrimination tasks in four types of learning environments: supervised, unsupervised, semi-supervised, and reinforcement learning environments. The project also explores visual learning based on correlated perceptual signals in multisensory or multi-cue environments, such as when a person both sees and touches surfaces. The computational studies compare people's learning performances with the statistically optimal performances of "ideal learners", and also with the performances of on-line learning algorithms from the Machine Learning literature. A key hypothesis is that people can visually learn with "unlabeled" data items (i.e., items that are not labeled by an instructor as examples of a particular category of interest) by transferring knowledge gained with "labeled" data items or by transferring knowledge gained from other sensory modalities. The work has important implications for the design of STEM training environments.

People can perceive the shape of objects accurately and reliably but how this occurs is not yet understood. This ability may stem, at least in part, from our use of both visual information and haptic information (information obtained when an object is touched or grasped). Moreover, if we learn to recognize an object based on visual information, we can often recognize the same object when our eyes are closed but we are allowed to grasp it. Similarly, if we learn to recognize an object based on haptic information, we can often recognize the object when we see it but cannot touch it. In other words, we exhibit cross-modal transfer of object shape information. How does information from the eyes and hands link up in the brain to yield a coherent representation of object shape? Insights obtained from this research can contribute both to our understanding of how humans perceive object shape using vision and/or touch and to development of improved robotic and other artificial intelligence systems operating in multi-modal settings in industrial, medical, military, and other applications.

The present project develops a theory of visual-haptic object shape perception in which people's notions of object similarity are not based on sensory features but rather on latent or hidden variables that represent object parts and their spatial relations in an abstract, modality-independent format. Object representations are formalized using a probabilistic "shape grammar" with Bayesian inference used to infer grammar-based object representations when an object is viewed, when it is grasped, or both. The model is tested using data obtained from behavioral studies of visual, haptic, and visual-haptic object shape perception by humans. The investigators will explore the types of representational change that underlie the transition from perceptual novice to expert (e.g, radiologists) and will assess whether perceptual expertise is well characterized as category learning, grammar learning, both, or neither. The research program will also develop a large public database of code for re-creating both visual and haptic features of complex objects. This will allow other researchers to fabricate the objects using a 3D printer, enhancing complementarity and comparison across research sites. Finally, training undergraduate and graduate students in the emerging field of computational cognitive science will contribute to a new generation of multidisciplinary scientists working across traditional boundaries between cognitive science and computer science.

This project will identify ways to improve education and training in the geosciences, building on fundamental research in cognitive science. Geoscience is a STEM discipline that is of growing importance to several national and global issues, including climate change, energy resources, and understanding earthquake activity. Expertise in geoscience depends heavily upon unconscious perceptual skills that are difficult or impossible to impart through traditional classroom education. Consequently there is a great need to improve or develop new methods for perceptual training (i.e., training students to visually detect, identify, and interpret geologic processes) and to study how educational practices in this area may be optimized. This project will apply a cognitive science approach to meet this goal, by conducting behavioral experiments involving geoscience students, and developing computational models of human cognition. This project is supported by NSF's EHR Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field.

The proposed research will focus on the role of visual working memory (VWM) in perceptual learning and perceptual expertise in the geosciences. VWM is a cognitive system that is critical to the geosciences. For example, when a geologist views or studies a landscape, he or she acquires visual information from multiple points in the scene by making numerous eye movements. The geologist can then (unconsciously) use his or her VWM to integrate information across eye movements to develop a coherent representation of the landscape. While there is much existing research on VWM, very little is known about the relationship between visual working memory on the one hand, and fundamental changes in perceptual ability in specific domains such as geoscience on the other hand. The proposed research will build upon, and extend a recently developed model of human perception that has the potential to connect these two domains. This model is based upon information theory, or the mathematical study of how physical systems can efficiently communicate information. This model suggests that training and expertise fundamentally change a person's perceptual system in a way that leads to more efficient use of cognitive resources. In order to test this model, a series of behavioral experiments will be conducted involving novice and experienced students in the geosciences. These experiments and computational modeling efforts will contribute fundamental knowledge that can be used to improve geoscience and education.