Richard M. Shiffrin - US grants

The Cognitive and Information Sciences (CIS) program offers a model undergraduate curriculum designed to expose students to the study of intelligent systems. It is intended as a model for the development of similar programs at other institutions as they begin to move into this emerging field. The core of the newly proposed major concerns formal theories of mind and information that integrate computer science, psychology, philosophy, neuroscience, and mathematics. The curriculum gives students training in the basic skills of this new field (writing, mathematics, experimentation, and computer programming), but also fosters the development of expertise in particular sub-fields such as human cognition, learning and instruction, logic, and computation. Newly designed courses will expose students to both basic scientific theories and to applications of these theories to education, system design, and database retrieval. The program will be assessed internally by pre/post exams within courses and exit interviews upon graduation, and externally by quarterly site visits by expert consultants and a comprehensive third-year review by consultants from Indiana University's School of Education. The tangible products of this proposal will include general information on the CIS degree and its requirements and materials for specific courses in the form of syllabi, lecture notes, slides, bibliographies, assignments, and exams. Most important will be the course software and laboratory software developed for all four of the core CIS courses, and several others; this courseware will emphasize interactive, hands on learning through doing, and will be in a form easily usable by other programs that are beginning or being planned in this area. These products will be disseminated through our existing technical report series and newsletter, through electronic media, web sites, and contacts at other universities, and will be provided upon request of other institutions starting such programs.

DESCRIPTION (provided by applicant): The relation of perception to memory is a fundamental characteristic of human cognition, each affecting and determining the other. This relation is explored empirically with a task known as short-term priming: Words (termed 'primes') are presented just prior to a trial, but are irrelevant to the main task. Then a target word is presented briefly and masked, and followed by two choice words, one of which had been the target. Performance is strongly affected when one or more primes are related to one or more of the choice words. Our research has been able to separate these priming effects into those due to 'preference' and those due to 'perception': Preference effects are indicated by the fact that the direction of performance change is determined by the way the primes are processed. The findings have recently been fit by a quantitative model called ROUSE that assumes: 1) Features of the primes are sometimes confused with those from the target flash. 2) The decision system matches the detected features to the choice word features. 3) The evidence from matching and mismatching features is discounted almost optimally when a choice word feature had also been in a prime word. This model fits many studies remarkably well, and suggests the priming task is a good model system for further investigations of the relation of memory to perception. The present project explores many aspects of this relation, including decision strategies and their determining factors, the nature of source confusion for features, the ways in which priming improves perception, the relation of accuracy to response time, and the relation to other attention and perception tasks. The data will serve as a base for further development of the ROUSE model for priming, and eventually a more complete model of perception,

Collaborative Research: Modeling Perception and Memory: Studies in Priming

David E. Huber, University of California-San Diego
Richard M. Shiffrin, Indiana University

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).

It is said that "seeing is believing", and we take it for granted that vision operates efficiently and accurately. This suggests that vision is easy. However, failed attempts at producing computer vision demonstrate exactly the opposite--vision is perhaps the most difficult operation performed by the brain, requiring one third of the neocortex. The NSF-funded research project being conducted by David Huber at the University of California, San Diego and Richard Shiffrin at Indiana University focuses on an important question in visual perception: How is it that we can keep separate what we are currently viewing from that which came immediately before? In truth, vision is constantly "blurring" together information over time, such as when viewing the smooth motion at the cinema that is produced by a sequence of still images shown in rapid succession. However, while reading, our eyes constantly move from one word to the next, and yet unlike a movie, we see each word separately and do not confuse it with the previous words. To accomplish this, the brain must have a trick for deciding when the previous image should be combined with the next image and when each should be kept separate. Huber and Shiffrin hypothesize that the process of identifying each word or each movie image causes it to be suppressed so as to reduce inappropriate blending with the next word or image. In the case of a movie, the images appear too briefly, and the blending produces apparent movement. In the case of reading, our eyes dwell on each word exactly the right amount of time to fully identify and suppress each word so as to reduce confusion with the next word. Huber and Shiffrin investigate this ability to separate visual images in a variety of tasks, including reading, face identification, and rapid detection of change, to name just a few examples. If their hypothesis is correct, manipulating the timing of stimuli should produce analogous behavioral effects in all of these situations. Beyond laboratory studies, this hypothesis may also improve computer vision systems in situations requiring rapid identification. For instance, computer controlled cameras at the airport might be used to identify faces of suspects, but this requires separating one face from another when there is a crowd of faces moving quickly past the camera. The results of this research may also be relevant to disorders such as autism, schizophrenia, and dyslexia, which often involve a component of distorted or abnormal perception. For instance, one account of dyslexia suggests that reading difficulties arise from an inappropriate blending of letters and words. Understanding the manner in which the brain separates visual information over time may help with the diagnosis, interpretation, and treatment of these perceptual deficits.

The human perceptual system receives a constant stream of continually changing information. For example, the eyes move several times each second, providing different views of different objects or words. This project investigates the dynamic process of separating in time and space information pertaining to previous sources (e.g., a previously viewed word) from information pertaining to the current source (e.g., the currently viewed word). Behavioral studies will address the process of discounting that serves to reduce perceptual separation errors due to source confusion. This discounting process can be understood at multiple levels of description and the proposed experiments test complimentary and related mathematical models at the causal and neural levels of analysis. Two causal models use Bayesian statistical techniques and focus on optimizing perception in a noisy world perceived with a limited capacity processing system; discounting is implemented as "explaining away" between competing sources. The neural model implements discounting through habituation that arises with the transient depletion of synaptic resources. In combination, these models demonstrate why perceptual discounting exists and the particular manner in which it is implemented. A wide variety of experimental paradigms involve the rapid presentation of visual objects and the proposed studies use these models to investigate whether perceptual source confusion and discounting may provide a unified account of these phenomena. Besides visual short-term priming with words, the proposed studies examine the popular perceptual and cognitive paradigms of repetition blindness, flanker effects, the attentional blink, negative priming, semantic satiation, and affective priming. All of these paradigms involve presenting a picture, word, or symbol on a computer screen followed by a second presentation that is either identical, positively related, or negatively related to the first presentation. An important goal of this endeavor is to provide a unified account of these perceptual phenomena that are currently considered in isolation by researchers.

This is a grant for student (undergrad, grad) and postdoc support for travel and attendance at the Sackler Colloquium May 2-3 2019 at the Beckman Center in Irvine CA. The title is: "The Brain Produces Mind by Modeling". Speakers include a large variety of world leaders in computational and behavioral modeling of the brain-mind connection. The colloquium will bring together various threads and approaches in neuroscience, cognitive science, and psychological science by focusing on computational modeling that attempts to bridge the gaps between these fields. It aims to present a coherent view of the brain as a model builder, using the model to maximize survival and utility in a complex world.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.