Daphne Bavelier - US grants

This project concerns the cognitive and neural mechanisms underlying the analysis of visual scenes. The aim is to uncover how the information initially gathered by the visual system is subsequently organized for use in thinking or acting. To interact appropriately with the visual world one needs to keep track of specific objects over time, even if they move or are blocked from view. The research focuses on the process whereby early visual information is organized into short term memory representations for objects (hereafter termed individuation). The studies test the assumption that the phenomena of repetition blindness and the attentional blink occur during individuation and so can be used as tools to study the individuation process. Repetition blindness is the failure to see or report the second occurrence of an item in a briefly-presented list. One set of studies tests the assumption that repetition blindness reflects a constraint on establishment of short-term memory representations: two separate memory representations cannot be established for identical objects during a short period of time. The attentional blink is the finding that the processing of an item hinders the processing of subsequent items for as long as a half-second. Several studies test the view that the attentional blink occurs because establishing a memory representation for the first item uses the limited resources available for individuation and thereby hinders the processing of subsequent items. The results of the studies will add to our understanding of how people identify, represent, and interact with objects in the visual world.

DESCRIPTION (provided by applicant): Cognitive skills and their underlying brain organization are not fixed but can be shaped by the experience of the individual. The extent of such experience-dependent plasticity provides valuable information about the ability of the nervous system to reorganize and adapt. To this aim, we have been studying individuals who are born deaf. The goal of our research is to document the effect of deafness and that of American Sign Language (ASL-the visuo-manual language used by the deaf) on vision and its neural organization. Our working hypothesis is that deafness alters visuo-spatial attention. Building on our previous behavioral and brain imaging work, we will test the hypothesis that deafness leads to enhanced peripheral visual attention, and we will characterize the consequences of this peripheral enhancement for the processing of centrally presented visual information. Attention is a multi-faceted concept and we will be interested in asking which other aspects of visual attention, beyond its spatial distribution, may be modified by early deafness. It may be that cross-modal plasticity proceeds primarily by reshaping the brain systems that typically support multimodal integration. Under this view, we expect little to no changes in other aspects of attention beyond spatial attention for which cross-modal links are well established. In contrast, the alternate view predicts that compensatory plasticity enhances many aspects of the remaining modalities, with deaf individuals possibly displaying enhancement on a wide range of visual and attentional skills. Deaf and hearing adults will be compared first;then, children (7-17 years old) will be included so as to characterize the effects of altered experience on the developmental trajectory of the skill under study. These studies will first contrast deaf signers and hearing non-signers to establish a population difference. Deaf non-signers and hearing signers will then be tested to separately assess the contributions of deafness and ASL use in the changes observed. This is important as deafness and ASL use have been documented to have different effects on visual skills. This work will allow us to document the range of plastic behaviors exhibited by the visual system as well as boundary conditions on this plasticity. In addition, it will provide insights for the development of environments better suited to deaf individuals'needs.

With support from a National Science Foundation Major Research Instrumentation award, Dr. Richard Aslin and his colleagues at the University of Rochester will establish the Rochester Center for Brain Imaging (RCBI). The overall goal of this new center is to assess the plasticity of the adult and child brain as it adapts to altered and varied experiences. One type of alteration is the loss of sensory input in a single modality (e.g., the loss of vision or hearing because of blindness or deafness). Previous research at Rochester has shown that congenitally deaf individuals who use sign language do so with the same parts of the brain (the left hemisphere) that are usually used for spoken language, despite relying on the visual rather than the auditory modality. Deaf individuals also have greater sensitivity to patterns of movement in the peripheral visual field because they rely more on signed language inputs delivered in the visual modality. These patterns of brain plasticity are the result of altered sensory input during early development and have important implications for the brain's ability to compensate for deprivation and injury, provided that it has time during early development to adapt to these unusual circumstances. Similar mechanisms of plasticity may be present in adults as they learn a new task or compensate for brain injury. The Rochester group will use functional magnetic resonance imaging (fMRI) to study both long-term (developmental) and short-term aspects of brain plasticity in adults, children, and non-human primates. The research will provide important insights into the neural mechanisms of learning and plasticity and the keys to the brain's ability to adapt to novel experiences.

Working with a team of cognitive scientists and neuroscientists, as well as magnetic resonance physicists and image processing engineers, Dr. Aslin will supervise the purchase, installation, and operation of a 3 tesla (T) fMRI system designed to measure the microscopic changes in blood oxygen level that occur in localized regions of the brain as participants perform a variety of tasks. This system, which is state-of-the-art in the field of human brain imaging, will provide a group of over 30 researchers from the University of Rochester and Cornell University (90 miles from Rochester) with the capability to explore a variety of issues in human brain plasticity and recovery of function after natural deprivation, injury, or disease. A key feature of the new center is a team of physicists and engineers who will develop new ways for fMRI to reveal even more fine-grained details about the functioning and visualization of the brain.

This project is important for several reasons. It will provide a first-class facility for non-invasive brain imaging to a group of researchers at Rochester who have already demonstrated their ability to conduct cutting-edge research in cognitive neuroscience. The RCBI will also play a significant role in the training of future scientists by actively involving graduate and undergraduates students from the University of Rochester and Cornell University, as well as undergaduates from the State University of New York at Geneseo (a non-Ph.D.-granting college located 30 miles from Rochester) in state-of-the-art brain imaging research. The new center provides an excellent vehicle to teach the principles of fMRI to a new generation of students who will become leaders in the field of cognitive neuroscience.

DESCRIPTION (provided by applicant): Video game playing has become a common activity in today's society;however, little is known about the effects of this activity on our perceptual abilities. Our previous work indicates that playing action video games modifies visual attention, such as the ability to look for a target in a cluttered environment or to process a rapid stream of visual information. Our broad aim is to further characterize the effects of video game playing on visual skills in adults and children. The study of the impact of video game use has broad implications, ranging from public policy to issues in health and performance training. Our aim is not to study the impact of casual entertainment on the society;rather we focus on health and performance training by determining the impact of video games on vision and on training for specialized visuo-motor tasks such as those performed by airline controllers. We will first ask whether the impact of game playing generalizes to various aspects of visual attention. These studies will be carried in adults and children. We will then test the possibility that video game playing may also modify early visual processes such as the bottom line on the eye chart. Finally, we will ask whether video game playing may also alter visual short-term memory or the ability to hold visual information in mind. The possibility that video game playing modifies several different aspects of visual processing is of theoretical significance as the literature on perceptual learning documents a great specificity of learning. Hence, learning a new task typically benefits performance on that very task, but fails to generalize to similar tasks. In contrast, the aim of this work is to test the possibility that video game playing benefits not only performance on the played video game, but also on a wide range of other visuo-motor tasks. This research holds therefore the promise to be of practical significance for rehabilitation of patients with visual deficits and/or the training of any personnel who work in environments that call for enhanced visual performance.

DESCRIPTION (provided by applicant): Video game playing has become a common activity in today's society; however, little is known about the effects of this activity on our perceptual abilities. Our previous work indicates that playing action video games modifies visual attention, such as the ability to look for a target in a cluttered environment or to process a rapid stream of visual information. Our broad aim is to further characterize the effects of video game playing on visual skills in adults and children. The study of the impact of video game use has broad implications, ranging from public policy to issues in health and performance training. Our aim is not to study the impact of casual entertainment on the society; rather we focus on health and performance training by determining the impact of video games on vision and on training for specialized visuo-motor tasks such as those performed by airline controllers. We will first ask whether the impact of game playing generalizes to various aspects of visual attention. These studies will be carried in adults and children. We will then test the possibility that video game playing may also modify early visual processes such as the bottom line on the eye chart. Finally, we will ask whether video game playing may also alter visual short-term memory or the ability to hold visual information in mind. The possibility that video game playing modifies several different aspects of visual processing is of theoretical significance as the literature on perceptual learning documents a great specificity of learning. Hence, learning a new task typically benefits performance on that very task, but fails to generalize to similar tasks. In contrast, the aim of this work is to test the possibility that video game playing benefits not only performance on the played video game, but also on a wide range of other visuo-motor tasks. This research holds therefore the promise to be of practical significance for rehabilitation of patients with visual deficits and/or the training of any personnel who work in environments that call for enhanced visual performance.

The ultimate goal of this project is to provide a novel model of the cognitive and neural basis of numerical cognition, and to use this knowledge to guide the development of new training methods that could improve mathematical abilities in children. The project is a collaboration among investigators at the University of Rochester, Johns Hopkins University, and Cold Spring Harbor Laboratories. Recent research suggests that acuity of numerosity judgments is predictive of success in formal mathematics education, and that similar cognitive processes can be trained by specific kinds of domain-general experience. The core idea is that the firing of neurons encodes a probability distribution, thereby representing simultaneously the most probable sample from the distribution and the variance (i.e., confidence) of the estimate.

This project will develop and test a formal Bayesian model that has the unique feature of naturally accounting for a number of metacognitive factors, a critical but undertested factor in the acquisition of expertise. The primary advantages of this Bayesian approach are its ability to provide a natural description of: 1) how the confidence of a learner relates to the precision of their number knowledge; 2) how a learner can combine information from multiple sources of information about number; 3) how intuitive preferences (also known as prior belief) predict learners' errors; and 4) how improvements in probabilistic inference may benefit the precision of the number sense.

This project focuses on development of number sense in 7 to 11 year old students through a 3D action game that will train the brain. The project tests a hypothesis that playing a number-sense action game can help children learn beyond the game, making them better overall at number-line sense, precision of numerosity, speed of numerical judgments, and ability to multitask numerical tasks. The innovation is twofold: a game, adapted from first-person shooter games, to train number sense, and a platform that makes it easy to vary aspects of the game (e.g., repetition, speed, number of possibilities) to be able to analyze what is it that is making a difference with respect to learning. Research is identifying the qualities of computer games that will train the brain to be automatic in its judgments and at qualities of experiences that promote such automaticity.

This project aims to promote fundamental mathematical competence in a way that makes the mathematics learning feel effortless and genuinely fun. In particular, the aim is to help children develop number sense, an ability to very quickly make estimates and numerical judgments. Such capability is key to keeping up in mathematics; children without that sense tend to lose interest in mathematics because mathematical computation becomes too frustrating. Training number sense, and, in turn, fostering mathematical performance, should allow more learners to think of themselves as "good at math." Cultivating a child's interest in mathematics from a young age has been identified as key to feeding STEM career paths.

The vehicle for promoting such mathematics learning is a new type of video game with foundations in first-person shooter games. Rather than shooting, players cast spells that require quick numerical estimates. There is reason to believe that such a game will be as engaging as first-person shooter games are. First-person shooter games have been shown to increase attention and executive control in adults, the kinds of cognitive skills known to foster academic achievement. Because engaging video games will reach and be played by a wide range of children; such a game has potential to reach at-risk populations who most need to better develop their number sense and those important skills. The intent is for the game to be used in informal settings, providing a complementary tool for educators to foster mathematics skills. Research that is done in the context of this game will identify the characteristics of game play that promote skill automation, foundational knowledge that is needed to provide a road map for creating child-friendly consumer video games to improve other important aspects of core cognition beyond number sense.

Behavioral training interventions have received much interest as an efficient and cost effective way to maintain brain fitness or enhance skilled performance with impact ranging from health to job training. In particular, neuroscience and behavioral research has documented the importance of explicitly training attentional control, in order to enhance perceptual and cognitive fitness, physical exercise, to promote physical fitness and rehabilitation as well as kindness and compassion, to produce changes in adaptive emotion regulation and well-being. During this same time period, video game play has become pervasive throughout all layers of society providing a potentially unique vehicle to deliver such controlled training at home in a cost-efficient manner. Given the potential impact of these interactive systems, it is necessary to understand the underlying scientific principles that govern the development and deployment of interactive computer games in healthcare and education. The search for the deeper understanding requires the involvement of expertise from a wide variety of areas ranging from neuroscience to computer science and engineering. To this end, the workshop brought together experts from a wide range of academic disciplines with engineers and software developers. The goals of the workshop was to identify the remaining challenges and roadblocks that represent barriers in making these sophisticated interactive systems to be among the key components supporting the future healthcare delivery with emphasis on prevention and quality of life. This workshop provided a unique opportunity to define new approaches to the design, assessment and distribution of video games for enhancing well-being and attention, opening a unique path to addressing key health care and societal concerns in a user-friendly and cost effective manner

? DESCRIPTION (provided by applicant): Amblyopia is a deficit that arises from abnormal visual experience early in life, most commonly when the two eyes are not aligned (strabismus) or have unequal refractive error (anisometropia). It has long been thought to develop into a permanent deficit unless properly treated early in life; however, recent studies call this into question [1-8]. Adult amblyopes can recover some visual functions. Thus, amblyopia provides a useful model for understanding how to unlock adult neural plasticity. Our main aim is to develop new interventions for restoring visual acuity and stereopsis in adults with amblyopia by embedding visual training in active and rewarding visuomotor tasks that require stereopsis. Our work during the current grant period indicates that playing action-packed video games enhances several aspects of vision in amblyopic adults. Training either monocularly or under dichoptic conditions results in improved visual acuity and reduced suppression in adults with amblyopia. However, it is now clear that both perceptual learning (PL) and videogame play (VGP), whether monocular or dichoptic, result in only a modest improvement in visual acuity (by one to two lines), and only limited improvement in stereopsis [1,3-8]. Much recent work has focused on the role of binocular suppression, as the key to recovering visual functions in amblyopia. Specifically, it has been suggested that treatment should be dichoptic - and aimed at eliminating suppression - rather than monocular [5-8]. However, our pilot studies call this approach into question, and suggest a more direct approach to improving both visual acuity and stereopsis: training under 3D conditions. By forcing the two eyes to coordinate and integrate their signal in the service of a common output, stereopsis provides a unique common teaching signal to the two eyes that may be central to overcoming amblyopia. Our emphasis on stereopsis is novel, but it is based on: (1) our pilot data, showing that in adults with amblyopia training stereopsis directly resulted not only in improved stereopsis, but also in improved visual acuity and contrast sensitivity, as well as reduced inter-ocular suppression and (2) the loss of stereopsis has demonstrable negative effects on everyday activities that can significantly impact individual's quality of life as well as limit their career choices and activities. The most noticeble qualitative deficit associated with amblyopia is impaired or complete loss of stereoscopic depth perception. Thus, the Aims of this proposal are: 1) To develop an integrated, immersive 3D videogame training program for adults with amblyopia; 2) Develop a battery of outcome measures to evaluate the effectiveness, mechanisms and real world impact of training 3) Evaluate the effectiveness of direct training of stereopsis on vision recovery in adult amblyopes.