Marlene Behrmann - US grants

[unreadable] DESCRIPTION (provided by applicant): The optimal analysis of a visual scene requires the efficient processing and representation of the objects in the scene as well as their spatial positions. There is, however, far too much information to be represented at any one point in time with the result that these multiple possible locations and/or objects compete for neural representation. Given this limitation, people are perceptually selective, experiencing and responding to only a subset of the sensory signals evoked by events in the local environment. Selection can occur from representations of spatial location but also from representations of the objects or features in a scene and can be determined top-down by the current behavioral goals of the observer or bottom-up by the properties of the stimulus. The psychological and neural mechanisms that mediate perceptual selectivity are termed "attention," and the focus of this proposal is on the nature and operation of these attentional mechanisms. The research adopts a multidisciplinary approach to the investigation of visual attention by combining behavioral, functional imaging and anatomical studies in normal and brain-damaged individuals. Three major questions are addressed. The first concerns the mechanisms underlying selection by spatial location and the role of parietal cortex (and subdivisions therof) in modulating selection in a top-down and bottom-up fashion. The second question parallels the first but investigates mechanisms associated with object-based selection and the hemispheric specialization of parietal cortex. The final question addresses the interaction of space and object-based attention in order to understand the unitary and seamless nature of perception. In addition to shedding light on the mechanisms of attention, this research examines how brain damage affects attentional function. Understanding the intact system (and its breakdown) will enable us to develop effective methods for remediation. Acquired attentional deficits are notoriously difficult to rehabilitate, with profound adverse consequences for the individual, and targeted intervention efforts are urgently needed. [unreadable] [unreadable]

Selecting visual information from complex, multi-object scenes for further analysis and action is necessary because of the imitations of the human visual system. Recent accounts of visual attention acknowledge that preferential processing may be afforded to information selected on the basis of its physical location (space- based) and/or on the basis of its shape or object properties (object- based). Our hypothesis is that both spatial and object attentional effects emerge in a unitary model from competitive and cooperative interactions between representations, and that these representations are significantly influenced by interactions with task demands. In this mechanistic account, then, attention which is defined as the selection of a small set of active units, arises as part of the process of scene interpretation in conjunction with fulfilling the demands of a particular task. We adopt two complementary experimental paradigms: an object cost paradigm that reveals the difficulty attending to two objects simultaneously, and a distractor paradigm that reflects the inability to ignore elements grouped together with relevant information. Simulations and experiments will address three main issues: (1) We will explore the nature of the representations derived from perceptual experience, how these representations transfer and generalize to novel stimuli and how they serve to complete patterns that are impoverished or degraded (including occlusion and amodal completion). We will also explore how varying sources of perceptual information combine and become grouped, and how grouping might bread down in neuropsychological patients. (2) The emergence of space- and object based effects. In simulation work, we will evaluate the differences between architectures with modular or distributed space and object representation, and assess whether they can account for a range of empirical data that reflect the simultaneous contribution of space- and object- processes. We will also examine how object representations modulate location based attention in patients with a spatial deficit and simulate the effect of a spatial bias in the ~lesioned~ network to provide comparisons between the network and the empirical data. (3) Modulation of visual selection by task demands. We will explore how task requirements interact with space and location based representations, how task knowledge is learned and how the visual representations are reshaped or reconfigured under the pressure of differing task demands. Moreover, we map out the time course of processing and evaluate whether the standard time course can be reversed by manipulating task complexity. Finally, the three-way interaction between space-based, object based and task based influences are explored in patients with brain damage to reveal the full interactivity of the selection process.

DESCRIPTION(adapted from applicant's abstract): We request support for a symposium to be held on June 2-4, 2000 at Carnegie Mellon University, Pittsburgh, on the topic of Perceptual Organization in Vision. The aim of the symposium is to being together researchers from a variety of disciplines to consider mechanisms of visuoperceptual organization. Both behavioral and neural approaches will be considered. Behavioral approaches will include those arising from research in cognitive psychology, developmental psychology, and animal behavior studies. Neural approaches will include evidence obtained from single neuron recording studies, from neuropsychological studies of patients with discrete lesions to the cortical visual system and from functional neuroimaging studies. Quantitative and computational approaches will also be incorporated. The plan is to hold a three-day symposium with approximately a dozen speakers and an expected audience of 150-200 attendees. The following types of questions will be examined in the course of the workshop: 1) What are the heuristics involved in visual perceptual organization? 2) Are these predetermined and/or are they experience-dependent? 3) Do they operate only feedforward or is the system interactive, combining bottom-up and top-down knowledge in deriving organizational structure? 4) Do they operate early or late or are they perhaps more parallel than sequential? 5) What is an object? 6) What is the role of attention in perceptual organization? 7) What specific neural mechanisms mediate perceptual organization? 8) Is there potential for rehabilitation of individuals with deficits in perceptual organization? The general goals of the meeting will be to review the current state of the field from a multidisciplinary perspective by bringing together researchers who might not ordinarily exchange ideas, to provide the opportunity for junior scientists to be brought up-to-date and to interact with the speakers, to identify future research directions, and to disseminate the results broadly. Participants will include established researchers as well as promising new investigators. A dozen of the leading researchers in the field will describe their own research on issues such as grouping principles, figure-ground organization,amodal completion, surface derivation, object superiority effects, and the role of attention in organization. The new investigators and students will be given the opportunity to interact with the speakers in less formal settings as well as in question-and-answer sessions held throughout the symposium. Discussants and the keynote speaker will introduce historical perspectives and will provide a critical as well as synthetic viewpoint with the intention of delineating the existing gaps in the field. The results of the symposium would be published as the 31st in the Carnegie Cognition Symposium volumes pursuant to the standing contract for such publications with Lawrence Erlbaum Associates.

Project: BCS 9983241 PI: Behrmann, M.

The project is to hold a symposium at Carnegie-Mellon University in June, 2000 on the topic of Perceptual Organization in Vision. The aim of the symposium is to bring together researchers from a variety of disciplines to consider mechanisms of perceptual organization. Both behavioral and neural approaches will be considered. Behavioral approaches will include those arising from research in cognitive psychology, developmental psychology, and animal behavior studies. Neural approaches will include results from single neuron recording studies, from neuropsychological studies of patients with discrete lesions and from functional neuroimaging studies. Qualitative and computational approaches will also be incorporated. The following types of questions will be examined in the course of the symposium: What are the heuristics involved in perceptual organization? Are these predetermined and/or are they experience-dependent? Do they operate only feedforward or is the system interactive, combining bottom-up and topdown knowledge in deriving organizational structure? Do they operate "early" or "late" or is it perhaps more parallel than sequential? What is an object? What is the role of attention in perceptual organization? What specific neural mechanisms mediate perceptual organization? The general goal of the meeting will be to review the current state of the field from a multidisciplinary perspective by bringing together researchers who might not ordinarily exchange ideas, to provide the opportunity for junior scientists to interact with leaders in the field, to identify future research directions, and to disseminate the results broadly. The results of the symposium will be published as the 31st in the series of the Carnegie Cognition Symposium volumes.

The majority of research on human perception and action tends to treat these as separate functions. What is less often considered in these research domains is that humans interact with a perceived world in which they themselves are part of the perceptual representation. Evidence has been mounting to show that self-representation is fundamental to both executing and understanding spatially directed action. It has been theorized to play a role in reaching and grasping, locomotion and navigation, infant imitation, spatial and social perspective taking, and neurological dysfunctions as diverse as phantom limb pain and autism. Behavioral research has revealed a number of tantalizing outcomes that point to a role for the representation of the body in basic human function; neuroscientists have identified multiple sensorimotor maps of the body within the cortex and specific brain areas devoted to the representation of space and place; and developmental researchers have identified neonatal behaviors indicating a representation of self and have traced the course of spatially oriented action across the early years. What is needed is a shared effort to merge perspectives of behavioral science, neuroscience, and developmental psychology in order to further our understanding of self-representation. With support from the National Science Foundation, the 2006 Carnegie Symposium will provide a forum by which researchers from these various perspectives can come together to share their findings, ideas, aspirations, and concerns.

Recent findings in the field of cognitive neuroscience have suggested that specific brain areas are dedicated to specific human behaviors. The current research examines a different brain-behavior organization in which cognition is supported by an interactive network of brain regions, with many brain regions participating in many behaviors in a graded, dynamic and cooperative fashion. Specifically, the investigators will examine the neural substrates responsible for two behaviors, namely face recognition and word recognition. The proposed research predicts that adult individuals who acquire an impairment in face perception (a disorder known as 'prosopagnosia') due to brain injury will also have problems to varying degrees with the perception of words, and conversely, individuals who acquire an impairment in word recognition (a disorder known as 'pure alexia') due to brain injury will also demonstrate some abnormality in face recognition. Additionally, this research investigates the performance of normal individuals with the expectation that if the right hemisphere is highly specialized for face perception, the left hemisphere will be highly specialized for word perception but that the balance between these two abilities might vary in a complementary fashion across a range of individuals. The investigators will use a combination of behavioral and neuroimaging studies of normal and brain-damaged individuals and use computational modeling tools to test these predictions. Findings from this research will provide support for the claim of partial co-mingling of face and word processing in a shared network as well as the hemispheric cooperation and competition in normal observers and associated individual differences for face and word representation. The empirical behavioral and imaging studies will generate data that guide the development of a computational model of how complex behavior is mediated by the brain and the model, in turn, will inspire further testable predictions of this account.

The critical focus of this research is to explore the correspondences between the brain and behavior with specific emphasis on the nature of the organization of the brain that supports rapid and efficient human behavior. What brain systems enable us to recognize words with such accuracy and speed? Are these the same systems that allow us to recognize faces so well? How do we explain individual differences in the ability to recognize such perceptually challenging visual input? To the extent this research can show how general computational principles lead to well defined functional specializations, it may transform our understanding of the functional organization and specialization in the brain in general. In addition to examining the fundamental mechanisms of visual object recognition and individual differences in normal and disordered visual recognition, the investigators expect that an understanding of the intact system and its breakdown will enable the development of more effective methods for intervention and rehabilitation for patients who suffer from visual recognition deficits. Finally, the project will contribute to research infrastructure not only by training students and junior investigators but also by disseminating the findings and the computer modeling tools to the broader research community.

DESCRIPTION (provided by applicant): A hemispherectomy involves the surgical disconnection or removal of a cerebral hemisphere in the service of treating certain intractable conditions (e.g. epilepsy). Despite the extreme nature of this procedure, the visual deficits introduced by hemispherectomy can undergo impressive improvement over time, especially when the procedure is conducted at an early age. The basis of this recovery lies in the ability of the intact hemisphere to 'take over' the functions normally performed by its counterpart in a remarkable display of neural plasticity. Surprisingly, rather little systematic exploration has bee directed at the mechanisms supporting this plasticity. The goal of our project is to characterize the remapping of visual processing in the intact hemisphere of these unique individuals by means of behavioral and neuroimaging (fMRI) investigations. First, at the level of the early visual cortex, we aim to characterize the (re)organization of retinotopic maps. One intriguing hypothesis that motivates this investigation concerns the ability of the intact occipital lobe to develop sensitivity over time to the affected ipsilateral hemifield. Second, at the level of higher level visual cortex, we aim to characterize the cortical topography involved in the recognition of prominent object categories such as faces and visual word forms. The ability of a single hemisphere to cope with multiple such categories is indicative of important adjustments in the architecture of high-level object recognition. For instance, a more compact integration of visual representations underlying different categories could mediate successful recognition processes at the neural level and we aim to clarify the neural underpinnings of this functional integration. Third, we propose to characterize the extent and the profile of the recovery as a function of a number of key variables pertaining to the specifics of the procedure. In particular, the side on which the procedure was performed (i.e. right or left hemispherectomy) could determine the differential recovery / development of different sets of abilities. This hypothesis is particularly relevant for certain visual categories commonly displaying hemispheric dominance / lateralization (e.g. faces in the right ventral cortex and visual word forms in the left). In sum, y virtue of our access to this rare population, our investigation provides an unprecedented opportunity to explore a radical case of cortical remapping and the consequences of this remapping for visual perception. The degree and the manner by which visual functions are regained under a drastic reduction in the availability of neural resources can shed light on the boundary conditions of neural plasticity. Critically, the differential remapping and improvement of different visual abilities speak to the key neural mechanisms underlying recovery and plasticity.

One outstanding question in the visual sciences is how different types of visual stimuli are recognized and processed. Humans rely heavily on vision and much of the cerebral cortex in involved with processing visual information. This project examines how adults recognize two classes of visual objects, faces and words, which on the surface seem rather different but may actually share some of the same underlying processes. The work also explores what goes wrong in individuals who have greater than typical difficulties with recognizing faces or words, arising from developmental or congenital problems. To address this, the investigators will explore both the behavioral skills and the brain systems involved in visual recognition of faces and words, in normally skilled individuals and in who have difficulty with face recognition or with word recognition.

This research is highly significant for a number of reasons. First, the research has the potential to contribute to our broad understanding of the brain and the processes that permit normal face and word recognition. Second, new techniques, software and analytical methods for measuring the brain and behavioral processes will be developed, and these new methods will be made available to other scientists and can be applied to studying other cognitive abilities, as well. Third, the studies will provide a better understanding of the processes that are affected when someone has difficulty learning to read or to recognize faces and thus may point to improved techniques for the diagnosis and remediation of such developmental deficits. Fourth, the project offers the opportunity for students to be trained in the field of cognitive neuroscience and to develop skills that will allow them to become independent scientific investigators in future. Finally, the investigators will use the research to teach middle school students about the brain and infuse them with an interest in scientific discovery from an early age.

Summary Lobectomy or, in extreme cases, hemispherectomy, involves the removal of part or all of a cerebral hemisphere as a means of treating certain intractable conditions (e.g. epilepsy). Despite the radical nature of this procedure, the resulting visual deficits can undergo impressive improvement over time, especially when the procedure is conducted at an early age. Surprisingly, rather little exploration has been directed at elucidating the mechanisms supporting this remarkable display of neural plasticity. The goal of this research is to characterize the remapping of visual processing in both cortical and subcortical regions from pre- to post-surgery and longitudinally thereafter, using (and correlating) fine-grained behavioral and neuroimaging methods (univariate and advanced multivariate techniques) in a relatively large group of children who have undergone posterior cortical resection. We first aim to characterize the reorganization of retinotopic cortex, by mapping meridians and by population receptive-field (pRF) modeling. One intriguing hypothesis that motivates this investigation concerns the ability of the intact occipital lobe to develop sensitivity over time to the affected ipsilateral hemifield. Second, at the level of extrastriate cortex, we will characterize the selectivity and topography associated with the recognition of visual categories such as faces, houses, objects and word forms during task and during resting state scans. The ability of a single hemisphere to cope with multiple categories is indicative of important adjustments in the architecture of high-level object recognition and successful recognition may be mediated by a more compact integration of visual representations underlying different categories. Third, we propose to explore functional and structural changes in the profile of subcortical structures (e.g., pulvinar/superior colliculus) as a potential mechanism for the cortical changes, and will examine structural connectivity between early and later cortical regions and subcortical structures using tractography methods. Last, we will examine patterns of recovery as a function of key variables such as age at test and at surgery, presurgical cognitive level and side of lesion. This last factor is particularly relevant for understanding hemispheric dominance, effects of lateralization and postsurgical reorganization. In sum, our investigation constitutes an unprecedented examination of a radical case of cortical remapping at the single case and aggregated group level. The degree and the manner by which there is restitution of visual function following drastic reduction in the available neural resources can shed light on the boundary conditions of plasticity. Understanding lobectomy and its consequences offers unique insights into neural plasticity that can contribute significantly both to a basic science understanding of the visual system and to translational approaches (possible rehabilitation methods and guidelines for surgery).

People recognize faces, words, and other common objects with remarkable speed and accuracy. This project addresses the way that visual knowledge of faces, words, and objects becomes organized in the brain through learning. Prior research has shown that the right cerebral hemisphere is specialized for the representation of faces, whereas the left cerebral hemisphere has a stronger representation for words, with the representation of objects approximately balanced across both hemispheres. These differences in hemispheric specialization are a matter of degree and vary across individuals. Investigation of these differences requires behavioral testing of recognition abilities, combined with neuroimaging of brain structure and brain activation patterns, to understand how the visual recognition system is organized both within and between the two cerebral hemispheres, how this organization emerges with visual experience, and how and why it varies across individuals. In parallel with human behavioral and neuroimaging studies, another facet of this project is the development of a computational simulation of the visual recognition system, using an artificial neural network that learns to recognize faces, words and objects as well as people do. This simulation model is designed to mimic the properties of the brain. Variants of the model can reproduce the behavior and even capture the underlying differences in brain organization of different individuals. This project advances our understanding of the neural basis and underlying brain organization for acquisition of face and word recognition. This research has profound therapeutic implications for millions of Americans who require remediation of developmental disorders like dyslexia, who need assistance to overcome difficulties in letter and word recognition, and remediation for recovery of reading and language abilities after stroke or neurosurgery.

Visual recognition is supported by a network of brain regions in both cerebral hemispheres, that starts with some initial structure in early childhood and then develops through experience to have graded specialization for different types of stimuli. To understand better how recognition occurs, this large-scale study investigates brain structure (e.g., white matter connectivity), brain function (e.g., selectivity of neural activation) and behavior (e.g., with visual stimuli such as faces, words, and objects presented to one hemisphere) over a large group of subjects. This multi-pronged approach allows us to test the prediction that participants with greater within-hemisphere and/or weaker between-hemisphere connectivity will show greater lateralization for faces to the left hemisphere and words to the right hemisphere (but no difference for objects). A second study tests predictions that variability in language lateralization of the brain (across right- and left-handed individuals) explains individual differences in word lateralization, and this, in turn, influences face lateralization. A final study examines fine-grained changes in brain activity as individuals are exposed to, and learn, novel visual object categories, allowing us to test predictions about competition in representing new information compared with known categories (faces and words). In parallel with these studies, this research includes the development of a computational model of the visual recognition system by training a spatially constrained multi-layer artificial neural network to recognize a large number of faces, words, and objects. The model’s performance is evaluated against data of the human studies to determine whether this model can re-create and explain aspects of the system under investigation, including the distinct patterns of different individuals. In identifying the principles underlying the organization of high-level visual information, the work has important implications for knowledge acquisition and learning, both in typical individuals across the lifespan, and in those facing difficulties due to developmental or neuropsychological disorders.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.