Daniel A. Butts - US grants

An understanding of how the brain processes visual stimuli is confounded by the complexity of the natural visual world, combined with the intricate neuronal processing that occurs within and across multiple cortical areas. Over the last few decades, substantial progress has been made by using simple laboratory stimuli, such as moving bars or dots, to develop simple descriptions of neuronal tuning to these elements. This approach has provided a reasonable functional description of lower cortical areas, but it is unlikely to be sufficient to characterize the regions of the extrastriate cortex whose responses are thought to represent elements particular to the natural visual world. Though a joint Canadian-US collaboration, this project couples new experimental approaches based on a set of complex stimuli approaching natural vision with appropriately complex models, in order to understand how neurons in successive stages of cortical processing are tuned to more natural visual features.

Most previous work with natural stimuli has focused on the ventral pathway in the cortex, which is concerned with computing object shape and identity. This is an extremely challenging problem, as the dimensionality of shape space is unknown. This project focuses instead on the dorsal stream of the primate visual cortex, which is primarily identified with motion processing. The advantage of this approach is that motion, particularly that seen in natural vision, can be locally decomposed into a low-dimensional optic flow space that can be sampled using naturalistic stimuli designed for this proposal. The development of such stimuli will extend both the spatial and temporal complexity of probes to areas in the dorsal stream, while providing the necessary constraints for a novel nonlinear modeling framework that will be developed. These models will then be applied to motion stimuli derived through simulation of natural three-dimensional virtual environments, allowing the complex processing uncovered to be linked to natural visual features. Furthermore, by performing this study across successive areas comprising the dorsal hierarchy (V1, MT, and MST), this project aims to expose general principles of cortical processing, namely how higher level abstractions are derived from lower-level visual features.

This project is jointly funded by Collaborative Research in Computational Neuroscience and the OISE Americas program. A companion project is being funded by the Canadian Institutes of Health Research.

The function of sensory neurons is typically defined by the relationship between sensory stimuli and their responses; however, in the cortex of awake animals, sensory responses account for only a fraction of neural activity. While activity not driven by the stimulus is often considered "noise" and neglected in experiments, such ongoing cortical activity has been linked to a number of processes related to cognition, and can be influenced by attention, tasks, and perception itself. It remains unclear how to relate such ongoing cortical activity to the processing of sensory stimuli, and more generally why it appears to play such a prominent role in sensory neuron function.

The goal of this project is to establish a new framework for understanding stimulus processing in the context of ongoing cortical activity, and thereby derive a much richer understanding of sensory neuron function. This work will leverage the wealth of information about activity within the cortical network that is now typically available from multi-electrode recordings, using experiments performed by collaborating laboratories in the awake visual cortex using tailored visual stimuli. The first aim is to develop new statistical approaches for identifying relevant modulatory signals detectable from these multi-electrode recordings, and perform detailed characterizations of stimulus processing in the context of these signals. The second aim is to study specific contexts where cortical activity is shaped by known network inputs, such as during saccadic eye movements, in order to directly link the modulation of stimulus processing to larger descriptions of sensory neuron function.

This work will provide potentially transformative insights into the relationship between sensory processing and cognitive function. The educational component of this proposal will integrate computational and quantitative approaches into general neuroscience coursework, and involve students at the graduate, undergraduate, and high school levels, in computational analyses of complex neurophysiological data.

PROJECT SUMMARY Despite decades of research studying primate primary visual cortex (V1), remarkably little is known about cortical processing in the region representing the central few degrees of the visual field: the fovea. This is particularly notable given that a disproportionately large fraction of V1 is dedicated to foveal processing, and the ?central? role of the fovea in both human visual perception and behavior. Studies of foveal V1 function have been greatly limited by the inability to track an animal's eye position with sufficient accuracy to reconstruct the position of the visual stimulus on the retina. We have recently developed a model-based eye tracking method to address this problem, which can continuously infer eye position with sufficient accuracy to study foveal V1 processing. We thus propose to perform the first detailed studies of stimulus processing in foveal V1. Simply put, it is not known whether the enormous body of work studying V1 outside the fovea is applicable to foveal V1, and whether there are critical elements of cortical function that only might be observed in the fovea. Due to the different dynamics of visual input to the fovea that are driven by eye movements, the different composition of retinal inputs, and the distinct functional roles the fovea plays in visual perception, we expect to identify fundamental differences in foveal V1 function. We will perform multi-electrode recordings across eccentricity in awake macaque, and target analyses to three aspects of visual processing expected to be different: (1) tuning for spatiotemporal elements of vision; (2) tuning to binocular disparity; and (3) color processing. We expect that this proposed study will reveal substantial differences in the stimulus processing of foveal neurons, potentially transforming our current understanding of visual processing in the cortex. Health Relevance. Understanding the unique functional role of foveal V1 in visual processing will provide critical insight into how human visual perception is constructed, and will likewise inform our understanding of the effects of many disorders that impact central vision. For example, age-related macular degeneration (AMD) is a leading cause of vision loss of Americans 60 and older. Patients with advanced AMD retain their peripheral vision, but lose vision in the fovea, resulting in severe difficulties with a wide range of important tasks requiring high-acuity vision, including reading, writing, driving, and object/facial recognition. Likewise, a range of disorders such as nystagmus can disrupt normal eye movements and fixations, thereby disproportionally affecting foveal vision. A better understanding of neural function subserving foveal vision could yield insights into their effects on visual perception and potentially facilitate the development of novel therapeutic strategies.

An urgent issue facing today's researchers in the life sciences is coping with the data explosion resulting from the advent of powerful new technologies. More data does not yield better information without the interdisciplinary tools required for such a transformation. This National Science Foundation Research Traineeship (NRT) award to the University of Maryland, College Park will build an innovative, cross-disciplinary model for graduate education that addresses this challenge by preparing students to pursue a range of STEM careers at the nexus of the computer, physical, and life sciences. Trainees will learn to combine physics-style quantitative modeling with data processing, analysis, and visualization methods from computer science to gain deeper insights into the principles governing living systems. The project anticipates training approximately sixty (60) PhD students, including thirty-five (35) funded trainees, from the physical, computer, and life sciences.

Understanding how data-derived interaction patterns can give insights into complex biological phenomena is the research focus of this program. Through an innovative combination of cross-disciplinary training, collaborative research, and outreach activities, NRT trainees will become experts in the process of transforming raw biological data into useful information from which new biological insights can be inferred. Participants will receive training in four different areas of network analysis: quantitative metrics for biological networks; mechanistic models of biological networks; network statistics and machine learning for biological applications; and visualization techniques for large, complex, biological datasets. This training will provide the foundation for research in one or more of three application areas, covering a wide range of biological scales: biomolecular networks; neuronal networks; and ecological/behavioral networks. Research experiences, interdisciplinary coursework, peer-to-peer tutorials, and internships with partners will provide graduate students with the skills needed to communicate complex scientific ideas to diverse audiences in order to maximize impact. Outreach activities will extend the benefits of the program to undergraduates, middle/high school students, and to the public at large.

The NSF Research Traineeship (NRT) Program is designed to encourage the development and implementation of bold, new potentially transformative models for STEM graduate education training. The Traineeship Track is dedicated to effective training of STEM graduate students in high priority interdisciplinary research areas, through the comprehensive traineeship model that is innovative, evidence-based, and aligned with changing workforce and research needs.

Vision is a process by which the image falling on the eyes is processed by specialized neurons within visual brain areas. Neurons in the early stages of visual processing convey information about small bits of the visual scene, like pixel-detectors in a camera. For example, a neuron in visual cortex might respond best to a small white bar at a particular location in visual space. Should this example neuron respond differently when the white bar is part of an object that we have seen before, or one that we are moving towards? Psychology might suggest so, but for almost 60 years, most scientists studying the neural basis of visual perception have implicitly assumed that responses of neurons in visual cortex depend only on the visual image falling on the eyes. It is increasingly clear that neurons in the visual cortex do indeed care about behavioral context – as well as the state of the brain itself. These external, internal, and contextual factors influence how neurons process the visual scene. Exactly how much these “non-visual” factors influence visual cortical neurons remains a significant open question that this project aims to address.

The experiments will record from neurons in the visual cortex of ferrets as they freely explore a naturalistic environment. Using position and eye-tracking cameras, the project will both recreate a movie of what the ferret saw within the environment, and track other observable variables related to behavior. The movie will then be replayed to the ferret while it is anesthetized, thus directly measuring any differences in neuronal responses to the same visual stimulation in these two very different contexts. Analysis will compare the physiological quality and statistical properties of neuronal responses across naturalistic and anesthetized conditions to quantify the contribution of natural context to neuronal responses. Results will relate the differences in the freely moving context to specific sources, like motor actions such as eye and head movements, familiarity with specific visual features, and their behavioral relevance. Experiments will inform models for how these sources influence neuronal activity, setting the stage for understanding the function of non-retinal inputs for sensory perception. The project will provide a foundation for long-term studies of natural vision.
This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NCS), a multidisciplinary program jointly supported by the Directorates for Biology (BIO), Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Color and form are often treated as separable features of an image. One can recognize shapes in achromatic photographs and conceptualize the color of an object abstracted from shape. Yet color-specific processing is embedded throughout the visual pathway from the first stage of the visual pathway, where three different types of light sensors (“cones”) with sensitivity to different parts of the visual spectrum initially convert light into electrical impulses. The color of a given point can in principle be determined by comparing the activation of the three different cone types, but the separate color channels are maintained until the primary visual cortex (V1), where they are finally combined in neurons that concurrently have sensitivity to different spatial patterns. Indeed, while it was initially thought that color and form were processed through separate pathways within V1, recent experiments have highlighted that a surprising fraction of V1 neurons mix them together in a diversity of ways. Exactly how the mixing occurs, and for what purpose, are critical open questions in understanding human vision, and have been difficult to answer because such mixing is too complicated to characterize using traditional approaches. This project combines large-scale recording of V1 neural activity during tailored “spatio-chromatic” visual stimulation with new computational approaches that offer an unprecedented high-resolution description of color processing within V1 while allowing determination of the underlying function of spatio-chromatic mixing in supporting natural color vision. The project also provides opportunity for cross-disciplinary training in neurophysiological and machine-learning based statistical modeling of undergraduate and graduate students.

This project is a tight combination of visual neurophysiology, data-driven computational modeling, and simulation. The investigators perform large-scale multi-electrode recordings across cortical lamina to determine the transformations of spatio-chromatic representations from cortical inputs (where color channels are separate) to cortical outputs (where they are mixed). These recordings are interpreted using nonlinear data-driven models that can provide high-resolution spatio-chromatic maps of the stimuli driving each V1 neuron, and distinguish the underlying computations being performed at each stage. Such characterizations are pushed to achieve cone-resolution by leveraging novel model-based eye-tracking that can account for small eye movements with an order-of-magnitude finer sensitivity than standard approaches. The first Aim determines the set of principles governing how spatial and chromatic information is combined in V1, which sets the foundation for processing throughout the visual pathway. The second Aim determines whether these rules are the same in the area of cortex processing the center-of-gaze (fovea), which is responsible for high-acuity color vision. Finally, the last Aim establishes a population decoding framework for linking spatio-chromatic sensitivity of individual V1 cells to the larger systems-wide goals of the visual cortex in processing natural color vision.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.