Daniel A. Butts - US grants

An understanding of how the brain processes visual stimuli is confounded by the complexity of the natural visual world, combined with the intricate neuronal processing that occurs within and across multiple cortical areas. Over the last few decades, substantial progress has been made by using simple laboratory stimuli, such as moving bars or dots, to develop simple descriptions of neuronal tuning to these elements. This approach has provided a reasonable functional description of lower cortical areas, but it is unlikely to be sufficient to characterize the regions of the extrastriate cortex whose responses are thought to represent elements particular to the natural visual world. Though a joint Canadian-US collaboration, this project couples new experimental approaches based on a set of complex stimuli approaching natural vision with appropriately complex models, in order to understand how neurons in successive stages of cortical processing are tuned to more natural visual features.

Most previous work with natural stimuli has focused on the ventral pathway in the cortex, which is concerned with computing object shape and identity. This is an extremely challenging problem, as the dimensionality of shape space is unknown. This project focuses instead on the dorsal stream of the primate visual cortex, which is primarily identified with motion processing. The advantage of this approach is that motion, particularly that seen in natural vision, can be locally decomposed into a low-dimensional optic flow space that can be sampled using naturalistic stimuli designed for this proposal. The development of such stimuli will extend both the spatial and temporal complexity of probes to areas in the dorsal stream, while providing the necessary constraints for a novel nonlinear modeling framework that will be developed. These models will then be applied to motion stimuli derived through simulation of natural three-dimensional virtual environments, allowing the complex processing uncovered to be linked to natural visual features. Furthermore, by performing this study across successive areas comprising the dorsal hierarchy (V1, MT, and MST), this project aims to expose general principles of cortical processing, namely how higher level abstractions are derived from lower-level visual features.

This project is jointly funded by Collaborative Research in Computational Neuroscience and the OISE Americas program. A companion project is being funded by the Canadian Institutes of Health Research.

PROJECT SUMMARY Despite decades of research studying primate primary visual cortex (V1), remarkably little is known about cortical processing in the region representing the central few degrees of the visual field: the fovea. This is particularly notable given that a disproportionately large fraction of V1 is dedicated to foveal processing, and the ?central? role of the fovea in both human visual perception and behavior. Studies of foveal V1 function have been greatly limited by the inability to track an animal's eye position with sufficient accuracy to reconstruct the position of the visual stimulus on the retina. We have recently developed a model-based eye tracking method to address this problem, which can continuously infer eye position with sufficient accuracy to study foveal V1 processing. We thus propose to perform the first detailed studies of stimulus processing in foveal V1. Simply put, it is not known whether the enormous body of work studying V1 outside the fovea is applicable to foveal V1, and whether there are critical elements of cortical function that only might be observed in the fovea. Due to the different dynamics of visual input to the fovea that are driven by eye movements, the different composition of retinal inputs, and the distinct functional roles the fovea plays in visual perception, we expect to identify fundamental differences in foveal V1 function. We will perform multi-electrode recordings across eccentricity in awake macaque, and target analyses to three aspects of visual processing expected to be different: (1) tuning for spatiotemporal elements of vision; (2) tuning to binocular disparity; and (3) color processing. We expect that this proposed study will reveal substantial differences in the stimulus processing of foveal neurons, potentially transforming our current understanding of visual processing in the cortex. Health Relevance. Understanding the unique functional role of foveal V1 in visual processing will provide critical insight into how human visual perception is constructed, and will likewise inform our understanding of the effects of many disorders that impact central vision. For example, age-related macular degeneration (AMD) is a leading cause of vision loss of Americans 60 and older. Patients with advanced AMD retain their peripheral vision, but lose vision in the fovea, resulting in severe difficulties with a wide range of important tasks requiring high-acuity vision, including reading, writing, driving, and object/facial recognition. Likewise, a range of disorders such as nystagmus can disrupt normal eye movements and fixations, thereby disproportionally affecting foveal vision. A better understanding of neural function subserving foveal vision could yield insights into their effects on visual perception and potentially facilitate the development of novel therapeutic strategies.

Vision is a process by which the image falling on the eyes is processed by specialized neurons within visual brain areas. Neurons in the early stages of visual processing convey information about small bits of the visual scene, like pixel-detectors in a camera. For example, a neuron in visual cortex might respond best to a small white bar at a particular location in visual space. Should this example neuron respond differently when the white bar is part of an object that we have seen before, or one that we are moving towards? Psychology might suggest so, but for almost 60 years, most scientists studying the neural basis of visual perception have implicitly assumed that responses of neurons in visual cortex depend only on the visual image falling on the eyes. It is increasingly clear that neurons in the visual cortex do indeed care about behavioral context – as well as the state of the brain itself. These external, internal, and contextual factors influence how neurons process the visual scene. Exactly how much these “non-visual” factors influence visual cortical neurons remains a significant open question that this project aims to address.

The experiments will record from neurons in the visual cortex of ferrets as they freely explore a naturalistic environment. Using position and eye-tracking cameras, the project will both recreate a movie of what the ferret saw within the environment, and track other observable variables related to behavior. The movie will then be replayed to the ferret while it is anesthetized, thus directly measuring any differences in neuronal responses to the same visual stimulation in these two very different contexts. Analysis will compare the physiological quality and statistical properties of neuronal responses across naturalistic and anesthetized conditions to quantify the contribution of natural context to neuronal responses. Results will relate the differences in the freely moving context to specific sources, like motor actions such as eye and head movements, familiarity with specific visual features, and their behavioral relevance. Experiments will inform models for how these sources influence neuronal activity, setting the stage for understanding the function of non-retinal inputs for sensory perception. The project will provide a foundation for long-term studies of natural vision.
This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NCS), a multidisciplinary program jointly supported by the Directorates for Biology (BIO), Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.