Eero P. Simoncelli - US grants

DESCRIPTION (provided by applicant): The goal of this research is to discover the cortical computations embodied in the responses of neurons in ventral visual area V2. Since V2 is heavily dependent on input from V1 for its visual responsiveness, we will develop a two-stage model, in which responses are constructed from a suitable combination of V1 afferents, with the design of each stage following a common canonical form. This model is intended to account for the visual response properties of neurons as economically as possible, while not necessarily reflecting the details of neuronal circuitry. This simplicity is deliberate, as it will allow the mdel to be fit to data recorded from single neurons, and, when assembled into a population, to predict perceptual capabilities. The motivation for the structure of the model, and our confidence in its success, comes from the convergence of three strands of previous work: (1) we have developed, fit, and validated a similar two-stage model for neuronal responses in area MT, a dorsal stream area which also receives primary afferent drive from area V1; (2) we have developed a two-stage model for visual texture representation that captures perceptually recognizable structures of natural images using spatial integration regions matched in size to those of V2 cells. We've shown that images synthesized to have matching model responses are indistinguishable to human observers; and (3) we've obtained preliminary data indicating that most of V2 cells respond more vigorously to synthetic texture stimuli than to spectrally matched noise stimuli, whereas V1 cells do not. The research is divided into three parts. First, we will gather electrophysiological data to dissect those model-generated features that underlie the increased responsiveness of V2 to texture stimuli. We will, in parallel, gather evidence for the increased responsiveness using fMRI, which will allow us to compare simultaneously measured responses averaged over neural populations in V1 and V2. Second, we will develop a physiologically plausible instantiation of the texture model and develop the methods to fit it to data from single neurons. Finally, we will link the novel functional response properties we have discovered to perception by simultaneously measuring neuronal responses and perceptual judgments in awake behaving macaques. To explore sensitivity to naturalistic features, we will relate psychometric and single-neuron neurometric functions, and use choice probability to link responses to behavioral performance.

Vision begins in the retina, where light is converted into electrical signals, processed to extract and compress visual information, and transmitted through the optic nerve to the brain. Despite decades of research, a full understanding of these transformations remains incomplete. In particular, most studies have documented specific properties of the responses of single retinal cells in isolation, using specialized artificial visual stimuli. The research performed under this grant aims to develop a full, unified computational model of retinal processing, including spatial and temporal filtering, nonlinear transformations, and adaptation to local luminance and contrast, in complete populations of neurons. The model will be tested by comparing its predictions to data from large-scale multi-electrode recordings of primate retinal ganglion cells (RGCs), verifying that it can mimic known retinal responses, and critically, testing its ability to explain responses to natural visual images, including the effects of fixational and saccadic eye movements. The resulting model will provide a compact encapsulation of the "neural code" of the retina, which will serve as a substrate for understanding all subsequent visual processing in the brain. In addition, the model will provide an essential component in the development of high-acuity retinal prostheses for people blinded by diseases of photoreceptor degeneration. Finally, the model will offer a useful tool for the development and testing of new display technologies.

The research has two main aims: (1) Develop and test a model of nonlinear subunits in RGC populations-- No current model captures the effects of nonlinear computations in a complete sensory neural circuit. The researchers will develop a model incorporating nonlinear subunits that captures the stimulus encoding properties of complete populations of RGCs at the resolution of photoreceptors, and will quantify the implications of these nonlinearities for encoding naturally-occurring visual stimuli. The researchers will develop methods to reliably fit the model to RGC responses to targeted stimuli that stringently constrain model structure, and verify model predictions in closed-loop experiments. (2) Incorporate adaptation; test model with targeted and naturalistic stimuli-- RGC responses adapt to luminance and stimulus contrast. No current model of the RGC population response incorporates adaptation with subunit nonlinearities, natural scenes, and eye movements. The researchers will incorporate adaptation in the model, fit the adaptive model using stochastic stimuli with varying mean and contrast, and test the model using stimuli that produce adaptation within and across subunits.