Ruben Coen-Cagli - US grants

Project Summary Signals from the natural environment are processed by neuronal populations in the cortex. Understanding the relationship between those signals and cortical activity is central to understanding normal cortical function and how it is impaired in psychiatric and neurodevelopmental disorders. Substantial progress has been made in elucidating cortical processing of simple, parametric stimuli, and computational technology is improving descriptions of neural responses to naturalistic stimuli. However, how cortical populations encode the complex, natural inputs received during every day perceptual experience is largely unknown. This project aims to elucidate how natural visual inputs are represented by neuronal populations in primary visual cortex (V1). Progress to date has been limited primarily by two factors. First, during natural vision, the inputs to V1 neurons are always embedded in a spatial and temporal context, but how V1 integrates this contextual information in natural visual inputs is poorly understood. Second, prior work focused almost exclusively on single-neuron firing rate, but to understand cortical representations one must consider the structure of population activity? the substantial trial-to-trial variability that is shared among neurons and evolves dynamically?as this structure influences population information and perception. The central hypothesis of this project is that cortical response structure is modulated by visual context to approximate an optimal representation of natural visual inputs. To test the hypothesis, this project combines machine learning to quantify the statistical properties of natural visual inputs, with a theory of how cortical populations should encode those images to achieve an optimal representation, to arrive at concrete, falsifiable predictions for V1 response structure. The predictions will be tested with measurements of population activity in V1 of awake monkeys viewing natural images and movies. Specific Aim 1 will determine whether modulation of V1 response structure by spatial context in static images is consistent with optimal encoding of those images, and will compare the predictive power of the proposed model to alternative models. Specific Aim 2 addresses V1 encoding of dynamic natural inputs, and will test whether modulation of V1 activity by temporal context is tuned to the temporal structure of natural sensory signals, as required for optimality. As both spatial and temporal are present simultaneously during natural vision, Specific Aim 3 will determine visual input statistics in free-viewing animals, and test space-time interactions in V1 activity evoked by those inputs. This project will provide the first test of a unified functional theory of contextual modulation in V1 encoding of natural visual inputs, and shed light on key aspects of natural vision that have been neglected to date.

To understand and navigate the environment, sensory systems must solve simultaneously two competing and challenging tasks: the segmentation of a sensory scene into individual objects and the grouping of elementary sensory features to build these objects. Understanding perceptual grouping and segmentation is therefore a major goal of sensory neuroscience, and it is central to advancing artificial perceptual systems that can help restore impaired vision. To make progress in understanding image segmentation and improving algorithms, this project combines two key components. First, a new experimental paradigm that allows for well-controlled measurements of perceptual segmentation of natural images. This addresses a major limitation of existing data that are either restricted to artificial stimuli, or, for natural images, rely on manual labeling and conflate perceptual, motor, and cognitive factors. Second, this project involves developing and testing a computational framework that accommodates bottom-up information about image statistics and top-down information about objects and behavioral goals. This is in contrast with the paradigmatic view of visual processing as a feedforward cascade of feature detectors, that has long dominated computer vision algorithms and our understanding of visual processing. The proposed approach builds instead on the influential theory that perception requires probabilistic inference to extract meaning from ambiguous sensory inputs. Segmentation is a prime example of inference on ambiguous inputs: the pixels of an image often cannot be labeled with certainty as grouped or segmented. This project will test the hypothesis that human visual segmentation is a process of hierarchical probabilistic inference. Specific Aim 1 will determine whether the measured variability of human segmentations reflects the uncertainty predicted by the model, as required for well-calibrated probabilistic inference. Specific Aim 2 addresses how feedforward and feedback processing in human segmentation contribute to efficient integration of visual features across different levels of complexity, from small contours to object parts. Specific Aim 3 will determine reciprocal interactions between perceptual segmentation and top-down influences including: semantic scene content; visual texture discrimination; and expectations reflecting environmental statistics. The proposed approach models these influences as Bayesian priors, and thus, if supported by the proposed experiments, will offer a unified framework to understand the integration of bottom-up and top- down influences in human segmentation of natural inputs. RELEVANCE (See instructions): This project aims to provide a unified understanding of perceptual segmentation and grouping of visual inputs encountered in the natural environment, through correct integration of the information contained in the visual inputs with top-down information about objects and behavioral goals. This understanding is central to advancing artificial perceptual systems that can help restore impaired vision in patient populations.