C Edward Connor - US grants

stimulus /response; attention; visual fields; Anthropoidea;

Shape processing is an essential visual function that underlies our ability to identify familiar items, interpret facial expressions, and comprehend written language. Understanding the neural mechanisms behind shape processing would constitute a key insight into visual perception. Previous studies have shown that shape is represented at the earliest levels of visual cortex (V1 and V2) in terms of oriented edges, and that cells at the highest levels (CIT and AIT) are selective for behaviorally relevant 3- dimensional (3D) stimuli like hands and faces. Little is known about the intervening stages where information about oriented edges is transformed into selectivity for complex 3D objects. The long term objective of this proposal is to fill this gap by investigating the neural mechanisms of shape processing that lead from oriented edges in 3D objects. The proposed approach to this issue is based on prior experiments in which we tested the responses of macaque area V4 cells to a large set of gradually varying 2D shape features. The systematic tuning profiles observed int these experiments suggest that shape is represented at intermediate levels in terms of simple contour features like angled corners and curve segments. The proposed experiments would distinguish between two hypotheses: (a) That these 2D contour features are indeed the essential shape primitives extracted at intermediate stages, to be combined later at higher stages into complex 3D percepts, and (b) That the observed 2D tuning profiles are actually cross-sections through more specific 3D tuning volumes, so that the true shape primitives are not (for example) 2D angles and curve segments but 3D corners and rounded volumes. These two hypotheses will be explored by extending the shape features stimulus set into the 3rd dimension, to test whether cells respond consistently to a given 2D profile (regardless of 3D information) or instead show more selective tuning for a specific range of 3D shape features. These tests will be carried out at intermediate and higher level stages in the object processing pathway (V2, V4, PIT and CIT) to determine how and what point 3D shape information is generated.

DESCRIPTION (provided by applicant): Our ability to understand and interact with our visual environment depends critically on brain mechanisms for transforming the eye image into knowledge about objects. Understanding this transformation would provide a rational basis for designing future generations of prosthetic neural implants for blind subjects that could deliver rich and useful object information. It would also impact clinical approaches to higher-order perceptual impairments produced by stroke or genetic disorders like autism. The neural transformation of object information can be studied at a detailed, mechanistic level by means of electrode recording from individual neurons in the brains of non-human primates, in which the organization of the object-processing pathway is similar to humans. Previous studies have focused separately on different levels of object information at early, intermediate, or late processing stages in this pathway. The long-term goal of this study is to discover how each of these levels is transformed into the next by synthesizing signals for simple image elements into increasingly complex and abstract representations. The novel approach proposed for this funding period is to study these different levels of object information simultaneously in order to infer transformation mechanisms. This requires searching an unprecedented range of stimuli, which can be accomplished by using an evolutionary search strategy. Stimulus evolution is controlled by a genetic algorithm for creating mutated descendants from stimuli that evoke strong responses from the neuron being studied. This moves the experiment quickly toward the region of interest in stimulus space. We will use this strategy to explore (1) how simple orientation/frequency signals are synthesized to produce neural selectivity for complex contour geometry, (2) how geometric part signals are synthesized to produce neural selectivity for familiar objects like faces and bodies, and (3) how geometric shape information is transformed into online knowledge about object characteristics. These experiments will help link previous results into a more coherent picture of the multi-stage transformation from eye images to object knowledge. RELEVANCE TO PUBLIC HEALTH: This study will help explain how the brain transforms eye image information into knowledge about visual objects. The results are relevant to the design of prosthetic neural implants for blind subjects and to understanding visual impairments due to stroke or genetic disorders.

DESCRIPTION (provided by applicant): Long-term Objective: To discover how higher-level visual cortex is specialized to exploit the shape statistics of our natural environment. Natural objects have a very specific statistical structure imposed by conditions in our world: gravity, lighting, physics, biology, architecture, and standard observer viewpoints. The visual system is highly specialized to take advantage of natural shape statistics and focus on the most useful and available shape information. This specialization helps make biological vision far superior to current computer-based vision systems. Understanding this specialization will shed new light on neural mechanisms of human object vision and could suggest new strategies for computer-based visual recognition. Specific Aims: (1) Use high-throughput computer-based photographic image analysis to characterize shape statistics of natural objects, (2) Measure the distribution of tuning for the same quantitative shape measures in neurons recorded from high-level ventral pathway visual cortex of awake macaque monkeys, (3) Compare the resulting image statistics and neural tuning distributions in order to discover how high-level visual cortex is adapted to natural image structure. Public Health Relevance: This novel analysis of brain specialization for natural shape statistics provides a fresh approach to understanding the neural mechanisms of human object vision. Understanding these mechanisms will elucidate disease states in which visual object perception is compromised, suggest new strategies for designing computer-based vision systems that could compensate for loss of object vision, and could ultimately provide the critical knowledge base for designing and optimizing microelectrode-based devices to provide prosthetic sensory inputs to higher-level visual cortex.

DESCRIPTION (provided by applicant): Our ability to live within and interact with a world composed of 3D objects depends largely on our spectacular capacity for visual shape perception. This is what makes vision so critical to our health, happiness, and survival. The long-term goal of this project is to understand 3D object perception by discovering the neural code for complex 3D shape in the primate ventral visual pathway. After decades in which neurophysiological studies of object representation in the monkey ventral pathway have focused exclusively on 2D shape, recent reports indicate a robust representation of 3D shape, although the nature of that representation remains completely unknown. We will address this issue using the same techniques we have recently applied to produce the first quantitative descriptions of complex 2D shape representation. We will combine dense, parametric exploration of 3D shape space with intensive computational analysis to test hypotheses about 3D shape coding dimensions, tuning functions, integration mechanisms, and population coding principles. The stimuli will be complex, smooth (spline-based), abstract, randomly generated 3D shapes. Successive generations of random shape stimuli will be determined with a genetic algorithm, using neural responses as feedback to guide sampling toward the most relevant regions of 3D shape space. The resulting data will be used to test hypotheses about coding dimensions relating to 2D boundary contours, 3D surface patches, and 3D medial axis shape, all described in terms of absolute and relative position, 2D and 3D orientation, 2D and 3D curvature, curvature orientation, and curvature derivative. We will test tuning functions ranging from simple Gaussians to complex manifolds describing highly specific part shapes. We will test a variety of mechanisms for integrating information across object parts, ranging from single-part tuning through multi-part tuning to holistic tuning for overall object shape. The hypotheses surviving from these individual cell analyses will then be tested at the population coding level.

DESCRIPTION (provided by applicant): Long-term Objective: To discover how higher-level visual cortex is specialized to exploit the shape statistics of our natural environment. Natural objects have a very specific statistical structure imposed by conditions in our world: gravity, lighting, physics, biology, architecture, and standard observer viewpoints. The visual system is highly specialized to take advantage of natural shape statistics and focus on the most useful and available shape information. This specialization helps make biological vision far superior to current computer-based vision systems. Understanding this specialization will shed new light on neural mechanisms of human object vision and could suggest new strategies for computer-based visual recognition. Specific Aims: (1) Use high-throughput computer-based photographic image analysis to characterize shape statistics of natural objects, (2) Measure the distribution of tuning for the same quantitative shape measures in neurons recorded from high-level ventral pathway visual cortex of awake macaque monkeys, (3) Compare the resulting image statistics and neural tuning distributions in order to discover how high-level visual cortex is adapted to natural image structure. Public Health Relevance: This novel analysis of brain specialization for natural shape statistics provides a fresh approach to understanding the neural mechanisms of human object vision. Understanding these mechanisms will elucidate disease states in which visual object perception is compromised, suggest new strategies for designing computer-based vision systems that could compensate for loss of object vision, and could ultimately provide the critical knowledge base for designing and optimizing microelectrode-based devices to provide prosthetic sensory inputs to higher-level visual cortex.

DESCRIPTION (provided by applicant): Our world is composed of 3D objects, and successful interaction with that world depends on neural processing of 3D object information. This is what makes vision so critical to our health, happiness and survival. Our long- term goal is to understand how complex 3D object information is processed in perception, memory, and cognition. Understanding these issues at a neural level will impact clinical approaches to visual agnosias, altered visual processing in neurological conditions like autism, and altered memory and decision functions in neurological conditions like Alzheimer's disease. We recently developed a novel adaptive sampling strategy for neural recording experiments, in which tests of object shape responses gradually adapt based on neural feedback. This highly efficient sampling strategy allows us to measure the specific object information signaled by neurons, which was not possible with previous experimental strategies. We now plan to leverage this approach to investigate the neural basis of 3D object perception, memory, and cognition, by measuring neural responses in inferotemporal visual cortex (IT), memory-related perirhinal and entorhinal cortex (PR and ER), and decision-related dorsolateral prefrontal cortex (PFC) of monkeys performing visual memory and discrimination tasks. These areas are the homologues of high-level object vision, memory, and decision areas in the human brain; only in monkeys can they be studied at the neural coding level. The unique aspect of our experiments is the use of adaptive sampling to identify the specific information signaled by individual neural responses, a critical element missing from previous research in this area. We will use this approach to address three specific questions: (1) Perception: Are 3D objects represented in terms of their medial axis shapes? This is a long-standing theory about object representation in the brain that has never been directly tested. (2) Memory: How are memory associations between 3D viewpoints formed in IT and PR/ER? Associating different views of the same object is the most computationally difficult aspect of vision. We will perform the first direct test of the prevailing theory that viewpoint association is learned through exposure to rotating objects during natural vision. (3) Cognition: How does 3D shape information in IT and PFC relate to behavioral decisions? The ultimate utility of 3D object perception is its use in guiding decision and behavior. Our experiment will be the first attempt to show how neural coding of complex 3D shape is related to object discrimination behavior.

DESCRIPTION (provided by applicant): The LONG-TERM GOAL of this project is to understand how the brain processes visual information about places-landscapes and interiors-and how that compares to brain processing of visual information about objects. We will measure object- and place-related responses of neurons in higher processing stages within two major brain pathways: (1) The ventral visual pathway is traditionally considered to be an object pathway, but our preliminary data show that it is also strongly engaged in representation of landscapes and interiors, with a dorsolateral-ventromedial gradient from object-related processing to place-related processing. (2) The recently recognized parieto-medial temporal pathway primarily processes place information and terminates in parahippocampal cortex. We will use mathematical analysis to understand how object and place information is encoded by neurons in these two pathways. We expect the results to have the following scientific impacts: 1. This first analysis of complex place shape is encoded at the level of individual neurons and groups of neurons will deepen understanding of brain mechanisms for perceiving, understanding, interacting with, and navigating through landscapes and interiors, complementing previous research at the brain imaging level. 2. Analytical comparison will elucidate how coding mechanisms are specialized for the very different structural characteristics and ecological roles of objects and places. 3. Analytical comparison of shape coding mechanisms between the ventral and parieto-medial pathways will increase understanding of how different visual functions are distributed across the brain. 4. Combined study of object and place coding will lead to future research on how these two elements combine in overall perception of visual scenes.

DESCRIPTION (provided by applicant): In this application we investigate how to provide realistic sensory feedback to subjects with upper limb prosthetics. Thousands of soldiers are returning from the war in need of prosthetic limbs that not only can move but also allows them to seamlessly interact with objects. The optimal approach for providing feedback is to tap directly into the neural circuits that underlie tactile perception. Recent studies have shown that areas 3b, 1 and 2 of SI cortex play different roles in processing cutaneous information, with neurons in area 3b responsible for coding orientation, area 1 for motion and area 2 for curvature. In this application we will investigate how to provide sensory feedback by directly stimulation neurons in these cortical areas in animals trained to perform tactile match-to-sample discrimination tasks. There are two specific aims. The first is to investigate how to provide sensory feedback of intensity, orientation, motion and curvature on a single finger pad. After training non-human primates (macaca mulatta) to perform the feature discrimination task, a chronic recording array will be implanted into somatosensory cortex. The goal of the recordings is to simultaneously record from many neurons that are tuned to the feature being studied. The animal will then perform the task under three conditions. 1) Matching a mechanical stimulus with a mechanical stimulus (this is the condition that the animal was trained to perform) 2) Matching a mechanical stimulus with a mechanical and electrical stimulus and 3) Matching a mechanical stimulus with an electrical stimulus. The electrical stimuli will be given to populations of feature tuned cells with the aim of changing the population response of the neurons. This stimulus will be modified in several ways 1) change in current intensity, 2) change in frequency of pulses 3) change in statistical properties of the firing (i.e., biomimetic, uniform, Poisson) and change the degree of correlated firing between cells being modulated. We will develop method to systematically alter the animal's behavior and to determine whether we can produce a population response that mimics the response of the tactile stimulation. The second aim is to train animals to perform a tactile one-back task. This task will determine how to stimulate cortex so that the stimuli can be delivered with minimal spatial and temporal smearing of the electrical signals across fingers. The results from the two aims will provide and understanding of how to provide sensory feedback of complex features to the hand.

? DESCRIPTION (provided by applicant): Learning to discriminate new shapes is a fundamental visual ability for humans and other primates. It depends on long-term changes in shape computations in the ventral pathway of primate visual cortex, especially at final stages in IT (inferotemporal cortex). Our goal is to investigate these changes at the level of individual neurons and neural circuits, by (i) analyzing progressive shape computation changes in continuously identified neural populations across long timescales (weeks to months) and (ii) correlating these changes with improvements in shape discrimination accuracy and speed. We would achieve this goal by combining methodologies developed in our two laboratories. The Connor lab has developed mathematical analyses of neural shape computations, based on large-scale adaptive stimulus sampling guided by genetic algorithms and multi- dimensional parameterization of stimulus geometry. The Leopold lab has developed the use of microwire bundle implants for long-term electrophysiological recording from populations of IT neurons, continuously identified by their signature response patterns across 100s of stimuli. Adaptive sampling can leverage the order of magnitude increase in sampling time with microwire bundles, offering a new paradigm for high- throughput testing of mathematically tractable object stimuli in ventral pathway cortex. Based on our previous investigations of shape coding and shape processing dynamics, we hypothesize that learning to discriminate a new shape accurately and rapidly is based on a progression through distinct combinatorial computations operating on that shape's constituent fragments: (i) Initial low-accuracy behavior reflects linear combination of shape fragment signals, present in the untrained state, yielding only ambiguous information about complex shape configurations; (ii) Increasing accuracy during early learning reflects recurrent network nonlinear computations, yielding slow but unambiguous signals for shape fragment combinations; (iii) Increasing speed during late learning reflects feed-forward nonlinear computations, yielding accurate, fast performance. Chronic microwire recording will allow us to track this computational progression, for dozens of individual neurons, and correlate computational changes with behavioral improvements through time. This would be the first continuous observation of computational changes in individual IT cells during extended periods of visual learning (weeks to months). Whether or not the specific hypotheses are verified, this will provide the most direct insights to date into how specific changes in IT circuit-level information processing relate to shape learning, which is critical to our understanding of symbols and objects.

? DESCRIPTION (provided by applicant): Learning to discriminate new shapes is a fundamental visual ability for humans and other primates. It depends on long-term changes in shape computations in the ventral pathway of primate visual cortex, especially at final stages in IT (inferotemporal cortex). Our goal is to investigate these changes at the level of individual neurons and neural circuits, by (i) analyzing progressive shape computation changes in continuously identified neural populations across long timescales (weeks to months) and (ii) correlating these changes with improvements in shape discrimination accuracy and speed. We would achieve this goal by combining methodologies developed in our two laboratories. The Connor lab has developed mathematical analyses of neural shape computations, based on large-scale adaptive stimulus sampling guided by genetic algorithms and multi- dimensional parameterization of stimulus geometry. The Leopold lab has developed the use of microwire bundle implants for long-term electrophysiological recording from populations of IT neurons, continuously identified by their signature response patterns across 100s of stimuli. Adaptive sampling can leverage the order of magnitude increase in sampling time with microwire bundles, offering a new paradigm for high- throughput testing of mathematically tractable object stimuli in ventral pathway cortex. Based on our previous investigations of shape coding and shape processing dynamics, we hypothesize that learning to discriminate a new shape accurately and rapidly is based on a progression through distinct combinatorial computations operating on that shape's constituent fragments: (i) Initial low-accuracy behavior reflects linear combination of shape fragment signals, present in the untrained state, yielding only ambiguous information about complex shape configurations; (ii) Increasing accuracy during early learning reflects recurrent network nonlinear computations, yielding slow but unambiguous signals for shape fragment combinations; (iii) Increasing speed during late learning reflects feed-forward nonlinear computations, yielding accurate, fast performance. Chronic microwire recording will allow us to track this computational progression, for dozens of individual neurons, and correlate computational changes with behavioral improvements through time. This would be the first continuous observation of computational changes in individual IT cells during extended periods of visual learning (weeks to months). Whether or not the specific hypotheses are verified, this will provide the most direct insights to date into how specific changes in IT circuit-level information processing relate to shape learning, which is critical to our understanding of symbols and objects.