Michael A. Paradiso - US grants

DESCRIPTION: Research on visual processing in the past thirty years has revealed that neurons are usually best driven by patterns of luminance contrast rather than luminance itself. Progressing from primary visual cortex to the numerous visual areas that stretch toward the parietal and temporal lobes, optimal stimuli tend to involve more complex arrangements of luminance contrast as receptive fields increase in size. The responses of visual neurons to patterns of luminance contrast suggest that we see visual form by the pattern of activity across a great many neurons sensitive to patterns of luminance contrast. A great mystery that remains is how we see anything other than form, specifically, the surface properties of objects such as their brightness, color, and texture. A large number of psychophysical experiments have demonstrated that the perception of these surface attributes is strongly dependent on information at luminance and chromatic boundaries. However, the neural processes by which the surface properties are computed are almost entirely unknown. The research proposed in this application aims to uncover these processes by focusing on brightness perception. The long-term goal of this research is to understand the manner in which neurons in visual cortex respond and interact, giving rise to visual perception. To achieve this goal, stimuli traditionally used in psychophysical experiments are used to examine the responses of visual neurons to surface properties. The representation of brightness information will be investigated in cortical areas V1 and V2 as well as the LGN and optic tract. Recordings made at each of these sites will establish at what stage in visual processing neural activity correlated with brightness originates. Cytochrome oxidase staining will be used to determine whether brightness associated neural activity is restricted to particular compartments of V1 and V2 thought to be components of parallel processing streams. By measuring the time course of responses to various patterns of uniform illumination, the hypothesis will be tested that there are distinct fast and slow components in the cortical computation of brightness. By employing a range of visual stimuli borrowed from perceptual studies, it will be established whether there are neurons that fire in a manner correlated with brightness irrespective of the type of stimulus.

IBN-9720320 PI: ANDERSON This project is being funded through the Learning & Intelligent Systems Initiative. This study investigates functional interactions among groups of neurons (nerve cells) in the brain. The cerebral cortex of the brain is a dynamic ensemble of groups of neurons with activities that coalesce and dissolve in the performance of particular tasks. A computational model called 'network of networks' describes the operation of computations based on neurons interacting in intermediate groupings, in the size range between single neurons (only one computing element) and entire brain regions (to hundreds of millions of computing elements). In the intermediate scale groupings, the model makes predictions about the behavior of both its component single neurons and the overall nature of the cortical computation, manifested as behavior and perception. Experimental tests utilize the mammalian visual system because so much is known about cortical processing of visual information at the level of single neurons, and also there is a large body of related experimental results for visual perception. There are three inter-related parts to this project. 1) Computer simulations and mathematical analysis further develop the 'network of networks' model itself. 2) Simultaneous activity of multiple neurons in visual areas are recorded physiologically to examine long-range transfer of information across visual cortex, of the type suggested by the model. 3) Analyses of previously obtained psychophysical behavioral data from human subjects are combined with computer simulations to try to understand the surprising effectiveness of silhouettes in object recognition, and to provide a test system for the network model. Results will have an impact because of the importance of linking cognition with neuroscience to understand mechanisms that underlie learning and perception, and because understanding how the brain handles complex computations will provide insights for the design of artificia l recognition and decision-making systems.

Representation and Computation in Natural Vision

In natural environments, objects are viewed under a wide variety of lighting conditions, poses, backgrounds, and juxtapositions with other objects. Artificial vision systems that are sufficiently invariant to accommodate such variations are never sufficiently selective. The rich structure of real images offers a multitude of chance arrangements, many of which cause systems to falsely detect an object that is not there. On the other hand, systems that are highly selective are at the same time highly prone to missed detections in the face of natural variability. The visual systems of humans and animals, in contrast, are able to see accurately under a wide range of viewing conditions--how is it that biological systems are both selective and invariant?

The pursuit of this question leads to an analogous question about complex cells and other invariant cell types that are ubiquitous in the ventral visual pathway. Their strength would appear to be their weakness: How is it possible for the visual system to build selectivity out of invariance? Models of complex cells suggest an explanation. Complex and other invariant cell types, by virtue of their nonlinear response characteristics, necessarily possess a functional connectivity whereby these cells become functionally connected to a generally small subset of their inputs. This commitment is circumstantial, inasmuch as it depends on the particular pattern in the receptive field. Functional connectivity is a demonstrable mathematical property of virtually all of the non-linear models put forward to date for complex-cell receptive-field properties. What is more, these observations lead to the conclusion that pairs of such cells that possess overlapping receptive fields will demonstrate a functional common input. This too is circumstantial, and in fact functional common input is high exactly when the patterns in the respective receptive fields "fit together"---correspond to pieces of a larger whole.

These observations suggest a solution to the dilemma of invariance versus selectivity: pieces that fit properly together generate a high degree of functional common input, which manifests itself by a statistical dependence between otherwise invariant representations, most likely in the form of partial synchrony, thereby signaling a composition of parts to cells deeper in the visual pathway.

In search of experimental confirmation of this proposed answer to the selectivity/invariance dilemma, the investigators employ new statistical and methodological techniques to study new questions about the receptive-field properties of invariant cells, and to measure new variables in the joint statistics of invariant cells with overlapping receptive fields.

Project Summary The Interdisciplinary Vision Training Program (VTP) at Brown University trains graduate students to become leaders in vision research. The goal is to train students in research and give them essential career skills, while also familiarizing them with vision disorders and their treatment, so that their future research will factor in public health needs. Students are admitted into either the interdepartmental Neuroscience Graduate Program or the graduate program in the Department of Cognitive, Linguistic, and Psychological Sciences. In these graduate programs, students take required and elective coursework to build a solid scientific foundation. By the second graduate year, students settle on a research lab and they have acquired the necessary scientific knowledge to conduct their dissertation research. At that stage, graduate students pursuing vision research in one of the preceptors' labs are encouraged to apply for the Vision Training Program. Students are selected based on their potential for a successful and productive vision research career. Students are generally admitted to the Vision Training Program in their 2nd or 3rd year, with 3 graduate students participating at a time. The program lasts 2-3 years so that students experience the full range of training activities. Once admitted to the VTP, trainees receive specialized training and experiences aimed at strengthening and broadening their understanding and abilities in vision research. The core of the vision program is the laboratory research training they receive. Research conducted in the preceptors' labs investigates a wide range of topics including retinal structure and function, development of the visual system, visual processing in the brain, visual perception and learning, computational models of vision, and visually-guided behavior. A key part of the program is a group of activities through which the students learn about visual disease and disorders so that they can appreciate the public health needs. Through regular interactions with the Department of Ophthalmology, students learn the vocabulary and procedures used by ophthalmologists and neuro-ophthalmologists. Trainees also observe eye surgeries and visual assessment as conducted in the vision clinic. At all stages, students learn essential skills for a successful independent research career in vision research. These include critical thinking and reasoning, effective science writing and oral presentation, knowledge of the scientific review processes, and training in ethics. To ensure a successful training program, we have selected a broad training faculty with productive track records and experience training students; a couple are relatively junior but are highly active in research and student training and they show great promise as research mentors. In addition to the preceptors in the vision training program, there is a rich intellectual environment at Brown. The Institute for Brain Sciences has over 130 faculty. The Center for Vision Research consists of 40 faculty sharing interests in vision research ranging from molecular biology to artificial systems to philosophy.

[unreadable] DESCRIPTION (provided by applicant): To understand the computations that take place in the human visual system and ultimately palliate disorders, it is essential to know how the system works in natural situations. Despite the immense success of neurophysiological research over the past forty years, it is the case that virtually all data were collected in greatly simplified behavioral situations and with simplified visual input. The long term goal of the proposed research is to elucidate the neural processing responsible for perception in natural visual situations that humans experience. Preliminary data show that key elements of natural vision including saccades and natural visual context have significant effects on the representation of information in primary visual cortex. It appears that fundamental aspects of visual processing can only be fully understood with natural visual input and behavior. The proposed experiments will establish the critical roles of natural context and patterns of eye movements and fixations in two specific aims: 1) Explain why saccades play a fundamental role in shaping V1 responses. 2) Identify the spatial and temporal aspects of visual context that are important in natural vision. The rationale behind the experiments is that in order to fill the gap in our understanding of natural visual processing we must conduct experiments that progressively build a bridge between conventional laboratory paradigms and natural vision. The significance of the proposed research is that it takes the next critical step bringing us significantly closer to an understanding of visual system function in situations encountered by humans. This is the knowledge needed to work toward solutions to visual deficits. [unreadable] [unreadable] [unreadable]

To understand the computations that take place in the human visual system and ultimately palliate disorders, it is essential to know how the system works in natural situations. Despite the immense success of neurophysiological research over the past forty years, it is the case that virtually all data were collected in greatly simplified behavioral situations and with simplified visual input. The long term goal of the proposed research is to elucidate the neural processing responsible for perception in natural visual situations that humans experience. Preliminary data show that key elements of natural vision including saccades and natural visual context have significant effects on the representation of information in primary visual cortex. It appears that fundamental aspects of visual processing can only be fully understood with natural visual input and behavior. The proposed experiments will establish the critical roles of natural context and patterns of eye movements and fixations in two specific aims: 1) Explain why saccades play a fundamental role in shaping V1 responses. 2) Identify the spatial and temporal aspects of visual context that are important in natural vision. The rationale behind the experiments is that in order to fill the gap in our understanding of natural visual processing we must conduct experiments that progressively build a bridge between conventional laboratory paradigms and natural vision. The significance of the proposed research is that it takes the next critical step bringing us significantly closer to an understanding of visual system function in situations encountered by humans. This is the knowledge needed to work toward solutions to visual deficits.

DESCRIPTION (provided by applicant): Over the past 50 years a tremendous amount has been learned about how neurons in different parts of the brain respond to incoming sensory signals and send out commands to move our bodies. Despite these achievements, we are far from a complete understanding of brain function and alleviation of most brain disorders. A fundamental challenge that must be addressed is the need to study the system simultaneously at more than one scale. On the one hand, it is essential that processing by single neurons be understood. Yet it is also critical to elucidate the ways in which information is distributed across neurons and brain areas and integrated to give rise to unitary percepts and motor acts. Microelectrode recordings of individual neurons are highly "local";it is difficult to extrapolate from single cell recordings to interacting populations. On the other hand, whole brain imaging techniques such as functional magnetic resonance imaging (fMRI) sacrifice both spatial and temporal resolution to achieve their global view. While such methods have helped reveal patterns of brain activity associated with various perceptual or motor states, the loss in spatial resolution (millimeters vs. microns) and temporal resolution (seconds vs. milliseconds), while good for human brain imaging, make direct examination of neural circuits by these more global techniques extremely difficult. This application seeks funding to obtain a state-of-the-art recording system that will bridge the local-global gap. The device gives the capability to record simultaneously from hundreds of individual neurons. At the same time, because there are two synchronized recording subsystems, it is possible to examine brain activity and interactions between populations of neurons in any two areas of the brain. The neural activity to be studied by the major users is patterns spread across individual visual and motor areas of cerebral cortex, patterns between two visual or two motor areas, and patterns between a visual area and a motor area. The experiments will be conducted using non-human primates trained to perform visual and/or motor tasks. Two microelectrode arrays, each holding 100 electrodes will be implanted in brain regions of particular interest. Recordings will be made while animals perform trained tasks. The data will reveal single-cell spiking and LFP relationships with perception and behavior, temporal relationships between active neurons, and the relationships between areas within and between the visual and motor systems. These studies, made possible by the shared instrument, will lead to major advances in our understanding of visual perception, motor coding, and visuo-motor integration for behavior. It is this type of information that is critical if one is to alleviate brain disorders associated with trauma or disease.

Humans are highly visual animals and our daily experiences, memories, and dreams are dominated by our visual sense. To create the continuous and seamless visual experience, humans explore their environment by making three to four eye fixations every second, each followed by a rapid saccadic eye movement to the next object of interest. However, most experiments on vision are conducted with research participants trained to keep their eyes still to remove these saccadic eye movements. Such an approach may not tell us in full how vision works in its naturalistic context where eyes are freely moving. In this project, Dr. Michael Paradiso of Brown University will use single unit recording from monkeys when they make visual decisions about objects appearing during or just after saccadic eye movements. The data are expected to reveal how eye movements alter visual processing and how the brain is able to parse the neural continuum into discrete perceptual block using signals associated with saccades. These experiments are innovative in their integration of behavioral testing in naturalistic paradigms with brain recordings using leading edge multi-electrode technology. The researcher aims to bridge the gap between basic research and our understanding of human visual experience in the real world.

Numerous disorders ranging from autism, to dyslexia and schizophrenia exhibit abnormal eye movements. At the present it is unclear how the abnormal eye movements are involved in the disorder, but the proposed project will provide critical data that can serve as a foundation for further studies targeted at specific disorders. Through the project, the researcher will continue to give lectures to K-12 students, collaborate with K-12 teachers on science instruction, and host a discussion group in Brown's Catalyst Program for incoming minority students at Brown University. Finally, Dr. Paradiso will participate in the big data effort by making the data available to support other coordinated NSF efforts that aim to make use of real data in the teaching of STEM related courses and to enable participation in discovery science by those who would otherwise have no access to such data.