Sergei Gepshtein - US grants

When we look at the world, how do we know which parts of the visual input belong to the same object and which do not? The process known as perceptual grouping takes elements of the visual input and combines them into what we experience as a visual scene that contains objects, people, plants, shadows, and so on. Most of the time perceptual grouping is involuntary but it can come under voluntary control. For this reason, the study of perceptual grouping is a part of the larger effort toward understanding consciousness. Although phenomena of perceptual grouping are an essential foundation of perception, they are often described using a list of qualitative "principles," such as proximity, similarity, and good continuation, that are vague and unquantified. Michael Kubovy at the University of Virginia and Sergei Gepshtein at the Salk Institute for Biological Studies are proposing to clarify some of the fundamental processes of perceptual grouping, using rigorous methods of measurement and modeling. The researchers start with simple visual patterns that allow them to study one force of perceptual grouping at a time. The individual forces of grouping will then be combined, using more complex visual patterns, with the goal to derive general quantitative laws of perceptual grouping. The researchers will study the interaction of geometric factors (such as proximity between elements of visual displays) and intensive factors (such as the luminance and contrast of the elements) in perceptual grouping. The laws of combination of grouping factors will be compared with the laws of combination of other sensory cues, which have been intensively studied in the perception of visual depth and in multisensory integration.

Software to be developed for this research program will be made accessible to the public. In addition to addressing fundamental issues in visual perception, the work has the potential to influence developments in visual media such as art, animation, and film.

DESCRIPTION (provided by applicant): One of the fundamental tenets of sensory biology is that sensory systems adapt to environmental change. It has been argued that adaptation should have the effect of optimizing sensitivity to the new environment. To make this premise concrete and precise, the proposed research builds on a normative theory of visual motion perception, which argues that the visual system will adapt optimally by balancing stimulus and measurement uncertainties. This theory makes predictions about visual spatiotemporal sensitivity as a function of environmental statistics: Adaptive optimization should be manifested as a change in spatiotemporal sensitivity for an observer and for the underlying motion-sensitive neurons. The proposed research will test these predictions. The effects of adaptation on visual sensitivity will be examined in the context of the neuronal representation of speed of visual motion. Adaptation using specific speeds will be used as a means to effect a change in environmental statistics. Both perceptual and neuronal consequences of this adaptation will be recorded, with the expectation that sensitivity changes will reflect the modeled process of optimization. The proposed research constitutes a richly interwoven collection of psychophysical, neurophysiological and theoretical approaches to the topic of visual adaptation. Experiments have been designed to yield an unprecedented body of comprehensive data bearing on the spatiotemporal properties of the primate visual system and the effects of environmental change. These data will be used to further understanding of the phenomenology and mechanism of adaptation. PUBLIC HEALTH RELEVANCE: The long-term goal of this project is to contribute to the understanding of biological substrates of visual perception. Detailed knowledge of normal functions of visual cortex shall provide insights into neural events that underlie visual sensitivity and the effects of visual experience, which will ultimately aid the treatment and prevention of neurologic and neuropsychiatric disorders of vision. These aims are pertinent to development and use of prosthetic and behavioral therapies for the visually impaired.

Abstract One of the most important questions in neuroscience today concerns the mechanisms by which sensory neurons give rise to perceptual experience. There are many ways to address this question, which have long populated the field of visual neuroscience. Prominent among them is the study of visual selectivity. Observers are highly sensitive to some visual stimuli and less sensitive to others. Visual neurons are also highly selective: Each responds to a limited range of stimuli along several stimulus dimensions. The proposed research aims to understand how the selective pattern of neuronal responses accounts for the observer?s selective perceptual experience and discriminative capacity. This understanding will be achieved through experiments that first evaluate patterns of neuronal selectivity for visual stimuli that vary in their spatial and temporal properties. Second, these data, in combination with manipulations of stimulus context, will be used to develop a novel mechanistic account of neuronal selectivity based on activity within cortical visual circuits stabilized by inhibition. Finally, to understand how neuronal selectivity underlies perceptual experience, direct comparisons will be made between physiological measures of neuronal selectivity and behavioral measures of perceptual selectivity, assessed concurrently under identical conditions. The proposed research constitutes a richly interwoven collection of psychophysical, neurophysiological and theoretical approaches to understanding of spatial and temporal vision. The experiments will yield an unprecedented body of comprehensive data regarding the spatiotemporal tuning of the primate visual system. These data will be used to further understanding of the mechanisms of sensory processing and will provide insights into pathologies of vision caused by trauma, disease and developmental disorders of the brain. !

In addition to their traditional role as a source of emotional rewards and inspiration, the modern mission of art museums is education. Objects of interest are presented in the context of a narrative that the museum visitor follows to learn history, materials, technique, and function, as well as relationships to the natural world and human civilization. Narrative principles for exhibition design have for decades emerged from small scale observational studies of museum visitor behaviors, such as expressions of engagement and choice of path. Building on recent advances in scientific understanding of sensory processing and behavioral choice, in combination with sophisticated computational tools for characterization of fine details of behavior, a team of scientists and museum professionals will turn a designated gallery at the Los Angeles County Museum of Art into a laboratory for investigation of human perception, action, choice, and learning. The broad applied goal of this project is to obtain scientific knowledge that will further enhance the educational mission of museums. The intellectual impact of the project will be an improved understanding of environmental and social factors that guide human behavior under naturalistic conditions. More generally, the project will benefit the larger communities of architecture and design professionals, as a model of experimental methodology and by offering a unique multifaceted dataset for analysis and evaluation of the influence of the built environment.<br/><br/>The project builds upon several technological and computational innovations. One is the methods of computational ethology, which is a new approach to quantitative behavioral analysis that employs high-resolution 3D motion capture together with machine learning methods for behavioral classification. This approach will yield efficient non-invasive measurements of visitor locations, rates of movement, poses, social interactions, gestures and expressions that reflect transitory cognitive states, such as visual attention and engagement with works of art. Observations from tens of thousands of anonymous museum visitors will be subjected to descriptive statistical analyses, to gain insights into the relationship between the structure and content of exhibition design and the behavior of individuals and social groups, and to discover spatial and temporal contingencies between visitor behaviors at different locations in the gallery. Results of these descriptive analyses will be used to develop predictive models of visitor behavior, capturing the full gamut of individual styles of visitor interaction with works of art and other visitors, informed by visitors’ sensory operating characteristics as well as sensory and motoric affordances of the gallery space. In the final stage of the project, strategic modifications to gallery design will be used to test and further develop predictive models in forecasting visitor behaviors. Results will constitute a new empirical framework for exhibition design and its impact on visitor experience.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.