Michael J. Tarr - US grants

DESCRIPTION (provided by applicant): Visual object recognition is one of the most poorly understood mental faculties. Theories of object recognition are underspecified with respect to both the functional roles ascribed to different neural structures in the inferior- temporal (IT) cortex and the range of visual recognition behaviors exhibited by humans. The inadequacies of current theories stem from a theoretical status quo combined with methodological limitations inherent in both psychophysics and neurophysiology. To better understand object recognition we must discard both standard feed-forward, hierarchical models that bear little resemblance to the facts as we know them and the behavioral studies that test such narrow models. We must also develop new neuroimaging methods to complement neurophysiology, which severely under-samples object representation space and typically relies on ad hoc/a theoretic strategies for determining which features/objects yield maximal neural responses. This proposal introduces new tools for mapping feature and object selectivity across human visual cortex using functional Magnetic Resonance Imaging (fMRI). The effort is motivated by neurophysiological studies with similar objectives. However, the informativeness and generality of visual physiology has been limited by the low number of samples (~103 recordings) relative to the size of the neural representational space (~109 neurons). fMRI, which measures brain responses in voxels (~106 neurons), enables the study of neural codes at a macro level, yet at a resolution fine enough to capture meaningful functional differences between brain regions. To explore the feature selectivity of localized regions of IT, visual stimulation will be driven by real-time fMRI, in which accruing neural contrasts between conditions are computed instantaneously. This mapping approach will be enhanced by employing two principled strategies for moving through feature space: an a priori method that relies on an algorithm for automatically segmenting objects into features (which has been validated against human segmentations);and, an a posteriori method that relies on "mutual information" to identify features that carry more or less task-relevant information. The end result, a more complete and theoretically-driven picture of selectivity in the ventral pathway, will form the basis for a new model of visual object recognition. Two model assumptions will be tested using novel fMRI methods. First, reverse correlation ("superstitious perception") will be combined with trial-by-trial neural responses to assess whether object processing proceeds in a non-hierarchical manner in which larger numbers of relatively simple features (e.g., those encoded in V4) are combined in a non-linear manner to represent objects. Second, time-resolved fMRI will be used to examine the degree to which recognition is driven by top-down, context-dependent processes. An improved functional picture of IT in combination with a more refined model of visual object recognition will aid in the creation of more effective treatments and retraining strategies for individuals suffering from traumatic brain injury or life-long recognition impairments, including the face recognition deficits associated with Autism. This research will provide a much clearer (and systematic) picture of how the human brain creates the experience of object perception given the optical information arriving at our eyes. A better understanding of the neural mechanisms underlying object and face recognition may lead to more effective treatment and retraining strategies for individuals suffering from either traumatic brain injury, particularly to the brain structures supporting perception, or from life-long face recognition deficits ("congenital faceblindness"). Progress in this area may also afford more focused interventions for specific symptoms associated with certain developmental disease processes (e.g., impaired face recognition in Autism).

The proposed activity under this award will investigate the neural representation of affective valence during visual object perception. The objects used in these experiments will be consumer products that are currently under development by the companies that the team will be working with during this award. The experiments the team plans to conduct will rely on a combination of functional magnetic resonance imaging (fMRI) and behavioral psychophysics designed to develop strategies and analysis tools to most effectively predict, as well as understand why, which object-based products are most preferred by consumers. In doing so, the team will form a more neuropsychologically-based model of how specific subcomponents of the visual system interact with both affective processing, and with the choice and decision-making systems.

By applying neuro-scientific methods to consumer testing and product development, the team will be introducing well-grounded, cutting-edge science and technology to commercial sectors that have not typically employed such methods. Beyond specific product evaluation and assessment, in an effort to provide the best possible service, clients will be instructed regarding the general principles of the approach. Thus, the team plans to conduct workshops for any of their customers that wish to learn more about the brain and how it functions. This will allow customers to be more involved in the process and ultimately get the most out of the service. The team's innovations have the potential to create a significant commercial impact by increasing the number of products produced that are more human centered, preferred, and which better meet human needs. This has the potential to rapidly decrease the physical and financial waste surrounding products that are not preferred and are therefore not consumed.

How do our brains take the light entering our eyes and turn it into our experience of the world around us? Critically, this experience seems to involve a visual "vocabulary" that allows us to understand new scenes based on our prior knowledge. The investigators explore the nature of this visual language, exploring the specific computations that are realized in the brain mechanisms used for scene perception. The work combines data from state-of-the-art computer vision systems with human neuroimaging to both predict brain responses when viewing complex, real-world scenes, and to analyze and understand the hidden structure embedded in real-world images. This effort is essential for building a theory of how we are able to see and for improving machine vision systems. More broadly, biologically-inspired models of vision are essential for the effective deployment of intelligent technology in navigation systems, assistive devices, security verification, and visual information retrieval.

The artificial vision system adopted in this research is highly data-driven in that it is learning about the visual world by continuously "looking at" real-world images on the World Wide Web. The model, known as "NEIL" (Never Ending Image Learner, http://www.neil-kb.com/), leverages cutting-edge big-data methods to extract a vocabulary of scene parts and relationships from hundreds of thousands of images. The relevance of this vocabulary to human vision will then be tested using both functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) neuroimaging. The hypothesis is that the application of prior knowledge about scenes expresses itself through learned associations between the specific parts and relations forming the vocabulary for scene perception. Moreover, different kinds of associations may be instantiated within distinct components of the functional brain network responsible for scene perception. Overall, this research will build on a recent, highly-successful artificial vision system in order to provide a more well-specified theory of the parts and relations underlying human scene perception. At the same time, the research will provide information about the human functional relevance of computationally-derived scene parts and relations, thereby helping to refine and improve artificial vision systems.

? DESCRIPTION (provided by applicant): The superior temporal sulcus (STS) is a large region of cortex in the temporal lobe that has been implicated in a wide range of cognitive processes. Critically, many of these are critical to social perception: perception of human motion and actions, understanding the mental states of others (theory of mind), perception of animacy, perception of faces, integration of audiovisual information, and detection of gaze direction. Our goal is to leverage cutting-edge neuroimaging techniques to gain a better understanding of STS structure and function as it relates to social perception, as well as the role of STS in the more extensive cortical networks that support the cognitive and perceptual processes enumerated above. Aim 1 will map the functional sub-regions of the STS associated with social perception using high-resolution fMRI and a wide array of well- established experimental designs and stimuli. This will help reconcile the different roles attributed to the STS arising from neuroimaging data that is usually collected in disparate domains of cognitive neuroscience and in different subjects. Here we will ensure these domains and their associated patterns of STS recruitment are compared within the same subject population. The end result will be a comprehensive functional map of the STS explicating the separate and shared cortical regions that are recruited across different social processes. Aim 2 will use diffusion spectrum imaging (DSI) to map the white matter connectivity of the STS. The same subject population participating in the fMRI scans of Aim 1 will also undergo DSI scans. Functional regions identified from Aim 1 will be used as seeds for fiber tractography, thereby allowing us to map the white matter connectivity patterns of functional sub-regions of STS. The resultant functional/structural connectivity map of STS will help clarify its computational role across a wide range of socially critical tasks. This multimodal map will also establish a normal baseline for comparison with data from individuals with social deficits such as autism spectrum disorder. Our overall result will be a clearer understanding of the neural underpinnings of a wide variety of critical social and communicative functions.

Humans are social creatures with extensive neural systems dedicated to the skills required to navigate interactions with others. This includes decoding the actions of others to infer goals and intentions, and planning our own actions that are appropriate for the current context. Brain regions that support these skills are anatomically dispersed in the four lobes of the brain, organized as a network with communication via long-range white matter connections. One key hub of this network is the posterior superior temporal sulcus (pSTS). The work is this proposal will address an important outstanding question: how the long-range connections supporting action understanding are organized, and the nature of the information that is integrated through these connections. This work will combine structural and functional brain imaging to identify anatomical pathways connecting systems supporting action recognition, with particular attention to pathways through the pSTS, and will use computational statistical analyses to characterize the neural information that is carried through those pathways. This problem is of urgent scientific and clinical relevance: Neuroscience increasingly recognizes that brain regions do not function in isolation, but instead reflect the integration of neural signaling from many cortical sources. The work in this proposal seeks to advance brain science by explicitly modeling these sources in a targeted cortical network. The action recognition network holds additional importance to the public, as some neurodevelopmental disorders (such as autism) are linked to atypical development of the pSTS and poor communication within this neural network. Therefore the outcomes from this work may be critical for developing new clinical tools for diagnosis and interventions for these disorders. Implementing the work in this grant will also support the full engagement and promotion of under-represented and first-generation of young scientists training in neuroscientific research.

The problem of how information is communicated and structured within the action recognition network is an important one. Many competing scientific models exist as to the functional specialization of the posterior superior temporal sulcus and connected brain regions within the action recognition network. New empirical data and analytical techniques are required to advance these theoretical models. A key to understanding information structure within the pSTS and the larger action recognition network is to evaluate the sources integrated within the neural signals, which reflect both sensory-driven perceptual analysis of social cues and the top-down goal-directed signals modulate influences. The work in this proposal will combine innovative experimental design with advanced multivariate statistical analyses to extract structure from the rich regional brain activation response, and will decompose the contribution of sensory-driven and top-down signals on neural tuning. At the same time, one must consider where top-down goal-directed signals originate and the structural pathways by which they are transmitted. The work in this proposal is innovative in that it will characterize the network architecture, both structurally and functionally, using a combination of tools rarely implemented despite their clear complementarity.