Ernst Niebur, PhD - US grants

Selective attention is the process whereby a subset of the incoming sensory stream is selected for further processing. In the case of visual attention, this process can be subdivided into two distinct types of selection. The first type entails the establishment of a spatial focus, for preferential processing of a single, circumscribed area of the visual field as needed for object analysis or recognition. The second type involves the establishment of a filter which selectively enhances one or more distinct visual channels across the visual field as a whole, such as to facilitate perception of all stimuli of a specific color, spatial frequency, or direction of motion. The principal investigators will study the neurobiological mechanism of selective visual attention, through computer modeling of the correlates of attention at the single cell level and the integration of visual processing and attentional control at the systems level.

9720353 Niebur The ability to learn and to adapt to a changing environment is one of the fundamental characteristics of life itself. The purpose of the proposed work is to develop, implement and test adaptation mechanisms which are modeled after their biological counterparts. The centerpoint of the project is the development of adaptive "neuromorphic" VLSI (Very-Large-Scale-Integrated) devices. Neuromorphic devices are similar to common computer chips but they are designed and operated in somewhat different regimes (analog variables instead of digital, subthreshold voltages instead of suprathreshold, and parallel processing instead of serial) which gives them properties much more similar to those of nerve cells than can be claimed of standard computer chips. While neuromorphic circuits have been designed and fabricated for about a decade, this work is one of the first to systematically incorporate learning and adaptation into that technology. This is accomplished by making use of recently developed design techniques (''floating gates'') which for the first time allow storage of learned values for periods of time (years) similar to the lifetimes of retention mechanisms encountered in animals and humans. Furthermore, while most of the previous work was concerned with the processing of sensory information (good examples being the "silicon retina" and "silicon cochlea"), this project also includes motor control and thus closes the behavioral loop, from sensory input to motor output which then again influences sensory input. The adaptive neuromorphic devices are integrated in a realistic model of the eye-movement control of primates (monkeys and humans), which is one of the best understood biological sensorimotor systems. Different mechanisms of adaptation and learning have been studied in monkeys and humans and the performance of the adaptive control system is evaluated by comparing with experimental data. The development of a hardware model for eye movements promises significant insights into the function of an important sensorimotor control system. Understanding human (and primate) eye control in the real world and how it adapts to changing situations may lead to the development of practical tools, for instance, for target detection and pursuit. More importantly, however, synthesizing a complex behavior will help us understand this and other behavioral patterns to a degree which is impossible to achieve with analytical methods alone.

Niebur
9876721
Abstract

How is the external world represented in the brain? At the lowest level, we know that the basic computational units of the brain are nerve cells, or neurons. We also know that neurons use electrical impulses called "action potentials" to communicate with each other. What we do not know is how these impulses code information, for instance, the representation of the organism's environment. A common view is that properties of perceived objects are coded as the average firing rates of neurons ("rate code") but this is not only an inefficient procedure, it also creates problems when the brain has to deal with more than one object simultaneously, as is usually the case. One of the problems is that, if a certain property is coded by a corresponding spike rate, and if several objects have this property, how does one decide to which of those objects the property belongs?

The brain does more than just map out the external world; the behavior of an organism also depends on its internal states. One of the problems the brain has to solve is information overload, resulting from the multitude of stimuli impinging on the sensors at all times. Therefore, one of the brain's most important functions is to carefully select those stimuli that are important, and to consider only these in detail. For instance, while you are reading this text, you are focusing on the written words and you are probably not thinking about how your left foot feels in your left shoe. However, now that you try to "feel" it, you can "pay attention" to it. We know that the tactile sensors in your foot behave the same way whether you are paying attention to it or not, but for the brain the two situations are clearly different. What, precisely, changes in the activity of the brain when you direct your attention from one stimulus to another?

Theoretical models have predicted that the code used by the brain not only uses the average rate but also more fine-grained temporal structures of the sequences of action impulses. For instance, it has been predicted that paying attention does not necessarily change the average firing rate, but the relations between the firing in different neurons, for instance, make it more likely that neurons fire together. Dr. Niebur and his students will compare the activity of neurons in monkeys performing a task that requires them to attend to a stimulus with the activity when they receive the same stimulation but are NOT attending to it. Preliminary results, as well as mathematical modeling, have indicated that there is indeed, such a difference in the relative firing of neurons. In addition to analyzing the neural activity of the monkey working on the task, they will also develop mathematical models of attentional processing.

[unreadable] DESCRIPTION (provided by applicant): An important aspect of visual perception is figure-ground segregation. Because the eyes register only 2D images of a 3D world, the visual system needs to detect the contours of objects and assign them correctly to the object side, rather than to the background side. Our recent studies indicate that the assignment of "border ownership" occurs in cortical area V2. In a so-far separate line of work, we have studied selective attention, which is the mechanism by which the nervous system selects the most relevant information at any time. We have found a strong correlation between the attentional state and the temporal structure of neural responses, viz. synchrony between neural spike trains. In the proposed work, we suggest that attention may be viewed as a mechanism that works at a level of intermediate vision and perceptual organization. Specifically, we propose that the neural substrate that is observed to generate the border ownership coding signals may also provide 'handles' for attentional selection. Central processes of selective attention may then rely on the visual data structure provided by this area. Our preliminary results indicate that attentional control may be exerted by employing synchrony structures akin to those seen in our previous work. We will study this hypothesis both by computational modeling and by recording in awake behaving monkeys. The hypothesis generates clear predictions, of which some have been confirmed in preliminary studies and others are to be tested in the proposed work. [unreadable] [unreadable] Understanding the interplay between selective attention and perceptual organization will contribute significantly to understanding the vision process in particular, and the principles of neural information processing in general. Beyond the theoretical interest, understanding the function of the visual cortex is an important goal in view of clinical applications because large parts of the brain are devoted to visual processing, and many neurological diseases are accompanied by central visual disorders. [unreadable] [unreadable]

DESCRIPTION (provided by applicant): Humans and higher animals receive much more information from their sensory periphery than they can process in detail by their central nervous system. Selective attention is the triage process by which incoming information is divided into the behaviorally most relevant parts, which require detailed processing, and all others that are suppressed. The goal of this research is to understand the neural mechanisms underlying tactile selective attention which will be studied by single unit recordings and local field potential recordings in awake behaving monkeys. The first Aim is to determine the limits of granularity of attentional selection. What are the limits of selection in space (attend to different fingers or to different hands), submodality (attend to vibration or to form) and time (changes within the duration of one trial), and what are the neuronal correlations of this attentional selection? We will specifically measure changes in mean firing rate and in correlations between neurons (synchrony). Aim 2 is to determine the influence of attentional selection on the responses of identified neuronal subpopulations, including the functional populations defined by the cortical layers and by the postsynaptic effects of neurons (excitation or inhibition). We will also determine whether all neurons showing synchrony are part of one population or whether there are several subpopulations whose members are synchronous with each other but not with members of other subpopulations. Two theoretical models of the role of neural synchrony in perception (1: tagging of the attended stimulus, 2: tagging of different parts of a perceived stimulus) make different predictions for the number of subpopulations and this analysis will test those predictions. Aim 3 is to study the mechanisms underlying the generation of synchrony. One hypothesis that will be tested is phase locking of action potentials to a periodic function (oscillations).

DESCRIPTION (provided by applicant): The proposed research addresses the question of how perceptual organization interfaces with attentional selection in the visual system. While these questions are frequently considered separate, we believe that they are closely connected and may in fact share a common neural substrate. We propose that the neuronal mechanisms of figure-ground organization, that is, the neural representation of the borders of a visual object, relies on neuronal circuitry that is also used to represent whether these objects are attended or not. We will study single cell activity with multiple electrodes in extrastriate cortex. These data will be used to constrain large-scale detailed models of the underlying neuronal circuitry. Three specific aims will be pursued. The first Aim is to model the mechanisms of attention-independent figure-ground organization in cortical area V2. In previous work, we have developed a model of figure-ground segregation that explains mechanisms underlying border ownership selectivity. The model can only explain changes in mean firing rates. The new model will be based on a model of single neurons that includes spiking and will thus be able to model amplitude and time course of the border ownership signals as well as pair wise spike train correlation between neurons. The second Aim is to study short-term memory for figure-ground structure. We will perform multiple simultaneous single-unit recordings in area V2 to characterize the recently observed hysteresis effects in border ownership coding. We will also record in higher extrastriate areas (V3 and V4) since the fast time course of border ownership selectivity makes it likely that it is imparted by connections through the white matter. These electrophysiological recordings will be complemented with the development of a model of persistence and hysteresis of border ownership signals. We will expand the spiking neural network model by introducing more complex single-neuron models that can explain the mechanisms underlying the hysteresis effects. The third Aim is to study how selective attention interacts with mechanisms of figure-ground organization and feature binding. We suggest that the selectivity to side of foreground figure observed in extrastriate cortex arises from a recurrent bias from grouping cells, and that the latter are also used to attentively select the figure. We will record from single cells and pairs of cells in extrastriate area V2 and study the influence of selective attention on border ownership selectivity. These recordings will be combined with a model of the interaction of top-down selective attention with figure ground organization. We will expand the spiking neural network model developed under Aim 1 to include selective attention. The model will explain rate effects and pair wise correlation functions under a variation of binding conditions and attentional states. The proposed research will contribute to our understanding of some of the most fundamental mechanisms of primate vision which is of importance for understanding both normal and impaired vision in humans. The insight gained from this project will contribute to the understanding of the neural basis of cognitive disorders such as dyslexia and hemi-neglect. PUBLIC HEALTH RELEVANCE: It appears to us that seeing is easy. In reality, it is a very complex process, as can be seen by the fact that no computer has a performance in artificial vision comparable to even simple animals. The goal of the proposed research is to understand how a visual scene is dissected into visual objects, and how these visual objects are attentively selected for more detailed processing. Deficiencies in attentional selection are present in many neurological diseases, e.g. hemineglect, and elucidating how selective attention works with image understanding will be important for understanding the mechanisms underlying these diseases.

? DESCRIPTION (provided by applicant): Decision-making is one of the most central cognitive functions. Since the early days of the 20th century a body of mathematical work developed the modern axiomatic approach to rationality in choice behavior. These normative models revolutionized economics and mathematical psychology by describing the properties of choices consistent with maximizing an ordered, internal representation of value, termed utility. Experimental research, however, has demonstrated a wide set of non-rational behaviors (preference reversals) that deviate from these normative theories. A number of computational models where developed to account for the observed non-rationalities. Most of these models explain behavioral preferences as the outcome of a dynamic computational process and not of a static maximization process with fixed utility and probability weighting functions. However, the cognitive and neural processes that are at the heart of preference formation are still poorly understood. We will combine behavioral data, electrophysiological recordings in humans and monkeys, and computational approaches to develop a new theory of the neural mechanisms underlying complex, multi-attribute decision-making. Intellectual Merit (provided by applicant): The overall goal of the present proposal is to understand the neural code of decision-variables (such as reward amounts and probabilities) and of the dynamic process by which these variables are integrated to form subjective values (utility) and preferences and mediate nonrational behavior. Monkey and human subjects will work in a novel behavioral task that allows us to observe the focus of attention of decision makers while they evaluate the offers and select one of them. Together with these behavioral data we will record decision-related activity in several brain areas. This data set will allow us t test the predictions of various cutting edge computational models that have been suggested to explain preference reversals, but are based on different mechanisms. We will also use the experimental findings to develop a neural mechanistic theory (Aims 1-3, below) and to account for non-rational behaviors, such as preference reversal (Aim 4). Specifically, we have the following aims: (1) Understand how the decision-variables (outcomes, amounts and probabilities) are encoded in the brain. (2) Understand how the separate decision-variables are integrated to compute the overall subjective value of choice options. (3) Investigate whether, and if yes how, attention influences the value computation of choice options. (4) Use the decision model developed in aims 1-3 to explain preference reversals. The end point of these investigations will be a new neurocomputational theory that consistently explains behavioral and neural data in our experiments. This model will integrate decision and attentional selection processes and will generate novel predictions to be tested in future research. Broader Impact (provided by applicant): Some of the most important problems of modern societies are caused by non-optimal decisions made by people. Abuse of illegal drugs, alcohol and nicotine but also the current epidemic of obesity and metabolic disease in the population can ultimately be traced back to people making decisions that are not in their objective best interest. The research proposed here studies how the variables underlying decisions are represented and computed in the primate brain, in particular by understanding situations in which optimal choices are discarded in favour of inferior ones. The project also contributes to the training of the next generation of scientists. Four PhD students will be trained; two at Johns Hopkins University and two at Tel Aviv University, and undergraduates will be part of the research groups. All PIs are strongly committed to increase participation by women and underrepresented minorities. Niebur and Stuphorn have a long track record of training minority high school students in their labs, successfully preparing them for a future college career. In addition, existing connections with Morgan State University, a historically black college in Baltimore, will be extended.

The visual brain infers a three-dimensional world from twodimensional images and organizes the visual information in terms of objects in three-dimensional space, representing even objects that are partially occluded and appear fragmented in the retinal image. This organization is the basis for attentive selection, action planning and object recognition. A combination of experimental and theoretical studies together with model implementations in neuromorphic hardware will be used to elucidate the interface between visual feature representations and attentive cognitive processes. Previous findings on the neural coding of figure-ground structure can be understood in terms of grouping mechanisms that structure the incoming sensory information as proto-objects (objects as defined by the system at this stage). The grouping mechanisms also provide handles for top-down mechanisms to address and select object-related information. The proposed work will explain how neuronal circuitry organizes spatially disconnected visual features into perceptual objects. How is this implemented neurally to lead to a coherent representation? Detailed computational models of the underlying circuitry will be developed, both as standard numerical simulations and in fast, neuromorphic hardware, and then tested by multiple single-cell recordings in awake non-human primates. Specifically, while prior studies examined spike time correlations indiscriminately in all neurons, our recent studies differentiated neurons according to their role in the grouping circuits. The grouping hypothesis predicts elevated synchrony only in pairs of neurons that belong to the same grouping circuit, but not in other pairs. These model predictions were confirmed in a recent study which showed that spike-spike correlation functions are in qualitative agreement with the idea that perceptual grouping is implemented by feedback from populations of dedicated grouping cells. Quantitative understanding requires the development of explicit spiking models, which is one of the main foci of this proposal. Models will be implemented on neuromorphic spiking hardware since the complexity of the cortical circuitry makes realistic model simulations on CPU/GPU system impossible. Predictions of integrate-and-fire type models of this circuitry will be compared with rate and synchrony observed in our recordings and deviates used to fine-tune the models. We will pursue the educational and broader impacts aims on five fronts. 1) Students will be crosstrained and mentored in biological, mathematical and engineering sciences, which will lead to graduates with unique skill sets. 2) We will contribute to the development of the nascent neuromorphic engineering field, providing new research problems that can benefit from the crosstraining and collaboration. We plan to participate in the NSF sponsored Telluride Neuromorphic Cognition Engineering and Capo Caccia Neuromorphic Cognitive Engineering Workshops for this purpose. 3)We will provide an opportunity for undergraduate students to participate in the research as part of our Site REU (managed by one of the PIs). They are trained in communications, research ethics and project management, which are crucial for success in todays biotechnology and bioscience work and market place. 4) We currently host students from local high-schools who conduct STEM research practicum rotations in our labs. This project will provide a perfect venue for the rotators to get exposed and mentored on multi-disciplinary research problems. We will use a tiered mentoring structure, where undergraduates mentor K-12 rotators, graduate students mentor undergraduates, and faculty members mentor all participants. 5) Our student recruitment plans will build on our current partnerships with MARC, LSAMP, McNair, SWE, SHPE and other similar programs and minority-serving institutions and local community colleges, to help develop a pipeline of qualified, diverse individuals who will contribute to the workforce in the area of STEM.

Decision-making is one of the most central cognitive functions of importance at practically all levels of society. In many real-world decisions, which of the available alternatives is chosen is influenced by many different attributes. Such multi-attribute decisions are complex because they require the integration and comparison of many pieces of information. For instance, selecting the bundle of goods that maximizes value given a budget constraint in a supermarket that only stocks 100 different goods requires checking approximately 10^30 possible combinations. For this reason, humans do not use rational choice theory in all their decisions. In addition to having to combine the influence of all the different attributes, another complexity is that one alternative is often preferable on one set of attributes, but another is preferred on others. Making a choice then requires a trade-off, which further complicates the decision process. However, the cognitive and neural processes that are at the heart of preference formation are still poorly understood. This complexity is thought to tax limited cognitive resources in humans who therefore can pay attention only to a limited set of information, on which the decision is then based. In addition, task history often systematically changes decision biases. This research program takes advantage of the opportunity to obtain direct recordings from individual's brains while they perform such complex decision. It will study these activity patterns to determine whether they can be explained via mathematical models of decision making. Understanding which attributes are considered during decision making, and how they are weighted could explain decision making in typical and a-typical populations. Furthermore this integrative research program forms an opportunity to expose engineering students to dynamical systems and control theories in an interdisciplinary context.

This project combines behavioral data, neural recordings in humans (patients undergoing epilepsy evaluation) implanted with multiple depth electrodes covering many cortical and subcortical brain areas, and computational approaches to develop a new theory of the neural mechanisms underlying multi-attribute decision-making in complex environments. This is a unique opportunity to study brain circuits simultaneously across multiple brain areas while humans make these decisions. The overall goal of the present proposal is to understand the neural circuit involved in (1) representing the relevant decision variables, (2) integrating these variables to form subjective values, and (3) selecting one of the options in multi-attribute decisions. Participants, with implanted electrodes, will work in a novel behavioral task that makes it possible to observe their focus of attention while they evaluate the offers and select one of them. Data will constrain cutting edge computational models of multi-attribute decision making that will combine: (i) a procedural model of the decision in each trial, and (ii) a latent variable model of biasing influence on decision-making resulting from past trial history. The computational models will make it possible to identify neuronal activity that represents task-relevant variables and the dynamic flow of information across the different elements of the identified neural circuit.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Adaptive Behaviors; Address; Anatomy; Architecture; base; Behavior; Behavior Control; Behavioral; Behavioral Model; Biological Models; Brain; Cells; Complex; Computer Models; Computer Simulation; Data; Decision Making; Development; Disease; Drosophila genus; Equilibrium; experience; experimental study; fly; Frequencies; Image; imaging modality; in vivo; Individual; information processing; insight; Investigation; Learning; long term memory; Mediating; Memory; Modeling; Molecular; Nervous system structure; novel; optical imaging; Organism; Output; patch clamp; Pattern; Physiological; postsynaptic; Psyche structure; reconstruction; response; simulation; Synapses; Testing; Trees; two-photon; Work;