Michael A. Arbib - US grants

Our approach to analyzing mechanisms of visuomotor coordination is two-fold: "top-down", to offer a coordinated control program of interacting schemas (concurrently active programs suitably structured for action-perception systems) to model behavior noted by neuroethologists; and "bottom-up", to provide detailed models of neural networks which are consistent with known anatomy and physiology, but which involve additional assumptions, amenable to experimental test, to yield a network capable of exhibiting appropriate behavior. Where possible, we implement a family of models, rather than a single model, to facilitate computer experiments and animal experiments which may discriminate between alternative mechanisms. Specifically, we plan to conduct studies of the following subtopics: (1) Modelling tectum-pretectum interactions subserving pattern-recognition in anurans; (2) Detailed neural modelling of retina; (3) Modelling patterns of behavior in frog and toad while developing a theory of motor schemas and their coordination for orientation, sidestepping, snapping, etc.; and (4) Developing mathematical and computer methodology for exploring a model-family to determine which subset of models approximates satisfaction of a given set of constraints. The work will make modelling tools available for the analysis of large brain systems. Our research plan has build into it structures which ensure that these tools are accessible to neurophysiologists, neuroanatomists and neuroethologists, while also facilitating a full exchange of ideas with the computer science community (artificial intelligence, simulation methodology, interactive computer graphics, etc.).

This is a continuation of a project that started while Prof. Arbib was at the University of Massachusetts, and is being conducted in collaboration with Prof. K. Ramamritham there. Substantial progress has been made on a distributed formal model of robot computations, operating system and scheduling support for realizing that model, and control-theoretic approaches to task-level robot control. The work on the distributed model is centered at the University of Southern California, while the other topics are pursued mainly at the University of Massachusetts. The work centers on control of complex tasks involving grasping and manipulation with a dexterous robot hand (the Salisbury hand).

The frog has proved an invaluable "biological robot" because, when stimulated visually, it may exhibit complex patterns of behavior which reflect a subtle analysis of the environment. It has thus served as a testbed for studies of pattern recognition, depth perception, cooperative computation, obstacle avoidance, and simple forms of learning. We study visuomotor coordination in frog and toad at two levels: a) to analyze what schemas (information processing "modules") are employed in actual brains; b) to model how schemas are implemented in living neural networks. Here a central theme is the analysis to tectal circuitry. Results of this proposed study will lead to a better understanding of sensorimotor integration in vertebrates and the computational techniques with which neural function can be understood. We have made interaction with experimentalists an integral part of our research to date; we now propose to augment these interactions by work in our own laboratory in which collection of new data is closely integrated with modeling. In particular, we will study prey-catching, predator-avoidance and other natural behaviors of frog and toad, both with and without lesions, in a richly structured environment. This will yield models at the level of interacting schemas. Each schema will provide constraints on the neural circuitry within a given brain region. Models meeting these constraints will be tested both neuroanatomically and neurophysiologically.

In this research, neural network and schemalevel models of visual- motor conditional learning in monkeys will be extended to cover a wider range of reaching and grasping behaviors. These models will then be used as a basis for robot learning, including such paradigms as reinforcement learning, staged learning, focus of attention, and learning by showing. The resulting neural net strategies will then be used to enable a robot to learn to use visual and tactile input to grasp arbitrary objects, to construct assemblies of blocks, and to coordinate the motion of two arms. Existing tools for neural network simulation will be interfaced with software for robot sensing and control.

9411503 Arbib Fast movements appear to be "stereotyped," yet high speed videotaping reveals significant variability in stereotyped behaviors. Fast movements are usually composed of several separate patterns of muscle coordination called "motor synergies." For example, the rapid movements of a frog snapping at a worm, which take less than 1/6th of a second, is composed of a "lunge synergy" (jumping towards the worm) together with the "jaw-tongue synergy" (the jaw opens, the tongue projects to hit the worm, the tongue retracts bringing the worm into the mouth and the jaw closes). This study is designed to understand how the brain coordinates multiple synergies, and how synergies that occur later in the behavior use feedback to compensate for inaccuracies of synergies that occur earlier in the behavior. The study will involve analysis of the behavior by high speed video taping, and recording of signals from the muscles controlling the behavior and the parts of the brain and spinal cord controlling the muscles. Studies already done by these investigators show that both synergies are variable, and that tongue projection can compensate for inaccuracy in the lunge. Data collected from the study will be used in neural network modeling of adaptability and coordination of rapid movements. The results of this study could provide new insights into the control of rapid limb movements by the brain and control of robots.

The University of Southern California (USC) is collaborating with two institutions in Mexico City (ITAM and CINVESTAV) to advance the methodology of simulating complex systems, with especial application to neural models at different levels of analysis. To provide multi-level simulation we integrate our present neural simulation language NSL with the facilities for object-oriented management of hierarchical structures called schemas provided by the abstract schema language ASL. This will be linked to work in databases for the representation of biological data underway at USC as part of the Human Brain Project supported by several US agencies. Additionally, the project complements a second NSF-CONACyt collaboration, between Georgia Tech and ITAM, that studies the technological issues of `Ecological Robotics: A Schema-Theoretic Approach`. The methodology will provide simulation tools which will help neuroscientists gain a better understanding of the dynamical processes associated with biological phenomena, and a technology with implications far beyond the brain research community.

This proposal is a request for funds to support 3 predoctoral and 1 postdoctoral trainees in the area of Cognitive and Computational Neuroscience. Cognitive Neuroscience is an emerging discipline at the intersection of the cognitive and neurosciences. The goal of this research is to understand human behavior in terms of its brain bases. Computational modeling plays an essential role in this endeavor, providing a theoretical framework for understanding both high level cognition and basic brain mechanisms. The goal of the training program is to develop future cognitive neuroscientists who will be able to advance the goal of understanding brain- behavior relationships, using computational modeling as a primary tool. The preceptors are 12 faculty from the neuroscience doctoral program and the departments of psychology, linguistics, and computer science, from which trainees will be drawn. Research activities focus on three substantive areas: language; vision; and learning and memory. The training plan focuses on courses and other activities that will allow trainees to integrate cognition, computational, and neurosciences approaches in conducting research in these areas. The plan for the postdoctoral position focuses on permitting a student trained in psychology or linguistics to gain additional expertise related to cognitive and computational neuroscience.

EIA 0130900-Michael Arbib-Biological Information Technology Systems-BITS: Neural Computing, Round 3

There have been two main rounds of neural computing to date, the first focusing on adaptation and self-organization, the second on compartmental modeling of the neuron. This project will catalyze a third round of neural computing: Analyzing the architecture of the primate brain to extract neural information processing principles and translate them into biologically-inspired operating systems and computer architectures. This project will focus on analyzing and further developing computational neuroscience models concerned with grasping, recognizing and executing actions, and describing those actions with language, in terms of basic information processing principles. The intention is to create a new research effort, applying the latest advances in computational neurobiology to the design of a new generation of machines. In particular, the proposed research will catalyze research and development of unusually robust, versatile, and adaptive computer architectures, that can easily adapt, correct themselves, and blend diverse styles of processing.

With support from a National Science Major Research Instrumentation Award, Professor Irving Biederman and his colleagues at the University of Southern California will purchase a state of the art three Tesla functional Magnetic Resonance Imaging (fMRI) system for the scientific investigation of how cognitive, emotional, perceptual, memory, linguistic, and motor capacities emerge from activity of the human brain.

Joining Professor Biederman and his Co-PIs Z.-L. Lu, L. Itti, A. Raine, and M. Arbib as users of the fMRI system will be members of a variety of academic units including the Neuroscience Program, the Departments of Psychology, Computer Science, Biology, Gerontology, Biomedical Engineering, Kinesiology, Electrical Engineering, and the House Ear Institute, Currently the community of interested users includes approximately 30 faculty and over 100 graduate and post-doctoral students. This on-campus facility will not only allow these research programs to proceed but will provide the capability for the development of imaging expertise within this community. The magnet will be available to researchers from other institutions as well.

The ability to probe the activity-not just the structure-of the intact human brain has been one of the great methodological advances of neuroscience in the past decade. The instrument will provide high-resolution images of brain structures but its primary use will be to assess functioning of the brain as subjects experience various stimuli or perform various tasks while the system measures neural activity at specific brain loci in the order of a few millimeters. Among the first of the research projects that will be launched once the system is installed is one focusing on regions of the prefrontal cortex known to modulate restraint and an appreciation of the consequences of one's own actions for individuals with and without a propensity for impulsive violence. Other studies are designed to understand how an image of a scene, never perceived previously, could be comprehended in a fraction of a section. Another will assess whether brain-produced opiates in areas that mediate comprehension provide the perceptual and cognitive pleasure associated with novel but interpretable experiences. Another study is motivated by the finding that neurons in monkey cortex involved in the production of certain motor movements, such as grasping, also fire when the monkey views the grasp of another organism. This research will evaluate whether such "mirror" neurons might be the core imitative capacity fundamental to the evolution of language. Still another investigation will focus on where and how "episodic memory"-the mental diaries of our lives-are produced and stored in the brain.

Plans for the operation of the magnet, to be housed in the Dana and David Dornsife Cognitive Neuroscience Imaging Center, will include instructional courses designed to give hands-on training and research experience to undergraduate as well as graduate students.
Special outreach programs are designed to involve qualified high school students from the local community as part of an effort to provide opportunities for underrepresented minorities to be counted among the next generation of scientists advancing our knowledge of cognitive and behavioral neuroscience.

Humans are unique in that they can learn and use language. However, we share many brain mechanisms for perception and action with our primate cousins, the monkeys and apes. The proposed research will chart how these shared brain mechanisms provide the basis for the uniquely human brain mechanisms that support the learning, perception, and production of language. The key idea is based on the mirror system, a system for controlling both the execution and observation of action, found for grasping in moneys. This idea was extended "beyond the mirror" in humans to encompass mechanisms for language. The approach to the research is computational: to develop computer programs that make explicit the implications of ape and monkey data and human data for analysis of the human language-ready brain.

The study of language within a more general understanding of action and perception may ground new approaches to language education and the treatment of language disorders. In particular, linguists have long debated what it is about language that is innate and what reflects the child's social learning. The assumptions of different schools of thought clearly influence approaches to language education. By laying bare underlying brain mechanisms that include an account of neural plasticity, the proposed research has the potential to transform the innateness debate in ways that can inform educators. To support this, the research will also develop a new information infrastructure, the Brain Operation Database, which will provide a repository of summaries of the empirical data used to test and develop models, descriptions of the models themselves together with summarized simulation results, and tools for comparing the success of different models in explaining known data and supporting predictions for new empirical research.


This work is co-funded by SBE/BCS and CISE/IIS/RI.

This INSPIRE award is partially funded by the Perception, Action, and Cognition Program in the Division of Behavioral and Cognitive Sciences in the Directorate for Social, Behavioral, and Economic Sciences and the Robust Intelligence Program in the Division of Information and Intelligent Systems in the Directorate of Computer and Information Science and Engineering.

This research will address and bridge two grand challenges: (1) To understand how action, perception, and social interaction were supported by the brain of the last common ancestor of macaque and human, complementing modeling elsewhere on great apes, and (2) To build on evolutionary insights to better understand how different parts of the human brain work together when we use language. Key entry points will be signed and spoken languages and the use of hand gestures (e.g., novel hand gestures by apes) to convey meaning. Going further, a particular focus will be on systems that link the brain's capacities to generate as well as recognize actions, and their interactions with other brain systems.

An international group of scientists in linguistics, primatology, neuroanatomy, neurophysiology, and neurocomputational modeling of motor, cognitive and language processes will pool data on the anatomy, physiology, behavior and communication of the various primate species. To support this extended collaboration, the researchers will build a novel online collaborative environment ("Collaboratory Workspaces") to test, make predictions, and challenge both the modeling and experimentation. This infrastructure may catalyze a new style of collaboration between modelers, experimentalists, and clinicians.

The research also has the potential to support modeling of the damage that results in the clinical disorders of apraxia and aphasia. Integration of models of vision, action and language is also important for creating robots that can flexibly and usefully interact with individual people and for "neuromorphic architecture," in which a building's sensors and action systems adaptively adjust to the human inhabitants.