Michael A. Arbib - US grants

Our approach to analyzing mechanisms of visuomotor coordination is two-fold: "top-down", to offer a coordinated control program of interacting schemas (concurrently active programs suitably structured for action-perception systems) to model behavior noted by neuroethologists; and "bottom-up", to provide detailed models of neural networks which are consistent with known anatomy and physiology, but which involve additional assumptions, amenable to experimental test, to yield a network capable of exhibiting appropriate behavior. Where possible, we implement a family of models, rather than a single model, to facilitate computer experiments and animal experiments which may discriminate between alternative mechanisms. Specifically, we plan to conduct studies of the following subtopics: (1) Modelling tectum-pretectum interactions subserving pattern-recognition in anurans; (2) Detailed neural modelling of retina; (3) Modelling patterns of behavior in frog and toad while developing a theory of motor schemas and their coordination for orientation, sidestepping, snapping, etc.; and (4) Developing mathematical and computer methodology for exploring a model-family to determine which subset of models approximates satisfaction of a given set of constraints. The work will make modelling tools available for the analysis of large brain systems. Our research plan has build into it structures which ensure that these tools are accessible to neurophysiologists, neuroanatomists and neuroethologists, while also facilitating a full exchange of ideas with the computer science community (artificial intelligence, simulation methodology, interactive computer graphics, etc.).

The frog has proved an invaluable "biological robot" because, when stimulated visually, it may exhibit complex patterns of behavior which reflect a subtle analysis of the environment. It has thus served as a testbed for studies of pattern recognition, depth perception, cooperative computation, obstacle avoidance, and simple forms of learning. We study visuomotor coordination in frog and toad at two levels: a) to analyze what schemas (information processing "modules") are employed in actual brains; b) to model how schemas are implemented in living neural networks. Here a central theme is the analysis to tectal circuitry. Results of this proposed study will lead to a better understanding of sensorimotor integration in vertebrates and the computational techniques with which neural function can be understood. We have made interaction with experimentalists an integral part of our research to date; we now propose to augment these interactions by work in our own laboratory in which collection of new data is closely integrated with modeling. In particular, we will study prey-catching, predator-avoidance and other natural behaviors of frog and toad, both with and without lesions, in a richly structured environment. This will yield models at the level of interacting schemas. Each schema will provide constraints on the neural circuitry within a given brain region. Models meeting these constraints will be tested both neuroanatomically and neurophysiologically.

9411503 Arbib Fast movements appear to be "stereotyped," yet high speed videotaping reveals significant variability in stereotyped behaviors. Fast movements are usually composed of several separate patterns of muscle coordination called "motor synergies." For example, the rapid movements of a frog snapping at a worm, which take less than 1/6th of a second, is composed of a "lunge synergy" (jumping towards the worm) together with the "jaw-tongue synergy" (the jaw opens, the tongue projects to hit the worm, the tongue retracts bringing the worm into the mouth and the jaw closes). This study is designed to understand how the brain coordinates multiple synergies, and how synergies that occur later in the behavior use feedback to compensate for inaccuracies of synergies that occur earlier in the behavior. The study will involve analysis of the behavior by high speed video taping, and recording of signals from the muscles controlling the behavior and the parts of the brain and spinal cord controlling the muscles. Studies already done by these investigators show that both synergies are variable, and that tongue projection can compensate for inaccuracy in the lunge. Data collected from the study will be used in neural network modeling of adaptability and coordination of rapid movements. The results of this study could provide new insights into the control of rapid limb movements by the brain and control of robots.

The University of Southern California (USC) is collaborating with two institutions in Mexico City (ITAM and CINVESTAV) to advance the methodology of simulating complex systems, with especial application to neural models at different levels of analysis. To provide multi-level simulation we integrate our present neural simulation language NSL with the facilities for object-oriented management of hierarchical structures called schemas provided by the abstract schema language ASL. This will be linked to work in databases for the representation of biological data underway at USC as part of the Human Brain Project supported by several US agencies. Additionally, the project complements a second NSF-CONACyt collaboration, between Georgia Tech and ITAM, that studies the technological issues of `Ecological Robotics: A Schema-Theoretic Approach`. The methodology will provide simulation tools which will help neuroscientists gain a better understanding of the dynamical processes associated with biological phenomena, and a technology with implications far beyond the brain research community.

EIA 0130900-Michael Arbib-Biological Information Technology Systems-BITS: Neural Computing, Round 3

There have been two main rounds of neural computing to date, the first focusing on adaptation and self-organization, the second on compartmental modeling of the neuron. This project will catalyze a third round of neural computing: Analyzing the architecture of the primate brain to extract neural information processing principles and translate them into biologically-inspired operating systems and computer architectures. This project will focus on analyzing and further developing computational neuroscience models concerned with grasping, recognizing and executing actions, and describing those actions with language, in terms of basic information processing principles. The intention is to create a new research effort, applying the latest advances in computational neurobiology to the design of a new generation of machines. In particular, the proposed research will catalyze research and development of unusually robust, versatile, and adaptive computer architectures, that can easily adapt, correct themselves, and blend diverse styles of processing.

This INSPIRE award is partially funded by the Perception, Action, and Cognition Program in the Division of Behavioral and Cognitive Sciences in the Directorate for Social, Behavioral, and Economic Sciences and the Robust Intelligence Program in the Division of Information and Intelligent Systems in the Directorate of Computer and Information Science and Engineering.

This research will address and bridge two grand challenges: (1) To understand how action, perception, and social interaction were supported by the brain of the last common ancestor of macaque and human, complementing modeling elsewhere on great apes, and (2) To build on evolutionary insights to better understand how different parts of the human brain work together when we use language. Key entry points will be signed and spoken languages and the use of hand gestures (e.g., novel hand gestures by apes) to convey meaning. Going further, a particular focus will be on systems that link the brain's capacities to generate as well as recognize actions, and their interactions with other brain systems.

An international group of scientists in linguistics, primatology, neuroanatomy, neurophysiology, and neurocomputational modeling of motor, cognitive and language processes will pool data on the anatomy, physiology, behavior and communication of the various primate species. To support this extended collaboration, the researchers will build a novel online collaborative environment ("Collaboratory Workspaces") to test, make predictions, and challenge both the modeling and experimentation. This infrastructure may catalyze a new style of collaboration between modelers, experimentalists, and clinicians.

The research also has the potential to support modeling of the damage that results in the clinical disorders of apraxia and aphasia. Integration of models of vision, action and language is also important for creating robots that can flexibly and usefully interact with individual people and for "neuromorphic architecture," in which a building's sensors and action systems adaptively adjust to the human inhabitants.