Timothy K. Horiuchi - US grants

PROPOSAL NO: 0347573
INSTITUTION: University of Maryland College Park
PRINCIPAL INVESTIGATOR: Horiuchi ,Timothy
TITLE: Adaptive Neuromorphic VLSI for Improving Accuracy and Precision:
Modeling Attention for Bat Echolocation


Abstract

Echoes are a major part of our daily auditory lives, yet very few of us are even aware that we use them constantly to understand the space around us. Echolocating bats have made these echoes their primary sense (i.e. sonar), demonstrating agile flight in complex forest environments and successful insect capture in flight; all in darkness. Because the brain represents a computational engine that far surpasses any single man-made computer in both capabilities and power consumption, we are interested to understand the principles of neural computation.

For a number of years we have been constructing custom analog integrated circuits that mimic the basic neural processing of bat echolocation in an effort to demonstrate these principles on a small flying vehicle, requiring real-time performance, miniaturization, and power efficiency.

The 'neuromorphic' very-large-scale-integration (VLSI) approach (which implements neural computations by mimicking its structural morphology), offers advantages in size and power, but often lacks needed precision and flexibility. This has been one of the main concerns about this research field's long-term viability. In this proposal we pursue three main goals: 1) to expand the use of learning and adaptation techniques at the circuit level to increase computational precision and accuracy at the system-level, 2) to explore the use of 'virtual' wiring to mimic large-scale wiring changes akin to neural development, and 3) to adapt existing neural models of spatial attention for use in sampled sensory systems (e.g., sonar) to guide this learning process.

Through this research and our educational plan we hope to raise awareness to the rich acoustic cues around us and how we as humans and the machines we build can learn to make use of them.

The Telluride Workshop on Neuromorphic Cognition Engineering
Neuromorphic engineers design and fabricate artificial neural systems whose detailed architecture, design, and computational principles are based on those of biological nervous systems. Over the past 12 years, this research community has focused on the understanding of low-level sensory processing and systems infrastructure; efforts are now expanding to apply this knowledge and infrastructure to addressing higher-level problems in perception, cognition, and learning.
The annual three-week intensive Workshop (held in Telluride, Colorado) consists of background lectures (from leading researchers in biological, cognitive, computational, engineering and learning sciences), practical tutorials (from state-of-the-art practitioners), hands-on projects (involving established researchers and newcomers/students), and special interest discussion groups (proposed by the workshop participants). For researchers in this community, this is the premier workshop for training students, initiating collaborations, and in-depth discussions on scientific issues.
In this workshop and through the Institute for Neuromorphic Engineering (INE), the mission is to promote interaction between senior and junior researchers; to educate new members of the community; to introduce new enabling fields and applications to the community; to promote on-going collaborative activities emerging from the Workshop, and to promote a self-sustaining research field.
Specific Goals for the period of 2007-2012: While there is no question that the Workshop has been very successful in its mission, three new challenges have been identified for the Workshop: 1) with a rapidly expanding community in both the U.S. and Europe, the Workshop experience needs to reach more people without increasing the size of the Workshop, 2) as larger and more challenging projects are tackled, more opportunities for group interactions are needed throughout the year, and 3) as more complex questions are asked at the system-level, more voices from cognitive neuroscience are needed.
To meet these new challenges, a new version of the Workshop is envisioned with: 1) an expanded theme to focus on Perception, Cognition, and Learning, 2) an expanded constituency, educational mandate and research focus to incorporate members of the NSF Science of Learning Centers (SLC), 3) to create a two-part Workshop series (to allow yearlong collaborations and deeper investigation into large scale projects), one held in the U.S. and funded by U.S. resources and the other held in Europe and supported by European resources and 4) a modified Workshop schedule to emphasize training at the beginning of the workshop to provide a needed focus on education for both beginners and experts alike. The infusion of new researchers (from the SLCs) that focus on learning at multiple scales (from synapses to classroom) will provide the needed knowledge, new collaborations, and new perspectives to move the community towards cognitive-level neuromorphic systems.
Broader impact of the Workshop to the public: The Telluride Neuromorphic Cognition Engineering Workshop will continue its tradition of public interaction. In particular, there will be a continuation of the educational program for K-12 students (based on neuromorphic/robotics design kits), undergraduate and graduate students (Workshop courses, new classes/lectures at participants? universities and REU), and to established researchers (exposure to new areas in the field). The workshop will also continue to educate the Telluride community with public lectures on the latest developments/issues in the field.
Recruitment of minorities and women to the field will be continue by organizing lectures at various Universities, particularly HBCUs (Morgan State U., MD, Lincoln U., PA, Morehouse College, Atlanta, GA, and others). By sending presenters to institutions local to their home universities, minimal funding will be required and provide the most likely connections for future collaborations. The Institute for Neuromorphic Engineering, currently housed at the University of Maryland (College Park, MD), will arrange the logistics. The lectures and other teaching materials developed at the workshop will also be made available to all interested parties and posted on the INE website.
Lastly, the workshop will continue to develop the researchers and leaders for the emerging field of biologically-inspired systems, cognitive/learning systems, robotics and implantable electronics. Various agencies and governments have recognized that smart devices (such as interactive humanoid robots) that mimic living organisms will have great academic and commercial value in future.

The broad goal of this project is to understand the processes that support perception and action in complex settings. The research focuses on spatial perception and navigation in the echolocating bat, an auditory specialist that produces high frequency sonar calls and listens to echo returns to determine the location of objects in its environment. The echolocating bat modifies its sonar calls in response to echo information from targets (insect prey) and obstacles, and quantitative analyses of this animal?s adaptive vocal behavior will be used to infer its perception of a changing environment. The biological component of this research combines behavioral and neurophysiological experiments to gain insight to how sensory information from complex scenes is coded and used to guide behaviors. Analysis of behavioral and neural data will be coordinated with modeling efforts and the development of a robotic spatial navigation system. Together, the biological and engineering arms of this research project will generate new knowledge that contributes to a deeper understanding of perception and action in complex, natural environments. Students and postdocs working on this project will learn to translate knowledge and methodologies across biology and engineering, ranging from ethology and neurobiology to computational modeling and robotic demonstrations. These individuals will be poised to make major contributions that impact both basic science and future technology, enabling breakthroughs that cannot be achieved through work solely within traditional disciplines. This research project will contribute to a rich library of multimedia materials that will be made available to educators and scientists working in both the private and public sectors: http://www.bsos.umd.edu/psyc/batlab/movies.html. Collectively, this research has wide-ranging impact for neurobiology, interdisciplinary research training, neuroscience techniques, robotics, and the design of assistive devices.

This INSPIRE award is partially funded by the Science of Learning Centers Program in the Division of Behavioral, Cognitive and Social Sciences in the Directorate for Social, Behavioral and Economic Sciences; the Perception, Action, and Cognition Program in the Division of Behavioral, Cognitive and Social Sciences in the Directorate for Social, Behavioral and Economic Sciences; the Energy, Power, and Adaptive Systems Program in the Division of Electrical Communication and Cyber Systems in the Directorate of Engineering; and the Applied Mathematics and Mathematical Biology Program in the Division of Mathematical Sciences in the Directorate for Mathematical and Physical Sciences. This research project draws on knowledge from many disciplines (neuroscience, cognitive science, computational science, mathematics and engineering) to create cognitive systems capable of interpreting observed, complex human movements and actions. New design methodologies will be developed for the integration of sensory modalities (vision, audition, touch) and their support of higher cognitive function (language, reasoning). In contrast to existing approaches which tend to be assemblies of modular components each solving its task in isolation, this team takes a novel approach called Active Cognition which has the following features: 1) Instead of modeling the different perceptual processes (vision, audition, and haptics), cognition, and motor control in isolation, the modules are integrated and capabilities co-developed in the tradition of dynamical systems theory to obtain a reasoning system where "the whole is greater than the sum of its parts"; 2) instead of segregating the low level processing of signals from the processing of higher level symbolic information, they will interact in a continuous dialogue, such that high level knowledge will leverage perception; and 3) instead of separating physical embodiment from algorithmic considerations, biologically inspired real-time hardware will be developed that implements complex functions by integrating signals and symbols. The project is organized in two working groups. The first group will develop a cognitive robot that can recognize complex human activities using visual and auditory signals captured by biological-inspired hardware. The second group will study attention in humans by measuring human response to audition and vision through EEG and MEG, and subsequently implementing the findings in robots. A yearly three-week, hands-on workshop will educate students, serve as testing ground for the team's ideas, and stimulate new collaborations. This workshop will also engage the involvement of the interdisciplinary research community that has formed around the goal of building biologically inspired cognitive systems.

Success in integrating different components of a cognitive system (hardware, sensors, and software) has the potential to catalyze a new industry of biologically-inspired cognitive systems, including household and service robots, and systems for intelligent transportation and smart manufacturing. In addition, this interdisciplinary project will play a significant role in building capacity for a new emphasis area in engineering and training of cognitive systems engineers who need combined expertise in computer science, electrical engineering and cognitive neuroscience.

The motivation for this biologically-inspired approach is to design systems that perceive and act in cluttered and noisy scenes that they have never experienced. This stands in contrast with the state of the art in computational engineering systems that need to be re-trained each time they confront an unanticipated environment. The main reason is that current approaches to perception address specific problems in isolation and do not consider that the primary role of perception is to support systems with bodies in action. As a result, they are constrained to the situations for which they were trained and cannot react to changing tasks and scenes. By focusing on cognition primitives rather than specific applications, the work is expected to greatly advance the state of the art of machine perception and lead to the development of systems that can robustly and on-line adapt to new environments, react to novel situations and learn new contexts. To do so, novel theoretical formulations of perception and action and high-speed, low-power, hardware implementations with on-line learning capabilities will be studied while assimilating new insights from the neurosciences. Consequently, this work will network neuroscience, cognitive science, applied mathematics, computer science and engineering so as to lower one of the few remaining barriers that keeps interactive robots in the realm of science fiction. Beyond the scholarly contribution, the work is expected to provide know-how for the design of systems with adaptive perception in a modular fashion with reusable components. Such systems have applications in computational vision and auditory perception problems and can advance the industry of cognitive biologically-inspired robotics and assistive devices.

This proposal sets forward novel ideas in the design of intelligent perceptual systems and the development of synthetic intelligence. Just about any task which an intelligent system solves involves the interplay of four basic processes that are devoted to: (a) context, (b) attention, (c) segmentation and (d) categorization. The members of the proposed network will study these canonical cognitive primitives by combining neural modeling with neural and behavioral experiments, theoretical and computational modeling and implementation in robotics. The findings of theoretical insights will then be adapted to satisfy the demands of realistic behavior, and to develop technological solutions for applications of robust and invariant perception and action. The proposed collaborative network will consist of a small science and engineering research team to directly address the questions in robust adaptive perception and action. It will then direct personnel, and inject results and pedagogical content to a Summer Workshop that aims to include a global network of researchers.