Tomaso A. Poggio - US grants

The measurement and use of visual motion is a fundamental component of biological and machine vision systems that provides essential sensory information for tasks such as navigation, object manipulation and recognition. Significant advances have been made toward understanding how vision systems might solve the individual problems of detecting sudden movements, segmenting the scence into distinct objects on the basic of motion discontinuities, tracking objects of interest, recovering the three-dimensional structure and movement of object surfaces, and inferring their own movement relative to the environment. This research examines how solutions to these problems are integrated into a motion analysis that performs these functions with speed, accuracy, reliability and flexibility. Such a system must embody multiple computational strategies that combine fast and robust methods for deriving qualitative motion information with slower, accurate methods for deriving quantitative models of three- dimensional structure and motion. The approach taken in this project brings together theoretical analyses, implementation and testing of computer algorithms, and observations on human motion perception. This research will lead both to significant improvements in the performance of computer vision systems at analyzing dynamic images, and new understanding of motion analysis in the human visual system.

The proposed research focuses on building vision systems with supercomputer capability into fast, small, low-power, analog integrated circuits. Examples where visual acuity and dexterity would be useful in products include collision sensors for cars, ground speed detectors for anti-skid lock brakes, navigation systems for mobile robots, perception sensors for micro robots and image pre-processors for remote sensing instruments. Current supercomputers are too large, too general and too complex to be cost-effective for such applications. Vision algorithms will be implemented directly in silicon through analog networks. Computation performed this way is the ultimate in parallelism, is inherently low power and compiles to a very small package size because sensors can be integrated directly with computational networks. Single chip sensor systems to be useful in the real world however, must be adaptive and self-calibrating. Designing adaptive, flexible, smart sensors requires extensive simulation. In fact, for simulations to complete in any reasonable time frame, computational assets on the order of supercomputer capability are essential. The research proposed is to utilize today's general purpose supercomputers to develop the appropriate algorithms for designing tomorrow's application specific single- chip supercomputers (analog vision chips). The Connection Machine, a 64,000 processor supercomputer, will be used for algorithm simulation and device design of these self-calibrating, adaptive vision chips. Standard computer vision algorithms bog down even the fastest computers in the world. For most vision applications, commercial supercomputers would not be feasible. For example, an automobile collision detection system must be small, low-cost, and consume little power. For such applications, general-purpose supercomputers would not be satisfactory (even if they were fast enough). The solution is to utilize special-purpose custom analog VLSI chips. These analog chips are fast, low-power, cheap and small. I have successfully built and tested more than a dozen different analog VLSI chips during my Ph.D. work at Caltech. These chips perform various smoothing, segmentation and interpolation algorithms using input from on-chip photosensors or scanned-in test data. Recently I am investigating some simple motion and stereo ideas.

9504054 Poggio The research project consists of three parts. First, the investigator plans to sharpen the tools to analyze the sample complexity question -- how many examples does the learner need to generalize well? This is related to the complexity of the model the learner is using to fit the data and generalize to unseen data. Implications for model selection and data mining will be explored. Second, it is proposed to develop active algorithms which choose their own examples. This reduces the sample complexity of learning at the cost of an increased computational burden. Access to high performance computing will help greatly. Applications to function approximation, pattern classification and system identification will be explored. Finally, the tools developed earlier will be applied to the domain of natural languages. In particular, the sample complexity of learning grammar will be investigated. At the same time the evolution of human languages can be modeled as a dynamical system. This can be used as an evolutionary criterion to choose between different linguistic theories (models). ***

This work will exploit learning techniques recently developed to work towards computers that learn to detect and recognize people, estimate user's gestures and communicate visually with them via a photorealistic computer-generated human face. In particular, the plan is to use tow main theoretical and algorithmic approaches to learning: Support Vector Machines and Hidden Markov Models. With these tools, two key aspects of a trainable man-machine interface will be developed: An analysis module that can be trained to estimate in real time facial expressions of the user and associated physical parameters and a synthesis module that can be trained to generate image sequences of a real human face synchronized to a text-to-speech system. The significance of the work is three-fold: (1) The project will contribute to the development of a new generation of computer interfaces more user-friendly and human-centered than today's interfaces. Such interfaces will be of direct use in education and as components of prostheses for the disadvantaged; (2) The project will integrate recently developed learning techniques to real time vision and graphics applications; and (3) The project will explore the boundaries of what is possible to achieve using 2D representations of faces rather than the more common, physically-based, 3D-based models.

Modern statistical learning approaches are expected to play a key role in providing more powerful tools to harvest information from bits, a crucial and growing problem for the Internet. The goal of this project is thus to develop a new technology for the management, organization, and search of multimedia digital information by exploiting and extending new statistical learning theories and algorithms. In the process we expect to prototype key system components and to develop scientific insights. Anticipated outcomes of the research are (1) new learning algorithms and associated representations that can be applied to categorize text, images, and video, (2) new theoretical analyses of these learning algorithms and query-answering methods and (3) demonstrations and evaluations of prototype systems for classifying and routing email messages and searching, categorizing, and extracting information on the Web.

Smarter classification software for multimedia data is a prerequisite to enable a second, more intelligent wave of Internet technologies. Automatic techniques to route, organize and search information are needed to help individuals and organizations exploit the sea of data that the computer networks are creating. The success of projects like this will make such a step possible and accelerate the evolution of the Internet.

EIA-0218506
Burge, Christopher B
MIT

CRCNS: Bioinformatics of Alternative Splicing in the Nervous System

Almost every human cell contains a huge instruction manual called the genome with many thousands of pages (the genes), each of which tells the cell how to make a particular building block (protein) that it needs to live or grow or to perform its assigned function in the body. The cell uses this manual in a complicated way, first copying (transcribing) each page that it needs to a piece of scratch paper (the pre-mRNA), and then cutting and pasting (splicing) pieces of the scratch paper (the exons) together to form the final recipe (mRNA) for the protein product. Interestingly, this cutting and pasting is often carried out in different ways in different types of cells or under different conditions in a process called alternative splicing (AS), generating many different varieties of a protein under different conditions. Alternative splicing is particularly common in neurons, helping to generate protein variants whose properties are optimized to the local environment of the neuron. For example, AS is used to tune the electrical properties of ion channels which help different sensory neurons in the inner ear respond to different frequencies of sound. In addition, mutations that affect AS are associated with a number of neurodegenerative diseases. The goal of the proposal is to gain a better understanding of the signals in a gene that determine how that gene will be spliced when it is expressed in a particular part of the brain, and of how alternative splicing is used to modulate brain function.

The central focus of our Center is to develop a framework for studying the neural computations underlying object recognition and visual attention in visual cortex. The Center's framework is based on the collaboration of labs working on monkey physiology, cat physiology and human psychophysics with a quantitative computational theory providing the main conduit through which experimental results in one lab affect experiments in another lab. The model that flows from the theory provides a novel way to drive a collaborative enterprise, providing a way to integrate the data, to check their consistency, to suggest new experiments and to interpret the results. The theory itself, based on two existing models for recognition and attentional saliency, will not only guide the experiments and drive synergies between different labs but will also evolve as an effect of the experimental results. The research is organized into three main projects, defined by geographical location and scientific questions, rather than discipline. In the MIT project, the labs of Tomaso Poggio, Earl Miller and James DiCarlo will be guided by a quantitative hierarchical model of recognition, probing the relations between identification and categorization and the properties of selectivity and invariance of recognition, especially with natural image clutter, in IT and PFC cortex of behaving macaque monkeys. In the Northwestern project, the lab of David Ferster will test a key prediction of the model about the nature of the pooling operation (a max operation vs. a linear sum) performed by complex cells in area 17, using very similar stimuli affected by clutter. The experiments will be done in the anesthetized cat, intracellularly, to allow for a characterization of the underlying circuit and biophysical mechanisms. In the Caltech project, the lab of Christof Koch will collaborate with Ferster lab on biophysical simulations of Vl circuits. It will also test -- using human psychophysics with stimuli configurations similar to the ones used by Jim DiCarlo -- the conditions under which attention is needed in recognition of natural objects and scenes; from the data, in collaboration with Tomaso Poggio, Koch's lab will extend the basic model of recognition by integrating it with the existing model of bottom-up saliency. A unique aspect of our Center is that, the computational component, centered around a quantitative theory of recognition, is the generic tool that drives interactions between the investigators, in addition to the standard pairwise interactions: the model suggests experiments and guides their planning and interpretation; the experimental results from one lab impact, through the model, work done in another lab, including model development, as well as their interpretation and what to do next. Ultimately, the whole process should lead to a better and more coherent understanding of the neural mechanisms of visual recognition.

Normal vision is not static: time is a key dimension of the natural world we see. The eventual understanding of biological vision requires understanding the neural mechanisms used to recognize objects and actions over time. Thus the focus of the proposed research is to study how the primate visual system recognizes objects and actions in time sequences of images. A meta-goal of this project is to exploit the synergies between computational approaches and physiological experiments to lead to a better understanding of brain function and at the same time to develop better computer vision algorithms. Object recognition in time sequences of images presents a significant challenge for recognition systems, because it requires both selectivity to shape and invariance to changes of appearance in time..
This project will extend an existing computational model of the ventral stream by adding temporal dynamics in its model neurons and the ability to process video sequences. It will also expand a working model of the dorsal stream to understand the relative roles that it and the ventral stream play in dynamic visual recognition. At the same time, recordings from single units, and multiple single units, from high level visual areas including IT and regions of the STS will be made in order to characterize the tuning of single neurons to the shape dynamics of specific image sequences. By combining modeling and physiology, this work will search for a computational explanation for how the higher areas of the visual cortex recognize objects and actions over time and how they can learn.
This integrative effort, which is focused on processing of dynamic perceptual information, can have a significant and direct impact on current theories of autism, dyslexia, and effects of stroke, in addition to directly guiding modeling and engineering efforts in computer vision. The proposed research is tightly coupled to education and teaching, and resources used in the research, including databases of videos, visual stimuli, the modeling software and the experimental data will be made available to the broad scientific community. Information on the project and its progress will be available at http://cbcl.mit.edu/projects/NSF-CRCNS/index.html

Today's AI technologies, such as Watson, Siri and MobilEye, are impressive yet still confined to a single domain or task. Imagine how truly intelligent systems --- systems that actually understand their world --- could change our world. The work of scientists and engineers could be amplified to help solve the world's most pressing technical problems. Education, healthcare and manufacturing could be transformed. Mental health could be understood on a deeper level, leading in turn to more effective treatments of brain disorders. These accomplishments will take decades. The proposed Center for Brains, Minds, and Machines (CBMM) will enable the kind of research needed to ultimately achieve such ambitious goals. The vision of the Center is of a world where intelligence, and how it emerges from brain activity, is truly understood. A successful research plan for realizing this vision requires four main areas of inquiry and integrated work across all four guided by a unifying theoretical foundation. First, understanding intelligence requires discovering how it develops from the interplay of learning and innate structure. Second, understanding the physical machinery of intelligence requires analyzing brains across multiple levels of analysis, from neural circuits to large-scale brain architecture. Third, intelligence goes beyond the narrow expertise of chess or Jeopardy-playing computers, bridging several domains including vision, planning, action, social interactions, and language. Finally, intelligence emerges from the interactions among individuals ? it is the product of social interactions. Therefore, the research of the Center engages four major research thrusts (Reverse Engineering the Infant Mind, Neuronal Circuits Underlying Intelligence, Integrating Intelligence, and Social Intelligence) with interlocking teams and working groups, and a common theoretical, mathematical, and computational platform (Enabling Theory).

The intellectual merit of the Center is its focus on elucidating the mechanisms and architecture of intelligence in the most intelligent system known: the human brain. Success in this project will ultimately enable us to understand ourselves better, to produce smarter machines, and perhaps even to make ourselves smarter. The Center's potential legacy of a deep understanding of intelligence, and the ability to engineer it, is tantalizing and timeless. It includes the creation of a community of researchers by programs such as an intensive summer school, technical workshops and online courses that will train the next generation of scientists and engineers in an emerging new field -- the Science and Engineering of Intelligence. This new field will catalyze continuing progress in and cross-fertilization between computer science, math and statistics, robotics, neuroscience, and cognitive science. Sitting between science and engineering, it will attract growing interest from the best students at all levels. The broader impact of the Center program could be to revolutionize K-12, and also 0-K, and 12-life with a deeper understanding of the process of learning. The ability to build more human-like intelligence in machines will transform our productivity, enabling robots to care for the aged, drive our cars, and help with small-business manufacturing. The Center team is composed of over 23 investigators, many having already made significant accomplishments in multiple research areas relevant to the science and the technology of intelligence. The Center team has a mix of junior and senior researchers, bringing expertise in Computer Science, Neuroscience, Cognitive Science and Mathematics. The institutional partners include nine institutions (MIT, Harvard, Cornell, Rockefeller, UCLA, Stanford, The Allen Institute, Wellesley, Howard, Hunter and the University of Puerto Rico), three of which have significant underrepresented student populations. The academic institutions are complemented by the Center's industrial partners (Microsoft, IBM, Google, DeepMind, Orcam, MobilEye, Willow Garage, RethinkRobotics, Boston Dynamics) and by world-renowned researchers at international institutions (Max Planck Institute, The Weizmann Institute, Italian Institute of Technology, The Hebrew University).

A truly comprehensive theory of machine learning has the potential of informing science and engineering in the same profound way Maxwell’s equations did. It was the development of that theory by Maxwell that truly unleashed the potential of electricity, leading to radio, radars, computers, and the Internet. In an analogy, deep learning (DL) has found over the past decade many applications, so far without a comprehensive theory. An eventual theory of learning that explains why and how deep networks work and what their limitations are may thus enable the development of even more powerful learning approaches – especially if the goal of reconnecting DL to brain research bears fruit. In the long term, the ability to develop and build better intelligent machines will be essential to any technology-based economy. After all, even in its current – still highly imperfect – state, DL is impacting or about to impact just about every aspect of our society and life. The investigators also plan to complement their theoretical research with the educational goal of training a diverse population of young researchers from mathematics, computer science, statistics, electrical engineering, and computational neuroscience in the field of machine learning and of its theoretical underpinnings.

The investigators propose to join forces in a multi-pronged and collaborative assault on the profound mysteries of DL, informed by the sum of their experience, expertise, ideas, and insight. The research goals are threefold: to develop a sound foundational/mathematical understanding of DL; in doing so to advance the foundational understanding of learning more generally; and to advance the practice of DL by addressing its above-mentioned weaknesses. Of six foundational thrusts, the first two focus on the standard decomposition of the prediction error in approximation and sample (or estimation) error. Their goal is to extend classical results in approximation theory and theory of learnability to DL. These two are then supported by a research project that is specific to deep learning: analysis of the dynamics of gradient descent in training a network. The fourth theme is about robustness against adversaries and shifts, a powerful test for theories which is also important for practical deployment of learning systems. The fifth thrust is about developing the theory of control through DL, as well as exploring dynamical systems aspects of deep reinforcement learning. The final topic connects research on DL to its origins - and possibly its future: networks of neurons in the brain. The proposed research also promises to advance the foundations of learning theory. Success in this project will result in sharper mathematical techniques for machine learning and comprehensive foundations of machine learning robustness, broadly construed. It will also ultimately enable development of learning algorithms that transcend deep learning and guide the way towards creating more intelligent machines, and shed new light on our own intelligence.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.