David Mumford - US grants

Behavioral studies of pigeons probe the higher levels of biological vision. Preliminary studies with complex visual stimuli (including natural scenes) have shown great similarities between pigeon vision and early human vision, despite an evolutionary gap of 100 million years. This suggests the existence of extraordinarily robust biological solutions to visual analysis. Further experiments on visual shape categorization are to be performed on human and pigeon subjects. The goals are 1) to infer underlying algorithms in the two species; 2) to examine adaptation of these algorithms to the circumstances of their use; 3) to formalize the algorithms and their adaptive capacities mathematically; 4) to synthesize biological visual categorization on a computer; and 5) to develop experimental procedures with the long-term goal of creating artificial visual machines.

This one-year grant will permit Dr. Yuille and one student to extend their preliminary work in representation and visual recognition of human faces. Their current system can locate strong features such as eyes and mouths by using parametric adaptation of deformable templates. They are now applying robust statistical methods to identification of partially occluded or unresolved features, and will also study pyramid algorithms for matching a global face template. The goal is robust face recognition in the presence of noise and distractions.

9352905 Hallett The Calculus Consortium based at Harvard University, with funding from the National Science Foundation, has developed, tested and disseminated an innovative single variable calculus course. The course is currently being used at over 125 colleges around the country and abroad. In this project, the effort is being expanded to include precalculus and the second year of calculus. A major component of the proposed project is dissemination. Information about the Consortium's single variable materials has reached a large number of faculty at diverse institutions. The dissemination effort for the proposed project is being modeled on the workshops, minicourses, newsletters, test sites, and networking that have proved successful in the current project. ***

This project is being funded through the Learning & Intelligent Systems Initiative, and is supported in part by the NSF Office of Multidisciplinary Activities in the Directorate for Mathematical & Physical Sciences. Learning in many cognitive domains, including language and vision, involves recognition of complex hierarchical structure that is hidden or only indirectly reflected in the input data. In this project a multi-disciplinary group of applied mathematicians, cognitive scientists, computer scientists, linguists, and neuroscientists will study the learning of compositional structure in language, vision, and planning, and will also probe the neural mechanisms for identifying and exploiting such structure. The research involves three interacting lines of work. The first refines and extends statistical learning models; the second applies these models to language, vision, and planning; and the third develops and applies new experimental and analysis techniques for probing the neural mechanisms that learn and exploit compositional structure. The results of the project should significantly increase our understanding of complex learning, and should have implications for a wide range of topics in education (e.g., learning of complex knowledge structures in science and math) and technology (e.g., automated speech recognition, computer vision, robotics).

9870676 Johnson This Integrative Graduate Education and Research Training (IGERT) award will support the establishment of a broadly- based graduate training program in mathematical, cognitive and computational approaches to understanding how diverse cognitive processes, including language, movement and reasoning, are learned. The basic phenomena to be addressed are of significant commercial as well as scientific importance because, for example, they form the basis for design of speech and pattern recognition software, and for the design of robotic systems. The program is a joint effort of 13 faculty from the Departments of Applied Mathematics, Cognitive and Linguistic Sciences, and Computer Science. In combination with institutional resources, NSF funds will provide stipends for 12 graduate students, 2 postdoctoral students and 9 undergraduate students each year, as well as for related costs of student research training. Graduate students will be required to satisfy existing coursework requirements of their home departments and to take at least three new IGERT advanced topics courses meant to span the three participating disciplines. During the first two years of graduate school, each student will complete an interdisciplinary research project before beginning thesis research. All IGERT graduate trainees and faculty will also participate in a weekly research seminar, biannual retreats, a yearly week-long mini- course lead by a visiting researcher and a national conference to be organized during the second year of the project. Postdoctoral fellows will jointly organize one of the advanced topics courses and will themselves receive additional training to complement that received during their own graduate studies. Undergraduate students will become involved through active involvement in summer research projects in the groups of participating faculty. IGERT is a new, NSF-wide program whose goal is to sponsor the establishment of innovative, researc h-based graduate programs that will train a diverse group of new scientists and engineers to be well-prepared for varied careers in the private and public sectors. IGERT provides an opportunity for the development of new, well-focused multidisciplinary programs that bridge traditional organizational barriers, uniting faculty from several departments or institutions to establish a highly-interactive collaborative environment for both training and research. In its first year, the program will provide support to 17 institutions for new or nascent programs that collectively span all areas of science, engineering and mathematics eligible for support by the NSF.

The Division of Applied Mathematics at Brown University will purchase a high-end multi processor computing and visualization facility suitable for computationally intensive computing, postprocessing, and visualization of large data sets, which will be dedicated to the support of research in the mathematical sciences. The equipment will be used for several research projects, including in particular: Numerical Methods for Deterministic Optimal Control Problems, NSF Grant DMS-9704426 to Dupuis; High Order Methods for Shock Calculations and Computational Electromagnetics, NSF Grant DMS-9804985 to Shu and Gottlieb; Modeling Natural Image Statistics, NSF Grant DMS-9615444 to Mumford; Invariant Manifolds and Complex Behavior in Nonlinear Physical Systems, NSF Grant DMS-9800922 to Haller; and Continuum Physics and Systems of Conservation Laws, NSF Grant DMS-980352 to Dafermos.

With National Science Foundation support Dr David Cooper and his colleagues will develop a technology for the recovery of 3D free-form object and selected scene structure from one or more images and video. The technique is based on the development of 3D shape representation and a semi-interactive, mixed-initiative system, along with machine decision-directed Bayesian surface-estimation. The main focus of the effort is the development of tools useful in archaeological site and artifact reconstruction and architecture. This will impact low level shape models and how they are assembled to form either more complex objects or complete ones. The latter condition often occurs at archaeological excavation sites where objects are found in pieces, or have been damaged from erosion.

Despite recent substantial progress in automated scene recovery in computer vision, the challenges presented by free-form shape extraction and assembly are still great. The researchers will employ an interactive approach in which a user can guide the recovery process or can be available when requested for assistance by the machine, e.g., in the complex task of assembling many fragments to re-create a large or complicated object. The intent is to develop the fundamentals of a user-controlled technology for the benefit of the naive user. The ultimate goal is to enhance productivity in interacting with large amounts of complex visual data by modeling the underlying 3D structure.

This project will provide new practical tools. It will also provide an effective testbed for 3D shape reconstruction and recognition, more descriptive local and global models for working with 3D shapes, a better understanding of human/decision-making-machine interaction for free-form geometric modeling and for extracting 3D geometry from one or more images and video, and associated computational complexity issues. As applied to the field of archaeology, this technology will provide improved surveying methods, an advanced archaeological record and a means to conduct high-level analysis not heretofore available. It will further permit the transfer of reconstruction capabilities from the artist directly to the archaeologist. Since the problems resolved in the archaeological context are to a significant degree generic and thus transferable to the fields of medicine, industry, defense etc, the potential impact is extremely broad.

The investigator and his colleagues study mathematical, statistical, and computational questions motivated by the problem in computer vision of defining and computing "structural scene description," descriptions of the objects in a scene and their relations to each other. The first goal is to obtain a theoretical understanding of the problem of inferring objects and their compositional structure, both in standard optical images and in laser range imagery and motion images. The second goal is to test this understanding by developing effective algorithmsfor the statistical inference of this structure, algorithms that work with real data at reasonable speed on current computers. The general approach is to formulate mathematical representations of patterns, typically compositional with a hierarchy of structures, to encode the variability of these structures in stochastic models, and to use the models to infer information about the image. Standard stochastic models are rarely adequate, prompting the development of new classes of probability measures or completely new directions for traditional models. Additionally, in the context of this application the team aims to develop mathematical, statistical, and computational ideas that are of broader use. For example, compositional issues in vision are similar to ones in the grammars of language. One aspect of the project studies this connection.

The investigator and his five colleagues continue their mathematical, statistical, and computational investigations of a range of problems motivated by image processing and computer vision. The underlying question is simple enough: Here's an image, what is it an image of? Despite its simplicity, this is a hard question to answer. The approach taken here is to decompose the image into different components in some hierarchical structure and to use statistical analysis to infer information about the image from the relationships of the components within the structure. There are similarities with the grammatical structure of language, which the project explores. Recognition of objects in an image is a fundamental problem for both computer systems and biological systems. Advances are important for engineers developing computer vision applications, computer scientists seeking efficient algorithms in problems related to intelligent behavior of machines, and cognitive scientists studying human vision and language skills. Results of the project are published on CD-ROM, allowing demonstration of the dynamic behavior of algorithms in ways impossible through traditional publication modes and offering new ways to exploit multi-media communications for both education and research purposes.

The problems of 3D shape representation, analysis, manipulation,
and estimation from unstructured sensor data are central not only
to computer vision, but are also of key significance to geometric
modeling, computer graphics, computational geometry, medical imaging,
surveillance, among others. The goal of this proposal is to investigate
approaches to these problems in the context of tackling certain
key problems in archaeology. This area of application offers
the following three distinct advantages: (i) it is a rich source
of free-form shapes, e.g., pottery sherds, marble plates and reliefs,
sculpture, column capitals, building facades, etc., which are
easily accessible, in contrast to 3D free-form shapes obtained
from medical images which require expert segmentation; this data
is also richer and more variable than data available in a typical
manufacturing environ-ment; (ii) the problems are of a generic
nature and the proposed approaches can be applied to a variety
of other domains; (iii) this application area forms a key bridge
between the Physical Sciences and the Humanities.

This focused study proposes to analyze the structure induced on spaces of shapes by the action of groups of diffeomorphisms equipped with a right invariant metric. The project contains four components related to these spaces: their geometric analysis, the development of appropriate statistical methods, the required numerical analysis, and the application of the results to medical imaging and computational anatomy. The general framework is addressed by a new approach which in some sense formalizes the mechanics of shapes. Lie groups with right invariant metrics indeed are structures on which classical laws of mechanics can be shown to hold, and in particular the conservation of momentum along paths of minimal energy. It turns out that this momentum is a key to the representation and characterization of deformations in this context. It is albeit difficult to handle, because it is usually singular, as a measure, or a distribution on a singular support. This and the numerical difficulty it creates is probably one of the main challenges that we address in our study. Other important aspects are the study of the geometry such an approach induces on shape spaces, including a study of their curvatures, and the existence and stability of normal coordinates. This will be related to open issues in shape statistics, and applied in particular to biomedical imaging problems.

This approach is therefore designed to provide new tools for describing and analyzing shapes. Although shapes are prevalent in the outside world and in science, this is a difficult problem. For the human mind, there is an intuitive notion of what shapes are, why they differ or look alike, or when they present abnormalities with respect to ordinary observations. Sculpture is the art of rendering existing shapes, or creating new ones, and the fact that artists are still able to provide unambiguous instances of subjects through distorted or schematic representations is a strong indication of the robustness of the human shape recognition engine. However, an analytical description of a shape is much less obvious, and humans are much less efficient for this task, as if the understanding and recognition of forms work without an accurate extraction of their constituting components. We can recognize a squash from an eggplant or a pepper via a simple outline, and even provide a series of discriminative features which distinguish them, but it is much harder to phrase a verbal description of any of them, accurate enough, say for a painter to reproduce it. It is therefore not surprising that, for mathematics, shape description remains mostly a challenge. There are however very important applications which depend on progresses made in this field, one of them being the computerized analysis of biomedical shapes (computational anatomy), which analyzes the impact of diseases on shapes of organs, obtained from modern techniques of non-invasive 3D imagery. The last fifty years of research in computer vision has shown a amazingly large variety of points of view and techniques designed for this purpose: 2D or 3D sets they delineate (via either volume or boundary), moment-based features, medial axes or surfaces, null sets of polynomials, configurations of points of interest (landmarks), to name but a few. Yet, it does not seem that any of these methods has emerged as ideal, neither conceptually nor computationally, for describing shapes. An important aspect of our study will be to describe shapes with an indirect approach, from the way they can be deformed. This is not a new idea, and can be traced back to the seminal works of D'Arcy Thompson at the beginning of the 20th century, but its mathematical formalization and the design of practical algorithms is a comprehensive task, still offering many open problems, that the present group will try to address and convey to the scientific community.

Object recognition tasks in computer vision and particularly medical imaging, require a theory of shape. There are many mathematical theories of geometry- Euclid's theory of triangles, Gauss's theory of the obstructions to flattening a surface, Poincare's theory of the most abstract properties unique to a sphere. But none of these seem to have captured what humans mean when they say this object is heart-shaped but that one is kidney-shaped. Automated analysis of medical scans requires that these issues be addressed. Humans have no difficulty answering questions about how similar two shapes are. This suggests two types of mathematical models. On the one hand, one might define precisely a measure of dissimilarity of two shapes, a so-called metric on the set of all shapes. On the other, one might define a probability distribution whose values represent the probability that one shape is likely to belong to the same category as the first. Both constructions are, however, not simple extensions of known mathematics because the set of all shapes is infinite dimensional: this means that any given shape can be wiggled or stretched in infinitely many distinct ways, that no finite set of numbers can fully describe any given shape. Thus new mathematics is needed to create the tools to work with shapes. It is hoped that the ideas behind this study may impact work in recognizing, identifying and comparing the objects present in any type of imaging such as MRI's and CAT-scans.

More precisely, this study starts from the observation that a shape (a closed surface in 3-space or a curve in the plane) can be viewed as a point in an infinite dimensional manifold. Even though the space of shapes is not linear, i.e. two shapes cannot, in any natural way, be added, the set of small deformations of a shape does form a vector space, namely the vector space of normal vector fields along the shape. Having a manifold, differential geometry can be used to define and study metrics on the space of shapes. Recent work has shown that a rich family of Riemannian metrics exists on this space with many different properties. Each one has its own geodesic equations and curvature tensor which have strikingly different properties. This project proposes to study two questions in particular: (a) the curvature for the Riemannian metric inherited from Sobolev norms on the diffeomorphism group of the ambient space and (b) the relationship between the features of the plane curve and its representation in the arguably most natural metric, the Weil-Petersson metric. We also seek to study the probability distributions on the space of shapes obtained as the marginals of diffusion processes associated to these metrics. We finally propose to make a database of scanned 2D shapes on which to compare the many metrics which have been proposed and to study clustering and the fitting of probability models to data. At present, there seem to be several good candidate metrics for applications so a comparative study is very important.