Jitendra Malik - US grants

This Presidential Young Investigator award is for the support of Dr. Malik's outstanding work in computer vision. He has already brought deep mathematical insight to innovative problems in shape from shading, curved-object recognition using aspect graphs, and scale-space approaches to edge detection and early vision. He plans to continue work in image segmentation, shape recovery, and object recognition. Dr. Malik is also an excellent speaker and a promising teacher, having graduated one outstanding Ph.D. student and co-authored papers with several others.

This award is to purchase manipulation and sensor and control processing equipment for projects in intelligent distributed control of materials handling, object manipulation using tactile sensing, coordinated motion and grasp planning, and finding surface boundaries in images. The award will allow the University of California, Berkeley to expand its RobotWorld system, update its LYMPH multiprocessor, and acquire a frame grabber and color camera to support fast image acquisition and processing. University of California, Berkeley will be purchasing manipulation and sensor and control processing equipment to support research in robotics and computer vision. The equipment will include an expansion of its RobotWorld system, an update to its LYMPH multiprocessor, and a frame grabber and color camera.

9873086
Malik
The purpose of this award is to investigate the computational problems in sensing and control of a prototypical metropolitan area freeway transportation system, comprising a 1,000 lane mile network spread ver 100 square miles, used daily by 1,000,000 people in 500,000 vehicles. Technical challenges in sensing, state estimation, prediction, control and incident detection are addressed. Sensing requires real-time processing and fusing of data from loops, video cameras, probe vehicles, and cellular phones. Estimation of system state parameters (e.g. travel time distributions over numerous links) requires combining current sensor data, data from the immediate past, as well as historical data while maintaining spatial continuity constraints. Multiple scale models that can predict the evolution of system state over time and space will be developed and used to calculate real-time control actions for the available "actuators" (variable messages signs, ramp meters, signals) in order to improve performance. Based on learned characteristic signatures, the same models can be used to detect and classify incidents such as injury accidents with sufficient reliability to trigger appropriate responses.

The impact of this research will be to develop the core capabilities of a traffic management information system for the twenty-first century. While the current system is monitored by a variety of sensors and vast resources are dispatched to prevent system breakdown, in the absence of computational technology to estimate system state and to predict system behavior, both transportation managers and travelers make decisions as if they were blind-folded. Systems based on this research potentially offer huge benefits to both traffic managers and travelers. For traffic managers, they will provide a real-time overview of system performance and a quantitative analysis of the impact of various control options. For travelers, they can provide real-time information and advice during the planning and execution of their trips. The techniques developed in the project also potentially have application to other problems, such as air traffic control, which share many of its essential characteristics.
***

This is the first year funding of a five-year continuing award. This research program is geared towards making significant advances to the science and engineering of visual information processing, and addresses fundamental problems in the fields of computational vision, computer graphics, and human-machine interactions. Today, images and video clips are ubiquitous on the internet, digital video is changing the way entertainment is produced, distance learning is used in various facets of education, and advanced visual interfaces to machines are around the corner. However, at present there are severe limits to the extent to which a user can benefit from visual information, because virtually all of this information is presented in its raw form, that is, the way it was captured. The goal of this project is to develop the technical tools needed to achieve a variety of complex manipulations of visual data. These tools will enable a user to freely explore, interact with, and create variations of the physical world being presented. For instance, a user may remove and add objects to an image of a scene, vary lighting conditions, change the materials of surfaces, or view the scene from a novel perspective.

This project encompasses a comprehensive research program for creating the science and technology base required to enable such advanced manipulations of visual data. The general research problem may be stated as follows: Capturing, understanding, and predicting the appearance of our everyday world. Success in this domain of research necessitates a unified approach to open problems in two fields: computational vision and computer graphics. The research effort will focus on five pertinent areas: sensing, modeling, estimation, generation, and evaluation. The tangible contributions will be in the form of sensors that provide new types of visual information; complex models of materials, reflectances and textures; estimation algorithms that use the team's new models to recover scene properties from minimal data; advanced rendering techniques; and a set of comprehensive image/video databases for evaluation of work in this field. The results will impact numerous application domains, including digital imaging, entertainment, virtual environments, distance learning, e-commerce, interactive product design, art restoration, architectural modeling, restorative surgery, and surface inspection.

EIA-0139474
Sastry, Shankar
Univ. of California - Berkeley

Title: Summer Undergraduate Program in Engineering Research at Berkeley-Information Technology (SUPERB-IT)


The Department of Electrical Engineering and Computer Sciences (EECS) proposes a new Research Experiences for Undergraduates (REU) site for summers 2002-2004. The research focus of the REU site will be Information Technologies. Information technologies are crucial for manipulating large amounts of information in electronic formats. Berkeley is at the forefront of many areas of IT research, and is known for its large-scale interdisciplinary and experimental research projects. This area represents one of the most critical technological developments of our era. Information industries use these technologies to assemble, distribute, and process information in a wide range of media. The field of Information Technologies, including embedded systems, is by far the fastest growing job category in the U.S. The overall goal of this research is to demonstrate new architectural solutions for the converged information and communication networks of the next century. The REU site will bring students together in computer networking, wireless communication, embedded systems, and data acquisition through censoring, video and image processing, and network-enabled applications from the EECS Department.

This project is creating a novel paradigm for computer vision, termed "reconstructive recognition", that incorporates the strongest elements of previous machine learning-based recognition efforts and the strongest elements of previous reconstruction efforts based on radiometric reasoning. The goal is to provide a new foundation for machine perception, and the potential for a transformative advance in applications of computer vision. The project seeks novel physics-based methods for recognition as well as novel learning-based methods for interpreting pixel values in terms of the physics of a scene. The agenda is structured around four aims: Aim I develops generalized reconstructive processes that unify the recovery of shape, materials, motion and illumination. Aim II focuses on supervised visual learning methods that exploit such reconstructive image representations. Aim III pursues unsupervised discovery of reconstructive representations that converge to be similar to the engineered models of Aim I. Finally, Aim IV introduces well-defined challenge problems that focus the field and serve as measurable proxies for progress in computer vision applications that have high potential impact on society.

There is a significant broader impact to this project, not least being the improvement in computer vision pedagogy that ensues from a reunification of the currently divergent recognition and reconstruction views of the field. More broadly, this project pursues critical steps toward a future where machines can see, a future that will bring changes to robotics, human-computer interfaces, security, and autonomous navigation, to name a few.

This project exploits the benefits of RGB-D (color and depth) image collections with extra depth information to significantly advance the state-of-the-art in visual scene understanding, and makes computer vision techniques become usable in practical applications. Recent advance in affordable depth sensors has made depth acquisition significantly easier for ordinary users. These depth cameras are becoming very common in digital devices and help automatic scene understanding. The research team develops technologies to take advantage of depth information. Besides the published research results, the research team plans to distribute source code and benchmark data sets that could benefit researchers in a variety of disciplines. This project is integrated with educational programs, such as interdisciplinary workshops and courses at the graduate, undergraduate, and professional levels and diversity enhancement programs that promote opportunities for disadvantaged groups. The research team is closely collaborating with the industrial partner (Intel), involving interns and technology transfer in real products. The project is also applying the developed algorithms to the assistive technology for the blind and visually impaired.

This research develops algorithms required to perform real-time segmentation, labeling, and recognition of RGB-D images, videos, and 3D scans of indoor environments. Specifically, the PIs develop methods to: (1) acquire large labeled RGB-D datasets for training and evaluation, (2) study algorithms to recognize objects and estimate detailed 3D knowledge about the scene, (3) exploit the object-to-object contextual relationships in 3D, and (4) demonstrate applications to benefit the general public, including household robotics and assistive technologies for the blind.