Narendra Ahuja - US grants

Active computer vision exploits control of foveation, focus, and stereo vergence to generate surface maps of complex scenes, in much the same way that humans control gaze direction, lens deformation, and eye vergence. This grant will support two graduate students developing algorithms for real-time control of an active stereo range-mapping system. The work is motivated by Sperling's model of biological vision, and will augment a testbed system in which plausible techniques of active vision may be tested.

This award in the Joint NSF/DARPA Initiative on Image Understanding and Speech Recognition is for a study of the visual cues that humans need to guide vehicles and manipulate objects remotely. Telerobotic operators may be hampered by response delays, limited visual bandwidth, and insufficient time to attend to critical details. This study will identify essential visual cues that must be extracted or preserved to enable competent navigation, obstacle avoidance, landing, docking, and manipulation. Theories will be tested by human capabilities in synthesized dynamic visual environments.

An important problem in robotics research is how to build a robot that can move over steep, irregular, or otherwise rugged terrain in a flexible and agile manner. Because wheeled vehicles do not perform well over such terrain, engineers have attempted to design legged robotic vehicles for this purpose. The idea is to use such vehicles to explore places where humans cannot easily go, such as the surface of other planets, or rugged or dangerous places on earth. Insects represent an extraordinarily variable life form that has the characteristics of adaptability and versatility of locomotion on land that engineers desire. In this award to Drs. Fred Delcomyn, Narendra Ahuja and Mark Nelson, a robotic, legged vehicle will be modeled after an insect. First the control system for locomotion will be designed and simulated on a computer. The unique aspect of the present design will be a deliberate adherence to the functional operation of the insect, especially in its use of feedback from sense organs in the legs to help it deal with uncertain footing. By building into the robot a rich array of sensory feedback from the legs, this device should emulate the versatility and adaptability of the insect, and impart to the robot the fault- tolerance and agility of the biological system. This research wil contribute to engineering a machine which utilizes the biological characteristic of locomotory adaptability.

This award will support a nine month long-term visit by Professor Narendra Ahuja, Coordinated Science Laboratory, University of Illinois at Urbana, to Japan for a cooperative research project with Dr. Fumio Kishino, Communications Systems Research Laboratories, Advanced Telecommunication Research Institute International (ATR) in Osaka. They will be undertaking research in the area of integrated image analysis and visualization. Professor Ahuja's current research includes 1) integration of the information present in diverse image cues for deriving a three-dimensional (3D) interpretation and 2) visually realistic and compact depiction of scenes. The success of a depiction is measured by the extent to which a human observer realistically perceives the desired scene characteristics from the depiction. Dr. Kishino's research is also concerned with both image interpretation and image synthesis, but for images that contain humans. They use a range of image cues related to human body motion in performing the interpretation. They also emphasize greatly the hardware implementation and prototype development issues. The researchers will closely study and compare the two lines of research, and attempt to determine ways in which each PI's research could be enhanced in effectiveness by incorporating the pertinent strengths of the other's activity. Results of the research could provide great military and commercial potential. Improved simulator technology could also be used as a scientific research tool in robotics, eliminating the need for real mobile robots to be built at every robotics laboratory. These robots are expensive to purchase and even more expensive to maintain and program. Both Professor Ahuja and Dr. Kishino are noted for very productive, high quality research. Likewise, both the University of Illinois and ATR are respected in the community for their significant research in the area of computer vision and visualization. It is expected that this collaborative effort will continue after the formal agreement has terminated, and it is further anticipated that useful publications will result from this bilateral effort.

9319038 Ahuja This is the first year funding of a three-year continuing award. The research focuses on analysis, development, real-time implementation, and initial exploration of real-world application of a new image transform. The transform is intended for multiscale, low-level image segmentation (i.e., extraction and representation of image structure at all geometric and photometric scales present in an image). The transform detects contours and skeletons of image regions, and identifies the cross-scale relationships among them. Implementation of the algorithms on highly-parallel architectures will be studied, as well as development of special purpose hardware for real-time implementation.

This is the first year funding of a three-year continuing award. A demanding application area of virtual or augmented environment is multi-user collaborative environment where multiple users at either a local site or remote sites communicate with each other and interact with a synthetic or virtual scene. Among the necessary sensors and devices, an effective visualization device and a real-time image acquisition system are two main challenges. The objective of this project is to develop a novel visualization device referred to as head-mounted projective display (HMPD), build a multi-user interactive workbench by integrating the developed HMPD technology with a unique real-time image acquisition system known as an omni-focus camera, and evaluate and quantify the system as an effective tool for remote collaboration. The head-mounted projective display (HMPD) proposed is coupled with a supple, non-distorting and durable projection surface as an alternative to current visualization devices. Its novel concept suggests solutions to part of the problems of state-of-art visualization devices, such as large distortion with wide field of view, occlusion contradiction between virtual and real objects, and brightness conflict with background illumination. Several properties of the proposed HMPD make it extremely suitable for multiple-user collaborative applications. Research efforts will be made to design and implement a lightweight and compact head-mounted prototype by introducing diffractive optical element (DOE) and plastic materials, and investigate approaches to optimize the illumination of the display and retro-reflective material properties for imaging purpose. At one site, a multi-user interactive bench prototype with tele-presence capability will be built by using the HMPD concept and adding an image acquisition system developed from a unique omni-focus camera system. At the other site, a mural display system will be built with conventional stereoscopic video system located near the mural display, where one or several collaborators will also gather. Tele-collaborative work will be tested between the Beckman Institute at the University of Illinois--Urbana Champaign and the School of Optics-CREOL at the University of Central Florida through the Internet II connection linking our laboratories. Finally, the PIs will quantify the depth and size representation and perception accuracy, evaluate occlusion perception aspects, and set up a comprehensive calibration procedure for the HMPD and the workbench and mural prototypes. The results are expected to impact a wide range of applications such as collaboration/tele-collaboration, tele-presence, tele-manipulation, and visualized education/tele-education.

This proposal is aimed at producing novel images of a scene from arbitrary new
viewpoints using a sparse set of compressed panoramic snapshots or sample
images of the scene. The samples are taken from a relatively small number of
strategically placed cameras. A major application and evaluation testbed of
the proposed work is to enable walkthroughs of a 3D scene by generating the
images of the scene along a trajectory dynamically chosen by a remote user.
The focus of the proposed work is on the acquisition of panoramic stereo
images which serve as scene samples, and their interpolation (or
extrapolation) for producing a new stereo pair of images of the scene from an
arbitrary new viewpoint. The user follows a dynamically chosen arbitrary
trajectory and wishes to continuously teleview the scene from along the
trajectory points. It is assumed that relative locations of the objects in the
scene for which walkthroughs are to be generated are known; the object
locations are sparse; and that the strategic placement of the cameras does not
change significantly to warrant camera replacement. Development and
implementation of a stereo panoramic camera is proposed. Limited work is
proposed on joint image compression. The topology of the camera network also
allows the determination of which cameras are to be used to generate the
walkthrough image at any given point along the user drawn trajectory.
Applications of the proposed work include video surveillance, virtual museums
and video conferencing.

In natural environments, sensory signals often arise from clustered sources. For example, during the mating season, the auditory cues that guide a female frog to a particular male are embedded in a dense chorus arising from hundreds of calling males of different species. Clustered signals pose a significant challenge for biological systems as well as for intelligent neural prostheses and machine perception systems. For example, an intelligent hearing aid should exhibit robust performance in cluttered environments with other voices in the background. A multidisciplinary team of investigators will explore the neural mechanisms and computational algorithms that animals use to detect, identify, and localize individual signals embedded in an ensemble of similar signals. Experimental studies will focus on the auditory-mediated approach of female frogs to a mating chorus and the electrosensory-mediated approach of electric fish to a cluster of prey. These studies will include audio recordings of frog choruses at different distances from natural mating ponds and electrical recordings of active electrosensory signals arising from swarms of zooplankton. Approach trajectories will be recorded and analyzed using radio telemetry (frogs) and infrared video recordings (fish, frogs). Theoretical analysis will draw on algorithms from computer vision including multiscale grouping and segmentation, target detection and tracking, active vision, texture analysis, and motion and structure estimation. Neural correlates of clustered signal processing will be assessed through electrophysiological studies on auditory nerve and midbrain neurons in frogs and primary electrosensory afferent and hindbrain neurons in fish.
This project will incorporate interdisciplinary training (engineering and neurobiology) for students, a web site for sharing data, software and references in this field and could contribute to the development of improved machine devices for speech recognition and video surveillance.

Abstract

The goal of this SGER proposal is to investigate the feasibility of using a segmentation tree as a general purpose multiscale representation of image structure, and assess the value of this representation for higher-level tasks such as object recognition.
This objective requires demonstrating the stability of such a tree under changes in object viewing conditions, and developing robust algorithms for matching segmentation trees to find corresponding regions in multiple views of the same object. The motivation to explore this line of thinking has come from the recent work of the PIs, which has indicated that segmentation trees have the potential of making a significant impact on the state of the art in object recognition. This finding is controversial as it contradicts a widely held belief in the vision community that since low-level image segmentation varies somewhat with imaging conditions, algorithms that use regions as image features cannot offer a reliable basis for image understanding.
The main goal of this proposal is to address those concerns and obtain conclusive results to firmly establish or reject the PIs'
preliminary conclusions.

This project is about automated, visual object recognition. It is aimed at a computational approach which has two parts. First, it learns whether a given set of previously unseen images, say supplied by a user, contains any dominant themes, namely, subimages, that occur frequently and look similar. Such themes, and the associated subimages, are called categories and objects, respectively. Second, given a set of categories automatically inferred during the aforementioned training, and a new, test image, the approach recognizes all occurrences in the image of objects belonging to any of the learned categories. It delineates each such object in the image, and labels it with its category name. Both learning and subsequent recognition do not need human supervision. The subimages defining a category can be small or large, simple or complex. It is reasonable to expect that low-complexity categories, e.g., containing small/few/simple subimages are more common in real-world images. For example, the simple category of elongated shapes occurs as a part of legged animals, stools and scissors. More complex categories consist of large/many/complicated regions and are less common. Simple categories, e.g., the ``leg'' are thus shared by more complex ones, e.g., all legged animals, and, in turn, ``leg'' is an articulated combination of the category of elongated shapes (limbs). Therefore, category representation can be made easier by expressing it as a configuration of simpler categories, instead of subimages directly, thus yielding a hierarchical, subpart model. Accordingly, the proposed approach learns and recognizes categories as image hierarchies. The use of hierarchical embedding of regions as the defining image features results in several advantages the proposed approach offers over existing other methods which mostly use local features: (1) The proposed approach requires no supervision, e.g., labeling or segmenting of training images, or other input parameters from the user. (2) It simultaneously provides category detection and high-accuracy segmentation. (3) Training is feasible with very few examples, and not all training images must contain objects from the categories. (4) The use of hierarchical models makes explicit the relationship of a specific category to other categories of similar, lower and higher complexities; it also serves as a semantic explanation of why a category is detected when detected. Expected major contributions of the work include computational formulations of: (1) Accurate extraction of image regions; (2) Image representation by connected segmentation tree; (3) Robust image matching amidst structural noise in images; (4) Unsupervised extraction of hierarchical category models; (5) Efficient recognition of a large number of categories; (6) Unsupervised estimation of the relevance weights of subcategory detections to category recognition, and (7) Generalization of the proposed approach to extraction of texture elements, as an example of how the proposed work may impact other challenging vision problems involving hierarchy.

The progress made on this project can be seen at the website: http://vision.ai.uiuc.edu/ahuja.html

This project explores new directions to solving the following problem. Given an image, determine whether and where specific objects, or objects from a specific category, appear in the image. Visual category is defined as earlier, namely, as a collection of objects which share characteristic features that are visually similar, and occur in similar configurations. The visual nature of objects sought is communicated through (training) data containing them, and estimated using machine learning. The approach consists of two main parts. First, it learns whether a given set of previously unseen images (including videos), say supplied by a user, contains any dominant themes, namely, subimages, that occur frequently and look similar. Second, given a set of categories automatically inferred during training and a new test image, the approach recognizes all occurrences in the image of the learned categories. It delineates each such object in the image, and labels it with its category name. Both learning and subsequent recognition do not require human supervision. The approach learns and recognizes categories as image hierarchies. The impact of the project includes accurate high-speed extraction of image regions, image representation by connected segmentation tree, robust image matching, unsupervised extraction of hierarchical category models, efficient recognition of a large number of categories, unsupervised estimation of perceptually salient, relevance weights of subcategory detections to category recognition, and generalization of the proposed approach to extraction of texture elements. More broadly, the proposed approach is useful for applications in search engines, surveillance, video analytics, monitoring and data mining.