Larry S. Davis - US grants

This award is for the purchase of a multiple degree of freedom, high precision, light weight stereo camera head. The same equipment is being purchased at three other institutions in a shared research effort. Research topics include gaze control and target tracking, stereo and motion analysis, landmark-based navigation, automatic acquisition of object and environmental models, hand-eye coordination, dextrous manipulation with multi-fingered hands, real-time perception and manipulation. System software, algorithms and subsystems will be shared across the institutions. The University of Maryland will be purchasing equipment to support a multi-institutional shared research effort in active vision for robot navigation and manipulation. Other members of the consortium are the University of Rochester, University of Pennsylvania and the University of Massachusetts. Each institution will acquire the same state- of-the-art binocular camera head of high precision and high speed, and will share in the development of systems software and application software.

9318183 Davis The Grand Challenge Application Groups competition provides one mechanism for the support of multidisciplinary teams of scientists and engineers to meet the goals of the High Performance Computing and Communications (HPCC) Initiative in Fiscal Year 1993. The ideal proposal provided not only the opportunity to achieve significant progress on (1) a fundamental problem in science or engineering whose solution could be advanced by applying high performance computing techniques and resources, or (2) enabling technologies which facilitate those advances but also significant interactions between scientific and computational activities, usually involving mathematical, computer or computational scientist, that would have impact in high performance computational activities beyond the specific scientific or engineering problem area(s) or discipline being studied. The investigators will study the application of high performance parallel computing to a class of scientifically important and computationally demanding problems in remote sensing- land cover dynamics problems including generating improved- fine spatial resolution data for the global carbon cycle, hydrological modeling and global ecological responses to climate changes and human activity. The research is collaborative, including scientist from the University of Maryland, University of Indiana, University of news Hampshire and NASA's Goddard Space Center. The award will combine research on -new analysis procedures for remotely sensed data -the integration of multispectral, multiresolution and multitemporal image data sets into a unified global data structure based on hierarchical data structures (i.e., quadtrees) -the utilization of these hierarchical, parallel data structures for the representation of spatial data (maps and products developed from image analysis) and the development of a spatial data base system with a sophisticate d query language that scientist can use to control the application of biophysical models to global data sets -run-time support for constructing scalable and parallel solutions to problems involving the manipulation of irregular data structures such as quadtrees -parallel l/O,especially novel methods for mapping large arrays and quadtrees onto parallel disks and disk systems, and for accessing them using low overhead bulk transfers The development work will be conducted on a 32 processor Connection Machine CM5, installed at the University of Maryland, and on an IBM SP1 which we propose to obtain as part of the program. This award is being supported by the Advanced Projects Research Agency as well as NSF programs in geological, biological, and computer sciences.

9408472 Davis This workshop will define new research directions needed to advance high-performance computing in the designated National Challenge areas: health care, education, environment, manufacturing. The workshop will focus on software issues that differ from those raised by the Grand Challenge problems. To assure concrete discussion, the workshop will focus on health care applications. Participants have been selected to provide broad representation across the high-performance computation community. Application experts will also participate. ***

9401151 Davis This award provides support for the acquisition of a distributed memory parallel computer together with support hardware for scientific visualization and storage of large image databases. The proposed system will be an experimental facility designed as a testbed for operating system and algorithm development, and will consequently run a wide range of experimental operating systems. The research topics to be explored span a broad range of applied research in high performance computing in three general categories: programming tools for HPC systems, parallel algorithms for scientific computing, and symbolic coding. A key component of the research program is the development of scalable parallel data structure and algorithms for the representation and analysis of global data sets arising from environmental science problems.

9625668 Davis, Larry University of Maryland, College Park CISE Postdoctoral Program: Postdoctoral Research Associate in Computational Science & Engineering Science: High Performance Computing for Remote Sensing Applications This award supports CES associate David Bader. The main objective of the proposed training and research plan is to develop innovative computational methods and software for applying high performance computing to fundamental problems arising in the analysis of remotely sensed image data. This work will be conducted at the University of Maryland Institute for Advanced Computer Studies (UMIACS) in close cooperation with Professor Davis (Computer Science), Goward (Geography), Ja'Ja' (Electrical Engineering), and Townshend (Geography), who are pursuing a number of closely related efforts. The application to be developed is a high performance system for estimating net primary production (NPP) using the production efficiency model (PEM) developed at the University of Maryland. The overall aim is to build a family of software modules that make use of a variety of algorithms to carry out geometric correction of images, to eliminate signals from extraneous sources (e.g. atmospheric correction), to sample spatially and temporally, and to generate derived geophysical properties. The modules will be constructed using proven portable and practical high performance programming languages and libraries. Our applications development will be conducted on two high performance computing platforms at UMIACS; a 16-node IBM SP-2 and a Digital Alpha cluster consisting of 40 Alpha processors networked via a Digital ATM Gigaswitch. The Maryland facility has approximately 300GB of online disk storage and a multi-terabyte tertiary storage system. ***

99-01249
Davis, Larry S
University of Maryland

CISE Experimental Partnerships: High Performance Systems for Shape and
Action Modeling

Research is being performed on the design and development of parallel algorithms for obtaining accurate models of natural objects and actions. The goal of the work is to develop a suite of parallel, distributed algorithms executing on a network of personal computers that can integrate video images from multiple viewpoints to construct an accurate geometric model of a complex, rigid object, or detailed dynamic action models of moving, articulated and non-rigid objects - specifically people. The work integrates and extends previous research in computer vision, hierarchical data structures and high performance software systems. Geometric and statistical algorithms for extracting shape and action descriptions from single and multiple video streams are being used along with hierarchical spatial data structures (e.g. oct-trees and quadtrees) which constitute the basic data structures for the shape and action representations being constructed. The enormous quantity of data and computation associated with applying such techniques and structures to a large number of video streams makes it impossible for vision scientists alone to design, construct and evaluate the omputational performance of these distributed algorithms and systems. Vision researchers are working together with parallel and distributed computing researchers to articulate a solution that includes geometric, algorithmic, and systems-oriented perspectives.

This is the first 4 years funding of a five-year continuing award. Humans are very good at discerning the spatial origin of sound using a mixture of frequency-dependent interaural time difference (ITD), interaural level difference (ILD), and pinna spectral cues in disparate environments ranging from open spaces to small crowded rooms. This ability helps us to interact with others and the environment by sorting out individual sounds from a mixture, and helps us to survive by warning us of danger over a wider region of space compared to vision. These advantages of spatial sound are important for human-computer interaction.

While the frequency-independent ITD cues (delays) associated with the two ears are relatively easy to render over headphones, the ILD (level difference) and pinna elevation cues are not. For a given source location and frequency content, the sound scattered by the person's torso, head and pinnae, and is received differently at the two ears, leading to differences in the intensity and spectral features of the received sound. These effects are encoded in an extremely individual "Head Related Transfer Function" (HRTF) that depends on the person's anatomical features (structure of the torso, head and pinnae). This individuality has made it difficult to use the HRTF in the proposed applications. Recent research, including that of members of this team, has focused on measuring the HRTFs for individuals in specific environments, on constructing models of the HRTF, on understanding how the geometry of the body is related to the characteristics of HRTF, and how the brain processes the cues to derive spatial information. However, this research has also indicated that the brain is extraordinarily perceptive to errors in cues that result when sound is rendered with an incorrect HRTF.

In this project the PI and his team will use numerical methods to compute individualized HRTFs from accurate 3-D surface models of the body. They will use multiview, multiframe computational vision techniques to extract the surface models from imagery. They will then use boundary element methods employing fast multipole/ transform techniques and parallel processing to compute the HRTFs from the surface models. The resulting HRTFs will be evaluated both by objective comparisons with acoustically measured HRTFs and by psychoacoustic testing, and will be used in demonstrations of virtual reality, augmented reality, and teleconferencing. A major advantage of this vision-based approach is that it will allow the PI and his team to investigate and model the way that HRTFs change with body posture, providing the potential of tracking dynamic environments. Thus, the project will include fundamental research to extend the static HRTF measurements to dynamic situations in different environments, using a combination of visual tracking to locate the person in real space, and construction of in-room HRTFs from free-field HRTFs using fast iterative techniques. This will provide a scientific foundation for HCI applications of audio rendering. The research will in addition yield algorithms and understanding that will have an impact on varied fields, including computer vision based model creation; scientific computing; computational acoustics for noise control and land mine detection; neurophysiological understanding of human audition; etc.

An ever-increasing segment of the population suffers from low vision resulting from complications of disease and old age. Surveys conducted by one of the Co-PIs as part of a previous project have determined, that the key information which is not available to people with low vision is textual information, usually of a directive or warning nature. For example, shopping in a large department store in a mall might involve looking for signs indicating where the store is, reading aisle signs in the store, and looking at product names, at labels and prices. This research will develop a "seeing-eye" computer to help people with low vision to observe and receive such information, so that they can participate more efficiently and comfortably in every day activities, and thereby lead more fulfilling and productive lives. The system will be composed of a digital video camera, computer, user interface, and speech or magnified visual output that can detect textual information in the environment, understand it using OCR, and provide it to the user who either has low vision or is blind. To achieve these goals, the PIs will in collaboration with colleagues at Johns Hopkins University build, over the first six months and then over the first two years, prototype systems using mostly existing technology and extensions to vision algorithms we have developed for identification of text regions in images and OCR, which can be evaluated on volunteer patients at the Wilmer Ophthalmological Institute and the National Federation for the Blind.. The functionality and range of applicability of our prototypes will necessarily be limited. Simultaneously, the PIs will work on long-term research problems that must be addressed to develop next generation seeing-eye computers with greater scalability and capability. In year three patient-volunteers at Wilmer and at the NFB will perform evaluations of the developed prototypes Subsequently, successful results will be commercialized and brought to the larger patient body (as have previous developments at Wilmer). Fundamental research problems to be addressed include: real-time algorithms for detection and rectification of text on planes and cylinders subject to perspective distortions; OCR from digital video, and OCR for text on textured backgrounds; and more robust and efficient algorithms and systems for stabilization and super-resolution of text blocks from video streams.

In this two-year effort, we propose to address some of the basic research issues that arise in the interpretation of video streams, simultaneously collected by a set of indoor or outdoor cameras.
Specifically, we are interested in inferring movements and activities of one or more humans using
distributed cameras. We propose to develop novel methods for detecting and tracking humans using 3D models
for body parts, and quasi-invariant recognition of activities humans are engaged in. We will make use of the recent advances made in the computational aspects of estimating posterior probability density functions using
Monte Carlo Markov Chain techniques to infer human descriptions and their activities.

Although large parts of our brains are devoted to the processing of sound cues and sound plays an important role in the way we interface with the world, this rich channel has not been extensively exploited for displaying information. The mechanisms by which received sound waves are processed neurally to form objects with auditory properties in many perceptual dimensions, including three corresponding to the source location (range, azimuth, elevation) and three to qualities ascribed to the source (timbre, pitch and intensity), are beginning to be understood. There has been significant progress over the last decade in understanding the mechanisms by which acoustical cues arise and how the biological system performs transduction and neural processing to extract relevant features from sound, and in the way we perceive and organize objects in acoustical scenes. Our goal is to exploit this understanding, and uncover the scientific principles that govern the computerized rendering of artificial sound scenes containing multiple sound objects that are information and feature rich. We will test, use and extend this knowledge by creating auditory user interfaces for the visually impaired and the sighted. The work aims both at developing interfaces and answering fundamental questions such as: Is it possible to usefully map "X" to the auditory axes of a virtual auditory space? Here "X" could be an image (e.g., a face), a map, tabular data, uncertain data, or temporally varying data. Are there neural correlates that can guide natural mappings to acoustic cues? What limitations does our perception place on rendering hardware? How important is

The PIs propose to establish a five-year ITR program that will lead the development of the next generation distributed video sensing systems for understanding human movements. Novel models of human movement and structure will be used for modeling the movements of singe-joint and whole bodies with applications to animation, biomotion, and gait analysis for diagnosing and treating movement-related disorders. The interdisciplinary team includes leading researchers from three core institutions - the University of Maryland (lead institution), Stanford University and New York University. The researchers cover a broad spectrum of interests, including biomechanics, computer science and engineering, electrical engineering, and kinesiology. The proposed research efforts will enable novel approaches for realistic animation and the detection of subtle variations in movement, leading to better diagnostic tools and personalized programs for rehabilitation of movement disorders. Strong educational and industrial outreach programs will also enhance our research program.

This planning effort, a collaboration of teams at the University of Maryland, Cornell University, Carnegie Mellon University and the Sciencenter of Ithaca, deals with the development and testing of a unique methodology for educating youth in computer programming. Through a mobile robot that is cleverly disguised as a small animal, participants will learn to manipulate the system by physically moving it as well as setting variables via electronic buttons thereby learning programming and design. The eventual use of this system and methodology is in museum exhibits so preliminary survey data will be gathered from various venues that presently use less capable devices. Iterative testing will be done at the Sciencenter in its exhibits.

The planned research will take video analysis indexing and retrieval in a new and promising direction. The research is driven by the need for intelligence analysts to be able to express video queries more efficiently than traditional relevance feedback and to be able to "provide more expressive queries that include "nouns" and "verbs" as they would with human language. While still constrained, the approach goes a long way toward bridging the gap between traditional relevance feedback based only on assumed relationships in the image, and full human language queries. The graduate students involved in the project will be required to publish in international conferences and journals and will likely use this research as a basis for their dissertations. Other impacts of this work include the mentoring of graduate students and the inclusion of junior personnel in the management of the project. Research will be disseminated through local, national, and international meetings and journals. The team will also install a server and public interface for demonstration on existing datasets. The system will be accessible through the web on limited datasets and on the full dataset by request.

Structural similarity search and retrieval in images that include both printed text and handwritten text remains a challenging problem, especially with collections that are noisy, and heterogeneous. Approaches currently in use generally convert documents before filtering. This work provides triage as a way to filter very large collections through structural similarity with known attributes, then new clustering with broader terms and hashing to extend the scale of collections considered. The work will provide new directions for document image retrieval, especially in conditions where there is a wide variation in structure and layout and will be made scalable in cloud environments. Another approach to scaling, especially in the area of duplicate detection, will extend multi-level locality sensitive hashing and generalize it to other analysis indexing and retrieval issues. In addition to including graduate students, results and software will be made available through Creative Commons licensing to provide for replication and extension of the results.

Traditional video analysis research has been centered on detection and recognition tasks for objects and activities from known sources with a fairly narrow range of content. This effort would extend the predictable dual view hashing algorithm developed in previous work from images to videos. Many videos can be naturally associated with text annotations by the producer and consumer comments (tags), language derived from speech tracks using speech to text methods or the semantic words associated with applying vision models like human detectors and local activity detectors. The team will combine appearance based methods for video classification with language models derived from these text sources so that videos can be retrieved via a natural language like interface. This will involve investigating ways of fusing these different text sources in one vector space language model and then applying the dual view hashing methods to a database of videos. They can then investigate retrieval performance using the text codes for a form of zero shot category definition. The research is driven by the need for intelligence analysts to be able to express video queries more efficiently than traditional relevance feedback and to be able to provide more expressive queries that include nouns and verbs as they would with human language. While still constrained the approach goes a long way toward bridging the gap between traditional relevance feedback based only on assumed relationships in the image, and full human language queries.

Traditional approaches to document retrieval focus on conversion to electronic text followed by indexing of the text content. Recently some work in the community has focused on indexing document image content directly. Such techniques break down when text content is limited or highly degraded. Work on document quality estimation will be extended image quality to address structural quality, a factor that is important for determining if traditional document processing operations will succeed or not. Then,the team will explore the effects of enhancement on classification and retrieval and extend existing work to adapt to changes in quality. The research is motivated by the need for analysts to deal with very large collections of image data. The traditional goal of converting all documents on an electronic form and using traditional text analysis methods fails when dealing with heterogeneous collections and very noisy (possibly multilingual) content. The approach will allow document image retrieval systems to scale to orders of magnitude beyond current capabilities, and permit users to move beyond content features and use structural similarity to explore large collections. This will permit the users to mine large collections for clusters of similar content without knowing a priori specifically what the collection contains through classification. The result will be adaptive techniques that can learn from small numbers of samples without knowledge of sources of degradation.

The bombing attacks at the Boston Marathon in April 2013 presented the law enforcement community with significant challenges in terms of the volume and variety of video and still images acquired in the course of the investigation. Tens of thousands of individual media files in multiple formats were submitted from a variety of sources. These sources included broadcast television feeds, private Close-Circuit Television (CCTV) systems, mobile device photographs and videos recovered from the scene, as well as photographs and videos submitted by the public. Teams of analysts reviewed this evidence using mostly manual processes to determine the sequence of events before and after the bombing, ultimately leading to a quick resolution of the case. In the aftermath, it has become evident that the proliferation of video and image recording devices in fixed and mobile devices make it inevitable that a similar situation will occur in future events. As a result, it is incumbent upon the law enforcement community and the U.S. Government at large to further explore the use of automated approaches, available today or in the coming years, to better organize and analyze such large volumes of multimedia data. The findings of this workshop will help define the future research agenda.

The problem of searching for actionable intelligence information from unconstrained images and videos is an unsolved problem. Solving this involves addressing many sub-problems such as video summarization, shot detection/scene change detection, geo-tagging, robust face recognition, human action recognition, semantic description, image recognition and designing human in the loop systems. In addition, issues such as data collection and performance evaluation have to be addressed. Given that several hundreds of videos and a large collection of still images may be available for analysis, there is a great need to develop robust computer vision techniques. While many existing computer vision algorithms perform reasonable well in constrained acquisition conditions, their performance when unconstrained images and videos are given, is less than satisfactory. This workshop precisely addresses the challenges that arise in analyzing a large collection of unstructured image/video collection. This workshop explores the state of the art in algorithms being developed in academia that can support forensic analysis and identification in large volumes of images and videos (e.g., multimedia). The workshop informs long- and near-term research and development efforts aimed at optimally addressing this situation in the future. The workshop identifies those video and image analysis problems which are: (1) Considered solved (i.e., ready to deploy in specific operational scenarios); (2) Nearly solved (i.e., could lead to solutions with one to three years of development); and (3) Over-the-Horizon problems (i.e., those challenges requiring concerted effort over the next 3-5 years and beyond).