Zygmunt Pizlo, Ph.D. - US grants

The proposed work focuses on human-robot interaction, namely on the robot physically sensing the human hand. Specifically, the PIs will study the microstructure (texture) of the contact surfaces between a robot and a human hand, to infer the perceptual dimensionality of haptic texture sensing (perceptual model), and establish the mapping of relevant spaces. Methods for producing intuitive and efficient synthetic textures will be investigated. Rendering algorithms will be developed for synthesizing textures with desired perceptual qualities. The work is expected to contribute to various areas of haptic perception, texture studies, and multimodal rendering of information.

Object recognition in Computer Vision, though being a main processing step in many tasks of robotics, surveillance, and other fields of automation, is still an unsolved problem. The recent results in human visual perception strongly suggest that contour extraction is a key step to object recognition. A development of a contour-based system for object recognition is proposed. The first step of the new approach concentrates on extraction of object contours from edge images that correspond to contours as perceived by humans. Since the extraction of complete contours may not be possible (e.g., due to occlusion), extraction is focused on meaningful parts of contours. The proposed approach uses a mixture of bottom up and top down processing for edge grouping. After each step of bottom-up processing in a pyramid architecture, top-down evaluation is applied to select the most promising grouping constellations. A promising grouping constellation is defined using cognitively motivated constraints. In accord with the cognitive simplicity principle known from Gestalt psychology, partial shape similarity will be used as a primary building block of such constraints. In accord with the newest results in human perception, grouping of edges to parts of object contours and recognition of the parts using shape similarity play a key role in object recognition. This means that object recognition is possible if only part of a contour is constructed, and the construction of the whole contour is not necessary for recognition. In particular, object recognition works in the presence of occlusion and segmentation errors.

The proposed solution to the object recognition problem can make a significant step to improve the application scope of vision systems. The results of this work will be applicable to vision systems, large image databases, and video analysis systems. The proposed research to find interdependence and structural information among visual parts may lead to further understanding of human visual perception and cognition. The proposed research will provide an excellent resource for interdisciplinary work for graduate and undergraduate students in computer science and psychology. The PIs will offer courses and seminars on proposed research topics that will bring the state-of-the-art knowledge and technology to the classrooms.

During the last several years, there has been growing interest in studying how humans solve difficult optimization problems. The common feature of these tasks is an enormous search space. Prior research on human problem solving was based on two key assumptions: (i) that search through a problem space is a key element in solving problems, and (ii) that human thinking and problem solving are suboptimal. Recent results obtained independently in three laboratories have demonstrated the fundamental inadequacy of these two assumptions. It has been shown that humans produce optimal, or near-optimal solutions to difficult optimization tasks by performing minimal amounts of search. Existing decision research paradigms are not able to account for this sort of result; some new approaches are needed. For example, it appears that the mental representation of a problem is a key to peoples' effectiveness in solving difficult optimization problem, but mental representation is not much studied in current decision research.

We will hold a three-day workshop where 15-20 scientists (psychologists and computer scientists working on optimization) from the United States, Canada, Australia, and Europe will discuss the current status of the field, future directions and ideas for collaborative work. The workshop will help establish a framework for research in this area over the coming decade and will also provide delegates the opportunity to discuss potential applications of recent findings about optimization by humans.

Last Modified Date: 07/21/08 Last Modified By: Daniel F. DeMenthon

Abstract
With the ever faster growing number of images and videos, the main bottleneck in extracting the information contained in them is their analysis (indexing) and retrieval. Nowadays image and video search engines are based on textual descriptions, since visual cues are at too low level to provide useful retrieval results when dealing with a large variety of images and videos. For example, if a human submits a query image with the request to find similar images, she focuses on a certain object or a group of objects in the query image. Thus, the meaning of similarity is given by the images that contain similar objects. Therefore, extraction of objects in images (and videos) is a key factor for true progress in content based image/video retrieval (CBIR). However, object extraction belongs to unsolved problems in Computer Vision (CV). This fact led to the development of a huge number of approaches that try to do CBIR without object extraction. However, although such approaches may be successful in some restricted application domains, in which case low level features may be sufficient to replace object extraction, they have not been successful in general purpose CBIR. The PIs believe solving the object extraction problem will lead to a breakthrough in CBIR. Therefore, the PIs propose to work on object extraction in images. There have been a large number of attempts to solve the object extraction problem in CV, and none provided a satisfactory solution. Why will our approach provide a good solution? A new methodology and a computation framework proposed by the PIs provide solid evidence that the breakthrough in object extraction is possible.
On the cognitive and geometric modeling side, the PIs propose to use a higher level knowledge of shape similarity and a mid level knowledge of local and global symmetry as cognitively motivated constraints for object extraction. Constraints are essential because object extraction is known to be an ill-posed inverse problem. The human visual system solves this problem very well and we are getting close to a full understanding of how this is done.
On the computational side, the PIs propose a new framework for a simultaneous estimation of medial axes and the contours. The proposed approach is inspired by the SLAM (Simultaneous Localization and Mapping) approaches in the field of robot mapping. Recent breakthrough solutions in robot mapping are based on the SLAM computation with particle filters. SLAM computation iterates over the processes of localization of the robot in the existing partial map (trajectory estimation), followed by a map update based on new observations and the estimated trajectory. The PIs treat the medial axis as trajectory of a virtual robot and the partial boundary as the map that is composed of edge segments associated with the medial axis. A first successful application of this framework is demonstrated by the PIs in the preliminary results.

Project URL: http://knight.cis.temple.edu/~shape/

Collaborative Research: Recovery of 3D Shapes from Single Views

Zygmunt Pizlo, Purdue University
Longin Jan Latecki, Temple University

The human eye, like a camera, produces 2-dimensional images of a 3-dimensional world. How does the human brain succeed in interpreting these impoverished 2-dimensional images, allowing us to see the world as it actually is "out there?" This fundamental question, whose significance has been appreciated for 300 years, has not been answered despite the efforts of many scientists, engineers and mathematicians. Conventional approaches, which have not been successful, tried to recover the 3-dimensional shapes of objects and scenes from their 2-dimensional images by analyzing the depths of surfaces in multiple images (such as might be obtained from two eyes or from moving images) and by emphasizing the role of learning and familiarity. The approach taken by Zygmunt Pizlo at Purdue University and Longin Jan Latecki at Temple University is very different. It uses only a single 2-D image to recover the third dimension by applying a priori constraints (assumptions about the world built-in to the human visual system) that reflect important visual properties that are generally present in the physical world, properties such as the symmetry and compactness of 3D objects.

Pizlo and Latecki's research has the potential of encouraging theoretical changes in the study of human perception because it uses an entirely new approach to a classical unsolved problem in vision. It could support breakthroughs in machine vision because human beings are known to be much better than any machine confronted with recovering the 3D world from 2D information. Machine vision has important applications to many domains, including law enforcement and national security. Pizlo and Latecki's research attempts to solve the visual 3D shape problem by combining the results of experiments on human observers with state-of-the-art computational modeling. The project will provide an excellent opportunity for the interdisciplinary education of graduate and undergraduate students in psychology, computer science and engineering.

DESCRIPTION (provided by applicant): The project will study how humans see three-dimensional (3D) objects and 3D scenes under natural viewing conditions. Note that any laboratory experiment is always a simplification of what actually pertains in real life for two related reasons: (i) the experimenter must have a full control over the experimental conditions: this will not be possible if the experiment contains too many parameters, and (ii) the interpretation of the results is easier and more convincing if the experiment is simpler. In most, prior research, a number of different criteria were used to decide how to best simplify the experimental conditions. In this project, unlike in all prior work, the criterion used is that huma 3D vision in the lab will be veridical as it is in our everyday life. Veridical, here, simply means that we see the 3D shapes and 3D scenes the way they are out there, that is, we see them accurately. This has never been done before because no available theory could explain how veridical vision is possible from a mathematical and computational point of view. Such a theory has finally become available. The project's goals are three-fold: (i) provide empirical evidence about the nature of veridicality of 3D vision, (ii) determine and characterize the limits of veridial vision: this will be done by specifying the geometry of the stimuli, as well as viewing conditions, for which vision ceases to be veridical, and (iii) formulate computational models explaining veridical vision and its failures. Achieving these goals will be instrumental in (a) designing machine vision devices that can help and assist the blind and visually impaired, (b) assessing implications of visual impairments in everyday life as well as in job related activities, and (c) explaining brain mechanisms responsible for 3D vision: this is essential for evaluating the effects of brain injuries and exploring possibilities for compensation by the brain. There will be four sets of behavioral experiments: half will use real objects in real scenes and the other half, 3D models of objects and real scenes rendered by means of virtual reality devices. The first set of experiments will examine the role of the a priori constraints normally operating in our natural environment, such as the symmetry of objects, the presence and direction of gravity, and the orientation of the ground surface. The second set of experiments will evaluate the effect of degradation of the visual stimulus by lowering its luminance, contrast, and spatial resolution. The third set of experiments will examine Figure-Ground Organization; specifically, the ability of a human observer to detect and locate objects in naturalistic 3D cluttered scenes using both foveal and peripheral vision. The fourth set of experiments will examine 3D scene recovery; specifically, the ability of a human observer to see the positions, sizes and orientations of 3D objects, as well as the empty spaces among and behind the objects. The results of these experiments will be used to formulate and test a computational theory of 3D vision. This theory will take the form a Regularization or Bayesian inference in which a priori constraints are optimally combined with the visual information.