Alan C. Bovik, Ph.D. - US grants

The purpose of this fundamental bioengineering research is to determine the means for providing three-dimensional images of single, living cells. The PI will conduct a research program of experimental and theoretical investigations designed to reconstruct the cellular anatomical three dimensional image. Such studies should illuminate anatomical and physiological processes in real time. Information from such studies could have a significant influence upon the investigation of cellular physiological processes.

ABSTRACT

"ACTIVE STEREOSCOPIC VISUAL SEARCH DRIVEN BY NATURAL SCENE STATISTICS"

Alan C. Bovik, Lawrence K. Cormack, J. Ghosh

The primary thrust of this proposal is to develop methods based on the natural statistics of stereoscopic images that will enable the design and implementation of the next generation of foveated, fixating machine vision systems that are capable of efficient and intelligent visual search, by exploiting and applying knowledge about human fixation and search mechanisms.
We summarize the intention of our proposal via the following key goals:
Goal 1: To develop a quantitative description of human active stereo vision as a function of natural scene statistics in a variety of three-dimensional visual search and learning tasks. Our emphasis will be on developing statistical models of stereo primitives that attract low-level visual attention based on a unique and in-depth statistical analysis. We feel that statistical models based on natural scene statistics have a very good chance of succeeding where deterministic models have failed.
Goal 2: To train a state-of-the-art foveated, fixating active computer vision system (named FOVEA) to search and to learn to search 4-D (space-time) scenes. To do this, back-end artificial neural networks trained on telepresent human search patterns will be used. The statistical models and extracted statistical stereoprimitives discovered as part of the research in Goal 1 will be used as a priori knowledge to improve the configuration and learning of the networks. We envision that these experiments will result in smart active machine vision protocols for exploring, searching, and interacting with 4-D environments, while giving new insights into visual cognitive processes.

PROJECT ABSTRACT
0310973
Alan Bovik
University of Texas @ Austin

Current methods for automatically assessing the quality of image and video data emphasize measuring fidelity relative to a reference. Thus a "reference" image/video is assumed available, and loss of quality is measured as deviation from the reference. However, it is desirable to dispense with the reference video for practical applications, such as video-on-demand, streaming web video, video services to wireless units, and digital television. Thus No-Reference (NR) quality assessment (QA) is important. However, little progress has been made on NR QA since the models used have been simplistic and largely limited to applications involving block-based compressed visual data. However, successful algorithms for correctly predicting the quality of signals that have been distorted with other types of artifacts, such as ringing and blurring resulting from JPEG2000 image compression, remain nonexistent.
We are working on a new approach that makes use of the fact that natural scenes belong to a small set in the space of all possible image/video signals. We are developing and adapting innovative statistical models that describe natural scenes. We have shown that distortions in image/video processing systems are unnatural in terms of such statistics. Thus we are applying Natural Scene Statistics (NSS) models for the NR QA of visual signals that are assumed to derive from the sub-space of natural scenes. We have already shown that NSS model are effective for NR QA of still images compressed by wavelet-based methods (e.g., JPEG2000). We are developing new NSS models for both wavelet-based video compression and for modeling distortions in wireless video streams from channel burst errors and fast-fading channels.

Project Abstract

This study is directed towards developing flexible, general-purpose Visual Search systems capable of Searching for objects in real, cluttered environments. The research will include extensive psychophysical and physiological experiments on humans and primates that will prototype artificial systems that mimic this behavior. The goals of the study can be conveniently divided into four Aims:
Aim 1: Develop and prototype a revolutionary camera gaze control device dubbed Remote High-Speed Active Visual Environment, or RHAVEN. RHAVEN will allow telepresent control of the gaze of a remote camera using eye movements as rapidly and naturally as if viewing the scene directly.
Aim 2: Develop optimal statistical bounds on Visual Search, by casting it as a Bayesian problem, yielding a maximum a posteriori (MAP) solutions for firstly, finding a target in a visual scene using a smallest number of fixations, and secondly, for next-fixation selection given a current fixation.
Aim 3: Construct models for Visual Search based on Natural Scene Statistics at the point of gaze. Visually important image structures can be inferred by analyzing the statistics of natural scenes sampled by eye movements and fixations.
Aim 4: Conduct neurophysiological studies on awake, behaving primates during Visual Search tasks. Measure and analyze search performance in awake, behaving monkeys, while measuring the responses of neural populations in the brain's frontal eye fields (FEF) which help control saccadic eye movements.
Broader Impact: The results of this research should significantly impact numerous National Priorities: Searching Large Visual Databases, Robotic Navigation, Security Imaging, Biomedical Search, Visual Neuroscience, and many others. It is easy to envision scenarios that would benefit by a fundamental theory of Visual Search. For example: searching for suspect faces in airport security systems; examining internet streams for questionable material; semi-automatic search for lesions in mammograms; steering robotic vehicles around obstacles in hostile environs; navigating huge visual data libraries, etc.

The arrival of the Internet and modern handheld communication devices is ushering in a remarkable revolution in the consumption of digital video technologies and products, and a resulting sea change in the nature and needs of the global information infrastructure. Digital video acquisition, networking, storage and display devices have advanced to an extraordinary degree of sophistication, leading to the rapid rise of many popular and globally deployed networked applications as Internet Video, Interactive Video on Demand (VoD), Video Telepresence, Video Phones, PDAs and other Wireless Video devices, Video Surveillance, HDTV, Digital Cinema etc. Monitoring and controlling the quality of broadcast video streams is essential towards improving quality of service (QoS). Yet, progress in methods for performing reliable video quality analysis has remained quite limited.

The research proposed here will create powerful Video Quality Assessment (VQA) algorithms that correlate highly with visual perception. The expected benefits of the proposed research are far-reaching. Methods for improving video Quality of Service (QoS) are a major emphasis of the world-wide cable, semiconductor, cell phone, and networking industries and considerable efforts are being expended on this topic. Successful VQA algorithms are likely to be deployed throughout the global wireline and wireless communication networks as well as in video acquisition and display devices. Breakthrough theories of video quality will enable the design of algorithms for video processing based on perceptual criteria - a decades-old holy grail of imaging science.

New, state-of-the-art video quality assessment (VQA) algorithms will be developed that are explicitly designed for use on High Definition (HD) video streams. These will be designed using perceptual criteria and taking into account such human factors as head and eye position. An open database of raw digital HD videos will be developed, along with multiple distorted versions of each video and human subjective scores on the distorted videos. Since HD videos are often resized for display on smaller screen, scalable VQA and IQA algorithms will also be developed that will for the first time, be able to assess the quality of images or videos, in a perceptually significant way that have been scaled or resized from their original dimensions. The development of successful HD Video Quality Assessment (VQA) algorithms that correlate highly with visual perception will represent a major advance in video engineering. The construction of an HD video quality database will be the first of is kind, and certainly heavily accessed by researchers around the world. Prior work by this group on non-HD VQA has resulted in some of the most-highly cited research in the image processing field in the past 20 years. It is anticipated that publications from this work will likewise be highly influential.
The equipment and developed algorithms will also be used as exemplars in the UT-Austin image and video processing educational program. The equipment will be used to generate numerous video processing teaching examples.

This project investigates two deeply commingled and significant scientific questions on the statistical distributions of range, disparity, chrominance and luminance in natural 3D images of the world: (1) developing a comprehensive database of co-registered luminance, chrominance, range, and disparity images of natural scenes; and (2) conducting eye movement studies on stereoscopic images.. On the acquired database, the research team studies and models the bivariate statistics of luminance, chrominance, range, and disparity . In the eye movement studies, the locations of visual fixations are measured as they land in range space against where they land in luminance, chromatic, and disparity space, making it possible to develop gaze-contingent models of the statistics of luminance, chrominance, range, and disparity. The results of these studies have broad significance in vision science and image processing. To exemplify this, new approaches to computational stereo and to stereo image quality assessment are developed. New computational stereo algorithms are developed using appropriate prior and posterior distribution models on disparity. Further, new algorithms are developed for stereopair image quality assessment using the statistical models that we will develop. These new algorithms dramatically impact the emerging 3-D digital cinema, gaming, and television industries, allowing for automatic assessment of 3D presentations to human viewers. The developed 3D range-luminance databases are made available via public web portals, and the results of the work are published in the highest-profile vision science and image science journals.

Determining the perceptual quality of video transmitted through complex networks and viewed on heterogeneous platforms, from cell phones to Internet-based television, is a key problem for the YouTube generation. It is also central to a variety of vision applications including face detection, face recognition and surveillance. Video is subject to numerous distortions: blur, noise, compression, packet/frame drops, etc. Quality assessment is non-trivial when an undistorted video is not available, and unsolved for multiple distortion types and in distributed, non-stationary viewing environments.

This project designs and creates intelligent video "quality agents" that learn how to determine perceptual video quality in heterogeneous networks, and assesses its impact on decision tasks such as face detection and recognition, all without the benefit of reference videos. It uses statistical properties of natural scenes, perceptual principles, machine learning, and intelligent adaptive agent collectives to handle videos simultaneously impaired by multiple distortion types. A primary application is novel face-salient quality assessment agents and quality-aware face detection algorithms. Multiple, co-operative video and face quality agents are trained using active learning based feedback mechanisms on mobile devices. This project yields adaptive, robust video Quality of Service assessment in real-life networks and provides new insights into human visual quality perception and visual distortion detection. The research team also creates two large, unique video quality databases: (a) A Mobile Video Quality Database of raw and distorted mobile videos and (b) A Distorted Face Database of undistorted and distorted face images, as gold standards for research and development in this area.

This project develops "distortion-aware" computer vision models and algorithms suitable for today's mobile camera devices, such as are found in cell phones. Today's mobile camera devices contain remarkably powerful computing capability, sufficient, in fact to contemplate performing sophisticated computer vision problems such as three-dimensional depth estimation, object detection and object recognition. However, mobile camera capabilities are much more limited due to distortions on capture, such as low-light noise, blur, saturation, over/under exposure, and processing artifacts such as compression. These distortions cause most computer vision algorithms to "break," making them unable to accurately recreate the 3D world or to find and recognize objects in it. This project creates computer vision algorithms with similar capability, using new and emerging models of visual neuroscience (how people see) and detailed and accurate statistical models of the three dimensional visual world (called natural scene statistic models). The project can impact many other camera devices, including low-cost surveillance and security cameras, mobile medical cameras, military cameras operating under battlefield conditions, and more.

This research develops principled approaches to using natural scene statistics models to solve difficult single-image visual tasking problems under poor imaging conditions. Specifically, the research team studies robust 'distortion-aware' statistical image models and algorithms for single-image 2D and 3D object and object category detection and synergistic 3D depth estimation. The research work includes (1) developing algorithms for fast, generic object detection and categorization "in-the-wild" that operate on single photographic images suffering authentic artifacts from digital cameras; (2) designing object and object class detection mechanisms augmented by 3D depth estimation processes, driven by powerful 2D and 3D prior natural image constraints; and (3) constructing a new annotated Color+3D database of HD precision-calibrated RGBD data using a Reigl VZ-400 Terrestrial Lidar Scanner on object categories of interest, yielding data of higher resolution and richness than existing datasets, complete with image labels as well as hand annotations of bounding-box object locations. This database is free to the community at large once it is available.