Xiuwen Liu - US grants

Robotics and Human Augmentation Program

ABSTRACT


Proposal #: 0307998
Title: Seeking Optimal Representations, Classifiers, and Generalizations for Image Based Recognition
PI: Liu, Xiuwen
Florida State University


Linear representations are ubiquitous in all areas of computational sciences. Many applications in image analysis and computer vision involve analysis of large-dimensional data by projecting them linearly to low-dimensional subspaces. In view of their computational efficiency such linear projections have become standard in certain applications. However, for recognizing objects from their images, there is seldom a discussion on finding "optimal" linear representations. Many societal, commercial, and scientific operations, such as homeland security and biometrics, rely heavily on image-based recognition and the recognition performance becomes a vital factor. This project aims to provide efficient algorithms for finding linear and non-linear representations that perform optimally in the context of object recognition. We propose to achieve this goal by: (i) formulating the search for optimal linear representations as that of optimization on Grassmann manifolds, (ii) using the geometry of Grassmannians to develop algorithms for finding optimal linear representations, (iii) analyzing the convergence properties and the theoretical limits of the proposed optimization techniques, and (iv) demonstrating the potential significant performance improvement on problems wherever linear representations are applicable. Since there are numerous applications utilizing dimension reduction using linear projections, including object recognition, image and text retrieval, subspace tracking, and nonlinear filtering, the potential benefits of the proposed research are tremendous. This research will be built on tools for stochastic optimization and statistical inferences on nonlinear manifolds, tools that will prove beneficial in many other applications.

This research will also enhance significantly the learning and research environment on computer vision at the Florida State University. Utilization of geometric and statistical approaches to applications in computer vision makes this a multidisciplinary effort to the benefit of all participants, including both graduate and undergraduate students. Outcomes of this research will be incorporated in recently designed courses on Computer Vision and Computational Statistics.

Imaging devices have become ubiquitous tools of surveillance of public areas, remote locations, areas of restricted access, and other sites where additional security is needed. The detailed analysis of collected images can provide invaluable information about people, objects, their characteristics and patterns of behavior. Thus, combined with other strategies, image analysis can contribute significantly to the prevention of terrorism and national security. However, the execution of this task poses challenging problems due to the vast amount of imagery generated by surveillance devices. To make this task feasible, advanced automated systems are needed to screen images and route to human operators only material that is very likely to contain relevant information. The proposed interdisciplinary research addresses problems on the interface of shape and digital image analysis, whose solutions will contribute to the implementation of such intelligent surveillance systems, and will be useful in numerous other applications. Images contain information about two main attributes of objects: their shapes and textures. The proposers will develop a novel framework to represent and analyze planar shapes quantitatively using methods and tools of differential geometry, differential topology, and statistics. Statistical texture analysis and synthesis will be combined with the study of shapes to produce finer models of imaged objects. New algorithms of shape and image analysis will be developed, implemented, and applied to: (a) the detection and recognition of objects in noisy images; (b) tracking dynamic shapes possibly subject to occlusions in video sequences; (c) the organization of large databases of shapes for efficient retrieval and processing of information. Current techniques of algorithmic shape analysis are somewhat limited in scope or performance: some represent shapes using coarse collections of landmarks whose selection may be difficult to automate, and some involve heavy computational costs. Computational efficiency issues also limit the use of existing methods of image analysis; in spite of the remarkable success that methods based on partial differential equations have had in many applications, computational costs associated with typical implementations are high and the performance is not adequate for applications in video surveillance. There is a pressing need for efficient, robust algorithms that can analyze, process, and simulate the dynamics of shapes of continuous closed curves. The main idea proposed here is the use of computational stochastic differential geometry to study shapes, i.e., the algorithmic analysis of differential geometric representations of continuous curves in a statistical framework. The proposers will: (i) analyze closed shapes by representing them as elements of infinite-dimensional Riemannian manifolds via their angle or curvature functions; (ii) develop geometry-based tools for statistical inference problems on shape spaces; (iii) derive techniques for nonlinear filtering and tracking of shapes in infinite-dimensional shape manifolds; (iv) study completions of contours and textures with the goal of discovering hidden geometric features that follow an observable pattern; (v) implement algorithms and apply them to the solution of problems in shape and image analysis. The key new element in this approach is the use of the geometry of spaces of curves to study shapes, not only the geometry of individual curves. Results originating from this research may have far-reaching implications in shape, image and video analysis. The proposed algorithmic approach to shapes has the potential to set a new paradigm for the treatment of curve evolution. The team has expertise in the areas of differential geometry and topology, statistics, computing, and image analysis. This grouping reflects the interdisciplinary nature of the proposed investigation and will further enhance the atmosphere of collaborative research that exists among the PIs and their graduate students. Moreover, the applications to be investigated will contribute to the education and involvement of more students in areas related to national security. The PIs will continue to develop and offer courses and seminars from the introductory to the advanced levels targeting a broad audience of science students with the goal of increasing the overall impact of this line of research. To encourage the participation of undergraduates and students from underrepresented groups, motivated students will have full access to the Florida State University Laboratory of Computational Vision, where a hands-on learning environment will allow them to explore the area with their own experiments. To disseminate research results the proposers will continue to publish articles in well-circulated journals, post results in various electronic preprint archives, produce multimedia presentations on CD-ROMs, write introductory articles in magazines or handbooks, and present results at regional, national and international conferences.

This award is supported jointly by the NSF and the Intelligence Community. The Approaches to Combat Terrorism Program in the Directorate for Mathematical and Physical Sciences supports new concepts in basic research and workforce development with the potential to contribute to national security.

ALGORITHMIC RIEMANNIAN GEOMETRY FOR A STATISTICAL
ANALYSIS OF IMAGES
Abstract

This project is concerned with the investigation of novel algorithmic representations of images and geometrical signal processing techniques for the automated analysis of image content. The investigators develop a new framework for an appearance-based analysis of imaged objects in terms of their shapes and textures using methods and tools derived from differential geometry and statistics. A statistical formulation is of the essence due to the large variability of shapes and textures frequently encountered in imagery of interest. The use of differential geometric methods in image processing is still incipient, but very promising, as solid evidence exists that such methodology is particularly well suited for the study of multidimensional, nonlinear features such as shapes and textures.

In recent years, the investigators have developed a statistical shape analysis program; shapes are viewed as elements of a shape space whose geometry is exploited for shape analysis. The investigators treat textures in a similar manner by creating a Riemannian manifold of textures and integrate both representations into a single shape-texture model for the algorithmic analysis of image content. Images are decomposed into their spectral components and local spectral histograms are treated as elements of an infinite-dimensional statistical manifold equipped with a geometric structure induced by non-parametric Fisher information. Differential geometric constructs are utilized to develop algorithms for: (i) statistical inferences and learning of shape-texture features; (ii) Bayesian detection and recognition of objects using shape-texture priors; (iii) dimensionality reduction techniques for efficient processing.

The main goal of this project is to develop novel computational models and strategies to analyze the shapes of spherical surfaces in Euclidean 3-space. In recent years, there has been a substantial progress in the computational study of shapes of curves with methodology based on the geometry of infinite-dimensional spaces of curves. However, attempts to extend these approaches to surfaces have encountered tall obstacles. In this project, an effective computational solution is proposed that encompasses all fundamental aspects of the problem. Shape spaces will be constructed equipped with geodesic metrics, which will provide a natural environment for the quantitative study of shapes of surfaces. A full set of computational tools will be designed and implemented to quantify shape similarity and divergence, to develop statistical models from samples, to synthesize shapes from learned models, and to analyze and simulate shape dynamics. Techniques will be developed to convert a noisy point-cloud representation of a surface of genus zero to a minimum-distortion parametrization over the standard sphere. Alignment algorithms will be designed to best match the geometric features of surfaces and to extract optimal parametrizations for modeling a family of shapes. Riemannian metrics inherited from weighted Sobolev spaces will capture geometric similarities and discrepancies between shapes to any desired order. The project will focus on first-order metrics, as they offer a good balance between geometric accuracy and robustness for computations. Due to the typical complexity of the geometry of surfaces, many algorithms will employ a coarse-to-fine approach both for the processing of point clouds and triangular meshes. Localization of spherical shapes in the frequency or spatio-temporal domains will also be employed for statistical modeling and to achieve computational efficiency.

The proposed research on shapes and forms of 3D objects is motivated by a series of problems arising in areas such as computer vision, medical imaging, and computational biology. Shape is a key attribute associated with patterns arising in geometric data and its effective computational representation and analysis will have an impact on application domains such as the recognition of objects or targets from various modalities of images, modeling brain anatomy and functions, the simulation of biological growth and motion, and anatomical changes associated with diseases and aging. As such, the proponents will make the tools of shape modeling and analysis developed under this project available to the broader research community and will also actively pursue collaborations with researchers in these areas.

This Federal Cyber Service: Scholarship for Service (SFS) project at Florida State University (FSU) is providing two-year scholarships to forty graduate students enrolled in computer science graduate degree programs with an information assurance focus. The students are being awarded scholarships in four cohorts of ten students each and the project includes activities to reinforce and invigorate the cohort structure. Through a formal relationship with Florida A&M University, the project is making a focused effort to recruit students from underrepresented groups into the program and has academic support programs to foster their success. FSU has received the National Security Agency and Department of Homeland Security (NSA/DHS) Center of Academic Excellence in Information Assurance Education (CAE/IAE) designation and the NSA/DHS Center of Academic Excellence in Research designation. The project includes an assessment and evaluation plan coordinated by an independent evaluator to ensure that the project achieves its objectives. The project's results are being disseminated through conference and professional presentations.

This collaborative research between Florida State University and Cornell University is to identify language-action features from text-based messages that can be used to dynamically infer a social actor's perceived trustworthiness. The team will investigate using optimal analysis techniques to calibrate trustworthiness reasoning, which can be used to computationally model actors' deceptive behaviors in cyber space and to infer actors' intent based on their words and actions.

This research will have a transformative impact in understanding the dynamics of trusting relationships through observing language-action features and psychosocial trustworthiness attribution mechanisms. This study serves as a precursor to a socio-technical schema that will facilitate national security and data protection for the general populace while also protecting the individual's right to privacy. This study will contribute to the science of cyber-security, and will help the cyber-security community to understand and enable trustworthy communication and collaborative information behavior among computer-mediated groups in a systematic way.

The Florida State University (FSU) proposes to add four new cohorts of undergraduate and graduate students to their existing CyberCorps(R) Scholarship for Service (SFS) program in cybersecurity with strong integration of education and research. The project will have an immediate impact on the information assurance and forensics capabilities of the federal workforce by providing graduates with general computing and security skills. The program components are designed to have a broad and lasting impact. Existing efforts to engage members of underrepresented groups and non-traditional students will be intensified. CyberCorps(R) students will collaborate on real-world projects to gain practical experience and enhance internship and job placement prospects. The proposed project addresses the current personnel gap in cybersecurity for the national information infrastructure. Through a formal relationship with Florida A&M University, the project is making a focused effort to recruit students from underrepresented groups into the program and has academic support programs to foster their success. The cybersecurity program is the most popular graduate program in FSU's computer science department and the prospect of a potential employment within the Federal Government has a significant impact on the number of applying students, including minority and female students.

The program is graduating skilled information assurance and forensics professionals with strong leadership skills and a commitment to public service. The program integrates rigorous, hands-on learning with research, professional development and outreach activities. The students will serve as summer interns to gain practical experience and will join federal agencies or other eligible entities upon graduation. The program has a well-established infrastructure for retention and mentoring. It includes a flexible selection procedure, offering admission by exception and choices of alternative degree programs, for strong cybersecurity candidates who otherwise do not meet the minimum computer science degree requirements. It is intended to broaden the scope of the cybersecurity program and attract minority and female applicants. FSU has received the DHS/NSA designation as the National Center of Academic Excellence in Information Assurance and Cyber Defense. The project includes an assessment and evaluation plan coordinated by an independent evaluator to ensure that the project achieves its objectives. The project's results are being disseminated through conference and professional presentations.

This project will contribute to the national need for well-educated scientists, mathematicians, engineers, and technicians by supporting the retention and graduation of high-achieving, low-income students with demonstrated financial need at Florida State University, a Carnegie I Research University. Over its five- year duration, this project will fund four-year scholarships to 33 unique Scholars who are pursuing bachelor?s degrees in computing. The project objectives include: (1) identifying and recruiting students with financial need and academic talent; (2) improving retention through cohort class enrollments, dedicated tutors, and academic support; (3) providing internship and research opportunities to Scholars; and (4) gathering feedback to refine the computing curriculum. A distinguishing feature of this project is the application of natural language processing, machine learning, and traditional analyses to examine fine-grained qualitative and quantitative data related to student success. These analyses are expected provide insights into student retention, the computing curriculum, the effectiveness of current support systems, and how to encourage women and other underrepresented groups to major in computer science.

The overall goal of this project is to increase STEM degree completion of low-income, high- achieving undergraduates with demonstrated financial need. Although a growing number of jobs require expertise in computing, only 10% of STEM graduates study computer science. This project seeks to increase the participation of low-income, high- achieving students in computer science. To achieve this goal, the project strategies include high school outreach, dedicated tutors, student support systems, cohort enrollment, and replacing student loans with scholarships. This project will investigate the effectiveness of these activities using randomized control trial experiments. As part of this study, the project will use natural language processing and machine learning approaches to analyze data from Experience Sampling Method surveys to identify and remediate gender and cultural biases in the computing curriculum. The expected outcomes of the project include identification of curriculum changes to encourage diversity and quantification of factors that contribute to student success in computer science. Project evaluation will include annual data collection and analyses of cohort demographics, academic performance, retention, use of support systems, reasons for separation, and placement. The research findings will be published at conferences such as SIGCSE, ASEE, FIE, and AERA. This project is funded by NSF?s Scholarships in Science, Technology, Engineering, and Mathematics program, which seeks to increase the number of low-income academically talented students with demonstrated financial need who earn degrees in STEM fields. It also aims to improve the education of future STEM workers and to generate knowledge about academic success, retention, transfer, graduation, and academic/career pathways of low-income students.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.