Siwei Lyu - US grants

This project studies natural image statistics, and their applications in diverse fields such as computational neuroscience, image processing, computer vision, and graphics. The centerpiece of this project is a new image representation based on a simple nonlinear transform that is statistically justified and biologically inspired. This representation provides a new language to describe image signals, and forms the basis to build statistical models to more effectively capture statistical properties of natural image. Built upon this new image representation, this project explores new paradigms to model and interpret visual neural responses and high-level perceptual properties, and provides new tools for image restoration, analysis and synthesis. On the other hand, by applying natural image statistics to the forensic analysis of digital images, this project facilitates forensic practitioners in criminal investigations, and contributes to national security and public safety. Moreover, this project contributes to education by making the learning of Computer Science fun and useful for undergraduate students, promoting the participation of women and undergraduate students in research, and improving the early learning of mathematics and sciences for local high school students.

Robots cannot currently grasp objects or perform other contact tasks in unstructured environments with speed or reliability. This project is developing techniques for accurate real-time perception in support of contact tasks. In the proposed method, sensor data tracks the continuous motions of manipulated objects, while models of the objects are simultaneously updated. Particle filtering, a kind of Monte-Carlo simulation, ensures consistency of this tracking and updating.

The strongest impact of this work will be in robotic grasping and manipulation. Because of the synthesis of modeling and probabilistic inference, further impacts can be expected, for example in real-time haptics for telepresence.

Noise, which corresponds to random variations extrinsic to the signals of interest, is an ubiquitous aspect that affects the performance of many tasks in signal processing. Even with the improving quality and sophistication of the modern acquisition devices, digital signals still carry noise due to many incontrollable factors. This research focuses on the fundamental problem of estimating parameters of the random noise model directly from a noise corrupted signal. As an immediate consequence, the results of this investigation will be applicable in a wide range of fields, including the forensic analysis of digital images, automatic processing of medical images, spectrum sensing in wireless communications and data processing in sensory neuroscience.

The technical approach taken in this research exploits the regular statistical properties of the original signals in multiple signal representations and their relationship with the noise parameters. Specifically, we will investigate the use of domains constructed from random band-pass filters that are more effective in revealing ?typical? statistical properties of the signals, especially when compared with deterministic representations such as Fourier, DCT, and wavelet. Concurrently, we will investigate the mathematical relationship between the observed statistics of noisy signal and the noise parameters. Drawing on these theoretical findings, this research is expected to lead to more effective and efficient algorithms for blind noise estimation. More generally, the proposed work will also explore efficient algorithms for blind local noise estimation in the presence of non-stationary noise statistics.

A current weakness of robots is their inability to quickly and reliably perform contact tasks in unstructured environments. The goal of this project, which represents a collaboration between faculty at two partner institutions, is to alleviate this shortcoming by developing techniques that will afford robots accurate real-time perception in tasks exhibiting intermittent contact. Project outcomes will have a strong impact in manipulation tasks, as robots become more capable and autonomous. The PIs also expect successful applications in other areas, for instance to drive real-time haptic displays in augmented reality systems, to extract human manipulation strategies from observed kinesthetic demonstrations, and to identify model parameters to improve simulation accuracy, not to mention in advancing the level of autonomy for space and undersea exploration. Additional applications outside of robotics are anticipated in situations where a system experiences abrupt state transitions and the goal is either state estimation or real-time feedback control (e.g., chemical, financial, and geological systems). The PIs' labs have a track record of supporting women and under-represented minorities, and the research will be integrated into a variety of pedagogical activities at the graduate and undergraduate level on both campuses.

In previous work the team proposed the DBC-SLAM framework, in which continuous states (i.e., poses, velocities and contact impulses), and discrete contact states (i.e., contact-noncontact and stick-slip) of the manipulated objects, are tracked and important model parameters are estimated. In this research, they will extend that work significantly in two directions. First, they will design new parallel, anytime complementarity problem (CP) solvers in order to attain real-time performance. Second, they will enhance the dynamic Bayesian models in DBC-SLAM to allow the use of point-cloud observations and more complex geometric models of the objects, robot links, and environment. The intellectual merit of the project lies in three main activities: first, the creative, yet rigorous, technical process of designing perception algorithms based on fundamental first principles of nonsmooth mechanics and Bayesian estimation in a way that can utilize point-cloud data; second, achieving real-time performance by exploiting the mathematical structure and properties of both the nonsmooth multibody dynamics and CPU/GPU computing systems; and third, pursuing the first two activities in a way that sheds light on the trade-offs between estimation accuracy and speed.

Although existing machine learning (ML) methods are effective for analyzing data that are static in nature, many of today's applications from information retrieval to web searching require real-time processing of massive amounts of streaming data. Often, the data is available in an online fashion, meaning the entire dataset is not available at once, but rather, individual data instances arrive sequentially. Area under the Receiver Operating Characteristic Curve (AUC) has been proven to be an effective evaluation metric and learning objective in many application domains, but it is difficult to optimize directly. To date, there are no satisfactory approaches to incorporate AUC into online ML algorithms for classification and ranking problems. This project will address the fundamental theoretical and algorithmic challenges in ML algorithms based on AUC maximization for processing streaming, high-dimensional data with efficient algorithms and theoretical analysis. The results of this project are expected to have broader impact in intrusion detection for cyber-security, fault detection in safety critical systems, information retrieval and cancer diagnosis. The planned research will also integrate with educational activities, including developing new undergraduate/graduate courses on optimization and machine learning, organizing a workshop and making software tools freely available to the public. The principal investigators also plan to undertake outreach activities to improve STEM learning of high school students.

A central topic of this project is to develop efficient online learning algorithms for AUC maximization and bipartite ranking, making them amenable for online processing of high dimensional and large volume of streaming data. This project also establishes a rigorous statistical foundation and thorough theoretical analysis of the online AUC maximization algorithms developed. The primary technical challenge in developing online AUC maximization algorithms and theory is that the objective functions involve statistically dependent pairs of instances while individual instances continuously arrive in a sequential order. This is in contrast to standard classification based on accuracy, where the loss functions only depend on individual instances and the related algorithms and theory are well developed. The project will address this gap by sufficiently exploiting the structures of the population objective involved in the problem of AUC maximization, rather than an empirical objective on finite data, through novel interaction of machine learning, statistical theory and applied mathematics. This will help to remove the pairwise structure in the original objective function and facilitate the development of efficient online learning algorithms for AUC maximization.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Machine learning is instrumental for the recent advances in AI and big data analysis. They have been used in almost every area of computer science and many fields of natural sciences, engineering, and social sciences. The central task of machine learning is to ?train? a model, which entails seeking models that minimize certain performance metrics over a set of training examples. Such performance metrics are termed as the aggregate losses, which are to be distinguished from the individual losses that measures the quality of the model on a single training example. As the link between the training data and the model to be learned, the aggregate loss is a fundamental component in machine learning algorithms, and its theoretical and practical significance warrants a comprehensive and systematic study. The proposed work will focus on several fundamental research questions concerning the aggregate loss: are there any other types of aggregate loss beyond the average individual losses?; if so, what will be a general abstract formulation of these new aggregate loss?; how can the new aggregate losses be adapted to different machine learning problems?; and what are the statistical and computational behaviors of machine learning algorithms using the general aggregate losses?.

The technical aims of the project are divided into four interrelated thrusts. The first thrust explores new types of rank-based aggregate losses for binary classification and study efficient algorithms optimizing learning objectives formed based upon them. The new aggregate losses will be applied to problems such as object detection, where rank-based evaluation metric is used dominantly. The second thrust aims to deepen our understanding of the binary classification algorithms developed using the rank-based aggregate losses and will be focused on a study of their statistical theories such as generalization and consistency. The third thrust will extend the study of new types of aggregate losses to other supervised problems (multi-class and multi-label learning and supervised metric learning) and unsupervised learning. The fourth thrust dedicates to the theoretical aspects of aggregate losses, in which an aggregate loss will be abstracted as a set function that maps the ensemble of individual losses to a number. This abstraction will be exploited to study the properties of new aggregate losses that make them superior than the average loss and propose new aggregate losses beyond rank-based ones.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

The unprecedented spread of disinformation, false information intentionally created to manipulate public opinions, is the flip-side of the Internet’s promise of universal access and information democratization. The presence of false and/or misleading information in the media ecosystem erodes trust in legitimate sources of information and poses a significant threat to society. We posit that enhancing user awareness and building resilience are the keys to combating disinformation, as ‘inoculated’ users can form the first line of defense against the spread of corrupted and misleading information. The overarching goal of our Disinformation Range (DRange) project is the development of a research/educational platform with integrated digital tools, advanced pedagogical techniques, and timely materials to increase disinformation awareness and improve user resilience, so as to inoculate them against the impact of harmful disinformation, and further prevent its spread.

DRange will facilitate the pursuit of high impact goals in three overarching categories: 1) developing flexible technologies and culturally responsive group learning activities to facilitate communal examination and discussion of false and misleading information and inauthentic online behaviors in safe and familiar settings; 2) conducting transdisciplinary research to advance our understanding of the impact of dis/misinformation; and 3) identifying and implementing preventive (‘immunization’) strategies and mitigation practices. DRange is envisioned as a comprehensive learning process that interweaves facilitated discussions, collaborative games, and group activities, supported by a flexible and adaptable technical platform that uses simulated (or de-toxed) disinformation to both encourage critical conversations about online risks and vulnerabilities, and cultivate user resilience. DRange will be designed, developed and structured in collaboration with community partners to foster group interactions in diverse settings (e.g., classrooms, after school activities, public libraries, summer camps, senior and community centers, etc.).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Strengthening American Infrastructure (SAI) is an NSF Program seeking to stimulate human-centered fundamental and potentially transformative research that strengthens America’s infrastructure. Effective infrastructure provides a strong foundation for socioeconomic vitality and broad quality of life improvement. Strong, reliable, and effective infrastructure spurs private-sector innovation, grows the economy, creates jobs, makes public-sector service provision more efficient, strengthens communities, promotes equal opportunity, protects the natural environment, enhances national security, and fuels American leadership. To achieve these goals requires expertise from across the science and engineering disciplines. SAI focuses on how knowledge of human reasoning and decision-making, governance, and social and cultural processes enables the building and maintenance of effective infrastructure that improves lives and society and builds on advances in technology and engineering.<br/><br/>Online abuse is a pressing and growing societal challenge. Online hate and harassment, cyberbullying, and extremism threaten the safety and psychological well-being of targeted groups. Understanding the problem and developing ways to address it is the active focus of many fields of research in the social and behavioral sciences and in computer science. Machine learning and the use of artificial intelligence (AI) offers great potential to support research in this area. Still, researchers face fundamental challenges in leveraging emerging machine learning techniques for innovative studies and scientific discoveries in online abuse. This SAI research project strengthens and transforms the current disperse machine learning software infrastructure. It develops a scalable, customizable, extendable, and user-friendly Integrative Cyberinfrastructure for Online Abuse Research (ICOAR). The new infrastructure advances the research capability for scholars in different fields of science to leverage advanced machine learning methods for online abuse research. The ICOAR software infrastructure can be utilized by a large and growing number of researchers on online abuse detection and is a stimulus to research and innovation in AI for social good.<br/><br/>This project enables easy access to state-of-the-art machine learning techniques and datasets for rapid online abuse analysis. It supports and advances future investigations of new concepts and phenomena, assessments of prevalence, measures of causal effects, predictions, and evaluation of online abuse detection algorithms. ICOAR offers a modular and user-centered approach, ensuring future enhancements and long-term sustainability. The open software infrastructure consists of three major layers: a data layer, a capability layer, and an application layer. The data layer includes tools for automatic data collection and preparation of online social media data from different sources, and access to public benchmark datasets. The capability layer is composed of modularized machine learning-based capabilities and algorithms for the study of online abuse. The application layer allows researchers to easily develop different applications based on their research priorities. The ICOAR resources are open-source and provide an easy-to-use learning platform for curriculum development and workforce training.<br/><br/>This award is supported by the Directorate for Social, Behavioral, and Economic (SBE) Sciences.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.