Percy Shuo Liang - US grants

Natural language understanding, automatically computing the meaning of text, is key for allowing citizens to deal intelligently with the vast amount of digital information surrounding us, from the fine print on credit cards to science textbook chapters or online instructional material. The goal of this project is to develop systems that can build richer understandings of text than current systems. Humans have an incredible ability to integrate the structure of meaning --- how the meanings of sentences can be built up from the meanings of words --- with statistical knowledge about how words occur together with other words. Humans also effortlessly integrate meaning with 'reference', knowing which people or events in the world the text is talking about. But these tasks are quite difficult for computational systems. This project builds new computational models that integrate deep neural networks --- computational models with great power for representing word meaning in a statistical way --- with computational methods from logic and semantics. These new models allow word meanings to be combined together to build sentence meanings and also allow meanings to be linked with entities and events in the world. The resulting representations should help enable such societally important language understanding applications like question answering or tutorial software.

This project develops compositional forms of deep learning that bridge between lexical and compositional semantics. This includes new kinds of embeddings that can be used to perform better meaning composition, computing for example that a student with a plaster cast is similar to an injured person just as earlier embeddings computed that injured is similar to hurt, and extending the virtues (such as lexical coverage) of embeddings to represent the denotations of logical predicates. Another focus is enriching models of meaning with models of reference, building entity-based models that can resolve coreference in texts to handle problems like bridging anaphora or verb and event coreference, with algorithms for entity-based coreference based on tensors that capture similarity of reference rather than similarity of lexical meaning. And it includes developing vector space lexicons that represent both natural language dependency tree fragments and logical fragments in a shared vector space, and representing meaning as general programs that can model the effects of events and processes on resources in the world. The new models are brought to bear on the end-to-end task of learning semantic parsers that map text to a semantic denotation.

With the increase in popularity of virtual assistants such as Siri, there is a renewed demand for deep and robust language understanding. Statistical semantic parsing is a promising paradigm for addressing this demand. The key obstacle in building statistical semantic parsers is obtaining adequate training data. This CAREER project aims to develop a new interactive framework for building a semantic parser, where the system, acting like a foreign speaker of English, asks users to paraphrase utterances that the computer already understands into ones that the computer doesn't. The framework opens up intriguing applications in education. One such application is a bidirectional tutoring system, in which the system poses questions to the student. The student must both answer and paraphrase the question, thereby both practicing the course material and providing training data to the system. Natural language is a universal entry point, which can increase engagement and promote diversity. High-quality semantic parsers can drastically improve the way humans interact with computers. In the longer term, this work can also have a significant impact on the way natural language processing systems are built. Currently, the prevailing paradigm is very much a train-and-deploy one, whereas there are many more opportunities for improvement and personalization if deployed systems were to learn on-the-fly.

This project develops a new interactive framework for building a semantic parser, which aims to obtain complete coverage in a given domain. The key idea is for the system to choose logical forms, generate probe utterances that capture their semantics, and ask users to paraphrase them into natural input utterances. In the process, the system learns about linguistic variation and novel high-level concepts. The data is then used to train a paraphrasing-based semantic parsing model. Existing paraphrasing models are either transformation-based, which excel at capturing structural regularities in language or are vector-based, which excel at capturing soft similarity. The project develops novel models to capture both. The framework developed in this project improves the state-of-the-art of natural language processing and machine learning in three ways. First, the framework departs from the classic paradigm of gathering a dataset and learning a model; instead, an interactive system interleaves the two steps. Second, the framework learns high-level concepts, which is crucial for natural language understanding, since words often represent complex concepts. Finally, it resolves a classic tension between the rigidity of logical representations and the flexibility of continuous representations by capturing both in a unified model.

This frontier project establishes the Center for Trustworthy Machine Learning (CTML), a large-scale, multi-institution, multi-disciplinary effort whose goal is to develop scientific understanding of the risks inherent to machine learning, and to develop the tools, metrics, and methods to manage and mitigate them. The center is led by a cross-disciplinary team developing unified theory, algorithms and empirical methods within complex and ever-evolving ML approaches, application domains, and environments. The science and arsenal of defensive techniques emerging within the center will provide the basis for building future systems in a more trustworthy and secure manner, as well as fostering a long term community of research within this essential domain of technology. The center has a number of outreach efforts, including a massive open online course (MOOC) on this topic, an annual conference, and broad-based educational initiatives. The investigators continue their ongoing efforts at broadening participation in computing via a joint summer school on trustworthy ML aimed at underrepresented groups, and by engaging in activities for high school students across the country via a sequence of webinars advertised through the She++ network and other organizations.

The center focuses on three interconnected and parallel investigative directions that represent the different classes of attacks attacking ML systems: inference attacks, training attacks, and abuses of ML. The first direction explores inference time security, namely methods to defend a trained model from adversarial inputs. This effort emphasizes developing formally grounded measurements of robustness against adversarial examples (defenses), as well as understanding the limits and costs of attacks. The second research direction aims to develop rigorously grounded measures of robustness to attacks that corrupt the training data and new training methods that are robust to adversarial manipulation. The final direction tackles the general security implications of sophisticated ML algorithms including the potential abuses of generative ML models, such as models that generate (fake) content, as well as data mechanisms to prevent the theft of a machine learning model by an adversary who interacts with the model.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.