Thomas L. Griffiths - US grants

Language is one of the most complex aspects of human behavior, and provides the foundation for many kinds of social interaction. The question of how people learn and use language is a subject of extensive research in several behavioral sciences, including cognitive science, psychology, and linguistics. There is a long tradition of using formal approaches to explore answers to this question, and recent work has begun to emphasize the importance of statistical models. With support from the National Science Foundation, Dr. Griffiths at UC Berkeley and Dr. Johnson at Brown University will develop and investigate new methods and models for learning and analyzing natural languages based on Bayesian statistics. In Bayesian statistics, the information about the structure of language provided by linguistic data is combined with a "prior" distribution that constrains the structures under consideration. This approach can make it easier to learn the properties of a language from limited amounts of data, and has a direct connection to theories of human language acquisition that emphasize the role of constraints in learning. This research project aims to integrate the statistical models used for learning and analyzing language with two methods from modern Bayesian statistics: Markov chain Monte Carlo algorithms and nonparametric Bayesian models. These methods make it possible to apply Bayesian inference in complex models of the kind that people typically work with in cognitive science and linguistics. The results of this project will provide new ways of working with traditional models of language, and lead to new models that are potentially of relevance to explaining how people acquire language. By exploring how contemporary statistical methods can be applied to the probabilistic models used in computational linguistics, this project will build closer connections between statistics, linguistics, and cognitive science, and provide opportunities for students to receive training in topics at the intersection of these disciplines.

This award was supported as part of the fiscal year 2006 Mathematical Sciences priority area special competition on Mathematical Social and Behavioral Sciences (MSBS).

People are able to learn new concepts much faster than computers, often requiring only a handful of examples where a computer might require hundreds. This remarkable ability is partly the consequence of extensive experience with the world, resulting in strong prior knowledge about the kinds of objects that are likely to form categories. This research project bridges the gap between human and machine learning by developing probabilistic models of human category learning, connecting psychological data with the latest theories from computer science and statistics. These mathematical and computational models are used to explore how people learn categories so quickly, to capture the effects of prior knowledge on categorization, and to build a catalogue of human concepts that can be used to test psychological theories and to train machine learning systems. In each case, the research combines the ideas, methods, and sources of data used in psychology and computer science, using hierarchical Bayesian models and Markov chain Monte Carlo algorithms to model human cognition, laboratory experiments to test these models, and large databases as a source of statistical information that guides model predictions. This research program is integrated with an educational plan that incorporates undergraduate and graduate teaching and mentoring, development of a textbook on probabilistic models of cognition, tutorials and workshops aimed at increasing contact between the computer science and psychology communities, and outreach through talks and a website.

One of the oldest problems in linguistics is to reconstruct ancient protolanguages on the basis of their modern descendants. Identifying ancestral word forms makes it possible to evaluate proposals about the nature of language change and to draw inferences about human prehistory. Currently, linguists painstakingly reconstruct protolanguages by hand, using knowledge of the relationships between languages and the plausibility of sound changes. This research project develops statistical, computational methods that automate or augment the reconstruction process. Unlike past computational approaches, these new models use detailed phonological representations to infer hidden sound changes. Moreover, they automatically infer which words are co-descendent (cognates).

These advances, combined with new algorithms for large-scale statistical inference, enable the analysis of orders of magnitude more data than prior work. The models from this project significantly expand the computational tools available to linguists; large-scale reconstructions make it possible to collect quantitative data to help answer long-standing questions about language change. Beyond word reconstruction, the models and tools from this project will be useful for other related applications, such as machine translation, where reconstructions can be used to fill gaps in the mapping between the vocabularies of different languages, and the alignment of biological sequences, which requires considering which regions in those sequences are co-descendent. In addition, the technical advances in probabilistic modeling and approximate inference methods will have cross-cutting implications for a range of modeling problems in computational linguistics, bioinformatics, statistics, machine learning, and cognitive science.

How do young children learn so much about the world so quickly and accurately? And how can they learn so much when, at the same time, they often seem so irrational and unpredictable? The research in this proposal will help answer these questions by bringing together ideas from computer science with research on very young children. The basic idea is that young children learn in some of the same ways as the most powerful machine-learning programs. Both the children and the computers explore a wide range of more and less likely possibilities. Moreover, children may sometimes actually explore more widely than adults and so be smarter or at least more open-minded learners. Some of their apparently irrational play, like their wide-ranging pretend play, may really reflect powerful learning methods.

This work should have significant broader impact for educational practice. If we understand children's basic natural rational learning mechanisms we can use those mechanisms to help teach science more effectively. In particular, there are significant practical questions about how we can leverage children's spontaneous play to help them learn. Similarly, this research has impact for studies of developmental disabilities such as autism and mental retardation. There is reason to think that children with these syndromes may have particular difficulty with the kind of learning about possibilities that is facilitated by pretend play, and understanding that learning may help us understand and remedy these difficulties.

Memory and culture form the foundation of human knowledge. The accumulation of knowledge across generations drives scientific progress, technological advancement, and the maintenance of societies through shared social norms. Understanding the fundamental cognitive processes that shape memory and cultural transmission has traditionally fallen within the purview of psychologists and anthropologists. However, advances in fields beyond the social sciences, including statistics, machine learning, evolutionary dynamics, and network science offer a rich set of computational techniques for describing information transfer in individuals and groups that can be fruitfully applied to understand memory maintenance and cultural transmission. The proposed project, if successul, will create a software platform for running experiments on personal and cultural memory online, thereby enabling others to conduct their own experiments using these paradigms. Additionally, the public looks toward memory research with the hope that it will one day provide techniques to help them remember more. Better understanding the processes that affect maintenance of personal and collective memories will reveal the conditions under which memories and cultural innovations are best maintained and possible techniques for improving them. Any progress towards this goal would provide a major benefit to society. Planned dissemination of the results of the proposed project are aimed at increasing public understanding of science through visualization of cultural memory dynamics. The Fellow has a history of public engagement and activities supporting inclusion and broadening participation, and continues to be engaged in these activities.

The goal of the proposed project is to construct computational models that capture the cognitive dynamics of memory and culture, with the ultimate goal of understanding (and eventually controlling and improving) the processes that shape them. The proposed project uses behavioral experiments, simulations, and analytic calculations to determine the cognitive dynamics of personal and cultural memory in three settings. Objective 1 considers the dynamics of updating in short-term memory. Objective 2 considers the dynamics of updating culturally-transmitted knowledge when individuals have varying amounts of information about the social context in which they are placed. Objective 3 considers the dynamics of learning in cultural memory, exploring whether individuals can adjust their own behavior to better sustain memories in a group. Each objective makes connections between cognitive processes and formal tools from machine learning, evolutionary dynamics, and network science.

Along with the human brain, human cognition is a product of evolution. The complexities of human cognition, both individually and collectively, include sharing information, innovating, and developing complex technologies, institutions and norms. Why and how these aspects of human cognition and behavior came to be is a question that crosses many disciplines, including biology, psychology, linguistics, archaeology and anthropology. It is also an important one, as understanding how various aspects of human cognition and behavior came to be what they are could have implications for engineering artificially intelligent systems, enhancing and augmenting human capabilities, and improving upon societal conditions.

Traditionally, the evolution of human cognition has been investigated in either of two ways: 1) through computer simulations of possible evolutionary processes that took place over long periods of time and many generations, or 2) through laboratory experiments that examine the behavior of modern humans to test evolutionary hypotheses about aspects of human cognition. This research project develops an innovative third approach that combines the two existing approaches. In this approach, rather than needing to simulate hypothetical human cognitive processes on a computer, a real human being can complete a relevant task online. This allows for real human cognitive processes to be measured and then incorporated into the computer simulation of evolutionary processes. Each participant will perform a computerized task much like in many cognitive science experiments. To incorporate these real human data into computer simulations of evolutionary processes, each person's data will influence the next iteration of the computer simulation. Specifically, the task given to the next set of participants will be affected by the performance of the participants that preceded them. For example, the nature of the available information for a later set of participants might be altered based on the performance of the earlier set of participants. Across many sets of participants and corresponding iterations of the simulation, hypothetical evolutionary processes can be examined to better understand how genes and culture interacted over time and generations to produce present-day human cognitive capabilities and behaviors. The investigators will use crowdsourcing technology to recruit large numbers of participants, and will explore questions about the evolution of language, how complex learning processes evolved, and how genes and culture interact. The software developed to run these simulations will be distributed freely, making it possible for other researchers and educators to run similar simulations to address a wide range of questions about the evolution of human cognition. A version of the software will also be created for educational purposes to teach students about evolutionary processes.

The last few years have seen significant breakthroughs in artificial intelligence and machine learning, resulting in systems that approach or even exceed human performance in interpreting pictures and words. This project explores the implications of these breakthroughs for understanding how the human mind works. Focusing on artificial neural networks, a key technology behind many recent breakthroughs that is capable of discovering novel representations for complex stimuli, the project has two goals. First, assessing the degree of correspondence between human and machine learning by examining whether the pictures or words that are similar in the representations discovered by neural network models are also judged to be similar by people. Second, developing methods for increasing this correspondence, with the goal of being able to use neural network representations to generate good predictions about how people learn and form categories using real images or text.

This research project will answer basic scientific questions about how the representations discovered by contemporary neural networks relate to human cognition. It will then explore what architectures and training regimes produce representations with these properties. In addition, the project will address the methodological question of how one can modify these representations to produce better alignment with human cognition. Answering this question will lead to powerful new tools for making models of human behavior in naturalistic contexts, leveraging the latest results in machine learning to broaden the scope of experimental research in cognitive science. By building stronger links between human and machine learning, this project will have implications for both fields. Even if current neural network systems turn out to differ significantly from human learning, they provide state-of-the-art representations for images and text that can be used as a starting point for developing better accounts of human representations. By discovering the ways in which the representations learned by artificial neural networks differ from those of humans, one can identify new algorithms and training methods that will result in a closer alignment. Since human beings remain the best examples available of systems that can solve certain problems, such an alignment offers a path toward expanding the capacities of current artificial intelligence systems and making them more interpretable by people, which is critical in settings that require human-machine interaction.