Gary Lupyan, Ph.D. - US grants

This INSPIRE award is partially funded by the Perception, Action, and Cognition Program in the Division of Behavioral and Cognitive Sciences in the Directorate for Social, Behavioral, and Economic Sciences and the Mathematical Biology Program in the Division of Mathematical Sciences in the Directorate for Mathematical and Physical Sciences.

This work explores the role of selection and adaptation in two radically different domains, 1) molecules and 2) languages. Consider, for example, human languages. The 6,000-7,000 languages spoken by people display a dazzling variety of sounds, words, and grammatical forms. This diversity is typically explained as a product of random drift: As a single population splits and drifts apart, the accumulation of small random changes eventually produces mutually unintelligible languages. An alternative is that some of the variation we see among human languages is due to selection. In this account, languages adapt to some extent to the different social and ecological environments in which they are spoken. Similarly, researchers have only recently considered the role of selection and adaptation in the study of prions (self-replicating proteins)and how they propagate within and across generations of cells.

Though biological structures and human languages are radically different domains, they share properties that suggest they may be described by a common mathematical framework. Specifically, (i) they are both epigenetically inherited, (ii) they both capitalize upon a pre-existing biological substrate, and (iii) they both propagate in a system of agents (cells, people). A highly interdisciplinary team of cognitive scientists, linguists, biologists, and mathematicians seek to connect and inform these domains by using mathematical models and large-scale behavioral experiments to understand the selective processes. They will assess convergent tests of the idea that selection acts as an organizing principle of systems at different scales. This work has important implications for issues such as the role of environmental context in the spread of structures such as prions or linguistic elements. For example, results could help explain how a structure newly infects populations, such as when a new word "invades" a linguistic environment, or when a prion structure successfully propagates and infects cells in its environment. The interdisciplinary nature of this award will provide a unique training experience for graduate students and will include outreach efforts to local schools.

To what extent is human cognition (e.g., visual perception, memory, and categorization) augmented by language? Does language only allow us to communicate with more flexibility or does language transform cognition, allowing humans to represent and manipulate information in novel ways? A series of studies previously conducted by this investigator has shown that language has pervasive and surprising effects on a range of abilities, such as learning new categories, using knowledge about familiar categories, visual memory, and even perception: Hearing a word can literally change what one sees. The proposed studies build on this earlier work by exploring the design features of language that make it an especially useful tool for constructing and manipulating mental representations. The studies use a variety of behavioral paradigms together with noninvasive brain stimulation to explore the behavioral and neural processes involved in the interaction of language, learning new categories, and retrieving information about familiar concepts and categories.

The proposed studies are critical not only for understanding the broad issue of language-augmented cognition, but also for the potential to understand applied issues related to linguistic and cognitive development. Hearing less language in early childhood leads not only to negative outcomes in using language to communicate, but appears to extend more broadly to other cognitive abilities. By understanding the mechanisms by which language augments cognition, we will better understand the broad consequences of language disorders, which may inform the development of effective interventions. The work will also enhance infrastructure for research and education by implementing and disseminating tools for efficient and cost-effective data collection over the web using crowd-sourcing services, which allow for generalizability to broader populations and will allow undergraduates to more easily conduct original research projects.

Background: We now know that children's language learning is strongly influenced by their linguistic environment including the type and quality of the language they hear. We also know that larger vocabularies during early childhood are associated with positive academic, economic, and even health outcomes later in life. Lacking, however, is a mechanistic understanding of the nature of the causal links between children's word learning and long-term outcomes. The current research addresses this gap by examining whether learning certain ?seed words??words with high inductive potential?accelerates word learning to a greater degree than other words and causes higher performance in non-linguistic categorization tasks. Specific Aim 1: To identify ?seed? words that are associated with consistently faster vocabulary growth Specific Aim 2: To determine whether teaching ?seed? words yields positive language outcomes. Specific Aim 3: To test when teaching seed words promotes category induction in nonverbal tasks. Methodology: We will examine children's language and cognitive development in a microgenetic longitudinal study. One-hundred twenty 30- to 36-month-old children will be randomly assigned to one of three experimental conditions: (a) a high inductive word training condition, (b) a low inductive potential word training condition, and (c) a control condition in which children do not receive language training. We predict that children trained on previously identified ?seed words? will have faster vocabulary development and higher performance on a nonverbal category induction task than children in the other two experimental conditions. Significance: The proposed studies will contribute to a greater understanding of children's language development. By undertaking a theory-guided identification of early-learned ?seed? words and measuring the effect that learning these words has on subsequent language development, the proposed work will inform the design of future language interventions. By investigating the relationship between word knowledge and category learning, the proposed work aims to understand why language abilities are so closely linked to other cognitive abilities.

Humans have remarkable abilities to find deep similarities in what are superficially different experiences and to use those similarities to help categorize their experiences. This process of categorization allows people to learn from examples and to generalize that knowledge to new situations. Categorization is also a fundamental aspect of human reasoning. This work investigates whether a key contributor to human categorization success is the ability to name the features and dimensions that compose the categories. Testing the causal link between naming and categorizing helps us understand why children with poorer language skills often go on to have poor academic outcomes, even in domains that appear to be nonverbal in nature like geometry. This work enhances scientific infrastructure by developing a freely available online tool for measuring the nameability of any visual or auditory set of items for adults and children, which will help researchers and educators more effectively describe learning strategies and allow for better understanding the sources of children's reasoning errors.

To explain what makes some categories harder to learn than others, researchers have typically posited a fixed set of features that are available to the learner. But where do the features come from? This work tests the hypothesis that an important source of features is the words people learn when learning a language. On this view, the words of a language do not simply map onto pre-existing conceptual distinctions, but are one of the contributing factors that create the distinctions. This hypothesis is tested using a series of category-learning and category-induction experiments with adults and a unique population of linguistically deprived children. The experiments systematically test the extent to which ease of naming predicts categorization success and tests the causal involvement of language by using verbal interference protocols. The behavioral work is guided by computational modeling using convolutional neural networks to help disentangle whether a category is easy to learn because it is nameable or whether it is nameable because it is easy to learn. This work is among the first to use deep-learning models to understand human categorization.

This project tackles a previously unexplored problem in the relationship between human and machine learning. Many problems that challenge human intelligence (chess, Go) have yielded to modern computer algorithms. Yet some tasks that are easy for humans, or even animals ? such as flexible locomotion and rapid and robust visual understanding of the surroundings are still at the cutting edge of artificial-intelligence research. Computers calculate without error. Yet, for example, quite a few people who know the difference between odd and even will say that 798 is odd, perhaps because two thirds of its digits are odd. Are there fundamental differences between the way computers learn and way humans learn? Can they be found with a rigorous study of games where the player must learn a rule by trial and error? This project uses games involving the learning of rules to explore similarities and differences between human and machine learning. It will seek new insights into human learning and may improve understanding of machine learning as well. Long-term, it aims to better integrate algorithms and humans for solving real-world problems; humans and computers work together best when they can complement each other, This project will seek generalizable distinctions between rules that are easy for humans and rules that are hard for humans; the special focus is to find problems where the order of difficulty is exactly reversed for machines. Finding the principles behind these reversals will help to triage problems. The long-term goal is hybrid systems, human and machine learning integrated to achieve goals such as medical diagnosis, treatment planning, etc. This project if successful will contribute to rigorously defining how and why some learning problems that seem relatively easier for humans are nonetheless more difficult for machines, and vice versa. With a focus on the specific activity of rule finding, this research may even shed new light on the scientific process, which has been characterized as ?discovering the rules of nature.?

This project explores complementarity between Machine Learning and Human Learning with a rigorously balanced approach, using a ?rule induction? challenge that is presented to both humans and computers. Computers will use state-of-the art deep neural networks, and explore the hypothesis space of rules describable in the project?s coding language. The psychological research investigates crucial problems such as transfer learning across rules, and the role of language and naming in rule discovery. Both human and machine ?players? learn the rules by trial and error. The rule encoding language, reinforcement-learning processes, and scoring systems ensure symmetry of human and machine learners. Performance measures will include discounted reward and convergence to error-free play. Learning curves will be used to measure the difficulty of learning each rule. Experimental conditions will be systematically varied, including not only the rule to be learned, but also parameters such as the minimum and maximum number of different shapes displayed, the maximum number of ?boards? that a user may use in attempting to learn a given rule, and the incentive/reward structure by which players earn rewards for their performance. The research will seek identify pairs of classes of rules such that the class that is easier for humans is more difficult for computers, and vice versa. The project will involve extensive experiments using diverse machine-learning approaches, as well as Amazon Mechanical Turk for data on human learning performance."Comparing the learnability of different rules sheds new light on human learning biases, may prove useful for structuring curricula, and may help identify which gaps in knowledge are most detrimental to human problem solving. The goal is to interpret or explain what distinguishes these anomalous pairs of rule classes from others where the relative degree of difficulty is the same for humans and computers.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Much of our world knowledge (also known as "semantic" knowledge) comes from our direct experiences with the world. For example, people can learn that snow is cold through touch and that sparrows fly through sight. But analyses of language reveal that a large amount of information about the world is contained within the structure of language itself. That is, by tracking which words occur with which other words, it is possible to learn things that seem to require direct experience. Moreover, the co-occurrence of words in different languages seems to reflect somewhat different bodies of knowledge. The project's principal aim is to 1) explore the scope of semantic knowledge embedded in the structure of different languages and 2) understand the extent to which people use this embedded information to learn about the world. Our educational systems depend on the ability to transmit knowledge via language in both its spoken and written forms. Understanding the kinds of semantic knowledge typically learned through language can help reveal the consequences of inequities in language exposure, such as those caused by reading difficulties.

To understand the relationship between people?s knowledge and information embedded in the structure of different languages, the investigators will compile a corpus of semantic features and generic statements from native speakers of eight languages: English, French, German, Dutch, Italian, Spanish, Mandarin Chinese, and Russian. They will then compare this information, generated by people, to information automatically derived from the distributional structure of each of the eight languages. This will allow them to determine whether cross-linguistic differences in people?s knowledge are predicted from differences in the information embedded in the different languages. The investigators will estimate the causal impact of language on people?s semantic knowledge using a quasi-experimental approach, rather than the more typical correlational analyses. The data will be compiled into a user-friendly, large-scale data resource (all with open source code and data) and integrated with existing multilingual text resources. The multilingual feature norms and ratings of generic statements will be an important resource for artificial intelligence and NLP research and may help identify sources of biases in training sets used in machine learning. This approach brings computational and empirical methods to the study of one of the oldest of human questions: how do we know what we know?

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.