James S. Magnuson - US grants

[unreadable] DESCRIPTION (provided by applicant): Language deficits have devastating effects on one's ability to function in society. Designing appropriate interventions depends in part on understanding spoken language processing in healthy adults. Indeed, similarity metrics based on spoken word recognition research have allowed the design of more sensitive tests for hearing and language deficits. In this proposal, four projects examine the effects on spoken word recognition of the temporal distribution of similarity in spoken words, learning, and top-down knowledge. Time course measures are obtained from eye tracking during visually guided tasks under spoken instructions. The eye tracking is complemented by more traditional paradigms, allowing direct comparisons of the measures and providing data for items not amenable to eye tracking. Natural English stimuli and artificial lexicons are used as stimuli. Real words do not fall into conveniently balanced levels on the dimensions of interest, while artificial lexicons allow precise control over phonological similarity and frequency, and therefore competition neighborhood. They also provide a paradigm for studying learning, whether in the case of new words or changes in the relative frequencies of competitors. The results of the projects are used to refine similarity metrics for spoken words and develop a computational model of spoken word processing and learning.

Context changes the way we interpret sights and sounds. A shade of color halfway between yellow and green looks more yellow when applied to a picture of a banana, but more green when applied to a lime. An acoustic pattern halfway between "p" and "b" is interpreted as "p" following "sto-" but as "b" following "sta-". But does context actually alter perception of sights and sounds, or only their interpretation? Cognitive scientists have long debated when and how "bottom-up" input signals (such as speech) are integrated with "top-down" information (context, or knowledge in memory). Do early perceptual processes protect a "correct," context-independent record of signals, or do perceptual processes immediately mix bottom-up and top-down information? One view is that accurate perception requires early separation of bottom-up and top-down information and late integration. An alternative is that early mixing of bottom-up and top-down information would make systems more efficient, by allowing context to immediately guide processing. In studies of language comprehension, this timing question is unsettled because of conflicting evidence from two measures of moment-to-moment processing. Studies tracking people's eye movements on objects upon verbal instructions support immediate integration: helpful information appears to be used as soon as it is available. Studies using ERPs (event related potentials, which measure cortical activity via scalp electrodes) suggest delayed integration: early brain responses appear to be affected only by bottom-up information. Results from the two measures have been difficult to compare because they have relied on very different experimental designs. In the proposed research the investigator will study the timing of top-down integration in human sentence processing using experimental designs that allow simultaneous comparisons of eyetracking and ERPs, with the goal of determining when and how top-down context is integrated with bottom-up signal information.

The proposed work has important implications for the design of language technology. In contrast to computer systems, humans efficiently exploit top-down context, and quickly learn to adapt to new contexts. An obstacle to making computer systems as adaptable as humans is that we do not fully understand how humans balance bottom-up signals and top-down context. The proposed research also has implications for understanding and treating language impairments. For example, understanding how normal perceivers balance and integrate signal and context may help identify subtle bottom-up impairments that lead to unusual reliance on context. The investigator is committed to integrating research and training activities in this CAREER project, and will actively involve undergraduate and graduate students in the research. The investigator will also develop courses designed to prepare students for independent research by providing hands-on training in cognitive theories and time course methodologies.

This Integrative Graduate Education and Research Traineeship (IGERT) award establishes a unique interdisciplinary training program that prepares Ph.D. scientists from cognitive (linguistics, psychology, communication disorders) and biological fields (molecular genetics, behavior genetics, neuroscience) to achieve a unified cognitive-biological understanding of human language development. Innovations from this approach will address societal challenges in education, technology, and health.

Intellectual Merit: A full understanding of human language ? including genetic, neural, cognitive and environmental influences that shape language development and recovery from brain injury ? requires linking a vast array of biological and cognitive sciences. Conventional approaches to graduate training produce cognitive and biological language scientists with little technical overlap ? a barrier to communication. Trainees will develop expertise in their home domains and rigorous interdisciplinary training, providing broad and deep knowledge of theories and methods of other domains to communicate, collaborate and innovate in multidisciplinary teams. The training program uses a team-based model, in an environment designed to foster creativity and innovation ? along with integrated coursework and training in how to translate that research to educational, technological, and health applications.

Broader Impacts: This IGERT award will prepare a new generation of leaders to conduct (and train others to conduct) the team-based basic and applied research needed to achieve a unified biological-cognitive science of language. Key components include: training to communicate with educators, policy makers and the public in innovative ways that address societal challenges in technology and education; comprehensive efforts to increase participation among historically underrepresented groups in science; and preparing trainees for the increasingly international scientific community through on-site and internet-based opportunities to connect with leading cognitive and biological centers of language research around the world.

IGERT is an NSF-wide program intended to meet the challenges of educating U.S. Ph.D. scientists and engineers with the interdisciplinary background, deep knowledge in a chosen discipline, and the technical, professional, and personal skills needed for the career demands of the future. The program is intended to establish new models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries, and to engage students in understanding the processes by which research is translated to innovations for societal benefit.

Learning is the basis of human mental development and the process of learning continues throughout our lives. Perhaps more than anything else, how and what we learn shapes who we are. This project has the dual aims of achieving a deeper scientific understanding of learning, and communicating a deeper understanding of the science of learning to scientists and the public. Over several decades, diverse fields including genetics, neuroscience, linguistics, education, and psychology have generated a wealth of knowledge about myriad aspects of how humans learn, but a grand challenge remains: to integrate that knowledge into a unified understanding of learning based on these diverse fields, and on scales ranging from the gene to the neuron, brain, and human behavior. This National Science Foundation Research Traineeship (NRT) award to the University of Connecticut will help prepare the next generation of researchers to meet this challenge. The traineeship, focusing on the science of learning, will train fifty (50) PhD students, including twenty-five (25) funded trainees, from education, genetics, linguistics, psychology, neuroscience, and speech-language-hearing sciences. By emphasizing a problem-based approach (learning by doing), embracing collaboration across a broad a spectrum of disciplines, and integrating hands-on training in communication and leadership skills that enable effective multidisciplinary project design, management, and scientific discovery, this program will offer trainees unique preparation for a range of careers in academia, industry, and the public sector.

The training program has five major components: an intensive one-year seminar that surveys the science of learning across all participating fields, a hands-on practicum where trainees learn to design and implement multidisciplinary research, faculty-student research interest groups that serve as brainstorming launch pads for new scientific challenges, data stewardship modules, and integrated training in outreach and communication. The seminar provides a cross-disciplinary introduction to the science of learning and the challenge of technical communication, while the practicum emphasizes practical skills crucial in academic and nonacademic careers that graduate education too often lacks: project design and management, budgeting and resource allocation, and external communications. The research interest groups, which will evolve as promising research proposals emerge from the practicum, serve as both focal point for research and organizational structure for the participants. From their first day in the program, students face the challenge of how to clearly and effectively share ideas without assuming prior knowledge or relying on technical jargon, a skill that not only enables excellence in research, but empowers trainees to become ambassadors for their work to society as a whole. This program will also promote diversity in careers requiring advanced training through best practices in recruitment and retention.

The NSF Research Traineeship (NRT) Program is designed to encourage the development and implementation of bold, new potentially transformative models for STEM graduate education training. The Traineeship Track is dedicated to effective training of STEM graduate students in high priority interdisciplinary research areas, through comprehensive traineeship models that are innovative, evidence-based, and aligned with changing workforce and research needs.

More than half of the world's population speaks two or more languages fluently. Speaking more than one language allows you to communicate and interact with individuals in other countries and cultures through conversation and reading. It also has economic benefits, and possibly even cognitive and health benefits. Scientifically, there are many open questions about bilingualism for education (how can we best train people in new languages?), health (how can you best treat a bilingual person with language difficulty after a brain injury?), and technology (how can we make speech recognition systems as flexible as multilingual humans?). To better understand multilingual language processing, researchers will record neural responses to speech in people who know either one, two, or three languages, while they listen to the languages they know. The project is a transatlantic collaborative effort with US researchers partnering with Spanish researchers. High school, undergraduate, graduate, and postdoctoral trainees will receive training in cutting-edge computational and cognitive neuroscience and psycholinguistics, including data science and modeling skills transferable to a range of academic and non-academic STEM careers.<br/><br/>One of the fundamental questions that researchers will explore is how two or more languages co-exist within a single language system and how each is represented in the brain. Some prior research suggests there is a deep continuous coactivation of all the languages a person knows even when they are in a single language context, while other research suggests that under many circumstances, only the language relevant in the moment is activated. The project will use the tools of computational neuroscience to develop cognitive theories and implemented models of bilingual and trilingual language processing, which the research team will compare to neuroimaging data with high temporal resolution (magnetoencephalography or MEG). MEG will be collected while monolingual, bilingual, and trilingual individuals process speech from languages they know under conditions designed to promote attention to a single language (isolated words or continuous speech from only 1 language) or two languages (random interleaving of isolated words from 2 languages, or more ecological 'code-switching' between 2 languages). Researchers will use a state-of-the-art neural network model of human speech recognition developed with previous NSF support. They will use continuous speech tracking to relate neural activity to both theoretically-generated hypotheses regarding potential impacts of language co-activation and the behavior and internal activity of neural network models. In this way, researchers will be able to compare human brain responses and neural network model responses to statistical predictions of the expectation level for each successive speech sound (consonant or vowel) and word during presentation of continuous speech. Comparing different models to neural responses will help researchers address fundamental questions, such as whether all the languages a person knows are active whenever they hear any language, and whether this is helpful or causes interference. This research promises to deepen our understanding of multilingual language development and processing in the human brain.<br/><br/>A companion project is being funded by the State Research Agency, Spain (AEI).<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.