Lori L. Holt - US grants

Exciting new research has promoted a vital sub-field of speech perception research concerned with describing the function of categories in the development and maintenance of language- appropriate perception. Recent work has suggested that at least part of the formation and function of phonetic categories is a result of general perceptual categorization mechanisms not specific to speech or language. Thus, there now appears to be opportunity for an integration of general categorization research with work on first and second language acquisition. Unfortunately, much of what is known about perceptual categorization has been derived from examination of categories that are fundamentally different from phonetic categories. Moreover, it is empirically difficult to examine influences of categorization using speech stimuli because it is extraordinarily difficult to determine a detailed history of experience. Pilot work by the PIs has suggested the utility of using complex non- speech sounds in probing the learning mechanisms that drive auditory categorization. These sounds can be synthesized to mimic complexities of phonetic categories and distributions of stimulus presentation can be theorectically derived to model aspects of phonetic categories while maintaining full experimental control over experience. The main goals of this work are threefold. The first goal is to provide a detailed database of the formation and structure of complex auditory categories. There is a dearth of research in this area and the proposed work will be useful in developing a taxonomy of auditory learning and testing extant models of general perceptual categorization (which have been based primarily on data from visual tasks). Experiments using explicit and incidental learning procedures will map the development of categorical response structures as listeners gain experience with novel stimuli. The second goal is to compare the resulting structures that arise from these categorization tasks to structures typical of speech categories such as categorical perception and the "perceptual magnet" effect. The third goal is to develop efficient methods of exposure and training to teach non-native contrasts to second-language learners. Learning the sound contrasts for a non-native language is an extremely difficult task. Exposing the mechanisms of complex category learning could illuminate potential aids to training individuals to discriminate these complex speech categories. These aids could extend easily to other complex learning tasks such as musical training, acoustic warning systems or auditory data displays.

When a child acquires a language, they must make sense of the language they hear around them. But this language is not always the same - it varies from speaker to speaker and situation to situation. Different people the child hears speak differently, and his/her mother will also use different kinds of language when playing or disciplining him/her. Professors Suzanne Curtin and Scott Kiesling of the University of Pittsburgh and Lori Holt of Carnegie Mellon University are linguists and psychologists who are exploring how children manage to organize speech sounds in their minds, and why they end up speaking the way they do. How much influence do caregivers have in this process? Researchers know from previous studies that children do not reproduce their parents' accent exactly, but that their parents' accent usually does make a difference in how they talk. It has also been observed that children of parents who have foreign accents are sometimes not aware that their parents speak differently from native speakers. What aspects of the child's accent are influenced by the caregiver, and what are determined by other forces (especially peers), and at what age? Understanding how individual children develop their own unique speech pattern will provide a greater understanding of the role of the language that children hear in shaping how they acquire language. Most research in this area has compared the speech produced by adults and children who have parents from different language backgrounds. This study explores how children perceive the language they hear around them, how they represent it in their minds, and how that knowledge changes as they mature.

Vowels are more likely to be pronounced differently in dialects than other speech sounds, even in the same speaker. For example, the way someone from Chicago says the vowel in "hat" is noticeable to someone from Tennessee, but there are no differences in the consonants between these two cities. Since vowels are more variable in pronunciation, and are also more easily measured with phonetic equipment, we focus on how children perceive vowel differences. First, children 6 to 24 months will be tested to see whether they can hear different ways of pronouncing the vowel in the word 'hay,' and in the word 'hoe.' In a second experiment, children are tested to see if they prefer one way of pronouncing the 'hoe' vowel over another instance. Methods to record children in naturalistic interaction with same-age children will be tested, as will methods to record interactions with a caregiver. These methods will help determine how children sort out all the variable information they receive in the speech around them and come up with their own way of speaking, and will be used in a future study that will follow children from infancy to school-age.

DESCRIPTION (provided by applicant): This collaborative research program investigates processes underlying the formation and tuning of complex sound categories. The overall goal is to provide a model of auditory categorization that can be readily applied to challenges of speech perception and communication disorders. Language learners form (phonetic) auditory categories of native-language sounds from the distributions of experienced speech sounds produced by many talkers. However, these averaged categories may not be appropriate for the speech produced by a specific talker. For example, non-native speech may not adhere to the patterns typical of native speakers. The aim of the current project is to develop and test a theoretical and practical model of how listeners use context to normalize, or tune, speech perception to the characteristics of a particular listening situation. The proposed experiments will move the model beyond mere demonstrations of normalization to make quantitative predictions of performance as a function of the content and temporal extent of the context. Such a practical model can be used to develop signal processing strategies for hearing aids and implants as well as to predict intelligibility of disordered speech. Building on the empirical outcomes of the previous project, the present research tests predictions arising from the hypothesis that a general auditory mechanism sensitive to the spectral interactions that occur between context and target sounds can account quantitatively for patterns of speech perception that appear to require extraction of vocal-tract-specific talker information. Another set of experiments will test the influence of perceptual learning of talker-specific patterns of speech in supporting this mechanism. A final series of experiments will bridge the gap that often exists between tests of speech perception phenomena and understanding real-world speech intelligibility and comprehension. Such a linkage is critical for deriving theory- and evidence-based clinical approaches in treatment of communication disorders. PUBLIC HEALTH RELEVANCE: Public health requires therapies developed based on detailed knowledge of the underlying mechanisms. Understanding how listeners encode the complex acoustic structure of speech across many talkers is critical to developing and evaluating therapies for individuals affected with language processing disorders, hearing impairment and developmental disorders like autism.

The growth in globalization across traditional language boundaries suggests a need for efficient second language (L2) acquisition training regimens. One of the most significant challenges for adult language learners is learning to hear fine distinctions among non-native sounds not used in the native language; such learning may require decades of experience with the second language. A classic example is the difficulty native Japanese have learning English /r/ and /l/, a sound contrast not present in Japanese. With prior NSF support, Drs. Holt and Lotto have uncovered principles of auditory learning using controlled experiments with non-speech sounds and have used these principles to design optimal training regimens. This project uncovered how characteristics of training, feedback and presentation mode affected auditory learning.

The present project will apply these findings to adult learning of non-native speech sounds, with the aim of producing more efficient L2 learning. One series of studies will investigate the benefits of video-game-based training (found to foster non-speech category learning) in learning non-native speech sounds. Another series of experiments will test whether manipulation of the variability of sound cues, found to be important in non-speech auditory learning in prior research, is effective in shifting the attention listeners give to these cues in second-language learning. Such shifts appear to be important for many cases of L2 learning, such as native Japanese speakers learning English /r/ and /l/. Beyond practical application in adult second language learning, the project has important theoretical implications for understanding human auditory perception and language processing. Such understanding is a prerequisite to developing rehabilitative techniques for disorders such as autism, dyslexia, central auditory processing disorder and specific language impairment.

To become fluent native speakers and listeners, children ultimately must perceive and produce speech that is consistent with the local dialect. For example, English spoken in Calgary, Canada sounds slightly different than English spoken in Pittsburgh, Pennsylvania. The variation in these two English environments is patterned and, in learning language, children must acquire these regularities. Seemingly at odds with this, children must also maintain some flexibility in mapping speech sounds to meaning. For example, infants sometimes encounter talkers whose speech deviates quite drastically from their community's dialectal norms, as when addressed in English by a non-native speaker from Italy. Thus, while learning the normative patterns of their local dialect, children also must maintain enough cognitive flexibility to accommodate talkers who do not fall within these norms. How do infants and toddlers accomplish these seemingly competing tasks in language learning? Answering this question will reveal general characteristics of children's learning at multiple time scales: from spontaneous interpretation of fleeting speech sounds, to short-term adaptation to deviations from expectation, to longer-term learning of the regularities of the local dialect. This research maps the range of acoustic variability to which infants are exposed in their native speech environment in different social interactions with adults. It will create a detailed acoustic speech production corpus with which to understand the challenges confronting infant language learners. The research will also study how infants' and toddlers' speech perception is shaped by regularities of the local dialect, revealing the mechanisms of long-term sensitivity to dialect norms and short-term adaptation to deviations from dialect norms. It will also investigate how toddlers' growing vocabularies influence adaptation to variability in spoken language.

The research takes an innovative cross-disciplinary approach to these questions. Understanding early development of the knowledge of the sound structure of English and its interaction with developing vocabularies will contribute to our understanding of typical language development. It will also further our understanding of accent development in multilingual learning situations and early communicative deficits. An important outcome of the project will be the dissemination of a speech corpus, providing a rich acoustic sample of speech for other researchers to use in studying English dialects.

Program Summary The objective of the proposed predoctoral training program is to train the next generation of behavioral science researchers to skillfully incorporate neuroscience perspectives and methods into their programs of research, based on an understanding of brain structure and function that extends across traditional areas of behavioral research. The Behavioral Brain (B2) Research Training Program has the specific aim of annually providing 8 graduate student trainees committed to research at the interface of the behavioral and brain sciences with foundational training in neuroscience methods and perspectives, through coursework and laboratory-based research experiences. Over the course of two years of support typically occurring early in the graduate career, trainees also will receive deep training in behavioral science research, via courses and independent programs of research. Finally, we aim for continued infusion of cross-cutting perspectives, through co-mentoring, laboratory rotation experiences, and program forums that foster exposure to behavioral and brain science research. We believe that basic research focused on the interface between behavior and the brain is crucial for understanding the mechanisms and treatment of a large number of human health issues that cut across NIH Institutes. Because NIGMS has a broad mission, it is a natural home for a training program that aims to bridge behavioral and biomedical approaches across traditionally separate lines of inquiry in the behavioral sciences. By focusing upon the brain as a common substrate, we believe progress in different subfields of behavioral research can be most effectively integrated, thus leveraging advances in one area into other domains of study. This training program focuses on three major research themes to accomplish integration: Representation & Communication; Evaluation & Control; Learning, Memory, & Plasticity. The training program is jointly coordinated by the University of Pittsburgh and Carnegie Mellon University. Situated within blocks of one another and possessing excellence in both behavioral research and neuroscience, the institutions share a long history of collegiality and cooperation in graduate training that can be leveraged to broaden and deepen the neuroscience training of the next generation of behavioral science students.

Globalization has increased the importance of information transfer across traditional language boundaries. This has created a demand for new approaches to adult language learning that are both efficient and effective. This need is real; there are more second-language speakers of English in the world than native speakers. Bilingualism also confers a competitive advantage with respect to enhanced cognitive abilities and protection from neurodegenerative decline, so developing next-generation approaches to promoting bilingualism may have even broader impact. One of the significant challenges to this endeavor is teaching adults the sounds of a new language. The speech sounds that form the building blocks of language vary considerably across languages. A long history of laboratory research demonstrates that the brain "commits" to the speech sounds of the native language. This supports efficient native speech processing but confers a cost to learning a new language that uses a different sound system. Perceiving non-native speech sounds in adulthood is a case where it has been thought that the adult brain simply cannot be trained to learn. This consensus has arisen from decades of attempts to devise effective learning regimens for training adults to improve non-native speech perception. To summarize this literature very briefly, learning gains are modest even with extensive training over the course of weeks. Moreover, learning does not generalize well to unfamiliar instances of the sounds or to new voices.

Under the direction of Dr. Lori Holt, Mr. Liu will target a promising new approach to this problem. Prior research from this team led to the counterintuitive finding that directing attention away from the task of language learning can lead to more efficient learning gains than the traditional approach of making overt judgments and getting explicit feedback. Examining adult learning when training is "hidden" in other engaging tasks is a novel and potentially more effective approach to teaching adults new language sounds. Mr. Liu's dissertation project builds on proof-of-concept evidence within this implicit training task. This study will examine whether extended (1-week) implicit perceptual training targeting improvement of speech sound representations impacts underlying neural representations (assessed via electrophysiological brain responses) and transfer to improvements in vocabulary-learning, modeled after classroom learning. Parallel studies will investigate conditions that optimize robust adult learning and generalization to new talkers and words. Results from this research will contribute to the understanding of the limits of adult perceptual plasticity and the nature of language learning. It also will inform the development of effective and engaging next-generation educational tools.

An estimated 43% of Americans read at or below a basic level of skill necessary to perform simple and everyday activities. This leads to widespread social-economic problems including lower participation in the labor force, heightened reliance on public assistance, poorer outcomes among offspring, and lower civic involvement. It is estimated that this costs the US more than $255 billion annually in lost tax revenue due to unemployment, lowered productivity, and crime. There is a critical need to understand the fundamental cognitive building blocks involved in learning to read, in order to improve the well-being of individuals in society and to improve economic competitiveness. The present project will lay the necessary scientific groundwork to advance understanding of how this learning develops across childhood and into young adulthood. The research team has discovered a connection between poor reading in developmental dyslexia and phonetic acquisition, a foundational aspect of learning to read. Building from this, the project will advance understanding of the developmental course of this learning in relation to mathematical, cognitive and language skills. The research will advance theoretical models of spoken and written language and will support evidence-based approaches to improving the level at which Americans read. The project's broader impact is enhanced by its implications for developmental dyslexia, a common developmental disorder that affects approximately 7% of the population. The research team will also be active in communicating their scientific discoveries to the public to promote STEM literacy in the community. An additional outcome will be training a diverse group of graduate and undergraduate scientists.

We do not yet adequately understand the cognitive and perceptual mechanisms that support learning to read, or how they may falter in poor readers. We do know that low literacy is not an isolated limitation in learning written language. Rather, it is associated with diverse learning challenges extending broadly to other aspects of language and even to visual and motor learning. In this context, the core hypothesis of the present project is that domain-general procedural learning supports development of robust phoneme categories, with associated benefits for literacy acquisition. The research team's prior work established that poorer reading skill among adults is associated with poor incidental learning of nonlinguistic auditory categories. The objective of the present project is to characterize procedural auditory category learning and its relationship to reading skill across development. The project involves a bi-national US-Israeli research team with cross-cutting expertise to examine procedural auditory category learning (1) from early childhood to young adulthood to capture earlier and later stages of literacy acquisition and maturation of memory systems; (2) in relation to developing language, cognitive and mathematical abilities, and individual differences; (3) in English and Hebrew, for which literacy demands are distinct; (4) in the context of procedural learning impairments of children and adults with developmental dyslexia; and (5) across periods of offline learning to characterize consolidation and retention of learning. In sum, the project attempts a comprehensive examination of procedural auditory category learning. It will advance understanding of the basic building blocks of literacy with the potential to transform research and theory in the field and to advance evidence-based approaches to addressing low literacy in the US.

This project is being supported by a partnership between the National Science Foundation and the U.S.-Israel Binational Science Foundation.

PROJECT SUMMARY/ABSTRACT Human communication and other listening behaviors often take place in acoustically complex, or noisy environments like schools, restaurants, and workplaces. Much of daily life requires us to select behaviorally- relevant auditory dimensions, and potentially suppress irrelevant dimensions, so that the information conveyed can be remembered and responded to appropriately. Unfortunately, this vital everyday ability is affected by many neurological conditions resulting in marked decreases in quality of life. Despite the importance of auditory selective attention, its cognitive and neural mechanisms are poorly understood. For example, although auditory selective attention is widely presumed to involve both a selective enhancement of behaviorally relevant auditory dimensions and suppression of dimensions outside this attentional focus, evidence for suppression is scant. The long-term goal of the proposed research is to arrive at a mechanistic understanding of auditory selective attention. The present project pursues the central hypothesis that human auditory selective attention is a result of processes related to both enhancement (of task-relevant sounds) and suppression (of task-irrelevant sounds). Preliminary studies establish a nonspeech experimental paradigm for engaging - and improving ?auditory selective attention directed to specific frequency bands, and for non-invasively mapping it across auditory cortex using multimodal MRI. A parallel preliminary study establishes that attention training drives improvements in behavioral and electrophysiological measures of auditory selective attention. Aim 1 will determine the fine- grained ?listening window? through which auditory selective attention prioritizes and selects behaviorally relevant auditory dimensions, and potentially suppresses irrelevant dimensions. These studies also will determine the extent to which tasks and expectations created from input regularities shape the listening window. Aim 2 will assess changes in the spectrotemporal shape of the auditory attentional filter as listeners learn to more efficiently deploy auditory selective attention to specific dimensions. Aim 3 will identify the neurobiological underpinnings of auditory selective attention and their changes across improvements in selective attention. In all, the proposed research will weave together classic psychophysical approaches, behavioral training as a means to introduce targeted demands on selective attention, and newly-developed human neuroimaging tools to examine human auditory selective attention along the primary axis of auditory representation ? frequency. This will build a bridge from perceptuo-cognitive assays of human auditory selective attention to mechanistic electrophysiological and cellular/molecular studies thus far only undertaken with invasive nonhuman animal work, thereby compounding understanding and building a natural path toward future evidence-based approaches to the remediation of auditory attention impairments.

PROJECT SUMMARY/ABSTRACT In recent years, human cognitive auditory neuroscience has made rapid strides due to advances in human neuroimaging, the advent of innovative machine learning/big data analytic approaches, and a greater mechanistic understanding of cognitive-sensory interactions in animal models. The dynamic landscape of this emergent field necessitates a highly interdisciplinary, human and translation-centric symposium that brings together expertise across academia and industry. This application requests partial funding for the Symposium on Cognitive Auditory Neuroscience (SCAN) to be hosted in Pittsburgh, PA in July 2020 and 2022, as a joint venture between Carnegie Mellon University (CMU) and University of Pittsburgh (Pitt). As a biennial meeting, SCAN aims to become the premiere intellectual and professional venue for current research in the emerging field of human cognitive auditory neuroscience. SCAN will incorporate elements typical to academic conferences (research talks, posters) as well as novel ideas that promote ?blue sky? thinking in this rapidly evolving field. SCAN will assiduously and innovatively work towards inclusivity and creating an atmosphere that encourages intellectual and professional engagement from women, underrepresented minorities, and individuals with disabilities. Another critical aim of the SCAN is to foster industry-academic partnerships with an eye towards translation of basic research and fostering career opportunities for trainees. Pittsburgh is uniquely situated to launch SCAN. With an enviable concentration of co-located auditory neuroscience expertise, Pittsburgh is also an intellectual hub for industries/start-ups engaged in in machine learning, natural language processing, and speech recognition. SCAN will leverage these advantages to foster growth and innovation tied to core missions of the National Institutes of Deafness and Communication Disorders.

PROJECT SUMMARY/ABSTRACT Human communication and other listening behaviors often take place in acoustically complex, or noisy environments like schools, restaurants, and workplaces. Much of daily life requires us to select behaviorally- relevant auditory dimensions, and potentially suppress irrelevant dimensions, so that the information conveyed can be remembered and responded to appropriately. Unfortunately, this vital everyday ability is affected by many neurological conditions resulting in marked decreases in quality of life. Despite the importance of auditory selective attention, its cognitive and neural mechanisms are poorly understood. For example, although auditory selective attention is widely presumed to involve both a selective enhancement of behaviorally relevant auditory dimensions and suppression of dimensions outside this attentional focus, evidence for suppression is scant. The long-term goal of the proposed research is to arrive at a mechanistic understanding of auditory selective attention. The present project pursues the central hypothesis that human auditory selective attention is a result of processes related to both enhancement (of task-relevant sounds) and suppression (of task-irrelevant sounds). Preliminary studies establish a nonspeech experimental paradigm for engaging - and improving ?auditory selective attention directed to specific frequency bands, and for non-invasively mapping it across auditory cortex using multimodal MRI. A parallel preliminary study establishes that attention training drives improvements in behavioral and electrophysiological measures of auditory selective attention. Aim 1 will determine the fine- grained `listening window' through which auditory selective attention prioritizes and selects behaviorally relevant auditory dimensions, and potentially suppresses irrelevant dimensions. These studies also will determine the extent to which tasks and expectations created from input regularities shape the listening window. Aim 2 will assess changes in the spectrotemporal shape of the auditory attentional filter as listeners learn to more efficiently deploy auditory selective attention to specific dimensions. Aim 3 will identify the neurobiological underpinnings of auditory selective attention and their changes across improvements in selective attention. In all, the proposed research will weave together classic psychophysical approaches, behavioral training as a means to introduce targeted demands on selective attention, and newly-developed human neuroimaging tools to examine human auditory selective attention along the primary axis of auditory representation ? frequency. This will build a bridge from perceptuo-cognitive assays of human auditory selective attention to mechanistic electrophysiological and cellular/molecular studies thus far only undertaken with invasive nonhuman animal work, thereby compounding understanding and building a natural path toward future evidence-based approaches to the remediation of auditory attention impairments.

Eavesdropping on gossip in a bustling cafe, tracking the quality of a sick child’s breathing through a nursery monitor, and listening for the high-pitched squeak of a mouse amongst rustling leaves all rely upon auditory selective attention. Humans and animals must prioritize, select, and sustain attention to relevant sounds – and potentially suppress irrelevant sounds – to guide decisions and actions. This selective attention guides our ability to listen, and is crucial to everyday behavior, especially in noisy environments, and its breakdown greatly impacts quality of life for many listeners, particularly as we age. Yet, despite being a fundamental human ability, we know relatively little about the cognitive and neurobiological basis of auditory selective attention. A better understanding of selective attention will have broad societal impact by guiding improvements to machine listening and next-generation hearing aids. This transatlantic partnership will also develop multiple educational outreach programs that include the creation of videos about cognitive neuroscience and the importance of auditory attention in hearing. These videos will be widely disseminated and linked to NSF’s Classroom Resources and the UK’s BBSRC Schools and Young People sites. <br/><br/>This binational, collaborative project links five laboratories from the United States and the United Kingdom to undertake cross-cutting behavioral and neurobiological research on auditory selective attention. The project capitalizes on interactions between learning and attention: when sounds occur frequently, listeners detect them more readily in noise and respond to them more quickly in decision making. This suggests that the rich patterns of sensory signals structuring the natural world may help to direct attention. The project will include studies of auditory attention in humans and also in an animal model (the ferret - whose audiogram and hearing is very similar to humans). The project unifies cross-species approaches with a pair of simple auditory tasks (duration detection and interval detection across a range of spectral frequencies), each of which provides unique opportunities to understand how input regularities in acoustic stimulus statistics direct attention to specific sounds or qualities of sounds. A novel aspect of this research is that parallel studies, using an array of complementary neurobiological techniques, will provide measurements over multiple levels of analysis from single neurons to whole brain activity, that will be integrated into a comprehensive picture of selective auditory attention. The project will use electroencephalography (EEG), magnetic resonance imaging (MRI), functional MRI (fMRI), high-density electrophysiology, and intracranial recordings from neurosurgical patients, to test competing hypotheses about how the brain’s cortical response to sound is shaped by acoustic context and experience. This research is likely to lead to new insights and advances in our understanding of the neural basis of auditory selective attention, which is crucial for hearing.<br/><br/>This proposal is awarded under the SBE-UKRI Lead Agency Opportunity.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Whether chatting with a friend at a café, discussing a topic in a classroom, or debating politics with a relative, we depend on our ability to communicate with many different people. Only 5 percent of the 7.5 billion English speakers are native talkers and there are also tremendous dialect differences in regional English. Thus, we quite often encounter a speaker whose speech differs from our own. Prior research demonstrates that links in how the brain coordinates listening and speaking can lead conversation partners to sound more like one another. The influence of what one hears on how one talks is often referred to as “transfer.” Transfer provides a window through which to better understand the mechanistic links between speech perception and speech production. This is important because it provides a foundation for broader impacts across a variety of domains, such as: 1) developing new technologies for effective education approaches in classrooms with language diversity; 2) assisting individuals with communication challenges like stuttering; 3) providing constraints to refine artificial intelligence and machine speech recognition systems; and 4) engineering brain prostheses that can restore speech lost to stroke.<br/><br/>The current project focuses on understanding how the brain links what we hear with how we speak, even when changes are subtle and not consciously identifiable. The investigators take a novel approach to the study of transfer by manipulating the make-up of the speech stream in subtle but systematic ways. This method allows clear predictions about the expected changes to production if it is implicitly influenced by perception. Preliminary data suggest this is the case: creating a subtle accent by manipulating the pitch of a voice induces robust changes in how speakers produce the very same dimension. This influence goes beyond immediate imitation of heard speech (for example, repeating a just-heard word). Rather, it implies a more fundamental change to the speaker’s production system that likely involves a form of incremental learning through small, but persistent, neuroplastic changes to the brain. The current project will systematically investigate when and how transfer occurs in order to reveal the responsible mechanisms.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.