Joseph S. Perkell - US grants

This proposal aims to characterize speech changes in patients who lose hearing as a result of bilateral acoustic neuromas (Neurofibromatosis 2). We will monitor patients who have already lost hearing in one ear from this disorder, and are at risk of losing hearing in the other ear. Because rapid hearing loss in the functioning ear will occur in some of these patients, their speech can be studied before and after deafening, providing a unique opportunity to detect subtle and progressive changes to speech parameters following deafening. While gross speech impairments do not commonly result from loss of hearing in adulthood, there are numerous subtle changes. The careful study of the speech parameters that do and do not deteriorate, and of the different time courses that deteriorating speech parameters follow, should illuminate the role of audition in underlying control mechanisms in speech. With a carefully selected population of patients, referred to us from the Neurogenetics Unit of N.I.N.D.S., Epidemiology Branch, and the Neurofibromatosis Clinic at the Massachusetts General Hospital, we will initially make acoustic and physiological recordings of speech production and test speech perception to establish baseline values. Those values will be used to assess changes in speech parameters in a subset of these patients whose hearing in the functioning ear begins to decline rapidly. On this subset of patients, we will perform repeated assessments including perceptual testing and acoustic and physiological recordings of their speech. We will seek to relate the measures of deteriorating perceptual function to the profile of degeneration of speech parameters. This work will be guided by experimental hypotheses concerning: timing of changes in speech parameter values and reliance on auditory feedback; relations among changing speech parameters; and relations between perception and production. To explore these hypotheses, we will measure within subjects, longitudinally: 1) acoustic characteristics of the production of speech segments (temporal and spectral variables) and prosody (fundamental frequency, intensity and temporal variables); 2) underlying physiological variables (parameters of laryngeal function, respiration and airflow, and coordination of oral, nasal and laryngeal articulations); and 3) speech reception (tests of phoneme, word, and sentence perception). The in-depth acoustic and physiologic analysis of the speech in combination with measures of the perception of this highly unusual group of subjects, from whom we can uniquely record pre-deafening measures of speech production, should provide otherwise unobtainable information on the role of auditory feedback in the control of speech production. From a clinical perspective, the results should also be useful in helping to understand and manage the communication problems encountered by these patients.

The long-range goal of this research is to contribute to an understanding of the motor control of speech production, including its variability, efficiency and communicative effectiveness. We will build on our initial findings of acoustic goals, clarity versus economy of effort, kinetic performance limits and vocal-tract modeling, and on the work of new collaborators. We will refine a theoretical overview with experiments based on the following hypotheses: (1) Segmental control mechanisms for the production of phonemic contrasts are based on acoustical goals. We will examine motor equivalence in reaching those goals. (2) The acoustic goals are determined partly by saturation effects (non-linear quantal relations) between motor commands and articulatory movements and between articulation and sound. We will study saturation effects in a bite block experiment. (3) To achieve the goals, speech movements are planned so that perceptual contrast is achieved with minimal effort. We will examine measures of speech clarity and articulatory effort under varying speech conditions. (4) Programming to achieve acoustic goals must utilize an internal model of the relation between motor commands and acoustic results. We will examine properties of that internal model with a speech sensorimotor adaptation paradigm. (5) The relation between motor commands and acoustic results includes biomechanical constrains, which we study with physiological vocal-tract models. This work will include: (a) further development and use of speaker-specific physiological/biomechanical models of the supra-glottal vocal tract, (b) collection of detailed acoustic, movement, force, EMG, MRI and aerodynamic data from several speakers to provide bases for model development and validation and for quantitative exploration of biomechanical saturation effects, and (c) further development of control models and their use in driving the biomechanical models in tests of hypotheses about acoustic goals, saturation effects, economy of effort and dynamical constraints. (6) The nature of relations between perception and production and their degree of refinement of the internal model may differ across speakers. We measure speakers' acuity in speech discrimination and relate those measures to differences among them in speech kinematics and acoustics. In these studies we will record and analyze the acoustic signals and movements of points on the mandible, lips and tongue using our EMMA movement transducer system; we also gather anatomical data with MRI and we make measurements of EMG, contact pressures, air pressure and air flow. We also conduct several kinds of perceptual experiments and compare the results with those from the production experiments.

This three-year award for U.S.-France cooperative research on speech production involves the research group at the Massachusetts Institute of Technology (MIT) led by Joseph Perkell and Pascal Perrier's laboratory at the Institut National Polytechnique de Grenoble, Grenoble, France. The objective of the research is to investigate variations in speech that depend on speaking style and phonological input within the same speaker. Articulatory, kinematic and acoustic data will be collected on simple utterances at MIT. The French laboratory will use these data to guide control of a simulation of the speech production apparatus. The project takes advantage of a 2-D biomechanical/physiological model tongue developed by the French investigators. It will advance fundamental understanding of speech production.

[unreadable] DESCRIPTION (provided by applicant): The goals of this application for continuation funding are: to deepen significantly our understanding of how a speaker's auditory acuity influences his or her speech motor planning; to describe the speech perception and production of hearing-impaired adults and the effects of cochlear prostheses; and to evaluate and help refine a quantitative model of the role of hearing in speech. We pursue these goals by conducting experiments with normal-hearing speakers, and with postlingually deafened adults who receive cochlear implants. Our experiments measure the effects of the implants on speech in recordings made before implantation and up to 2 years after, as speakers' auditory acuity evolves. According to our model of the role of hearing in adult speech motor control, many of the goals of speech movements are in the auditory domain. Consequently, a central theme of this research is the role of auditory perception in the feedback and feedforward control systems that are used to achieve auditory goals during speech. Feedforward control is almost entirely responsible for generating articulatory movements in adults. However, when there is a mismatch between the speaker's intention and the resulting auditory feedback during production of a speech sound, that error leads to corrective motor commands that serve to update feedforward commands for subsequent movements. The speaker's ability to detect such a mismatch depends on his or her auditory acuity. The proposed research inquires into the role of auditory acuity when producing phoneme and lexical stress contrasts; when compensating for feedback perturbation of vowel formants or for mechanical perturbation of sibilant spectra; and when imitating synthesized vowels. To measure acuity, we present synthetic speech continuation for discrimination testing. To assess relations with acuity, we measure the degree of separation of contrastive phonemes and lexical stress; dispersion of productions around their phoneme means; compensation for formant shift and for mechanical perturbation of sibilant spectra; and imitation accuracy. In most of these experiments, we also block auditory feedback temporarily in order to reveal the state of feedforward commands. Analyses take demographic variables such as age at hearing loss and duration of implant use into account. Relevance to public health: Hearing oneself speak plays a 2-fold role in maintaining good speech-it may correct the pronunciation of speech sounds even as they are occurring and it can revise plans for producing the sound next time. When a speaker is seriously hearing-impaired, those 2 mechanisms do not function well and speech can deteriorate over time. This project studies the idea that the people most affected are those who cannot distinguish the different speech sounds well. The results can serve to stimulate research on new therapies in which certain speech disorders, including those of speech development, are alleviated by training clients to hear differences better among the sounds that they produce. [unreadable] [unreadable] [unreadable]

The National Science Foundation will provide partial support for a conference, "From Sound to Sense: Fifty+ Years of Discoveries in Speech Communication," (http://rledev.mit.edu/soundtosense/default.cfm), held on June 11-13, 2004, at the Massachusetts Institute of Technology in Cambridge, MA. The conference has two goals. The first is to survey the current state of the field, by providing an historical overview of what has been accomplished during the last five decades of speech research, and by sampling some of the most interesting recent work in this active and exciting research topic. Twenty-one invited speakers will present talks in six different areas: Phonology and Phonetics; Speech Acoustics; Speech Perception; Planning and Production; Development, Pathologies and Remediation; and Speech Technology. The second goal is to promote cross-fertilization among the subdisciplines, to benefit future work in this field of study, which has its foundations in the physical, engineering, linguistic, cognitive and medical sciences. To this end the conference is organized in single sessions, rather than multiple parallel sessions, so that everyone will hear all presentations and participate in the discussions together. Many of the presentations will be archived as written papers at an MIT web site and distributed as a CD, and all of the sessions will be videotaped (if funds permit), offering a rich resource for both students and established investigators.

Many of the historical overviews will be presented by senior researchers such as Gunnar Fant, Kenneth Stevens, Peter Ladefoged, John Ohala, David Pisoni, Bjorn Lindblom, Victor Zue and Ray Kent, who have done pioneering work in their subdisciplines, including advances such as
-the source-filter theory of speech acoustics, which models the way that changes in the size and shape of the vocal tract produce the sounds of a spoken utterance,
-the quantal theory of acoustic-articulatory mapping, which identifies the acoustically stable articulations that provide some of the bases for sound categories in languages of the world,
-models of human speech perception, production, development and pathologies, which relate models of the acoustics and articulation of speech to human language processing, and
-algorithms for the automatic recognition and synthesis of speech, which not only provide aids for the handicapped, but also serve as tools for testing our models of human speech processing.
The invited speakers who were responsible for these advances bring the valuable perspective of decades of thinking about and investigating speech to their historical overviews. The meeting thus provides an opportunity for students to hear first hand the insights of researchers who up to now may have been only names on classic papers, and to have personal contact with them.

Other speakers represent leading-edge current work, addressed to a broad range of questions. These include how individual languages make different use the special articulatory-acoustic mapping properties of the vocal tract, how the brain operates to process speech both as input and as output, and how automatic recognition and synthesis can take advantage of our increasing understanding of how speech is processed by humans, to create better tools for human-computer and human-human interactions by spoken language. A final summarizing address, followed by a discussion forum, will synthesize the ideas presented at the conference, with a focus on how future speech research can benefit from interaction among the represented sub-specialties.

DESCRIPTION (provided by applicant): A meeting, "From Sound to Sense: Fifty+ Years of Discoveries in Speech Communication," will be held on June 11-13, 2004 at MIT in Cambridge, MA. Significant research over the past five decades will be reviewed by investigators who have been major contributors over this period, much of it with support from NIH. Other invited speakers will present a sampling of current work in several areas, in ways that speak to scientists whose specialties may be outside of these areas. Poster presenters will be encouraged to follow the same theme. Participants include 20 invited speakers who are leaders in the field, up to 180 presenters at three poster sessions, and a broad sample of the speech research community as attendees -- with encouragement of student participation. Topics to be covered include Phonology and Phonetics; Speech Acoustics; Speech Perception; Planning and Production; Development, Pathologies, and Remediation; and Speech Technology. A final summarizing address and forum will synthesize the ideas presented at the conference, with a focus on how future research may benefit from cross-fertilization among the represented sub-specialties. Partial support for this conference is requested.

This project aims to systematically investigate relationships between structural anomalies in brains of persons who stutter (PWS) and functional properties of the speech motor system. The project's approach builds on recent important findings regarding white matter (WM) irregularities and other neuroanatomical anomalies in PWS by attempting to identify behavioral (kinematic and acoustic) correlates of these anomalies during speech and other auditory-motor tasks, and by providing mechanistic interpretations of these findings within an established computational modeling framework of the neural bases of speech. The project is comprised of two inter-related experimental investigations that utilize structural neuroimaging in combination with behavioral measures to investigate theoretically motivated hypotheses concerning possible neural deficits underlying stuttering. The neuroimaging involves the use of magnetic resonance imaging and diffusion tensor imaging in PWS and matched fluent controls to obtain subject-specific measures of a set of candidate neuroanatomical markers (quantitative measures of structural properties of white and gray matter) derived from prior findings. The first investigation studies correlations between the neuroanatomical markers and behavioral measures reflecting feedforward motor control: kinematic and reaction-time (RT) data collected from the subject groups while they perform carefully designed, simple RT and speech tasks. The second investigation studies correlations between the neuroanatomical markers and the use of auditory feedback in speech: data collected on the subjects'responses to real-time spectral and temporal perturbations in the auditory feedback of their own speech. Both investigations will also test for differences in the measures between the groups of PWS and controls, and will consider alternative hypotheses. Establishing relations between behavioral and neuroanatomical anomalies in stuttering should provide new insights into important structural substrates that could lead to or potentiate the expression of underlying neurofunctional deficits. Different combinations of neuro-structural and neuro-functional factors could be manifested in a range of influences on stuttering, from emotions to malfunctions of movement triggering and timing.

For normal adult talkers, articulating speech feels effortless and requires little, if any, conscious thought. However, this subjective impression belies the complexity of the neural processes underlying speech production. During speech, the articulators, including the tongue, the lips and the jaw, are moved in highly complex ways to achieve the rapid altering sequences of desired speech sounds. Our current knowledge of the brain mechanisms for the control of speech articulation remains primitive. This project aims to advance this knowledge by focusing on an important component of the speech system: auditory feedback. Auditory feedback refers to the sound of speech heard by the talker him/herself when speaking. It is has been shown to participate in correcting articulatory deviations online (with very short delays) during the production of single phonemes. However, it remains unclear how auditory feedback is used by the speech motor system to control the production of multisyllabic connected speech, the most frequently occurring type of speech in real-life communication. This is the main issue that will be addressed by this project. A computer setup based on fast digital audio processing will be used to introduce subtle perturbations of the auditory feedback of specific acoustic properties of vowel sequences when the subject utters a simple sentence. Two types of perturbations will be used to investigate separately the spatial (frequency-related, or 'spectral') and temporal aspects of feedback-based articulatory control. The compensatory changes in the subjects' articulation will be measured indirectly by extracting changes in acoustic parameters from the audio recording of the subject's speech. In addition to characterizing the role of auditory feedback in the online control of the spatiotemporal trajectories of articulation, this project will also use functional MRI to map out the brain regions involved in this feedback control process. The knowledge gained from this project will expand our knowledge of the normal process of speech production, and will also provide clues for addressing questions in future research about the mechanisms of disorders of speech production, such as stuttering.