Brian MacWhinney - US grants

Thirteen experiments are proposed to examine the production and comprehension of sentences by child and adult speakers of English, French, German, Hungarian, Italian, and Serbo-Croatian. The principle focus of the experiments is on the processing of devices involved with subjectivalization and reference-identification. The experiments seek to test the resilience of a plurifunctional, parallel-activation model of sentence processing called the "competition model." This model is presented as a performance model and not a competence model. As a model of adult processing, it seeks to account for performance in terms of relative cue weight. As a developmental model, it seeks to explain the order of acquisition of particular uses of grammatical devices in terms of the relative "cue validity" of those devices. The experiments include eight studies of sentence comprehension and five of sentence production. The studies of comprehension examine: 1) the impact of ungrammaticality on sentence interpretation, 2) the competition of case-marking with agreement marking in Serbo-Croatian and Hungarian, 3) the processing of multiple agreement cues in Hungarian, 4) the competition of clitic marking with agreement marking in French and Italian, 5) the processing of relative clauses in word-order varying languages, 6) late acquisitions of grammatical devices in English and Italian, 7) the interpretation of full and clitic pronouns in Italian, and 8) the acquisition of switch-reference marking in Hungarian. The studies of production examine: 1) on-going description of a film with a story-line, 2) use of lexical and syntactic primes and topical probes, 3) the impact of delay on retelling, and 4) developmental patterns in the control of discourse-based devices.

Sixteen experiments are proposed to examine the on-line processing of sentences by child and adult speakers of English, German, Hungarian, and Italian. The experiments focus on the processing of devies involved with role identification and anaphoric reference, seeking to test the resilience of a plurifunctional parallel-activation model of sentence processing called the Competition Model. This model is presented as a performance model, not a competence model. As a model of adult procesing, it seeks to acount for performance in terms of relative cue weight. Cue weight, in turn, is predicted to be a function of the objective validity of cues in the linguistic environment. Development is viewed as a process of acquiring a system of cue weights that correctly reflects various aspects of cue validity. The specific studies being proposed fall into five classes: 1. the collection of corpora of adult-adult and adult-child discourse in the four languages and the analysis of these corpora in order to compute cue validity values, 2. the examination of the on-line assignment of the subject role using auditory presentation of digitized speech with pictorial probes; 3. monitoring for the occurrence of errors in gender agreement, subject-verb agreement, word order, and case-marking; 4. monitoring for the occurrence of words whose shape is predicted by various morphosyntactic devices; and 5. recognition of a pictorial probe during the process of real time integration of anaphoric reference. These studies are the first extensive investigation of on-line sentence processing in children and the first application of on-line techniques to cross-linguistic research.

Summary The Child Language Data Exchange System (CHILDES) Project seeks to broaden and deepen our scientific understanding of language development by providing new ways of analyzing real world face-to-face interactions. The computational tools that had been developed in the previous phases of the project constitute the primary methodological basis for new empirical research on the development of spontaneous use of a first language. This work has resulted in over 8000 published articles examining all aspects of language development, including word learning, sound learning, grammatical development, and communicative development. All of the programs and data sets are provided over the web without charge to researchers. The database that has been collected using these tools is now the largest spoken language database available anywhere. However, we can achieve still greater efficiency and analytic precision by building even more powerful computational tools. The next phase of this project will develop new techniques to support analytic methods in the study of language development. These methods include rapid computer-assisted transcription of interactions, diarization of daylong audio recordings made in the home, automatic analysis of morphological and syntactic structures, a simple user interface for searches, web-based support for collaborative commentary between research groups, construction of standard comparison group norms, and methods for moving data between different programs for alternative analyses. In addition, we will promote the use of the database and programs by constructing web-based tutorials, by improving the current user interface, and by conducting workshops and presentations at conferences.

9451359 MacWhinney The laboratory microcomputer has become a crucial tool for conducting experiments in psychology. This project therefore supports the deployment of a computer program, called PsyScope, that will remove this technological barrier. A national prototype is being developed for the use of this program as the backbone for the teaching of Research Methodology in Psychology. Construction of this prototype involves completion of (1) an introductory manual, (2) a series of standard experiment types, and (3) the building of tools for linking experimental design to statistical analysis programs. Working through a consortium of universities and colleges, we plan to disseminate this prototype throughout the nation. It is also expected that successful deployment of this system nationwide will lead to a major improvement in the research training of the future generation of experimental psychologists.

The goal of this project is to deepen our understanding of the impact of early left focal lesions on linguistic information-processing. Using on- line experimental methods, this project examines the extent to which comprehension and production in real time are differentially impaired for children with different sites and types of early left focal lesions. The proposed studies investigate the effects of early focal lesions on (1) on- line processing during both comprehension and production, (2) processing differences across specific linguistic domains, and (3) processing profiles for individual subjects. Previous research on this topic has several limitations: (l) it has focused primarily upon production in contrast to comprehension, (2) it has made little use of experimental procedures, focusing instead on parental reports and spontaneous speech samples, (3) it has made no use of on-line methodology to study the processing of language in real time, and (4) it has failed to distinguish subjects fully according to both site and type of lesion. The present study focuses on early left focal lesions secondary to either cerebral infarct (CI) or periventricular hemorrhage (PVH) in 40 children. Magnetic resonance imaging (MRI) scans will be obtained on all subjects and analyzed using a standard procedure. The theoretical model used to guide the research is provided by the Competition Model -- a general model of linguistic information-processing, that has been used extensively in a variety of languages with both normal subjects and speakers with language impairments. Using the PsyScope experimental system, the current study measures real-time simultaneous processing and integration of verbal information across different linguistic domains. These tasks have been used extensively with both normals and adult aphasics. Recently, they have been piloted with normal children and children with language impairments. These on-line experimental techniques allow us to: (l) characterize the nature of performance deficits as they relate to site (anterior/posterior), and type of lesions (CI vs. PVH) for children with early focal brain lesions; (2) investigate the relationship between the comprehension and production deficits; (3) determine at what level the processing skills are impaired (e.g. lexical, sentence, or discourse) and; (4) determine the extent to which individual profiling techniques better characterize the nature of the deficits. Tasks being examined include base reaction time, signal detection, serial recall, auditory memory, rapid syllable production, picture naming, lexical decision, picture choice, constrained sentence production, error detection, cross-modal probe recognition, on-line sentence interpretation, constrained story production, and on-line story interpretation. Data gathered in these studies will be used to construct maximum likelihood models for individual subjects which then will be compared to control models in terms of theoretically defined mathematical "lesions" for particular parameters in the model.

The goal of this program is the successful training of the next generation of cognitive psychologists with a specific emphasis on basic processes and individual variation in cognition. NIH has underscored the importance of applying behavioral research to the study of a wide variety of health problems from autism to addiction and aggressive behavior. The training proposed here addresses that need by focusing on individual differences in cognition and learning. The application of cognitive psychology to these issues emphasizes the role of individual differences in cognition, training, and motivation as they interact with social and educational structures. In order to deal effectively with this wide range of applications, the next generation of cognitive psychologists needs to master a core set of research methodologies and theoretical approaches. This proposal focuses on three specific pathways for training cognitive psychologists interested in the study of individual differences in cognition. The first pathway introduces the trainee to the methods and theories of cognitive neuroscience. Methods here include imaging techniques (FMRI, ERP, PET), psychometric evaluation, lesion studies, and work with specific clinical populations. The second pathway exposes the trainee to the methods and theories of cognitive development. Methods here include microgenetic analysis, gesture-speech studies, naturalistic observation, modeling, and strategy choice analysis. The third pathway emphasizes understanding of variation in normal adult cognitive functioning. Methods here include protocol analysis, implicit learning paradigms, studies of working memory, and investigations of problem-solving. Each trainee will be committed to exploring in detail one of these three pathways. However, all trainees will also learn something about methods and theories in all three pathways. In this way, trainees will be given the tools needed to produce high quality research on the roots of individual variation in human cognition.

The goal of this project is the development of computational tools to facilitate the linguistic analysis of transcript data in the CHILDES (Child Language Data Exchange System) database. The three areas being targeted are lexicon, morphosyntax, and phonology. The goal of the lexical initiative is the construction of a keyed relational database that will serve as a dynamic lexical frequency dictionary for each of the major languages in the database. This database will be available to researchers in electronic form. A canonical form of the dictionary will also be published in book format. A series of demonstration studies based on the use of this database will be completed to illustrate to researchers how a lexical database can serve as the foundation for modern crosslinguistic studies of lexical development. The goal of the morphosyntactic initiative is the construction of tools to permit automatic tagging and parsing of the corpora in the CHILDES database. Using these parsed and tagged structures, additional programs will be designed to compute morpheme usage and omission, phrasal structure, clause structures, and sentence structure. These measures can be used to compare across languages, age levels, and clinical subtypes. The goal of the phonological initiative is the construction of a set of tools for developing and analyzing phonological databases. Although these data will initially be represented in standard CHILDES file format, they will also be converted into a relational database that will facilitate a variety of analyses of phonological processes. The completion of this work will markedly improve the usability of the CHILDES database for more advanced forms of linguistic analysis.

We will provide instruments in a new 3600-square-foot teaching laboratory located in the Department of Psychology at Carnegie Mellon University, in order to develop state-of-the-art facilities for the teaching of research methods in psychology. The educational process will occur both in formal classes, largely conducted in a new classroom facility, and in individual student projects, conducted in independent-study rooms within the same area. Our model is for students to take courses in the teaching laboratory that will prepare them for subsequent independent work. Instrumentation will comprise both software and hardware. A principal goal of the planned curriculum development is to enhance the department's research methods courses, which are a cornerstone of our undergraduate program. We currently teach research methods in cognitive psychology, developmental psychology, and social health psychology. New instrumentation will greatly improve those courses and allow us to add a methods course in cognitive neuroscience. Other curricular developments will occur in courses utilizing brain images, courses teaching computational methods, and faculty-supervised independent research projects. Course materials and tools that are developed will be disseminated over the internet, as a model for the application of particular tools to teaching of research methods. A Web site will be created to publicize results from classroom curricular development and, when appropriate, tools developed in student projects. The site will provide information about each methods course, including the syllabus, assignments that use the instrumentation (software and hardware), and faculty-developed tutorials for using the instruments. Local and national outcomes will be assessed.

In order to learn what it means to conduct empirical research in Psychology, students need to design and analyze their own experiments. However, students typically have few real interactions with the use of the microcomputer as a productive laboratory tool. The problem has been the lack of software that allows undergraduates to build experiments on their own. The shipping of the first version of the E-Prime system during the summer of 1999 opens up a fundamentally new way of addressing this problem. E-Prime is a robust, commercial experiment generation system. Because E-Prime is becoming recognized as the standard for building experiments in Psychology, it is now possible to construct a web-based resource that uses E-Prime as the delivery engine for a wide variety of instructional materials. We will call this system STEP (System for the Teaching of Experimental Psychology). There are seven specific goals in this project.
1. Building a package of classic experiments appropriate for introductory psychology classes.
2. Building 100 additional experiments as examples for the advanced undergraduate.
3. Building computer "wizards" and QuickTime videos to teach experiment creation.
4. Building tutorial materials to show students how to generate 30 classic experiments.
5. Writing instructor manuals with HTML links between sample E-Prime experiments and specific topics covered in current textbooks on experimental design and experimental psychology.
6. Working with the E-Prime development team to include new features that are important for the teaching of experimental psychology. In particular, these will include ability to display and control multimedia, administer surveys, and record ongoing observational behaviors.
7. Creating the social structures needed to support STEP. The project will be evaluated using a pretest-posttest design, item analysis, objective usage figures, and usability studies.

***
9978056


The goal of TalkBank is the creation of a distributed, web-based data
archiving system for transcribed video and audio data on communicative
interactions. These interactions will include mothers talking with their
children, family dinner table talk, classroom interactions, animal cries,
signed language, formal debates, phone calls, talk with foreigners, club
meetings, and dozens of other types of communicative interactions. The data
will come from speakers of many languages, professions, and ages. Some
speakers will have language disabilities and some will be language learners.
The formal specification for data in TalkBank will use an a system of
annotationa graphs called Codon. Tools will be created for the entry of new
and existing data into the Codon format; transcriptions will be linked to
speech and video; and there will be extensive support for collaborative
commentary from competing perspectives.

Researchers will be able to locate a particular segment of an interaction and
immediately play back that section on their computer monitors. TalkBank will
facilitate comparisons across social groups, languages, and situations. It
will also provide tools for working in detail on single populations, as well
as collaborative commentaries that test competing interpretations against a
constant set of data.

The initiative establishes an ongoing interaction between computer
scientists, linguists, psychologists, sociologists, political scientists,
criminologists, educators, ethologists, cinematographers, psychiatrists, and
anthropologists.

The goal of TalkBank is the creation of a distributed, web-based, data
archiving system for transcribed video and audio data on communicative
interactions. These interactions will include mothers talking with their
children, family dinner table talk, classroom interactions, animal cries,
signed language, formal debates, phone calls, talk with foreigners, club
meetings, and dozens of other types of communicative interactions. The data
will come from speakers of many languages, professions, and ages. Some
speakers will have language disabilities and some will be language learners.
The formal specification for data in TalkBank will use a system called Codon.
Tools will be created for the entry of new and existing data into the Codon
format; transcriptions will be linked to speech and video; and there will be
extensive support for collaborative commentary from competing perspectives.
Researchers will be able to locate a particular segment of an interaction and
immediately play that section back on their computer monitors. TalkBank will
facilitate comparisons across social groups, languages, and situations. It
will also provide tools for working in detail on single populations, as well
as collaborative commentaries that test competing interpretations against a
constant set of data.

The initiative establishes an ongoing interaction between computer
scientists, linguists, psychologists, sociologists, political scientists,
criminologists, educators, ethologists, cinematographers, psychiatrists, and
anthropologists.

The proposed effort will build upon prior work by participating institutions in audio and video capture, analysis and collaboration, allowing each institution to make progress more quickly by leveraging each other's respective strengths, capabilities and prior work. The project brings together NSF-funded efforts involving multimedia data in the human sciences: the DIVER project housed at Stanford University, and the TalkBank Project, housed at CMU and the University of Pennsylvania, as well as NSF Middleware and Grid projects. The proposed project will adopt a two-track research approach to designing and implementing the Collaboratory. The first project track is to develop a virtual video repository and video analysis community portal. The second track will produce an open community toolkit that greatly expands generality and capabilities for video analysis, video input and output. In addition to these goals of moving from local to distributed users, tools and data will evolve in an ongoing process of user testing, piloting, and software packaging, involving the community and refining the tools based on their experiences. Project results at each interim milestone will be disseminated broadly for enhancing scientific and technological understanding, especially to conferences and journals and complemented by an open website for sharing ongoing research, technical developments, technical reports and software releases to be downloaded to user communities. Video Collaboratory toolkits will be made available to participating institutions with efforts made to spur adoption that will facilitate multiple communities using collaborative commentary around video.

The initial focus of our project is the human sciences, with the aim of leveraging video records to enhance the learning sciences, education, teaching and training. We also expect that video application and infrastructure work can provide significant support to the physical sciences, since video is an important data type for scientific visualization and primary empirical data, and several Grid initiatives use video databases integrally. We anticipate producing open-source video software that can be utilized in the current Grid and Middleware platform initiatives. This could enable a new primary software layer, a set of standards for video analysis and collaboration middleware, and the potential for a substantial broader impact as an outcome of this work.

This project will create a digital audio archive that will enable
scientists in several fields to approach novel research issues in speech and language
studies, issues in group decision-making, and issues at the leading edge of
human communication scholarship. The Supreme Court of the United States
(SCOTUS) has been recording its public proceedings since 3 October 1955.
These recordings - now in the National Archives - span nearly five decades
and consist principally of oral arguments in which justices and attorneys
engage in various forms of persuasion and communication between bench and
bar and, obliquely, among the justices themselves. The arguments have been
transcribed professionally across the entire period, creating a matchless
collection of audio materials coupled with highly accurate transcripts. The
audio - along with other activities captured on audio such as the
announcement of opinions - offers a unique opportunity for researchers
across a wide spectrum of disciplines to engage in novel and transforming
research projects that were once thought beyond the reach of investigators.

The chief result of this work will be a complete and continuing archive of
more than six thousand hours of SCOTUS audio. It will provide synchronized
(i.e., time-coded) transcripts of the collection, identify and tag
individual speakers, build new mark-up tools for these new domains, and
share the corpus with researchers and faculty. The result of this
interaction among political scientists, legal scholars, linguists, and
computer scientists will yield: new knowledge in the modeling of
multi-party discussions with complex goals, novel strategies in small group decision
process analysis, and path-breaking approaches to extended collaborative
commentary addressing the dynamics of human communication.

The SCOTUS archive will be maintained as a shared public resource to
enhance study and understanding of the Supreme Court of the United States. It will
be available to anyone with World Wide Web access. Based on past
experience, principal audiences include: researchers across diverse domains, teachers
and students, lawyers and litigants, and the visually- and
hearing-impaired.

Today, more than a million unique users access selected SCOTUS materials
each month. With a complete and updated SCOTUS archive and improved ability
to query and search, the number of users should expand substantially.

By exploiting common interest and beneficial interactions among diverse
research communities, this project will create a vast collection of digital
objects. Working with partners experienced in data-sharing, the effort aims
at revolutionizing the ability to collaborate with physically distributed
teams of researchers and their students.

DESCRIPTION (provided by applicant): The goal of this program is the training of students who can use methods from neuroscience and experimental psychology to improve our understanding of mental function and dysfunction. NIMH has underscored the importance of applying behavioral research to the study of mental disorders. In order to deal effectively with this wide range of applications, the next generation of experimental psychologists needs to master a core set of research methodologies and theoretical approaches that can be applied to the study of mental disorder. This proposal focuses on four specific pathways for training psychologists in the study of mental disorders. The first pathway introduces the trainee to the methods and theories of cognitive neuroscience with specific applications to mental disorder. Methods here include imaging techniques (MRI, ERP, PET), psychometric evaluation, lesion studies, and work with specific clinical populations. The second pathway exposes the trainee to the methods and theories of cognitive, emotional, and social development, as they illuminate childhood mental disorders. Methods here include micro-genetic analysis, gesture-speech studies, naturalistic observation, modeling, and strategy choice analysis. The third pathway examines decision-making and risk-taking in social contexts, as they are influenced by mental disorder. The fourth pathway emphasizes the extension of the methods of adult experimental psychology to the study of adult mental dysfunction. Methods here include protocol analysis, implicit learning paradigms, studies of working memory, and investigations of problem-solving. Each trainee will be committed to exploring in detail one of these four pathways. In addition, each trainee will be engaged in an ongoing series of hands-on studies of particular issues in mental dysfunction.

This interdisciplinary effort develops, integrates, and refines a suite of activity and behavioral observation tools supporting the automatic collection, annotation, access, analysis, and archiving of behavioral data for individuals and groups. These tools capture a continuous audiovisual record of human activity in various settings and apply machine intelligence technology to automatically process that record for efficient use by analytical observers to monitor situational behavior over time. The annotated record provides a level of completeness not feasible with human observers, allowing, for the first time, large-scale longitudinal behavioral and clinical research based on continuously captured and processed data, enabled through extensible interfaces accessing such voluminous records in a user-friendly, yet utilitarian manner. The record will be processed with data reduction and extraction technologies that recognize faces and speech, track moving individuals, and identify social interactions, while protecting the confidentiality of the participant's identity in collaboratively accessible, computerized databases.

The tools focus on the automatic identification of features within the audiovisual record to improve the accuracy and completeness of manual, labor-intensive rating instruments. Through computer vision, speech recognition, sensor integration, and machine learning, multimedia data extraction technologies will be developed for individual behavioral measurements. Additional tools will be developed to mine the resulting annotated longitudinal datasets for insights into individual interactions and reactions. The relevance of the tools will be demonstrated, refined, and validated through an initial challenge application and environment: the elderly residents in a continuing care retirement community. Collaborating studies of parent-child, teacher-student and autistic children social interactions will validate portability and utility across domains of behavioral research. The tools will be extensible, enabling behavioral scientists to better accommodate novel situations and source material. Specifically, the scientist will be able to identify a need for a class of audiovisual detection, adeptly supply training material for that class, and iteratively evaluate and improve the resulting automatic classification produced via machine learning and rule-based techniques.

The project's broader impacts include supplying social and behavioral scientists with automated tools supporting novel approaches to creating and analyzing data in their endeavors to characterize human behavior. Research by sociologists, psychologists, anthropologists, and medical clinical investigators will be enabled in new modes with greater accuracy and precision of observation than has heretofore been possible or practical. Ethical issues are explicitly and proactively addressed to engage the larger social concerns.

The majority of research on human perception and action tends to treat these as separate functions. What is less often considered in these research domains is that humans interact with a perceived world in which they themselves are part of the perceptual representation. Evidence has been mounting to show that self-representation is fundamental to both executing and understanding spatially directed action. It has been theorized to play a role in reaching and grasping, locomotion and navigation, infant imitation, spatial and social perspective taking, and neurological dysfunctions as diverse as phantom limb pain and autism. Behavioral research has revealed a number of tantalizing outcomes that point to a role for the representation of the body in basic human function; neuroscientists have identified multiple sensorimotor maps of the body within the cortex and specific brain areas devoted to the representation of space and place; and developmental researchers have identified neonatal behaviors indicating a representation of self and have traced the course of spatially oriented action across the early years. What is needed is a shared effort to merge perspectives of behavioral science, neuroscience, and developmental psychology in order to further our understanding of self-representation. With support from the National Science Foundation, the 2006 Carnegie Symposium will provide a forum by which researchers from these various perspectives can come together to share their findings, ideas, aspirations, and concerns.

DESCRIPTION (provided by applicant): The study of phonological development has important implications for the diagnosis, understanding, and treatment of developmental language disorders. It also has implications for the understanding of language patterns in stuttering, disfluency, aphasia, bilingualism, second language learning, and dementia. Recent computational advances now make it possible for researchers to link high quality digital recordings to phonological and phonetic transcriptions. Using standards such as Unicode, IPA, and XML, the CHILDES database project now provides universal Internet access to large corpora of transcripts linked to audio for students of both first and second language acquisition, along with a wide array of tools for lexical, syntactic, and discourse analysis. However, the CHILDES Project has not provided effective tools for phonological and phonetic analysis. PhonBank seeks to bridge this gap by providing a new database on phonological development with transcripts linked directly to audio records. It also provides a program that automates creation and analysis of these new corpora. The construction of this database is being be supported by a group of 60 researchers and their students who have agreed to contribute already collected and transcribed corpora from children learning 17 different languages. Subjects include bilingual children, normally-developing monolinguals, and children with language disorders. The data are being structured to facilitate testing of models regarding babbling universals, variant paths in segmental and prosodic development, markedness effects, prosodic context effects, segmentation patterns, statistical learning, frequency effects, interlanguage transfer, diagnosis of disability, stuttering patterns, disfluency patterns, and the effects of morphology and syntax.

DESCRIPTION (provided by applicant): We propose the construction of a shared database of multimedia interactions for the study of communication in aphasia. The goal of this work is the important in patient-oriented treatment of aphasia. To reach that goal, we must solidify the empirical database supporting our understanding of communication in aphasia. Toward that end, we have organized a consortium that will develop a shared methodological and conceptual framework for the processes of recording, transcription, coding, analysis, and commentary. Our nine specific aims are: 1. Protocol standardization. We will develop a standardized data collection protocol that will be implemented at all consortium sites. 2. Database development. We will convert all transcriptions to the CHAT standard and link the transcripts to the digitized audio and video, placing emphasis on data from underrepresented participant groups. 3. Analysis customization. Using the current CLAN programs as a basis, we will construct a full set of tools for the analysis of multimedia transcripts on the levels of phonology, lexicon, morphology, syntax, discourse, and pragmatics. 4. Measure development. We will use the annotations produced by these tools to automatically compute measures that are currently being coded by hand. We will also develop new measures based on automatically constructed annotations. 5. Syndrome classification. Using these new measures and the growing database, we will work with consortium members to develop new approaches to syndrome-based patient classification and diagnosis. 6. Qualitative analysis. We will support qualitative analysis on three levels. First, the CLAN editor will support standard Conversation Analysis (CA) transcription. Second, we will formalize a set of coding systems specific to aphasic communication. Third, we will promote a system of web-based collaborative commentary. 7. Profiles of recovery processes. We will develop microgenetic methods such as time sequential analysis and growth curves to trace changes across time in both individual patients and patient groups. 8. Evaluation of Treatment Effects. We will develop methods that allow us to evaluate the effectiveness of specific aphasia rehabilitation treatments. 9. Johnny Appleseed. We will disseminate these new tools through personal contact, workshops, journal publications, and downloads over the Internet. We will place particular emphasis on dissemination of these tools to institutions serving minority populations.

EAGER: Workshop on Shared Infrastructure for the Social and Behavioral Sciences

Robert M. Groves
University of Michigan

SBE 0930160

Abstract


This project involves a workshop to be mounted in the spring of 2009 to address the shared infrastructure needs of the social and behavioral Sciences. The workshop addresses the fundamental question: What social and behavioral scientific questions of great import to human understanding cannot now be answered because researchers do not have the needed infrastructure to answer them? The infrastructure needs involve data needs and satisfactory methodological tools. The focus of the workshop centers on how human scientists can build the needed infrastructure.

Participants in this project are chosen to represent researchers at the cutting-edge of individual lines of inquiry. Viewpoints involve both new, bold initiatives, and key needed additions to well established thinking about current human science domains.

The product of this workshop is a listing of key questions that can be answered with new infrastructure, explicated in a language that can be understood across the SBE disciplines represented in the Directorate. The results are potentially transformative because the discussion has broad impact for researchers doing social and behavioral research across all subfields of the human sciences.

DESCRIPTION (provided by applicant): Aphasia Bank is a shared database of multimedia interactions for the study of communication in aphasia. The goal of this work is the improvement of patient- oriented treatment of aphasia. To reach that goal, we must solidify the empirical database supporting our understanding of communication in aphasia. Our seven specific aims are: 1. Core Protocol database development. We will continue to expand the core database to include additional participants, languages, bilingual types, and clinical types. 2. Additional database development. We will develop additional standards for creating databases that can include test data, imaging data, and data from participants with severe aphasia. 3. Automatic analysis. We will construct tools for automatic computation of scales sensitive to clinical diagnosis and the measurement of recovery processes. 4. Dissemination. We will disseminate the data, tools, and methods through personal contact, workshops, manuals, journal publications, and downloads over the Internet. We will construct materials for training and teaching. We will place particular emphasis on dissemination of these tools to institutions serving minority populations. 5. Cross-project linkage. We will link the English Aphasia Bank database to emerging databases in the Dementia Bank, TBI Bank, Fluency Bank, and AAC Bank projects. 6. Syndrome classification. Using these new measures and the growing database, we will work with consortium members to develop new approaches to syndrome-based patient classification and diagnosis. 7. Qualitative Analysis. We will develop methods for examining how people with aphasia achieve communication through gesture, scaffolding, and augmentative communication devices.

A child's language development in the first few years of life predicts long-term cognitive development, academic achievement, and expected income as an adult. Early language development in turn depends on linguistic interactions with adults. Increasingly, researchers are using daylong audio recordings to study child language development and child-caregiver interactions. Compared to short language samples, daylong recordings capture the full range of experiences a child has over the course of a day. Daylong audio recordings are also being used in applied settings. For example, studies show that by the time they enter First Grade, children from higher socioeconomic backgrounds hear tens of millions more words than children from lower socioeconomic backgrounds, perpetuating social inequalities. Multiple large-scale intervention projects targeting low socioeconomic households, including the Thirty Million Words Initiative in Chicago and the Providence Talks program, are using daylong audio recordings to provide automated, personalized feedback to parents on when and how often their child hears adult words and experiences conversational turns. The features of daylong recordings that are advantageous for researchers and practitioners also pose unique challenges. For one, their long durations are ideal for studying the temporal dynamics of child-adult interaction, but taking advantage of the long durations requires the enlistment of automated speech recognition technology. Current automatic speech recognition systems have difficulties with child speech and are challenged by the noisy and varied acoustic environments represented in the recordings. Another challenge is that the recordings capture private moments that require long hours of human listening to remove. This makes it difficult for researchers to share the recordings publicly, so that the potential value of the recordings collected by individual research labs is not fully realized.

This project will create a new resource, called HomeBank, that will have three key components: (1) a public dataset containing daylong audio recordings that have had private information removed by human listeners, (2) a larger dataset containing about ten to one hundred times as many hours of recording that have not had private information removed and will be free but restricted to those who have demonstrated training in human research ethics, and (3) an open-source repository of computer programs to automatically analyze the daylong audio recordings. HomeBank will take advantage of an existing cyberinfrastructure for sharing linguistic data called TalkBank. The daylong audio recordings included in the datasets will represent both typically developing and clinical groups, a range of ages from newborn infants to school age children, and a range of language and socioeconomic backgrounds. We expect the primary users to be basic and applied child development researchers as well as engineers developing automatic speech recognition technologies. The free-to-access database and the open source computer programs will ultimately improve both the data on which early interventions are based and the tools available for providing parents with feedback on the linguistic input they provide their children.

How children learn to produce speech smoothly without stops, restarts, errors, or omissions (known as speech fluency), has received little research attention. Some children who exhibit fluency difficulties, such as later-talking and bilingual children, are misdiagnosed with speech disorders such as stuttering as a result. However different fluency issues require different types of speech or language therapy. A better understanding of typical fluency development and of different types of fluency problems can inform the neural, linguistic and motor coordination demands that underlie speech fluency. The goal of the current project is to develop a comprehensive database archive of audio and video recordings of child speech and their corresponding transcripts that can be used for both basic and clinical research on the development of speech fluency in children.

This project examines fluency development during the preschool years in typical, late-talking, and bilingual children, by using existing data contributed to the CHILDES Project (www.childes.psy.cmu.edu), an open access archive of children's speech samples. Additional corpora from late-talking and bilingual children will also be included to establish a new, more comprehensive "Fluency Bank" archive. All corpora will be reformatted to validate and revise transcripts, index speech rate, characterize disfluencies, and code speech for grammatical features. Automatic and semi-automatic approaches will be used to link audio recordings and transcripts. The resulting Fluency Bank will become an important and unique research resource for those interested in fluency development in both typically and atypically developing children.

The goal of the FluencyBank project is the construction of a shared computerized database of multimedia interactions for the study of the development of speech fluency and disfluency through childhood. We will create a Gold Standard database collected from four groups of participants: Spanish-English bilinguals, late talkers, children who stutter, and normally-developing control children. Working with our Consortium members from 32 sites, we will extend this database with already available recordings and transcripts, as well as additional new data using the new core protocol. The Gold Standard data will be coded extensively for the training of computational linguistic methods for automatic and semi-automatic data analysis. Using these methods, we will create new measures of fluency that we can use to distinguish developmental patterns of recovery from stuttering. Where available, we will supplement this behavioral database with information from genetic and neuroimaging work. The results will be important for the development of the theory of typical and atypical patterns of fluency and disfluency. They will also allow us to better understand the course of fluency development, and to devise and evaluate methods for treating fluency disorders. The results will be important for both research and clinical practice. We will disseminate these data and methods to the wider community through conferences, networking, and technical support systems. All data and methods will be freely available to all interested researchers.

The study of phonological development has important implications for the diagnosis, understanding, and treatment of developmental language disorders. It also has implications for the understanding of language patterns in stuttering, disfluency, aphasia, bilingualism, second language learning, and dementia. Recent computational advances now make it possible for researchers to link high quality digital recordings to phonological and phonetic transcriptions. Using standards such as Unicode, IPA, and XML, and the infrastructure developed in the CHILDES Project, the PhonBank database project now provides universal Internet access to large corpora of transcripts linked to audio for the study of phonological developemnt. PhonBank also provides the Phon program that automates creation and analysis of these new corpora. The construction of this database is being be supported by a group of 60 researchers and their students who have agreed to contribute already collected and transcribed corpora from children learning 25 different languages. Subjects include bilingual children, normally-developing monolinguals, and children with language disorders. The data are being structured to facilitate testing of models regarding babbling universals, variant paths in segmental and prosodic development, markedness effects, prosodic context effects, segmentation patterns, statistical learning, frequency effects, interlanguage transfer, diagnosis of disability, stuttering patterns, disfluency patterns, and the effects of morphology and syntax.

AphasiaBank is a shared database of multimedia interactions for the study of communication in aphasia. The goal of this work is the improvement of patient- oriented treatment of aphasia. To reach that goal, we must solidify the empirical database supporting our understanding of communication in aphasia. Our six specific aims are: 1. Protocol database development. We will continue to expand the core database to include additional participants, languages, bilingual types, and clinical profiles (severe aphasia, RHD, dementia, AOS, PPA, and PPAOS). 2. Analysis Automation. We will construct tools for automatic computation of scales for clinical diagnosis and the measurement of recovery processes. Using these new measures and the growing database, we will work with consortium members to develop new approaches to assessment, diagnosis, and classification. 3. Johnny Appleseed. We will disseminate the data, tools, and methods through personal contact, workshops, manuals, tutorials, collaborative commentary, journal publications, and downloads over the Internet. We will construct materials for training and teaching. We will place particular emphasis on dissemination of these tools to institutions serving minority populations. 4. Cross-disorder comparisons. We will link the English AphasiaBank database to the growing databases in the DementiaBank, TBIBank, and RHD projects, as well as new data in AphasiaBank for AOS, PPA, and PPAOS. 5. Recovery and treatment evaluation. We will continue retesting of PWAs to evaluate the nature of recovery in the chronic period. We will evaluate the effects of programs such as teletherapy, script learning and repetition, and group conversation programs. 6. Functional Communication. We will develop methods for measuring and evaluating the ways in which people with aphasia, including those with severe impairments, achieve communication through gesture, conversational scaffolding, and augmentative communication devices.

NIH and NSF have provided support for the construction of a series of TalkBank databases on language disorders and development. These databases include a variety of multimedia samples of spoken language from specific target populations. There are currently separate TalkBank databases for 8 target populations including people with aphasia, apraxia, dementia, stuttering, right hemisphere damage, traumatic brain injury, autism spectrum disorder, and phonological disorders. The data includes both naturalistic samples and samples collected with a specified elicitation protocol. In addition, there are ongoing efforts to include representative samples from Latino and African-American participants across all of these databases. There are two major goals for this conference. First, we need to bring together the developers and users of these databases to share their interests, achievements, and desires for further developments. Second, we need to help both young researchers and more established researchers learn about a wide variety of new analytic methods that can lead to important advances in each of these content areas. We are requesting support for a two-day meeting to bring these groups together to maximize the use of the TalkBank resources that are being created.

Human communication happens both when we speak to each other and when we write things down. Researchers studying how people communicate can easily obtain many examples of written material. Written material is also easy to analyze because it is easy to scan it for computer use or to download it from the Internet. It is far more difficult to do research on spoken language because it is much harder to record speech and convert those recordings into a format a computer can use, especially speech that occurs in natural situations such as a family conversation around a dinner table or an argument between children in a classroom. This project will build a new system that will let researchers and students examine recorded conversations and provide additional data about details of those conversations, such as the talkers’ emotions, that cannot otherwise be measured.

The new system to be built will be part of the TalkBank system at Carnegie Mellon University, the largest publicly accessible databank for the study of spoken language in the world. TalkBank includes audio and video recordings from children, classrooms, bilinguals, second language learners, and persons with language disorders in 34 different languages, together with written transcripts of those recordings. The new system will allow scientists and researchers to study human communication by looking at patterns of speech in individual conversations, which is a level of detailed analysis that is not currently possible. Studying communication at the conversation level is important because fine details in speech patterns can help answer important questions such as how people resolve confusion and misunderstandings as a conversation unfolds. The new system will be based on a concept called Collaborative Commentary, which allows research groups and students to add comments about important conversation details, such as talkers’ motivations, directly into the database by adding links to the conversation transcript. This same system will also permit users to provide comments on media that has not yet been transcribed and create new transcripts through a web browser interface. This project will engage the public with the data through Citizen Science and provide a resource for teaching about language and communication through guided commentary. Completion of this work will open up a remarkable new window to the study of spoken language and communication.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

This project seeks to strengthen the compliance of the CHILDES (Child Language Data Exchange System) Project with the FAIR and TRUST standards. CHILDES provides data in the form of transcripts and linked media for the study of child language development. This supplement to the CHILDES grant will allow us to improve the findability, accessibility, interoperability, and reusability of the database. These improvements will transfer without further work to 13 other NIH databases for the study of normal and disordered spoken language.

This project seeks to restructure the software system supporting the CHILDES (Child Language Data Exchange System) Project. CHILDES provides data in the form of transcripts and linked media for the study of child language development. This supplement to the CHILDES grant will leverage best practices in software development and advances in cloud computing to improve the impact, reliability, sustainability, scalability, and openness of the CHILDES Project tools and database. The project will support the modernization and containerization of CHILDES software.