Lynn K. Perry - US grants

PROJECT SUMMARY Only 56% of children with hearing loss (HL) in elementary school and only 44% in high school are reading at grade-level (Geers et al., 2008). The literacy gap between children with HL and children with normal hearing (NH) is observable early: preschoolers and kindergarteners with HL score significantly lower on early literacy measures than children with NH (Easterbrooks et al., 2008; Nittrouer et al., 2012). Identifying the mechanisms underlying emergent literacy in the preschool years for children with HL is therefore crucial to begin to close the outcome gap. The long-term goal of this research program is to validate effective interventions for preschool children with HL that close the early literacy gap between preschoolers with HL and NH. Initially, we will identify the literacy acquisition mechanisms for children with HL and then build and validate interventions that address the identified mechanisms. Toward this goal, the aims of the proposed research are to determine the developmental influences between 1) phonological awareness and alphabetic knowledge, 2) phonological awareness and receptive vocabulary, and 3) expressive morphology and receptive vocabulary for preschoolers with HL. Children's emergent literacy skills will be assessed as they develop during preschool (Lonigan et al., 2000). The obtained longitudinal data will be analyzed using latent change score modeling, useful for identifying the developmental influences among co-developing constructs (McArdle, 2009). Latent change score modeling in conjunction with longitudinal data will enable us to identify the developmental influences among phonological awareness, alphabetic knowledge, receptive vocabulary, and expressive morphology for children with HL. Identifying the unique developmental influences for children with HL relative to children with NH will lead to an improvement of theoretical models of emergent literacy to account for the unique literacy mechanisms of children with HL. Improved theoretical models will allow us to infer the malleable factors in emergent literacy development for children with HL, leading to hypothesis-driven intervention creation and improved clinical outcomes for this population.

Project Summary This Administrative Supplement proposes implementation of a multimodal data pipeline to support machine learning of child language production in complex naturalistic environments. The Supplement builds on the parent R01 (DC018542) that gathers objective, longitudinal data to capture the vocal interactions of children with hearing loss (HL). Even with cochlear implantation, HL is a life-altering condition with high social costs. Inclusion of children with HL and typically hearing (TH) peers in preschool classrooms is a national standard, but it is not clear how early vocal interaction contributes to the language development of children with HL and their TH peers. The parent R01 employs computational models of child location and orientation to indicate when children are in social contact with their peers and teachers. An additional strategy for pursuing the broad goals of the R01? identifying interactive contexts in which children produce phonemically complex vocalizations and interactive speech?is machine learning. Machine learning algorithms can determine the contextual, individual, and interactive factors that predict children?s vocalizations and vocal interactions. However, the parent R01 does not propose machine learning, nor are data disseminated in a format designed to facilitate machine learning. To facilitate machine learning in the classroom, a rigorous diarization process is required to determine speaker identity, which is operationalized as the likelihood that each vocalization was spoken by a given child or teacher. We will integrate audio processing of each target child and teacher?s first-person audio recording with processing of their interactive partners? recordings. The influence of partner recordings will be determined by their physical distance and orientation relative to the target. This will yield a weighted speaker identification score for each vocalization. For 25% of the sample, the algorithmic score will be compared to speaker identification provided by trained coders to quantify intersystem reliability. Processed datasets will include 7,160 hours of multimodal recordings of child and teacher movement in classrooms synchronized with continuously recorded, child- and teacher-specific (first-person) audio recordings. De-identified output data will characterize vocalizations with respect to algorithmically computed speaker identification probabilities, coder-identified speaker identity (25% of sample), phonemic complexity and audio characteristics (e.g., fundamental frequency), as well as the position and relative orientation of all individuals in the classroom, and child demographics (including characterizations of HL). Over the course of the supplement, output data, Python processing code, and metadata descriptions of the processing pipeline will be disseminated in dedicated distribution portals including Github, Kaggle, and the UCI repository. Recordings will be released to certified investigators via NIH-funded repositories such as Databrary and Homebank.

Project Summary This project uses objectively measured, longitudinal data to capture the vocal interactions of children with hearing loss in order to understand everyday factors that facilitate their language development. Significance. Hearing loss (HL) is associated with delays and difficulties in social and language development. Even with cochlear implantation (CI), HL is a life-altering condition with high social costs. Inclusion of children with HL and typically hearing (TH) peers in preschool classrooms is a national standard, but little is known about how early interaction contributes to the language development of these children. The project premise, supported by preliminary data, holds that language-mediated interactions with peers and teachers in inclusive classrooms positively influences the language development of both children with HL and their TH peers. Innovation. Temporally intensive, objective measurements of child and teacher movement in classrooms will be synchronized with automated analysis of continuously recorded, child-specific audio recordings. Computational modeling of child location and orientation within classrooms will indicate when children are in social contact with their peers and teachers, fertile periods for language-mediated interaction. Approach. Participants will include 125 children (55 with hearing loss who have cochlear implants or hearing aids, and 70 with typical hearing) who are enrolled in three inclusive oral language classrooms. Children will be observed longitudinally in an early preschool (2-year-old), preschool (3-year-old), and pre-kindergarten (4-year-old) classes. Each class will be observed once per year over the 5 year project. (15 unique classes). Objective measurements of social contact and language use occur twice a month while standardized assessments of receptive/expressive language competencies occur at the beginning and end of the school year. Aim 1. Determine whether exposure to partner speech during periods of social contact longitudinally predicts speech to those partners during social contact. Overall, higher levels of socially mediated exposure to peer and teacher speech are hypothesized to predict higher speech to those partners. Socially mediated exposure to more vocally responsive partner speech and more phonemically complex speech is hypothesized to yield higher rates of more phonemically complex child speech. Aim 2. Determine the longitudinal influence of speech to partners during social contact on the development of children's language competencies. Children's speech to peers and teachers?particularly more phonemically complex speech?is hypothesized to be associated with increases in their assessed language competencies. Aim 3. Determine whether hearing loss directly or indirectly impacts the longitudinal development of speech to partners during social contact and assessed language competencies. In addition to testing for differences between children with HL and TH, Aim 3 explores whether associations between socially mediated vocal interactions and the development of language competencies differs among children with HL on the basis of factors such as hearing age (time since augmented hearing). Aim 3 also explores the role of other individual differences such as biological sex and SES indices on patterns of interactive speech with partners and assessed language competencies. Rigor and reproducibility will be enhanced by the use of objective measurement, data-driven computational models, and dissemination of study tools and de-identified data.