Lorraine E. Bahrick - US grants

The proposed research explores infants' perception of audible and visible events. Three studies will investigate the development of infants' sensitivity to audio-visual relations of 3 levels of specificity. These relations include 1) temporal macrostructure, in this case, the synchrony between sights and sounds of an object's impact, 2) temporal microstructure, here, audio-visual relations specifying the impact of a single object versus a group of objects, and 3) specific object-sound pairings, the relation between modality specific characteristics of a moving object and its impact sound. The research addresses the following questions: What is the nature and basis of the developmental course of audio-visual event perception? What is the developmental progression of infants' detection of the above audio-visual relations? What audio-visual relations do infants abstract from visual and acoustic exploration of an event? Prior studies of intermodal perception have generally tested infants of only one age group and thus have not investigated the nature and course of this developmental process. The proposed research addresses these questions by testing infants of 3 age groups (1 1/2-3 1/2 months) in 3 kinds of tests measuring detection of the audio-visual relations listed above. A new method of assessing intermodal capabilities in young infants is developed and used in the proposed research. The method involves training infants with single films and natural soundtracks of events. Then tests are administered to measure the nature of the audio-visual relations detected through training. In the tests, trials of 2 films and 1 soundtrack are presented. Infants' visual fixation to the 2 films will be recorded to determine whether a preference for the sound specified film is evident as a result of training. The research will test a set of theoretical predictions generated from a Gibsonian invariant detection model of perceptual development; that audio-visual relations are differentiated in order of increasing specificity. These studies can provide the basis for normative data on the development of intermodal perception, and may be used for diagnosing abnormal patterns of development.

The proposed research investigates long-term memor in 31/2- month-old infants using a novelty preference method. It explores memory for object motion, a topic that has been virtually ignored in the literature. Further, it evaluates a new model of infant attention. This model was developed on the basis of a preliminary study which assessed memory for object motion and demonstrated a preference for novelty after a 1-minute delay, no preference after a 1-day delay, and a familiarity preference after a 1-month delay. These results were interpreted as supporting a 3-phase model of infant attention: Phase A, Short-term memory, is characterized by a novelty preference, phase B, intermediate memory, is a transition phase in infant attention from novel to familiar stimuli, characterized by no preference, and phase C, long-term memory, is characterized by a familiarity preference. The first experiment proposed will replicate the preliminary study in a more carefully controlled design. Infants will be familiarized with an object undergoing one of two kinds of motion. A novelty preference test will follow after delays of 1-minute, 1-day, 2- weeks, or 1-month, where the familiar object will be presented undergoing the familiar versus a novel motion. A decreasing preference for novelty and an emerging preference for familiarity are predicted across retention intervals. In a second study, the effects of a retrieval cue on novelty preferences at phases B and C will be assessed. It is expected that the retrieval cue will reinstate preferences for novelty, at least at intermediate retention intervals, supporting the characterization of phase B as a transition phase in infant attention, as opposed to a period of total forgetting. This research challenges current interpretations of infant memory studies where novelty preferences have been taken at the primary index of memory, and null results have been interpreted as evidence of forgetting. It would support a view that preferences for novelty and familiarity interact with retention time, and would suggest a new principle governing information processing in infancy. An understanding of these relations will not only have methodological and theoretical significance, but will also have significance for evaluating delayed patterns of development. One can ultimately assess the effects of prematurity, low birth weight, mental retardation, and other risk factors on subsequent long-term memory and progression through the three phases of the attention cycle.

The primary goal of the proposed research is to further our understanding of the nature and basis of the development of intermodal perception of audible and visible events in infancy. Prior research has primarily focused on identifying the nature of cross-modal invariant relations detected by infants. Their findings have been consistent with Gibson's invariant-detection view of perceptual development, but they have left many important questions regarding the nature and basis of intermodal learning and development unanswered. The proposed research will fill this gap by examining the process of intermodal learning directly in a new intermodal training and transfer method. Infants of 2-6 months will receive training with natural, audible and visible events and then the audio-visual relations detected through training will be assessed under a variety of conditions. Three levels of nested audio-visual relations that characterize the stimulation from single events have been delineated: temporal synchrony uniting the sights and sounds of an object's impact, temporal microstructure specifying the composition of an object, and modality-specific relations between the pitch of a sound and the color and shape of an object. This research will identify the developmental progression of infants' detection of these nested relations, providing the first test of whether intermodal learning of nested audio-visual relations proceeds in order of increasing specificity, or in some other developmental sequence. Further, through a series of transfer of training studies, this research will provide the first assessment of how and under what conditions intermodal knowledge acquired in one stimulus context becomes flexibly extended to new event contexts. The attainment of this sort of flexible rule-based knowledge is the essence of intelligent functioning. An understanding of this process will promote the development of norms for intermodal functioning and transfer of training at various ages in infancy. Consequently, one can eventually diagnose abnormal patterns of development.

This research will investigate memory in young children and their mothers for events related to a major hurricane as a function of stress, operationalized as level of hurricane exposure. Ninety-six three- and four-year-old children and their mothers will be interviewed two months and again one year following the devastation of Hurricane Andrew in Miami, Florida. Thirty-six additional three and four year olds and their mothers will be interviewed at the one-year interval only, as a control group. Participants will be selected so as to represent equally low, moderate, and severe hurricane exposure conditions, defined by the extent of damage to their homes. All participants will be asked to recount events that fall into three time periods, 1) the prehurricane preparations, 2) the hurricane itself, and 3) the posthurricane recovery period. The memory interview will consist of open-ended questions, followed by specific category prompts for each period. The research will assess the amount of rehearsal of the information from the period before, during, and after the hurricane, as well as the degree of child and adult well being. Accuracy, amount, and content of recall will be coded, using a propositional analysis. Of primary concern is how these retention measures differ as a function of stress level and retention interval. The research will assess the effects of the first interview on retention at one year, using comparisons against the control group for the three stress levels. The research will also explore relations between content and amount of recall for children and adults. Of secondary concern will be relations between measures of memory and well being, and between adult and child well being. Findings of this study may improve our understanding of the relation between memory and stress in children and provide us with important information about the amount, accuracy and nature of recall for a traumatic event. In contrast with most research on traumatic events (e.g., physical, emotional, or sexual child abuse), this research is unique in that stress is so easily operationalized and memories can be corroborated.

DESCRIPTION: The primary goal of the proposed research is to further our understanding of the nature and basis of the development of intermodal learning and perception of audible and visible events in infancy. Prior research has focused primarily on identifying the intermodal capabilities of infants without examining the learning process itself or the origins and boundary conditions of these abilities. In the proposed research perception of both social and nonsocial events is investigated. Three levels of audio-visual relations that characterize the stimulation from single events have been delineated: 1) global - amodal temporal synchrony uniting the sights and sounds of an object's impact; 2) nested - amodal temporal relations specifying properties such as substance, composition, number, rhythm, and tempo; and 3) arbitrary specific relations between visual and acoustic properties such as pitch and color. The proposed research will systematically investigate the intermodal learning and generalization of learning in young infants on the basis of three levels of audio-visual relations available in face-voice events and events depicting objects striking a surface. It will employ an habituation/training phase followed by tests of intermodal learning and generalization. Predictions consistent with a principle of increasing specificity are tested: 1) Young infants should not learn to relate incongruent sights and sounds when they are synchronous; 2) Even newborns may be sensitive to global amodal relations; 3) Learning about amodal relations precedes and constrains learning about arbitrary relations, both developmentally and within a given episode of exploration; 4) Familiarity with an event should speed up intermodal learning; and 5) Generalization of learning about amodal relations should occur earlier and be greater than generalization of arbitrary relations. This research will provide the first systematic investigation of how intermodal learning and knowledge develops and how, once acquired in one context, it becomes flexibly extended to other contexts. The attainment of this sort of flexible rule-based knowledge is the essence of intelligent functioning. An understanding of this process will promote the development of norms for intermodal learning and generalization at various ages in infancy, which could contribute to the diagnosis of abnormal patterns of development.

DESCRIPTION (Adapted from applicant's abstract): The primary goal of the proposed research is to further our understanding of the nature and basis of the development of intersensory perception and attention and its consequences for learning and memory about audible and visible events. Prior research has primarily focused on identifying intersensory capabilities of infants without directly examining the learning process or developmental change and without integrating insights from psychobiological or neural research. The present proposal will assess the origins, nature, and development of intersensory perception in young infants and will evaluate predictions generated from our "intersensory redundancy" hypothesis which integrates findings from the comparative and neural research with those on human development. It is proposed that in early development, intersensory redundancy selectively recruits infant attention to amodal properties and facilitates further processing, learning, and memory for those properties. In contrast, unimodal stimulation facilitates attention, learning, and memory for modality-specific properties. Since most events are multimodal, this gives a processing advantage to amodal over modality-specific properties in early development. This has important consequences for learning and memory and for more complex processes such as language and social development, since the perceptual bases of these abilities emerge from a multimodal context in early infancy. In the proposed research eight experiments will evaluate these hypotheses using an infant-controlled habituation procedure. The research will assess the emergence and developmental change in infants' sensitivity to amodal and modality-specific properties in the context of multimodal as well as unimodal stimulation. The effects of intersensory redundancy on attention, perception, learning, and long-term memory will be assessed for social and nonsocial events. Together, these findings, regardless of whether they support out hypotheses, will shed light on the origins and function of the salience of intersensory redundancy. Results will be considered in light of those from Lickliter's psychobiological findings, Rochat's behavioral findings, and Mundy's social/clinical findings, with the goal of establishing general principles of development and providing a basis for evaluating atypical or delayed patterns of early perceptual organization.

DESCRIPTION (Adapted from applicant's abstract): The primary goal of the proposed research is to further our understanding of the nature and basis of the development of intersensory perception and attention and its consequences for learning and memory about audible and visible events. Prior research has primarily focused on identifying intersensory capabilities of infants without directly examining the learning process or developmental change and without integrating insights from psychobiological or neural research. The present proposal will assess the origins, nature, and development of intersensory perception in young infants and will evaluate predictions generated from our "intersensory redundancy" hypothesis which integrates findings from the comparative and neural research with those on human development. It is proposed that in early development, intersensory redundancy selectively recruits infant attention to amodal properties and facilitates further processing, learning, and memory for those properties. In contrast, unimodal stimulation facilitates attention, learning, and memory for modality-specific properties. Since most events are multimodal, this gives a processing advantage to amodal over modality-specific properties in early development. This has important consequences for learning and memory and for more complex processes such as language and social development, since the perceptual bases of these abilities emerge from a multimodal context in early infancy. In the proposed research eight experiments will evaluate these hypotheses using an infant-controlled habituation procedure. The research will assess the emergence and developmental change in infants' sensitivity to amodal and modality-specific properties in the context of multimodal as well as unimodal stimulation. The effects of intersensory redundancy on attention, perception, learning, and long-term memory will be assessed for social and nonsocial events. Together, these findings, regardless of whether they support out hypotheses, will shed light on the origins and function of the salience of intersensory redundancy. Results will be considered in light of those from Lickliter's psychobiological findings, Rochat's behavioral findings, and Mundy's social/clinical findings, with the goal of establishing general principles of development and providing a basis for evaluating atypical or delayed patterns of early perceptual organization.

The South Florida Research Consortium on the Development of Attention, Perception, Learning, and Memory will apply a developmental framework to foster the integration of biological, psychological, and social levels of analysis to the understanding of selective attention, perceptual processing, learning and memory during infancy and early childhood. The consortium will integrate research across species (utilizing both animal and human subjects), across developmental stages (from prenatal through early childhood), and across levels (neural, psychobiological, psychological, and social). A primary theme of the project is a focus on making research, theory, and application more ecologically relevant to multimodal natural learning contexts. Additional themes include: addressing the nature and basis of developmental change and growth from infancy to early childhood and investigating the mechanisms of early learning and generalization.

At the catalyst stage, this project will initiate and support workshops, planning meetings, and pilot research collaborations that can address a range of content areas including: the development of object and event perception, biological bases of early learning and memory, the development of language and communicative functioning, social and emotional development, and early cognitive development. This collaborative effort will contribute to theory construction and the development of an integrated knowledge base in the science of learning that can be easily translated to applications in education. Core and affiliated faculty concentrated at three universities in South Florida that will provide a rich and diverse set of resources and a critical intellectual mass for collaborative interdisciplinary research and for creating an infrastructure necessary for a future SLC proposal.

Broader impacts that will result from this proposed activity include applying newly discovered fundamental principles of learning and development from interdisciplinary efforts to education. The consortium will promote unique opportunities for training of investigators and students through the cross-fertilization of traditionally separate domains of knowledge and expertise, thereby fostering a synergistic effect on the growth of a science of learning. Further, the project's focus on the theme of ecological validity will facilitate easy translation between theory and application of findings to naturalistic learning contexts such as the social environment or the classroom setting.

DESCRIPTION (provided by applicant): The infant is faced with a barrage of complex information in the world, being conveyed through every sense modality. The infant must isolate streams of information specifying unified objects and events (e.g., object with impact sound, moving face with voice) in order to perceive an organized flow of information. However, research has suggested that information from different modalities may interact with one another to alter perception of an object or event. The objective of this project is to investigate audiovisual interactions in categorization and identify their developmental course. The central hypothesis for the proposed research is that auditory information plays an important role in categorization and that this role is modified during development, based on the extent of the individual's category knowledge. The proposed investigations are significant because they will provide a link between research in intersensory perception and categorization to demonstrate the role of multimodal information in the development of categories. Gender is a natural category that varies along a continuum and has been shown to have a long developmental trajectory that continues well into childhood. Previous research has demonstrated a typicality effect in gender categorization in which more masculine or feminine faces are categorized more accurately and quickly by both adults and children. Infants and adults will be tested to determine if categorization of facial gender can be biased based on the gender of a synchronous voice. The range, and development, of this effect will be tested by varying the typicality of the face which individuals are asked to categorize. Five- and ten-month-old infants will be tested in an infant-controlled habituation procedure in which gender-ambiguous or less typical faces of a single gender are paired with very typical voices and tested with very typical male and female faces. Preference for the opposite gender of the habituation voices indicates that infants categorized the habituation faces based on the gender of the synchronous voices. Adult participants will be asked to identify the gender of a series of gender-ambiguous and less typical faces synchronized with very typical voices. Accuracy and reaction time data will be analyzed for trials in which the gender of face and voice match versus trials in which the gender of face and voice are mismatched. Future research will include investigations of other categories and interactions of other modality combinations.

DESCRIPTION (provided by applicant): The proposed research investigates how and under what conditions various aspects of social events become salient, attended, and perceived and how this changes across development from infancy through early childhood. In particular, this proposal explores the developmental course of infants'perception of faces, voices, and amodal properties of speech (tempo, rhythm, and intensity) in unimodal auditory, unimodal visual, and multimodal, audiovisual stimulation using convergent measures of heart rate, eye tracking, and infant controlled visual habituation. Predictions concerning the role of redundancy across the senses in promoting and organizing the development of attention, perception, and learning about different properties of events in multimodal and unimodal stimulation, generated from of our model of selective attention (the intersensory redundancy hypothesis) will be tested. Five specific aims systematically explore the conditions that facilitate versus attenuate learning about faces, voices, and amodal properties of speech. By investigating multimodal and unimodal perception under a single framework, we will provide a basis for integrating separate literatures and reveal important interactions between modality of stimulation (unimodal, multimodal) and attention to properties of events (redundantly versus nonredundantly specified) that cannot be detected in separate research designs. We use a novel combination of convergent measures: visual habituation and recovery reveal what properties of audiovisual speech events infants detect (faces, voices, amodal properties of speech), heart rate indexes the depth and efficiency of processing, and eye tracking reveals what features of dynamic faces infants selectively attend under different conditions (redundant vs nonredundant). By including measures across different levels of analysis, critical controls for amount and type of stimulation, manipulations of task difficulty, and effects of repeated exposure, we will reveal much more about the nature, basis, and processes underlying the attentional salience of social events than can be revealed by separate studies or single measures. Patterns of selective attention and learning about social events that converge with those of our prior studies of nonsocial events will suggest that general perceptual processes govern attention and learning in this domain. Our goals are to advance developmental theory in the area of attention and perception, and to establish norms for infant sensitivity to intersensory and unimodal information about critical aspects of social events that can be used for assessing atypical patterns of development and can serve as a basis for interventions. Assessing multimodal conditions will also enhance ecological validity and foster translation of findings to real world learning contexts. PUBLIC HEALTH RELEVANCE: This research will reveal new information about the nature, basis, and development of attention to and perception of faces, voices, and amodal aspects of speech (tempo, rhythm, intensity) in unimodal visual, unimodal auditory, and multimodal audiovisual stimulation in infants and young children. Convergent measures of visual habituation, heart rate, and eye tracking, about the nature of selective attention to different properties of social events under different conditions, will provide a comprehensive and integrated picture of processes usually studied separately. Findings will provide a wealth of critical information about typical developmental patterns at a level of detail that is novel and necessary for identifying atypical patterns of development, including social attention deficits characteristic of autism. Findings are easily translated to real world settings and can serve as a basis for interventions for developmental delays.

The purpose of this KO2 proposal is to provide me with time and resources for a combination of career development and research activities designed to enhance my ability to advance developmental science in the area of early perception and attention. My overarching aim is to make basic research in infant and child development more easily translated to applications such as atypical development in disorders such as autism and to natural learning contexts such as home or school. I have created a plan for interweaving my knowledge, training, and basic research on the typical development of attention and intersensory perception with new knowledge, training, and research on atypical attention and intersensory perception in autism, in a way that will result in significant cross-fertilization and value added for each program. This synergistic effect will be accomplished by pursuing career development activities such as directed readings, traveling to conferences and research labs, creating a collaborative research network, learning advanced statistical methods, writing theory and concept papers, and training and mentoring the next generation of scientists, as well as by directly interweaving three research directions designed a priori to inform one another. These research directions converge to focus on how and under what conditions various aspects of social events (a primary basis for cognitive, perceptual, emotional, and communicative development) become salient, attended, and perceived and how this changes across development. The proposed program bridges gaps between research conducted on infants and children, unimodal and multimodal perception, social and nonsocial event perception, and typical and atypical development. The first research direction (my currently funded RO1) assesses the typical development of intersensory perception of social events in infants and young children with a focus on the role of intersensory redundancy in guiding and organizing the development of attention and perception. It uses a combination of infant controlled habituation, reaction time, visual preference, and eye tracking measures. A second research direction (funded by Autism Speaks) assesses attention and intersensory perception of social and nonsocial events in children with autism, typical development, and developmental delay. We have developed a new protocol assessing four basic building blocks of attention (disengagement, orienting, maintenance, intersensory processing) in a single test to clearly characterize attention skills and impairments in autism. Children with autism show a social orienting impairment, with reduced attention to faces, people, and speech. However, we know little about the typical nature, basis and developmental course of attention and perception of social events. In a third (new) research direction, I thus propose to provide the first systematic data base on the typical development of attention to faces, voices, and audiovisual speech, across infancy and early childhood. These studies will bridge critical gaps between knowledge of typical and atypical development and forge direct links between my two current research programs by using common methods, measures, and stimuli across age, from 2- to 36- months. This will provide a comprehensive developmental picture of typical development, serving as a basis for theory in developmental science and providing a developmental context and baseline for evaluating atypical development across infancy and early childhood. If the KO2 proposal is funded, this would allow me the time to pursue the newly proposed research direction and the career development activities designed to integrate my ongoing research program on typical development with that of atypical development. During the next few years, I am uniquely poised to accomplish this goal given the unique and fortunate circumstances of having a funded RO1 on typical development along with a small grant on autism, and trained research teams in place for each program, as well as access to a highly skilled and talented group of collaborators. The additional time afforded by the KO2 award would make it possible to take this integration to the next level through research, training, scholarship, collaboration and mentoring.

DESCRIPTION (provided by applicant): Disorders of attention, social, and communicative functioning have become a significant public health concern yet we lack a systematic data base characterizing the typical development of basic building blocks that support optimal developmental outcomes. Attention is the gateway to all we perceive, learn, and remember, however, it remains a significantly understudied topic in developmental science. Typically developing infants show heightened attention to faces, voices, and infant directed speech and this is critical for scaffolding cognitive, social, and language development. In contrast, children with autism show social orienting impairments marked by decreased attention to these stimuli as compared with nonsocial events, and serious deficits in social and communicative functioning. These capabilities depend critically on early attention and intersensory processing skills, the ability to integrate information across the senses, particularly in dynamic faces, voics and social events. However, there currently are no individual difference measures of intersensory processing for infants or young children and the pathways by which intersensory processing skills affect attention and later social, cognitive, and language development remain poorly understood, presenting serious obstacles to identifying the emergence of atypical developmental patterns in infancy. The present proposal addresses these needs. We have developed the first two individual difference measures of attention and intersensory processing that can be used with infants and young children. They assess attention orienting, disengaging, maintenance and speed and accuracy of intersensory processing for dynamic, audiovisual social and nonsocial events. The present proposal will provide the first longitudinal data sets revealing developmental trajectories for these basic building blocks of attention and intersensory processing in typical development across 3 to 42 mos of age (Aims 1 and 2). This will provide a critical basis for identifying atypical developmental trajectories. Using a structural equation approach to growth curve modeling, we will then test models of association between developmental trajectories for these measures with cognitive, social, and language outcomes at 18, 30, and 42 mos (Aim 3). This will elucidate the most viable models of influence through which intersensory processing and attention skills affect social, cognitive, and language outcomes, contributing to developmental theory, knowledge, and guiding interventions. Finally, we develop and test a procedure for training intersensory processing skills and assess its effectiveness in a transfer test (Aim 4). Improvement and the conditions that foster it will lay a foundation for developing intersensory interventions. These goals have high health relevance. They will reveal the typical development of infant attention and intersensory processing skills and their effects on child outcomes, providing the first systematic body of basic research designed to be relevant and easily translated to identifying early atypical trajectories of attentin and intersensory processing and guiding interventions.

Project Summary Disorders of attention, social, and communicative functioning have become a significant public health concern yet we lack a systematic data base characterizing the typical development of basic building blocks that support optimal outcomes. We developed the first two individual difference measures assessing these building blocks for infants and children. We now propose to implement the measures in 13 research labs to collectively build a well-planned, large-scale, shared database, mine the database to advance knowledge and theory in developmental science, and, at the same time, forge a new model for collaborative research. Faces, voices, and infant-directed speech are highly salient to typically developing infants and scaffold language, social, and cognitive development. In contrast, children with autism show social-orienting impairments and deficits in social-communicative functioning. These capabilities depend on early multisensory attention skills (integrating and attending to information across the senses) particularly in dynamically changing faces, voices, and audiovisual speech. Without measures to assess individual differences in these fundamental skills, the pathways by which they affect later language, social, and cognitive development remain poorly understood. Our new protocols assess individual differences in attention maintenance, disengagement, and speed, and accuracy of intersensory processing for audiovisual social and nonsocial events in preverbal children. Preliminary findings from my current R01 reveal exciting relations between these multisensory skills and language and social outcomes in 3-36-month-olds. The present proposal builds on this foundation. The Multisensory Data Network brings together 13 experts in developmental science. We will 1) integrate our new protocols into each of their research labs, and 2) implement an overall data collection plan, with each lab testing participants using both protocols along with standardized language and social outcomes. This will create a shared database of more than 1600 3-60 month old children. 3) Data will be uploaded to Databrary, an online data-sharing library. 4) We will guide and standardize data collection across sites and create a multisite aggregate dataset for easy sharing across investigators. 5) Capitalizing on advantages of large datasets, we will derive the first preliminary norms for multisensory attention skills across 3-60 months and, using cutting-edge, SEM-based analyses, develop models characterizing developmental cascades from multisensory attention skills to more complex language and social capabilities that rely on this foundation. 6) We then develop portable protocols, expanding potential applications to classroom and home settings. This project will provide the first dataset of multisensory attention skills and effects on later outcomes across the first 5 years of life, with implications for identifying atypical trajectories and infants at risk for delays and guiding interventions. This collaborative model will catalyze new research directions and is more time efficient, cost effective, and will generate a larger, more diverse dataset than possible in individual research labs.