Uri Hasson - US grants

DESCRIPTION (provided by applicant): Space and time are two fundamental properties of our physical and psychological realms. We recently proposed that the brain uses similar strategies for integrating information over space and throughout time. It is well established that neurons along visual cortical pathways have increasingly large spatial receptive fields (SRFs). This is a basic organizing principle of the visual system; neurons in higher-level visual areas receive input from low-level neurons with smaller receptive fields, thereby accumulating information over space. Drawing a parallel with SRF, we defined the temporal receptive window (TRW) of a neuron as the length of time prior to a response during which sensory information may affect that response. We argue that, as with SRFs, the topographical organization of the TRWs is distributed and hierarchical. The accumulation of information over time is distributed in the sense that each brain area has the capacity to accumulate information over time. The processing is hierarchical because the capacity of each TRW increases from early sensory areas to higher order perceptual and cognitive areas. Early sensory cortices such as the primary auditory or visual cortex have relatively small SRFs and short TRWs (up to hundreds of milliseconds), while higher-order areas have relatively large SRFs and long TRWs (i.e. can accumulate information over long periods of time). The goal of this proposal is to test this novel hypothesis by characterizing TRWs throughout the cortical hierarchy using temporally extended naturalistic stimuli. Using two complementary methods, functional magnetic resonance imaging (fMRI) and intracranial electroencephalography (iEEG), we will develop novel experimental paradigms and analytic tools to measure processing time scales and to probe the underline neural mechanisms by which brain areas accumulate information over time. A better understanding of how the brain accumulates and integrates information over time may shed light on various cognitive disorders as ADHD, learning impairments, and schizophrenia, which often involve difficulties with synthesizing information over time.

Communication is a dynamic process by which information is transferred across people. For the sake of experimental control, however, most cognitive neuroscience work on communication focus on either language production in the speaker's brain or comprehension in the listener's brain during highly artificial tasks. Communication, which by nature is a joint action embedded in a social context, is paradoxically studied in single individuals in isolation. In this proposal, we advance a new and versatile framework for understanding the neural mechanisms underlying communication in the real world. This framework argues that effective communication emerges when the neural activity of the two interlocutors are ?coupled? together. This coupling can take the form of (1) mirroring, when the listener's neural patterns match those of the speaker; (2) conditional transformations, when the listener's patterns reflect a lawful relation to the speaker's neural patterns; or (3) synergies, when the activities of the two brains dynamically influence and constrain each other to optimize information sharing. To test our theoretical framework, we propose developing both stationary and portable dual-brain imaging systems for measuring the neural activity of multiple individuals engaged in dialogue in the laboratory and in clinical settings. Two of the systems, fMRI hyperscanning and ECoG hyperscanning, take advantage of the high spatial and high temporal resolution of the respective methods to precisely characterize coupled neural dynamics during dialogue. For the third system, we propose developing a portable, dual-brain fNIRS system to characterize how two brains interact in real-life settings. Field work measuring the level of brain-to-brain coupling between a caregiver and a child could be used to study the acquisition of a first language; as an early preverbal biomarker for developmental disorder (e.g. lack of caregiver-child coupling as an early biomarker for autism); and as a temporally refined diagnostic tool for evaluating the effectiveness of behavioral and pharmacological interventions aimed at alleviating communication deficits (e.g. in autism, schizophrenia). Although the proposed research is technologically challenging, this laboratory has a track record of developing innovative analysis tools and theoretical frameworks for the study of cognitive functions in real-life contexts. The PI has substantial experience working with ECoG, fMRI and fNIRS, as well as studying both neurotypical and clinical populations. Our proposal is strongly grounded in prior work studying the extent of shared neural responses across subjects during the processing of real-life information. Thus, although ambitious, the research plan is both feasible and grounded, and has the potential to transform the way we understand and assess the neural processes by which we interact with others in everyday contexts. Ultimately, we believe that this work will lead to a novel brain-to-brain coupling biomarker for detecting preverbal communication disorders and assessing interventions in clinical settings.

Project Summary Social interaction in real-life contexts necessitates dynamical interactions among two or more brains as individuals listen, speak, act, and mutually adapt to one another to reach shared understanding. Elucidating how brains forge such shared understanding requires shifting from a ?one-brain? to a ?multiple-brain? frame of reference, as well as from artificial laboratory conditions to natural, real-life settings. Here we outline a novel framework to identify inter-brain dynamical coupling that underpins knowledge about social phenomena (e.g., mental states, social norms, emotions, etc.), henceforth referred to as social knowledge. The long-term goal of our laboratories is to understand how neural networks couple, within and across brains, to create and share information that enables social understanding and connectedness. We intend to develop a novel brain-to-brain coupling framework to better understand how social concepts are represented in the brain and how they are transferred across brains to achieve group cohesiveness. The overall objective of this application is to identify mechanisms that facilitate coupling across a speaker and a listener during real?life interaction. This approach is innovative because it uses new experimental paradigms optimize for studying social interactions in naturalistic contexts, employs both fMRI and fNIRS methods (single scanning and hyperscanning), and includes development of novel analysis methods for modeling shared brain responses to complex, natural social stimuli in real-life contexts. This contribution is significant because the proposed research will provide new insights into a central function of the human brain: the ability to connect with others by dynamically and interactively creating and sharing social knowledge. The work proposed in this application will advance knowledge of how brains process and share information in ways that promote social understanding and will produce new approaches for detecting and diagnosing communication and developmental disorders.

Project Summary Present and prior information converge often in everyday life. For example, during language comprehension, each syllable achieves its meaning in the context of a word, and each word in the context of a sentence. Despite the clear importance of such integration of past and present, most studies of memory use simple stimuli that are isolated in time. The long-term goal of this laboratory is to understand how the brain uses past information, gathered over seconds to hours, to make sense of a stream of incoming information. Previous work from the laboratory shows that many areas of cortex can accumulate information over time and use it for online processing. Furthermore, this research showed that early sensory areas use past information gathered over milliseconds, and this timescale increases to minutes in higher-order brain areas. These findings suggest that memories needed for online stimulus processing are topographically distributed in a hierarchy across the cortex based on their temporal properties. The overall objective of this application is to investigate the functional role of cortical areas at the top of the processing hierarchy; in particular, we will investigate to what extent these cortical areas have an intrinsic ability to accumulate information over minutes, and to what extent these long-timescale properties emerge from interactions with the hippocampus. This contribution is significant because the proposed research will provide new insights into a central function of the brain: the ability to accumulate information over minutes and use it to process an incoming information stream. This approach is innovative because it uses new experimental paradigms, both fMRI and ECoG methods, both neurotypical and brain lesioned amnesic patients, and includes development of novel analysis methods for brain responses to complex natural stimuli (stories and movies). The work proposed in this application will advance knowledge of how the brain combines information across minutes and will produce new approaches to the study of how memory is dynamically used during online stimulus processing.