Adeen Flinker, Ph.D. - US grants

DESCRIPTION (provided by applicant): The spatiotemporal dynamics of cortical activity underlying speech perception and production are not well delineated. Current neuroimaging techniques such as fMRI and PET provide high spatial resolution but lack sufficient temporal resolution to address the temporal dynamics of language. In contrast, electrophysiological techniques such as EEG and MEG offer high temporal resolution but lack the spatial resolution needed to ascertain what cortical networks are critically involved in speech perception and production. This proposal aims to address this gap in the literature by analyzing electrical signals with both high temporal resolution and excellent spatial resolution acquired during language processing tasks. The experiments involve recording electrocorticogrpahic activity (ECoG) directly from the cortex of patients undergoing treatment for refractory epilepsy. Multiple electrode grids are neurosurgically placed on the surface of the brain providing extensive coverage of left perisylvian language cortices. The high fidelity ECoG signal contains a rich spectrum of information. In addition to classical event related potential (ERP) analysis, time- frequency analysis techniques offer a window into local and global network connectivity. We will use the recently described high gamma oscillatory response (HG: 60-200 Hz) together with other frequency bands and ERPs as cortical markers for neural activity. We will employ two tasks aimed at elucidating the spatiotemporal dynamics of basic phonological and lexical processing. One task will consist of hearing auditorily-presented phonemes and then re-producing them while the other will use words in the same experimental structure. Comparing the two tasks will serve as a basis for ascertaining the timing dynamics of cortical regions involved in phonological and lexical processing. Furthermore, we will examine the role of the inferior frontal gyrus (IFG) in receptive processing. Specifically, we will address the controversial role of IFG in phonological processing. In addition to providing data relevant to linguistic theory, determining the spatial and temporal dynamics of language processing may lead to advances in public health. New brain mapping techniques may permit shorter and safer procedures during neurosurgery for several disorders (i.e. epilepsy, tumor resection) replacing the traditional cortical stimulation mapping with its associated morbidity related to seizure induction and length of the mapping procedure. PUBLIC HEALTH RELEVANCE: This proposal will help create a cortical map of where and when different brain regions are active during comprehension and production of language. This will expand current theories of how the brain processes language as well provides new brain mapping techniques. Such mapping techniques may permit shorter and safer procedures during neurosurgery for several disorders (i.e. epilepsy, tumor resection) replacing the traditional cortical stimulation mapping with its associated morbidity related to seizure induction and length of the mapping procedure.

DESCRIPTION (provided by applicant): Language processing remains one of the most striking examples of hemispheric asymmetry. While much research has focused on elucidating the functional differences, the exact underlying mechanisms of the laterality remain unknown. Language function is severely disrupted in left but not right hemisphere lesions while damage to the right hemisphere impairs the processing of prosody. Many studies in both humans and animal models have exhibited similar functional asymmetries, nevertheless, the exact neuronal mechanisms underlying hemispheric lateralization in language remain unknown. Several hypotheses have been proposed explaining the functional asymmetries, including a differing hemispheric capacity for temporal and spectral resolution, a difference in the temporal scale of information integration as well as global vs. local processing of stimuli. The aim of this study is to propose and test a unifying hypothesis whereby the left hemisphere processes a wide range of temporal modulations and a limited range of spectral modulations and the right hemisphere processes a wide range of spectral modulations and a limited range of temporal modulations. Using novel stimuli filtering techniques, we propose altering speech stimuli on a spectral modulation and temporal modulation axis and directly test our hypothesis using Magnetoencephalography (MEG) and Electrocorticography (ECoG). Specifically, we will address: 1) Left and right hemisphere sensitivity to spectral and temporal modulations; 2) Task-related modulations of auditory cortex; 3) Spectrotemporal auditory receptive fields in higher order auditory cortical fields. ! PUBLIC HEALTH RELEVANCE: Clinical observations dating back to the 19th century provide clear evidence that damage to the left but not right hemisphere impairs language processing. While there exist many hypotheses regarding the roles of the left and right hemispheres in speech, language, and other aspects of cognition, the exact neural mechanisms underlying hemispheric asymmetries remain unknown. The aim of this proposal is to elucidate the mechanisms underlying left and right processing of speech, in a hope to further our understanding of basic mechanisms of speech analysis and enrich the diagnostic and treatment tools for language disorders.

This Major Instrumentation Grant supports the acquisition of a 512-channel integrated stimulation and recording system for human neurophysiology research at the NYU Comprehensive Epilepsy Center (CEC). Each year, 50-80 patients at the CEC undergo intracranial electrode monitoring and surgical removal of brain tissue to treat medically refractory epilepsy. The surgery involves making a small craniotomy in the skull and implanting electrodes in the brain that are used (1) to monitor neurophysiological brain activity during seizures and (2) to stimulate different regions of brain to understand their importance for cognitive, sensory and motor tasks. With this information, clinicians make informed decisions on what regions of the brain can be safely removed to treat the patients' epilepsy, while sparing critical functions. Having patients with electrodes temporarily implanted in their brain enables the unique opportunity to study both human cognition and epilepsy in a manner that is not possible with non-invasive tools such as brain imaging. With rapid improvements in electrode technology, the number of electrodes in each implanted device continues to grow and offers researchers even more detailed information about brain function in normal cognition and disease. To maximize the information available from each electrode, it needs to be independently connected to a stimulation and recording system. At present, the clinical system at the CEC cannot support the total number of electrodes in both clinical and research devices, leading to compromises in the research data. Moreover, some newer electrode technologies are not compatible with the current system. Thus, valuable information is lost. This award will enable the CEC to purchase a piece of equipment that has the capacity to measure from and stimulate 512 isolated electrode channels, ensuring no information is lost and bearing the promise of a more detailed and complete understanding of brain function.

NYU has built an incredibly productive research program around the human intracranial electrode patient population. Research is broadly focused on understanding language, memory, sleep, and epilepsy in the human brain. While the clinical value of the research is clear, the work also provides insight into the "normal" functioning of the human brain and thus falls within the scope of NSF supported science. For example, scientists working with the CEC have been able to provide deep insight into questions such as how language is processed by the brain and how memories are formed and subsequently strengthened during sleep. The intracranial research program at the CEC is highly collaborative and involves partnerships with scientists and clinicians across NYU as well as at multiple research institutes around the country. Patients are engaged in sensory, cognitive, and motor tasks and researchers measure activity in the brain to understand the neural basis of cognition. This award will not only expand the amount of information researchers can measure from a single patient, but also enable them to stimulate and perturb the brain during task performance to uncover causal brain-behavior relationships previously inaccessible in humans. Moreover, the technical expertise and interests of the collaborative group of researchers working with the CEC span a wide range and include non-invasive neuroimaging in humans and novel electrode technologies in animals. The capability to compare many channels of human intracranial recording with cross-species electrophysiological recordings in animal models will expand capabilities for translational research and accelerate the pace at which medical technology develops in the lab and can be brought to the bedside to improve clinical care.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Despite significant advances in neural science, the dynamics by which neural activity propagates across cortex while we think of a word and produce it remains poorly understood. This proposal will develop novel, data-driven approaches for understanding functions and interactions of various brain regions by leveraging rare neural recordings obtained with electrocorticography (ECoG) sensors while neurosurgical patients participate in tasks involving language perception, semantic access and word production. This project will produce a set of validated novel computational tools for estimating neural representations and their dynamics as well as elucidate the cortical networks subserving perception, semantic access, and production of speech. Although these tools will be developed for ECoG data, the proposed frameworks are applicable to other neural data modalities including fMRI and EEG, and thus have broad applications in neuroscience. The ability to robustly translate between speech and its neural representations is vital to the development of speech prosthetics, which would allow patients with degenerative conditions (Amyotrophic Lateral Sclerosis) or neurological damage (locked-in syndrome) to drive a speech synthesizer via control from intact cortical structures. The network connectivity tools could shed light on the propagation dynamics of epileptic seizures as well as on how cortical communication, when impaired, gives rise to language aphasias and disconnection syndromes. Furthermore, the decoding and network connectivity tools could help develop novel language mapping approaches for brain surgery without the associated risks of electrical stimulation mapping.

The project consists of three core thrusts: developing neural decoders for language processing, developing directed connectivity models, and experimental validation. The neural decoders will be based on deep-learning architectures able to learn a transformation between neural signals and the speech heard by the patient, the speech produced by the patient, or the semantic concept represented by the stimulus word. The connectivity models will generalize and coalesce current approaches for estimating the task-dependent, time-varying directed connectivity between cortical regions. Lastly, these findings will be experimentally validated via clinical electrical stimulation data and cortico-cortico evoked potential (CCEP) stimulation experiments. Current modeling approaches of ECoG data have mostly focused on variants of linear models and on speech acoustics. This project will harness the potential of highly non-linear and deep networks for modeling neural responses to both speech acoustics and access to semantics. Additionally, tools for inferring direct connectivity and interactions among neural regions will provide a detailed characterization of the network dynamics, which is largely overlooked by most ECoG decoding studies.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Project Summary Speech is a uniquely human trait that is central to our everyday lives, but the mechanisms that enable retrieval and formation of articulatory sequences remain elusive. Several lines of evidence suggest that the left inferior frontal gyrus (IFG), or Broca?s region, is active prior to articulation per se and critically supports articulatory planning. However, it remains unclear how different anatomical parcellations of IFG support retrieval of semantic information and articulatory planning. To investigate this issue, we will leverage resources at NYU, assembling clinicians and researchers with complementary expertise to reexamine the role of the IFG in speech. We propose working in a cohort of neurosurgical patients who provide a rare and unique opportunity to collect direct cortical recordings and perturbations during speech production. Specifically, we will address the following questions: (1) what is the time constant of neural responses across IFG in relation to speech production, (2) what is the functional specificity of IFG regions, pars opercularis and pars triangularis, and (3) what are the functional boundaries Broca?s territory across the left and right frontal cortices. To gain traction on these issues, we have developed a battery of speech production tasks that is designed to mirror clinical electrical stimulation mapping and provides a within subject functional and causal comparison. We will use standard and high density electrocorticography (ECoG) to measure responses with high temporal precision within IFG. To determine causality, we will manipulate cortical activity using direct cortical stimulation and examine changes in behavior, timing of speech deficits and comparing with ECoG functional responses. The outcome of this study will greatly advance our understanding of the neurophysiologic mechanisms underlying speech production with a long-term goal of addressing a range of speech disorders and improving current language mapping techniques.

To avoid post-operative language impairments after surgery for drug-resistant epilepsy, electrical cortical stimulation mapping (ESM) is widely relied upon to localize language cortex, but ESM often elicits after- discharges (ADs), seizures, involuntary movements, and pain, which can limit mapping and impact patient safety. Furthermore, ESM is time-consuming and usually produces all-or-none results, providing limited insight into the function of stimulated sites. Finally, up to 30% of patients can have language dysfunction after resections that are guided by ESM. These limitations have motivated an alternative mapping method based on task-related power changes in high-gamma frequencies (?50-200 Hz), which are highly correlated with neuronal population firing rate changes. Although high-gamma mapping (HGM) overcomes the aforementioned limitations of ESM, its clinical validation has been limited to ESM-HGM comparisons in small case series with significant variations in technique across centers, yielding inconsistent results, with unexplained discrepancies between methods. Moreover, neither HGM nor ESM have been prospectively validated for predicting post-operative language impairments with either subdural electrocorticography (ECoG) or stereo-EEG (sEEG, increasingly a safer alternative to ECoG). The overall objective of this study is to use a small consortium of three large academic epilepsy surgery centers to demonstrate the diagnostic validity and safety of iEEG HGM for both ECoG and sEEG in a prospective series of 221 patients, using an innovative browser-based bedside HGM system to standardize methods and share data across centers. This study will test the hypothesis that HGM can accurately predict post-surgical language outcomes but requires a different framework for interpretation than that used for ESM. First, using traditional methods for interpreting HGM results, we will test the concordance between HGM and ESM, and compare their safety and feasibility. Based on previous studies, we predict that HGM will be a good, but imperfect, predictor of ESM results. However, we predict that due to ESM-related seizures and differences in mapping duration, HGM will be safer and better tolerated by patients. Second, we will test models that attempt to explain and reconcile HGM false negatives and false positives with respect to ESM, and we will test whether these models, incorporating both functional activation (HGM) and effective connectivity (measured with cortico-cortical evoked potentials, or CCEPs) can better predict ESM results. Third, we will develop and test models for prediction of post-operative language outcomes, based on the anatomical extent and volume of cortical resection with respect to HGM and ESM results, using voxel-lesion-symptom matching (VLSM). Since clinical decisions are currently based on ESM and clinicians will be blinded to HGM, we predict that the incidence of language deficits will be significantly higher when HGM+ electrodes are resected. This study will have a direct and profound impact on the clinical practice of language mapping for neurosurgical procures, while contributing valuable insights into the structure and dynamics of human language networks.

Project Summary Our ability to understand other people in terms of the underlying mental states that drive their actions, termed theory of mind (ToM), is essential to human social behavior. Neuroimaging research has found that this sophisticated, abstract reasoning ability relies on a specific network of regions across association cortex, with one region, the temporo-parietal junction (TPJ), showing a particularly selective response and ToM-relevant information content. However, because the ToM response in TPJ has only been studied using noninvasive brain imaging techniques with limited precision, our understanding of the detailed functional properties of this region?what information is represented, how these representations evolve over time, what computations are performed?remains very limited. Here we propose to study the role of TPJ in mental state reasoning using intracranial electrocorticography in humans, providing a direct measure of neural activity with combined spatial and temporal resolution. We will study TPJ responses using narrative comprehension tasks, in which subjects listen to a story describing the actions and interactions of human characters, and answer questions that require reasoning about their mental states. In Aim 1, we will probe basic aspects of the time course of TPJ response, leveraging the high temporal resolution of the electrocorticographic signal. We will test the following hypotheses: 1) TPJ has rapid, transient increases in activity in response to novel mental state information in a narrative. 2) TPJ responses to mental state content increase over longer time scales in a paragraph-long narrative, as more contextual information is available. In Aim 2, we will use high-density recordings to probe the precise spatial and functional organization of responses to theory of mind, semantic comprehension, and episodic recall. We will test two hypotheses about the functional specificity of ToM responses: 1) responses to ToM content and to more generic semantic content will be spatially segregated within TPJ. 2) Areas responsive to ToM will also be engaged during the episodic recall of richly social events, but not during more generic recall processes without a social component. This research will provide the first electrophysiological characterization of a region involved in uniquely human social cognition, and lay the groundwork for a research program that will characterize the neural basis of mental state reasoning with increasingly precise recording devices.

Project Summary Time is the fundamental dimension of sound, and temporal integration is thus fundamental to speech perception. To recognize a complex structure such as a word in fluent speech, the brain must integrate across many different timescales spanning tens to hundreds of milliseconds. These timescales are considerably longer than the duration of responses at the auditory nerve. Therefore, the auditory cortex must integrate acoustic information over long and varied timescales to encode linguistic units. On the other hand, the nature of the intermediate units of representation between sound and meaning remains debated. Focal brain injuries have shown selective impairment at all levels of linguistic processing (phonemic, phonotactic, and semantic) but current models of spoken word recognition disagree on the existence and type of these representational levels. The neural basis of temporal and linguistic processing remains speculative partly due to the limited spatiotemporal resolution of noninvasive human neuroimaging techniques which is needed to study the encoding of fluent speech. Our multi- PI proposal overcomes these challenges by assembling a team of researchers and clinicians with complementary expertise at NYU and Columbia University. We propose to record invasively from a large number of neurosurgical patients, which provides a rare and unique opportunity to collect direct cortical recordings across several auditory regions. We propose novel experimental paradigms and analysis methods to investigate where, when, and how acoustic features of speech are integrated over time to encode linguistic units. Our experimental paradigms will determine the functional and anatomical organization of stimulus integration periods in primary and nonprimary auditory cortical regions and relate the temporal processing in these regions to the emergence of phonemic-, phonotactic-, and semantic-level representations. Finally, we will determine the nonlinear computational mechanisms that enable the auditory cortex to integrate fast features over long durations, which is essential in speech recognition. Understanding of the temporal processing of speech in primary and nonprimary auditory cortex is critical for developing complete models of speech perception in the human brain, which is essential to understanding of how these processes break down in speech and communication disorders.