Bingni Wen Brunton - US grants

Sounds in natural environments are complex mixtures from many different sources. This project seeks to understand how humans organize mixtures of sounds into meaningful objects. Perceptions of auditory objects arise not from any particular part of the brain, but rather from coordinated activity across many brain regions; further, binding of sounds to auditory objects may switch very rapidly. Therefore, the study of how auditory objects are formed and how rapid switching occurs requires analyzing recordings of brain activity in humans across many brain areas and at very high speed. This project aims to develop new theoretical methods for integrating and analyzing complex dynamic data sets of brain recordings from large-scale electrode arrays. The modeling approach will provide insight in the understanding of human auditory perception in both normal and clinically impaired minds.

Significant advances have been made in the past three decades characterizing neural correlates of auditory perceptions localized to the auditory cortex. Nevertheless, these neural correlates are likely not restricted to the auditory cortex, or to any particular part of the brain. To understand the neural mechanisms of auditory perceptual representation and perceptual switching, the current project combines advances in both experimental design and theory. Large-scale electrocorticography (ECoG) recordings will be collected from human subjects as they self-report their perceptions during a bistable auditory task involving rapid perceptual switching. Next, spatial-temporal patterns of cortical activation during the task will be extracted from these large time-series datasets using a data-driven method novel to neuroscience known as dynamic mode decomposition (DMD). Features extracted by DMD will then be used to build data-driven, low-dimensional dynamic models that capture the temporal evolution of multiple cortical areas, encoding both the auditory stimulus and the perceptual state.

Much knowledge about how human brains process information and generate actions has been informed by carefully controlled experiments in laboratory settings. However, understanding the brain in action requires exploration of its functions outside structured tasks. The current project explores neural processing over many days using large-scale recordings of brain activity augmented with video, audio and depth camera recordings, all simultaneously and continuously monitoring a subject. Importantly, unlike the majority of existing studies, here the subjects receive no instructions but are simply behaving as they wish in their hospital room-including eating, sleeping, and conversing with family. The project will advance data-intensive science and human neuroscience, leveraging external monitoring of the subjects to interpret naturalistic neural activity. The results of this project will be catalytic in understanding of the human brain, opening the door to study of brain function outside the structured confines of laboratory experiments.

The neural decoding algorithms developed will be directly applicable to current Brain-Computer Interfacing (BCI) technologies, enabling the deployment of systems that can predict the user's needs and improve quality of life outside the laboratory. Further, ongoing collaborations with neurosurgeons focus on evaluating this novel data-intensive approach to ethological brain mapping and how it may complement existing clinical functional brain mapping. The project will support and enable the education of students at the intersection of data science and neuroscience, including training scientists at the undergraduate, graduate, and post-doctoral career stages. Results from the research will be distributed as open access publications and code repositories, supporting a commitment to reproducible science.

This proposal focuses on data-driven innovations to enable more accurate decoding and inference of actions from long-term, naturalistic neural recordings. The first aim proposes to develop algorithms for automated decoding of natural motor and speech behaviors. Unsupervised clustering will be used to discover coherent patterns in brain activity, and clusters will be annotated with behaviors automatically parsed from external monitoring streams. Motivated by the size of the dataset and substantial variety between individuals, this scalable computational approach circumvents tedious manual annotation and fine-tuning of parameters. The second aim proposes to infer networks of dynamic causality of cortical networks engaged in task-free, naturalistic behaviors. This aim focuses on testing the hypothesis that neural correlates of naturalistic behaviors differ from those of repeated, instructed behaviors. Functional networks and the dynamic causality of cortical areas will be explored using methods from nonlinear dynamical systems theory. These networks will be compared to results from clinical brain mapping. This project will improve state-of-the-art neural decoding in naturalistic contexts and uncover neural correlates of task-free behaviors in humans.