Scott Makeig - US grants

Do schizophrenics' brains respond abnormally to sensory stimuli? Adler, Freedman, and associates, using a stimulus sequence of paired-clicks, claimed that inhibition of a particular early feature of the auditory evoked response (P50) at 500 msec intervals is strong in normal adults but weak or absent in schizophrenic patients. However, this phenomenon has not been replicated in normal subjects, nor has its relation been investigated to the positive wave at the same latency (known variously as P1 or Pb) evoked in many experiments employing much higher repetitive rates of stimulation (1-10/s). Part I of this proposal will comprise three sets of intensive electrophysiological experiments using multiple scalp electrodes on a small group of normal adult subjects. In each experiment adequate numbers of responses will be recorded to explore within- and across-subject variability, as well as measurement of the topographic distribution and temporal stability of the phenomena observed. Part I will first explore the relation of the auditory P50 wave recorded using stimulus rates of 1 per 10 sec to the positive wave(s) near 50 msecs referred to by others as P1 of the Late Wave Sequence (using rates near 1/s), or Pb of the Middle Latency Sequence (rates near 10/s). Second, it will investigate the timecourse of P50 inhibition using a set of paired-click experiments similar to those of Adler and Freedman, et al. but with varying inter- and intra-pair intervals. Third, it will compare the timecourse of P50 inhibition to the timecourse of perturbation (inhibition, augmentation, phase shift) of the 40 Hz steady-state response following paired-dropouts in a 40 Hz auditory click probe train, as in the recent experiments of Makeig on the "40 Hz CERP." Part II will compare the P50 and/or CERP inhibition measured in a small pilot group of adult schizophrenics with that of the normal subjects in Part I. Selection of experimental parameters in Part II will depend upon the results of Part I.

DESCRIPTION (provided by applicant): A major shift in scientific perspective on the nature and use of electrophysiological brain data is now ongoing a shift from measurement and visualization of individual channel signals (in the 'recording channel space') to visualizing and interpreting the data directly within a suitable inverse model representing activity reaching the electrodes by volume conduction from a set of effective data sources in native 'brain source space'. An equivalent shift, via the development and exploitation of an appropriate inverse imaging model, made possible the phenomenon of structural and functional magnetic resonance imaging (fMRI). While the electrophysiological inverse problem is still difficult, dramatic progress has been and is being made through combined use of multimodal imaging and modern statistical signal processing methods. Recovering the considerable degree of spatial source resolution available in high-density scalp electroencephalographic (EEG) and other electrophysiological data, while retaining its natural advantage over other functional imaging methods in temporal resolution, has begun to yield a steady stream of new information about patterns of distributed brain processing supporting human behavior and experience. Relative to other brain imaging modalities, EEG has substantial and increasing cost and mobility advantages, making promotion of new EEG methods for source space analysis of increasing interest and importance for brain and health research. However, applying new source signal and signal processing models to electrophysiological data is complex and increasingly involves application of modern mathematical methods whose details are not within the training of most health research professionals. The EEGLAB signal processing environment, an open source software project of the Swartz Center for Computational Neuroscience (SCCN) at the University of California San Diego, began as a set of EEG data analysis running on MATLAB (The Mathworks, Inc.) released on the World Wide Web in 1997. EEGLAB was first released from SCCN in 2001. Now, more than ten years later, the EEGLAB reference paper (Delorme & Makeig, 2004) has over 2,350 Google scholar citations (increasing at above 1 per day), the opt-in EEGLAB discussion email list links over 5,000 researchers, the news list over 9,000, and a recent survey of 687 researcher respondents reports EEGLAB to be the software environment most widely used for electrophysiological data analysis worldwide. EEGLAB is thus now a de facto standard supporting a wide range of EEG and other electrophysiological research studies and teaching labs. At least 35 EEGLAB plug-in toolsets have now been released by researchers from many laboratories. Under NIH PAR 11-028 we propose renewal funding to further develop and maintain the EEGLAB software framework. We propose new and better tools for brain source and source network modeling and localization, an expanded online EEGLAB course and workshop, better statistical inference modeling of group data, and new support for automated source decomposition, measure computation, data duration, and data sharing.

he central active challenge we are constantly addressing in daily life is to correctly assess the intent of others ('What is she trying to do? ...') and the import of sensory events ('What - good or bad - may happen now? ...') based on active perception ('It looks to me like she is trying to ...') and retrieved associations (''And she was the one who ...'). The corresponding problem for cognitive neuroscience is to identify, ideally from non-invasive brain activity recordings, those patterns of distributed brain activity that accompany and support active human cognition and behavior. This problem has two parts: First, -What patterns of distributed brain dynamics follow from, accompany, and predict specific world events and subject behavior? -To fully understand the experience and behavior of subjects in performing a given task, we must take into account both the import of each task event to the subject and the intent of each of behavioral event. These factors cannot be known directly, but they may be accurately guessed or inferred, in many cases, from detailed recordings of subject behavior and from the specific historical context in which each recorded environmental or behavioral event occurs. In the case of electroencephalographic (EEG) and/or magnetoencephalographic (MEG) signals recorded non-invasively from the human scalp, a second part of the problem remains -Which brain areas generate the identified signal patterns?'

The usual approach to analyzing electromagnetic scalp data has been to separate recorded events and behavior into a few simple categories, to average the recorded brain dynamics time locked to each event category, and then to apply physical inverse source estimation methods to scalp maps of peaks in the resulting averages. This project will explore using new machine learning methods, including advanced independent component analysis (ICA) and sparse Bayesian learning (SBL) methods, to jointly model the recorded task event, subject behavior, and brain dynamic data recorded in a complex learning task. The project has two goals: First, to identify patterns in unaveraged EEG and/or MEG data that reliably accompany subject behavior in specific contexts, and second to determine the exact areas of the subject's cortical mantle that locally synchonize their electromagnetic activities to produce the identified scalp patterns. If successful, the project will enhance the value of noninvasive electromagnetic brain imaging for identifying and measuring, with high temporal and spatial resolution, complex, distributed patterns of locally synchronous cortical activity that support active human cognition.

Human infants must learn complex skills to interact effectively with parents and other humans, but these social skills emerge at somewhat different ages in different infants. How can we explain this variability? How do infants attend to their social world, and thereby learn routines to interact effectively with other people? This project follows a group of 45 healthy toddlers who have been tested extensively from 3 to 18 months of age on a variety of changing cognitive and emotional responses to social stimuli. The same infants have been observed regularly at home in interactions with their parents. The current project asks how these toddlers' emerging social skills reflect their individual differences in cognition and emotion as infants, and on the different social input provided by their parents. The project focuses on changes in language and imitation skills from 18 to 24 months of age, and the brain dynamics that underlie these skills. The toddlers who were tested and observed starting at 3 months of age will be invited to participate again at 20 to 24 months of age. New sessions will use a unique system at UC San Diego: a Mobile Brain Dynamics (MoBI) facility for recording EEG (electroencephalographic) and body motion-tracking data simultaneously from two people. The project will use this system to record toddlers and parents as they engage in three types of interactions: 1) toddlers following parent's pointing (or line-of-gaze), 2) toddlers reacting to words spoken by parents, and 3) toddlers imitating parents' simple actions. These interactions represent important social achievements for toddlers. Advanced EEG analysis will be performed on electrical potentials measured on toddlers' and parents' scalps. At the same time special cameras will record the positions of their heads and arms. This design will therefore yield a continuous record of changes in the toddlers' and parents' brain electrophysiology (reflecting their thinking and emotional reactions) and body positions as they interact. In addition, toddlers will complete a battery of behavioral and language tests. This project will pioneer a new paradigm for studying the social development of young children, and yield the most complex and complete data available on how early social-attention behaviors relate to early language and imitation, and brain processes underlying these relations. The results will have implications for early childhood education, treatment of developmental disabilities, and parenting practices.

de Sa
0756828

Brain computer interfaces (BCIs) translate basic mental commands into computer-mediated actions. BCIs allow the user to bypass the peripheral motor system and interact with the world directly through brain activity. These systems are being developed to aid users with motor deficits which can stem from: neurodegenerative disease (such as Lou Gehrig's disease, or ALS), injury (such as spinal cord injury), or even environmental restrictions which make movement difficult or impossible (such as astronauts in space suits). BCI systems typically require extensive user training to generate reproducible and distinct brain waves. Furthermore, until very recently, most BCI systems have interacted with the user in unintuitive or unnatural ways, such as moving a cursor or bar left and right by engaging in two unrelated forms of mental imagery, such as moving the right hand vs. the left foot. Realistic visual feedback of interpreted motor action should substantially improve usability and performance of BCI systems. This hypothesis is based on four observations: 1) humans have evolved to adapt their motor control in response to visual and proprioceptive feedback; 2) rapid motor adaptation is demonstrated in virtual reality experiments; 3) animals improve their neural signal when given visual feedback of their decoded neural activity; and 4) visual feedback of interpreted movement should activate the mirror neuron system, producing a stronger movement signal. The proposed work aims to improve upon current BCI systems based on motor imagery by providing more natural and lifelike feedback. This task can be broken down into 3 main objectives: 1) analyze motor imagery with visual feedback in an offline setting; 2) develop algorithms for real-time EEG analysis; and 3) construct a real-time BCI system utilizing lifelike motion animations as visual feedback. While results of objectives 1 and 2 should each in their own right contribute to the current state of the art in BCI systems, the largest BCI performance and usability gains should be made by introducing lifelike feedback into an online paradigm in the third objective. The proposed system can also be used to study learning and sensory-motor processing in normal subjects by studying their adaptation to the system. It may also inform more costly invasive recording experiments by helping to determine optimal placements of implants. All software written for EEG signal processing and analysis will be made available as add-ons to EEGLAB which is distributed in accordance with University of California policy for research, education, and non-profit purposes. The EEGLAB project is also developing an EEG database in conjunction with the San Diego Supercomputer Center. Representative data sets will be released via this database in accordance with University of California policy.

DESCRIPTION (provided by applicant): Current technology allows recording of brain electrical and/or magnetic activity from 256 or more scalp sites with high temporal resolution, plus concurrent behavioral and other psychophysiological time series, while dense human intracranial data are routinely acquired during some brain surgery and surgery planning procedures. Subject anatomic magnetic resonance (MR), computerized tomography (CT), and/or diffusion tensor (DT) head images may also be available. Standard analysis approaches extract only a small part of the rich information about human brain dynamics contained in these data. We propose a collaboration between the UCSD Swartz Center for Computational Neuroscience (home to the EEGLAB software environment development project), the UCSD Center for Research in Biological Systems (home to the Biomedical Informatics Research Network (BIRN) coordinating center), and leaders in six other human electrophysiological research communities to develop a public 'A Human Electrophysiology, Associated Anatomic Data and Integrated Tool (HeadIT) resource'. This framework will be built on the BIRN Data Repository framework (www.nbirn.net/bdr), thereby expanding its scope and capabilities. The HeadIT resource will share existing, high- quality, well-documented data sets, allowing their archival preservation and continued public availability for re-analysis and meta-analysis with increasingly powerful analysis tools. Initially, the HeadIT repository, extending a foundational database within the BIRN Data Repository will contain a rich collection of human electrophysiological data contributed by SCCN and others and physically distributed across storage nodes hosted by centers focused on seven research fields: epilepsy, neurorehabilitation, attention, magnetic recording, child development, neuroinformatics, and multimodal imaging. The HeadIT resource will include a software facility for accessing and analyzing repository data in the EEGLAB (sccn.ucsd.edu/eeglab) and other widely-used Matlab-based electrophysiological tool environments. EEGLAB will be extended to include a foundational tool set for performing meta-analyses across more than one archived HeadIT study. We will develop minimal information standards and quality assurance tests for contributed HeadIT data, a facility for interactive data visualization, and will test and validate the operability of the HeadIT resource via named ongoing research collaborations that will serve as the initial user community for tool and data framework development and testing. PUBLIC HEALTH RELEVANCE: The proposed 'A Human Electrophysiology, Associated Anatomic Data and Integrated Tool (HeadIT) Resource'will allow re-analysis of freely available recordings of brain activity and associated behavioral and physiologic measures using freely available analysis tools. This will allow large multi-study meta-analyses for patterns not visible in any single study, re-analyses to validate previously published conclusions from existing data, and application of successively more advanced tools to complex and costly data collected in a wide range of clinical and basic research areas.

Intellectual Merit: This project aims at combining cognitive and computational neuroscience, neuroengineering and system identification towards a transformative understanding of the way distributed brain dynamics interact with motor activity in humans. 3-D body and limbs movement kinematics, eye movements and electroencephalographic (EEG) spatiotemporal brain data will be recorded simultaneously during motor control and adaptation in healthy and Parkinson?s disease patients. In particular, altered and real world motor tasks will be simulated in 3-D immersive virtual reality technology with force feedback robots providing proprioceptive interaction and feedback. Cognitive, behavioral and kinematics data will constrain the design of large-scale computational models of motor control and adaptation based on known anatomy and physiology of the basal ganglia. Neuromorphic engineering will guide the design of mobile embedded computational systems for real-time emulation of the brain-body models and closed-loop sensory-motor control for Parkinson?s patients. We expect that the development of new machines for neuro-rehabilitation will result in a threefold synergetic interaction between engineering and neuroscience: human-machine interactions will transform the notion of movement control and provide new contexts to study embodied cognition that will benefit neuroscience; in turn, new knowledge in neuroscience and motor control will accelerate the development of adaptive machines for rehabilitation and/or enhancement. Finally, comprehensive and predictive mathematical models of motor control implemented in neuromorphic hardware are expected to lead to new intelligent neuroprosthetic tools.

Broader Impact: Outcomes of this research will contribute to the system-level understanding of humanmachine interactions and motor learning and control in real world environments for humans, and will lead to the development of a new generation of wireless brain and body activity sensors and adaptive prosthetics devices. This will advance our knowledge of human-machine interactions, stimulate the engineering of new brain/body sensors and actuators, and have a direct influence in diverse areas where humans are coupled with machines, such as brain-machine interfaces, prosthetics and telemanipulation. We anticipate that the confluence of cognitive and computational neuroscience, control theory and wearable, unobtrusive bioinstrumentation will provide novel non-invasive approaches or the treatment and neuro-rehabilitation of Parkinson?s disease and will potentially transform our understanding of brain/body interactions. The project draws graduate and undergraduate students across divisions and in the NSF Temporal Dynamics of Learning Center (TDLC) and Institute of Neural Computation (INC) at UCSD participating in interdisciplinary engineering and neuroscience aspects of the design, optimization, and training of largescale neuromorphic systems and their human interfaces. Through outreach channels on campus supported by the TDLC and the NSF Research Experience for Undergraduates (REU), the program will actively pursue increased participation in research and education of the next generation of scientists and engineers.

Brain computer interfaces (BCIs) translate basic mental commands into computer-mediated actions. BCIs allow the user to bypass the peripheral motor system and to interact with the world directly via brain activity. These systems are being developed to aid users with motor deficits stemming from neurodegenerative disease, injury, or even environmental restrictions which make movement difficult or impossible. One popular class of EEG-driven BCI systems is based on imagined movement. In these systems the user interacts with a computer through motor imagery such as the imagination of hand vs. tongue movement. But the ability of users to control such a BCI is very variable, and all the factors involved are not fully understood. For example, EEG signals can change drastically from offline training to online use. Unfortunately, drift in EEG can lead to loss of control of the BCI, which leads to user frustration and further drift of EEG signals from their training baselines.

The PI's goal in this project is to create a more robust BCI system by specifically addressing loss of control and system drift. Her hypothesis is that explicitly training on a signal that incorporates a user's satisfaction and, more importantly, dissatisfaction with the current performance may result in a more natural interface, and thereby lead to a reduction in loss of control and improved system usability and performance. The research will be carried out in three stages. First, active and passive EEG signals of dissatisfaction and satisfaction will be analyzed in a simulated online setting. Next, a real-time online system that recognizes dissatisfaction vs. satisfaction to control 1-D cursor movement will be constructed and system performance compared to that of a standard left/right motor imagery system. Finally, the best working parts of the dissatisfaction/satisfaction system will be integrated with the more standard left/right system, to create a better hybrid system. The (dis)satisfaction signals will be based on actively controlled motor imagery signals, interpreted emotion, and detection of error-like signals.

Broader Impacts: This project has the potential to vastly improve the robustness of EEG-based BCI systems, by responding to natural signals of satisfaction and dissatisfaction, by being resistant to drift, and by naturally taking advantage of frustration which is a common cause of loss of control. By training the BCI to recognize frustration the PI expects to turn this typically negative trait into a positive. The project will support and train an under-represented minority graduate student and a post-doc in this important interdisciplinary area, and it will create projects for under-represented REU participants as well as for high school students through the PI's partnerships with the NSF Temporal Dynamics of Learning Center (TDLC, where she is a member of the faculty governing and admissions committee for the REU program) and the Preuss School (a charter school for low income students with no college educated parent). All software written for EEG signal processing and analysis, as well as data from the experiments, will be made available as add-ons to EEGLAB which is distributed by co-PI Makeig.

Researchers, parents, and teachers all assume that infants and children learn from their social experiences. However, little is known about how social learning happens in infants and children. For example, even in a simple turn-taking game, how do toddlers and adults adapt to one another? How do they fall into a "rhythm," or affect one another's next turn? When and why do toddlers and parents show enjoyment of the game and of one another? We know almost nothing about how toddler's brains produce these social actions and emotions. The proposed research will measure moment-by-moment changes in the behaviors of toddlers and parents playing a turn-taking game, while also recording their electroencephalograms (EEG): changes in electricity on the scalp, partly caused by the brain's neocortex, which controls actions and communication. Two-year-old toddlers and parents play turn-taking games on a touch-screen table. They produce many unscripted social actions, and are sometimes rewarded for their cooperative actions. The positions of toddlers' and parents' hands and heads are motion-captured, so that their EEG can be precisely synchronized with their actions. State-of-the art analyses will derive the most meaningful information from the EEG of both toddler and parent, synchronized with their own actions and with their observation of the other's actions. Advanced statistics will be used to analyze patterns of actions and EEG events throughout the game.

This paradigm will be used to investigate two major topics: First, how do toddlers' brains register not only their own actions but also other people's actions? Toddlers' brain networks for processing people's actions are virtually unknown, and have not been studied during ongoing social interactions. The results will document toddlers' brain responses to perceiving a parent's actions. This might help researchers understand how parents promote children's engagement and learning during face-to-face activities. It might also indicate why some toddlers are more socially responsive than others. Second, the rewards obtained during face-to-face interactions are crucial for children's learning, but almost nothing is known about how their brains respond to rewards during social interactions. The results will document toddlers' and adults' behavior and brain responses to rewards, especially rewards for cooperation. This will help researchers understand how toddlers develop social skills for learning in social contexts. These skills are critical when children begin school. In sum, a new paradigm for measuring time-locked brain signals and actions of turn-taking toddlers and parents will shed light on the development of skills for social connection and cooperation. The project will also provide training for undergraduate and graduate students, and interns from a charter high school for non-traditional students. The participating laboratories will also host field trips for high school seniors from underserved neighborhoods to learn about neuroscience.

The perception of rhythmic patterns of events in time is central to our ability to find meaning in the sounds of language and music: the basis for much of human culture and communication. We do not passively receive temporal patterns, but actively engage with them by using a repeating 'pulse' or 'beat' to form an essential scaffold for our perception of time. This ability might be most obvious when expressed through dance, or simply tapping a foot to music, but it has deeper importance for how we comprehend sound even in the absence of movement. The scaffold provided by the beat cycle enables listeners to predict upcoming events, allowing more efficient encoding and learning of sensory patterns. How does this important perceptual mechanism work? New evidence suggests that perceiving patterns in sound doesn't depend only on the auditory system, but also involves activation of the motor system, even when the listener is not moving. This proposal tests the provocative and potentially transformative idea that motor planning activity is not only to help us move, but is also necessary for perception of patterns in the sounds we hear. This research has many potential societal benefits in both education and medicine. An understanding of the auditory-motor interactions underlying rhythm perception could explain a growing number of findings suggesting an important link between beat perception and language, including the development of reading in children, the perception of speech in noise, and attention, and may help drive improved educational interventions. The results could also provide a brain-based explanation for the growing use of rhythmic music in the treatment of movement disorders such as Parkinson's disease and possibly guide development of enhanced therapies and diagnostic tests.

This proposal addresses a critical, and difficult, open question within auditory cognitive neuroscience: Does motor activity play a causal role in beat perception and if so, what is that role? Establishing this would be a transformative breakthrough in our understanding of the perception of time. While there is strong existing evidence that motor regions are active during beat perception, the dynamic functioning and interaction among parts of the cortical network supporting beat perception is not fully understood. In particular, a causal role of motor activity on auditory processing has not yet been demonstrated directly. This program of research directly examines whether motor planning regions influence processing in auditory cortex and whether a dynamic network is activated during beat perception. To achieve these objectives the investigators use two interlocking approaches: 1) Advanced quantitative methods of cortical source-resolved electroencephalographic (EEG) brain dynamics during beat perception tasks to identify regions in the brain whose activity patterns mirrors the endogenously perceived beat and to examine the directional flow of influence between these beat perception areas and other auditory processing areas; 2) Non-invasive transcranial magnetic stimulation (TMS) to transiently suppress and/or facilitate activity in beat perception areas.

? DESCRIPTION (provided by applicant): Attention-Deficit Hyperactivity Disorder (ADHD), one of the most prevalent child psychiatric disorders, is associated with significant long-term impairment. Despite extensive research, the causes and brain basis of ADHD remain poorly understood. Attempts to delineate `core deficits' in ADHD have remained elusive, in part because of the heterogeneous nature of the disorder but also because of relative weaknesses in methods so far used to characterize cognitive function in children with ADHD. We propose a new EEG source imaging approach that identifies independent sources of EEG information in identifiable cortical areas and permits highly time-resolved network analysis, both at the group and individual levels. We will apply this approach to a large set of NIMH-funded existing EEG and behavioral data collected from children with and without ADHD. Our goal is to develop effective biomarkers that can both improve ADHD diagnosis and to advance the broader NIMH goal of better understanding ADHD pathophysiology at a non-categorical, individual subject level by identifying the position occupied by each ADHD subject in a broad landscape of individual differences linking brain function and symptomology. This will be possibly the most comprehensive look to date at EEG cortical network activation during cognitive performance in children. This comprehensive assessment, with near- millisecond time resolution, in a large sample will clarify the mechanisms underlying cognitive deficits in ADHD. The results will test and demonstrate the ability of emerging EEG source imaging to better characterize individual and group differences in brain and behavior. If successful, this new approach may enable more sensitive diagnosis, individualized treatment, and treatment monitoring for ADHD, and could be applied to study of other psychiatric pathologies, both by mining large existing but still under-exploited EEG data sets and by informing new study designs and analyses.

How do we learn our way around a new city, building, or other environment? Spatial learning occurs as we integrate the visual, auditory, and other sensory impressions we gather as we move through the new environment. This collaborative project will investigate the brain dynamics of human participants as they actively navigate new types of laboratory mazes. The goal is to observe and model, for the first time, the distributed brain dynamics that support spatial learning during active human navigation. To do this, the investigators will use an original, non-invasive "mobile brain/body imaging" (MoBI) data recording approach that combines simultaneous full body motion capture and brain electrical (EEG or "brainwave") recording. Advanced signal processing methods will allow them to use non-invasively recorded scalp EEG data to follow the time courses of electrical brain activity within the cerebral cortex as
subjects actively explore computer-defined mazes. Beyond introducing new methods and software to the field of cognitive neuroscience, the project could enable improved design of living and work environments, development of new and effective approaches to improving spatial navigation abilities of children and people with spatial disabilities, guide better training for and evaluation of first responder operations, enable more effective operation of remote observation vehicles, and spur development of methods to maintain spatial orienting abilities in the elderly.

The project will have three technical objectives: 1) To further develop and exploit a new computational approach to studying brain source network dynamics of freely moving individuals, 2) to observe and model transient cortical network interactions supporting physical navigation in 3-D space, and 3) to examine to what extent sensory information about the spatial structure of the physical environment is processed in brain networks that support modality-specific vs. supra-modal information processing. To accomplish these objectives, the project will capitalize on unique strengths of the collaborators and the unique facilities available in the investigators' pioneering MoBI laboratories. The project will use a novel "Sparse-AR" (perceptually sparse augmented-reality) approach in which information about the participant's spatial environment is delivered in the form of brief bursts of sound or light. The Sparse-AR approach will allow modeling of how our brain activities change as we receive and integrate new bits of evidence about unfamiliar or familiar features of the spatial environment.

A companion project is being funded by the Federal Ministry of Education and Research, Germany (BMBF).

The increasing prevalence of learning disorders, attention deficits, and lackluster appetite for reading across all walks of life, and particularly among school-age children, poses severe problems to humanity and, in the long run, burdens social and economic development. This Partnership for Innovation Building Innovation Capacity (PFI:BIC) collaborative project tackles the impending threats to humanity of illiteracy and faltering education heads-on by creating a new smart-service human-computer interface (HCI) neurotechnology platform as a highly effective, user-friendly, and fun-to-use tool aiding learning and stimulating cognitive development at home and in the classroom. The immersive HCI neurotechnology will allow directly measuring progress at the cognitive level and providing real-time feedback to guide the user in learning to read more effectively. The project is highly Science, Technology, Engineering and Mathematics (STEM) intensive both in its activities and in the targeted benefits of the developed technology, which extends directly to learning science and mathematics by probing cognitive performance of children while they solve puzzles. The development of unobtrusive neurotechnology further addresses a critical need for practical integrated and modular brain-computer interface (BCI) solutions in HCI promoting widespread consumer and clinical use in the marketplace. The partnership provides opportunities for students to gain practical experience in innovation in the marketplace through internships with the industrial partners.

The central aim is to develop and leverage new HCI technology as a learning coach and personal cognitive development assistant that facilitates learning to read and acquiring other critical skills in cognitive development. The immersive yet unobtrusive HCI technology testbed will comprise a dry-electrode electroencephalography (EEG) BCI, a tablet with touchscreen and integrated camera, and a suite of signal processing algorithms running in the cloud, for monitoring brain and gaze activity in children learning to read, and providing real-time neurofeedback on progress in cognitive performance to promote enhanced learning. The partnership will transition scientific advances of a previous NSF-sponsored UCSD project (NSF EFRI-M3C, ENG-1137279) in studying the distributed dynamics of human motor control, to development of neurofeedback training paradigms for learning enhancement, and to practical deployment on the unobtrusive immersive testbed implemented using Cognionics dry-electrode EEG wireless BCI neurotechnology and Syntrogi real-time cloud-based signal processing software pipelines. The potential for human empowerment by the technology will be demonstrated by evaluating effectiveness in enhancing learning capabilities and cognitive performance in simulated classroom settings and other targeted learning environments.

The lead institution for the project is University of California San Diego (UCSD), with investigators from the Institute for Neural Computation and Department of Bioengineering. The industrial partners in the effort are Syntrogi Inc. (dba Qusp, small business, San Diego CA) and Cognionics, Inc. (small business, San Diego, CA). The project also engages broader context partners Drs. Andrea Chiba and Leanne Chukoskie from the UCSD Temporal Dynamics of Learning Center, Dr. Barbara Moss from San Diego State University Department of Psychology, and Dr. Zewelanji N. Serpell from Virginia Commonwealth University Department of Psychology, in the human case studies and the assessment of the developed HCI technology in diverse learning environments.

Electroencephalography (EEG), the first function brain activity imaging modality, has several natural advantages over metabolic brain imaging modalities. EEG is noninvasive, low cost, and lightweight enough to be highly mobile. Two major shifts in scientific perspective on the nature and use of human electrophysiological data are now ongoing. The first is a shift to using EEG data as a source-resolved, relatively high-resolution cortical source imaging modality. The second is a shift from recording electrophysiological data with at best a scant record of behavior (e.g., latencies of occasional button presses) to concurrently collecting and combining EEG data with other data modalities (e.g., body motion capture, eye tracking, audio and video, ECG, EMG, GSR, MEG, fMRI, etc.), paradigms that we term Mobile Brain/Body Imaging (MoBI) to capture brain activities and subject actions during natural, motivated behavior.The EEGLAB signal processing environment, an open source software project of the Swartz Center for Computational Neuroscience (SCCN) of the University of California, San Diego (UCSD), began as a set of EEG data analysis running on Matlab (The Mathworks, Inc.) released by Makeig on the World Wide Web in 1997. EEGLAB was first released from SCCN in 2001. Now nearly twenty years later, the EEGLAB reference paper (Delorme & Makeig, 2004) has over 4,100 citations (now increasing by over 3 per day), the opt-in EEGLAB discussion email list links over 5,500 researchers, the EEGLAB news list over 15,400 researchers, and a survey of 687 research respondents reported EEGLAB to be the software environment most widely used for electrophysiological data analysis in cognitive neuroscience. Currently, at least 52 EEGLAB plug-in tool sets have been released by other researchers from many laboratories. Here we propose, first, to greatly augment the power of the EEGLAB environment by enabling it to perform time series, biophysical, and statistical analyses of multimodal as well as unimodal EEG data. However, ever more precise analyses of large and multimodal data sets and studies require increasing amounts of computational power, more than is readily available in many laboratories. Thus second, in collaboration with the San Diego Supercomputer Center (SDSC) we propose to expand the current Neuroscience Gatew? ay (?nsgportal.org) services to enable EEGLAB users to freely run EEGLAB processing scripts and pipelines on SDSC supercomputers. The proposed Open EEGLAB Portal will allow researchers to submit any amount of unimodal or multimodal EEG data for parallel processing using standard or custom EEGLAB processing pipelines. We will also develop and release first tools for meta-analysis of source-resolved EEG measures ?across studies. Multimodal EEG analysis and source-level EEG analysis accelerated by free use of supercomputing resources will give the EEG research community unprecedented abilities to observe and model distributed cortical dynamics supporting human experience and behavior.

To take advantage of recent and ongoing advances in intensive and large-scale computational methods, and to preserve the scientific data created by publicly funded research projects, data archives must be created as well as standards for specifying, identifying, and annotating deposited data. The value of and interest in such archives among researchers can be greatly increased by adding to them an active computational capability and framework of analysis and search tools that support further analysis as well as larger scale meta-analysis and large scale data mining. The OpenNeuro.org archive, begun as a repository for functional magnetic resonance imaging (fMRI) data, is such an archive. We propose to build a gateway to OpenNeuro for human electrophysiology data (EEG and MEG, as well as intracranial data recorded from clinical patients to plan brain surgeries or other therapies) ? herein we refer to these modalities as neuroelectromagnetic (NEM) data. The Neuroelectromagnetic Data Archive and Tools Resource (NEMAR) at the San Diego Supercomputer Center will act as a gateway to OpenNeuro for NEM data research. Such data uploaded to NEMAR at SDSC will be deposited in the OpenNeuro archive. Still- private NEM data in OpenNeuro will, on user request, be copied to the NEMAR gateway for further user processing using the XSEDE high-performance resources at SDSC in conjunction with The Neuroscience Gateway (nsgportal.org), a freely available and easy to use portal to use of high-performance computing resources for neuroscience research. Publicly available OpenNeuro NEM data will be able to be analyzed by running verified analysis applications on the OpenNeuro system. In this project we will build an application to evaluate the quality of uploaded NEM data, and another to visualize the data, for EEG and MEG at both the scalp and brain source levels, including time-domain and frequency-domain dynamics time locked to sets of experimental events learned from the BIDS- and HED-formatted data annotations. The NEMAR gateway will take a major step toward applying machine learning methods to a large store of carefully collected and stored human electrophysiologic brain data to spur new developments in basic and clinical brain research.