Ken A. Paller - US grants

The major objective of this research is to advance our understanding of the brain mechanisms responsible for human memory. The primary memory functions under study are those responsible for episodic recollection, the type of memory used to perform standard tests of recall and recognition. Results from neuropsychological studies of organic memory disorders have been used to (a) identify brain structures that are crucial for this memory function and (b) show that episodic recollection is biologically distinguishable from other types of memory - an insight that provides a key foundation for the proposed experiments. These experiments make use of a new technique that allows electrophysiological correlates of episodic recollection to be derived from EEG recordings in neurologically normal subjects. With this technique it is possible to monitor recollection in the absence of any concurrent behavioral indication to that effect. Recordings are made while words are presented under explicit or implicit memory testing conditions. Some of the words are also presented earlier in the experiment, but in implicit memory tests subjects are not informed that memory is being tested. Specialized performance measures in these implicit memory tests are sensitive to the influence of prior word presentations, an outcome referred to as "priming." In addition, subjects in these circumstances regularly engage in "incidental recollection," which is defined as recollection not required by task instructions. The extent to which subjects engage in conscious recollection can be systematically controlled using various experimental procedures. Behavioral measures demonstrate that certain procedures influence recall and recognition, but do not influence priming, whereas other procedures yield the opposite pattern. These memory dissociations support the conception of recollection and priming as two different types of memory. Furthermore, preliminary results showed that recollection was associated with a distinct pattern of brain electrical activity recorded from the scalp. Priming was associated with another pattern of brain electrical activity that occurred at an earlier latency and was more focally distributed over occipital brain regions, consistent with the fact that priming occurred within the visual modality. Further experiments will focus on three specific aims: (1) to provide converging evidence with several experimental methods supporting the contention that conscious recollection is associated with a measurable electrophysiological marker, (2) to determine whether this phenomenon is a general one that can be found using either implicit or explicit memory tests, auditory or visual stimuli, linguistic or pictorial information, and so on; and (3) to compare and contrast electrophysiological measures of recollective processes with those associated with priming so as to learn more about the distinction between these two types of memory and the responsible neural mechanisms. These new on-line measurements of the electrical activity accompanying recollection and priming will thus be useful for monitoring the time-course of memory functions and developing a better understanding of their cognitive and neural organization.

Humans are amazingly proficient at rapidly perceiving facial identity, extracting facial expressions of emotion, and remembering individual physiognomies. These skills are critical for effective human interaction. This vital realm of cognitive expertise is situated at the intersection of memory and perception. However, the fact that many memory processes occur concurrently when we see a familiar image places serious constraints on their investigation. The next stage of advancement in memory research requires disentangling the plethora of processes that in combination influence memory performance. A cognitive neuroscience perspective on this problem is emphasized in this research project, which takes the investigation of facial memory as key for elucidating perceptual expertise and memory in general. The exquisite temporal precision of EEG measures is combined with sophisticated behavioral measures of memory to allow multiple memory processes and corresponding neurophysiological signals to be characterized independently. The specific memory phenomena under investigation include: (1) Remembering a person's face together with biographical information about that person and relevant personal experiences; (2) Recognizing a face as familiar in the absence of any additional memory retrieval; (3) Facilitated processing of a facial image ("priming") due to prior visual experience with the same face and in the absence of conscious remembering; (4) Similar unconscious memory based on conceptual rather than perceptual knowledge; and (5) Unconscious influences due to emotional facial expressions. Behavioral and electrophysiological results obtained as these 5 phenomena are dissociated from each other will help to sharpen theoretical conceptions of these complex memory phenomena by providing neural validation of these cognitive distinctions. Independent neural signals being characterized in this research are intended to show how these processes unfold in time when a face is viewed. These measures are opening the door for further inquiry into the fundamental characteristics of various memory phenomena, including conscious and nonconscious memory more generally. Electrical signals of memory, once identified, can then be used to determine which memory functions are operative in different situations, and how the various components operate interactively. Fruitful cross-disciplinary extensions include investigations of these memory phenomena in elderly and patient populations who show cognitive impairments in a subset of relevant storage and retrieval processes. EEG analyses will be used to generate more detailed hypotheses about neural processes and structures, which can then be tested using other neuroimaging methods, both with respect to accurate and inaccurate memory.

A comprehensive scientific understanding of facial memory will have wide-ranging impact. By fractionating multiple aspects of facial memory, the research findings will provide insights into conceptions of various memorial influences on behavior, including but not limited to learning and memory in the classroom, in a court of law, in the acquisition of various cognitive skills, and in everyday interpersonal interactions. This project will publicize research to the general public, bring underrepresented groups into scientific research, train post-doctoral, graduate, and undergraduate students, and develop implications for treating disease-related memory disorders and normal memory decline in aging. Better use of eyewitness testimony, of computerized facial identification, and of memory for people generally, will come from improved comprehension of the multiple facets of facial memory examined and characterized in the research.

Project Summary This proposal describes the continuation of the Training Program in the Neuroscience of Human Cognition. A group of 25 faculty preceptors?investigating perception, attention, memory, emotion, problem solving, language, cognitive control, and the like?has been assembled to participate in training the next generation of cognitive neuroscientists at Northwestern University. The training program is led by Ken Paller (Director), Marsel Mesulam (Associate Director), and an Internal Steering Committee, with input from an External Advisory Committee. Each year, pre- and postdoctoral trainees are selected through a highly competitive process on the basis of proposed research, scientific potential, and excellence of prior scientific training. Funding from the Northwestern Graduate School is also provided to 3 predoc Affiliate Trainees per year. Trainees conduct their research under the guidance of one or more preceptors affiliated with many different departments and centers within the university. In order to provide thorough training in all facets of Cognitive Neuroscience, the training program brings together a diverse set of perspectives to facilitate a broad range of methodological, computational, and theoretical endeavors. Exceptional opportunities are provided for trainees to learn from and closely interact with local and visiting faculty members, and with other trainees, in the service of expanding their capacity for creative research on the neural substrates of cognition. Trainees often bridge the distinct areas of expertise of two mentors through novel interdisciplinary collaborations, thus expanding the breadth of both their training and of Cognitive Neuroscience research at Northwestern. Trainees gain from in-depth research discussions with a network of other scientists outside their home laboratory and department, including regular presentations (and critique) of their own work. Concerted efforts focus on including under-represented individuals and on fostering a diversity of perspectives, which also enriches the whole program. This T32 funding mechanism facilitates innovative strategies and novel research orientations, and solidifies a sense of community for all involved in human cognitive neuroscience at Northwestern. Value-added benefits of the program also include a rich array of activities that supplement standard research training, providing trainees with opportunities to expand their training outside of the home laboratory and beyond departmental boundaries. The community also facilitates training in specific research skills across individuals and supplies generally relevant knowledge to help trainees gain independence and become better scientists. Faculty preceptors monitor training through formal advising and evaluations, with ample examples of written and oral scientific communication, and with attention to responsible conduct and all aspects of career development. A chief goal of the program is to provide top-rate comprehensive training to young scientists who will become future leaders in human cognitive neuroscience, ultimately bringing their skills to bear on a variety of scientific endeavors pertinent to health and disease.

The ability to accurately recognize information that was previously learned is central to human success. This ability relies on a complex set of brain mechanisms. Traditionally, human memory is subdivided into memory with awareness of retrieval, or explicit memory, and memory without awareness of retrieval, or implicit memory. Surprisingly, it is possible for a person to respond with high accuracy on a memory test in the absence of explicit feelings of familiarity or remembering. With support from the National Science Foundation, Dr. Ken Paller and colleagues at Northwestern University will characterize the circumstances under which implicit recognition is possible. Implicit recognition is found when people have negligible confidence about their memory, whereas explicit memory shows the reverse pattern. Implicit recognition is stronger for stimuli that are learned under conditions that place severe limitations on information processing, even though these conditions lead to very poor explicit memory. By recording brain activity from electrodes placed on the scalp while people perform memory tests, Dr. Paller will investigate which specific brain potentials are associated with implicit memory and which are associated with explicit memory. The results of this research will provide new information about mechanisms of implicit and explicit memory and about how both types of mechanism can drive accurate memory judgments.

This research project will provide training opportunities for undergraduate, graduate, and post-doctoral trainees in cognitive neuroscience. Specific findings are expected to be disseminated widely and will lead to numerous insights into how explicit and implicit memory influences everyday behavior. Such knowledge will be important for enhancing our understanding of learning and memory in the classroom, in a court of law, in the acquisition of various cognitive skills, and in interpersonal interactions. The work will also aid efforts to devise rehabilitation strategies for people with memory difficulties due to aging or neurological disorders.

The ability to accurately remember information is central to human success. Yet, as new information is added every day, the human brain's expanding storehouse of memories requires continual restructuring. How can the massive influx of new information be used to best advantage? With funding from the National Science Foundation, Ken Paller, Ph.D., of Northwestern University is investigating the intriguing hypothesis that brain events during sleep are instrumental for effective memory storage. Although some memories rapidly fade away, others endure for years, presumably because of a progression of changes in brain networks. It remains unclear how memories are enduringly stored in the human brain, and to what extent sleep might be helpful. This project explores a novel way in which memory storage can be improved. Specific sounds related to information learned previously are presented softly while an individual sleeps. These sounds can in theory prompt memory rehearsal during sleep, leading to an improvement in memory tested a short time after awakening. The procedures thus provide an opportunity to observe a potential benefit of sleep -- memory strengthening that could underlie lasting memory storage. This project explores the procedures whereby these "sleep-sounds" are effective, including (a) determining whether long-lasting memory improvement can be produced by sleep-sounds, (b) comparing learning of spatial locations, of words, and of specific manual actions, and (c) showing which stages of sleep are amenable to this memory strengthening. Experiments also compare the memory strengthening that is possible when participants are awake versus asleep. Electroencephalographic measures of brain activity are included in the project to further elucidate the brain mechanisms of memory improvement during sleep. Although it is widely thought that new information cannot be learned during sleep, information learned before going to sleep can apparently be rehearsed during sleep. This project provides new evidence about how a period of sleep can lead to memory improvement and how that improvement can be focused on specific information.

Results obtained through this project provide new leverage on understanding memory storage in general, and memory processing during sleep in particular. This project is expanding knowledge about the special role that sleep can play in the stabilization of memories and in the formation of interconnections among these memories that can be the basis for creative thinking. By advancing understanding of how memories are strengthened during sleep, implications will ultimately arise for promoting optimal educational outcomes in the classroom, enhancing the daily acquisition of new information and skills, and evaluating memory in legal contexts, among other applications. An appreciation of sleep and learning will also aid efforts to emphasize the importance of suitable amounts of sleep for public health.

Coercion attacks that compel an authorized user to reveal his or her secret authentication credentials can give attackers access to restricted systems. The PIs are developing a new approach to preventing coercion attacks using the concept of implicit learning from cognitive psychology. Implicit learning refers to learning of patterns without any conscious knowledge of the learned pattern. Using a carefully crafted keyboard-based computer game the PIs plant a secret password in the participant's brain without the participant having any conscious knowledge of the trained password. This planted secret can be used for authentication, but participants cannot be coerced into revealing their secret since they have no conscious knowledge of it.

This project explores three directions for using implicit learning in computer security. First, the PIs are developing implicit learning tasks designed to be used in challenge-response authentication. Second, the PIs are experimenting with methods to demonstrate implicit knowledge by measuring electrical activity along the scalp using off the shelf EEG devices. Third, the PIs are conducting user experiments to demonstrate that participants are able to properly authenticate, but cannot consciously recognize the trained secret. This project is a collaboration between computer security researchers and cognitive psychologists. Ultimately, the project aims to understand how the brain represents implicit knowledge. This in turn will lead to new coercion resistant security mechanisms for high-security applications.

The ability to remember is central to human success. Whereas most memory research focuses on the formation and/or recall of memories, events intervening between the acquisition of new information and its subsequent retrieval may play a central role in shaping memory storage and determining what can be remembered. In particular, recent research suggests that memories are reactivated during sleep. This insight has led researchers to hypothesize that reactivation during sleep may have a profound influence on whether a memory persists or fades away. However, investigating this question in humans has been hindered by difficulties in measuring reactivation. Furthermore, merely measuring reactivation is insufficient for showing that reactivation improves memory. To establish causality, one must directly manipulate reactivation instead of just observing when it happens. An overarching objective of this project is thus to monitor, characterize, and manipulate memory reactivation during sleep. These efforts at the intersection of memory research and sleep research will expand our understanding of how memories are stored in the brain, providing knowledge that can then guide efforts to improve learning and memory in a variety of contexts, including in education, in the legal system, in public service training, for individuals with memory dysfunction due to disease, for older individuals with age-related memory decline, and for a wide variety of on-the-job uses of memory. The research will also highlight the relevance of sleep for effective learning, bolstering societal appreciation of the need for sleep.

This project will take the innovative approach of combining auditory stimulation during sleep with extensive analysis of EEG recordings. Through systematic arrangement of learning and testing procedures, auditory stimulation will function as a prompt for memory reactivation in the brain while relevant neural activity is simultaneously monitored. These procedures make it possible to analyze neurocognitive processing during sleep and to relate this processing to later memory performance. By manipulating which memories are cued during sleep, causal inferences about how reactivation affects stored memories will be made. Dr. Paller and his team will focus on established physiological signals (e.g., N400 and sleep spindles) as well as on multivariate analyses. By applying sophisticated pattern classifier analyses to sleep EEG data collected immediately after the auditory cues are played, a time-varying neural measure of how strongly the associated memory is reactivated during sleep will be derived. This neural signature of memory reactivation can be related to features of sleep physiology and also subsequent memory to shed light on exactly which aspects of sleep promote strong memory reactivation and, ultimately, accurate recall. This work thus provides a powerful new set of approaches for investigating the fundamental brain events that enable memory storage to be enduring for the vast amount of information we all need to remember.

Identifying the cognitive, computational and neural mechanisms responsible for determining why some memories survive when others fade is one of the many grand challenges facing researchers of the human mind and brain. It is widely understood that sleep plays a critical role in long-term remembering, yet what exactly happens during sleep to affect the persistence of memories remains largely unknown. This project brings together a team of researchers who will integrate multiple independent lines of work in cognitive neuroscience, cognitive psychology, and computer science in order to investigate the precise mechanisms undergone by recently-formed memory representations as a person sleeps and how these mechanisms determine which memories survive and which fade. The proposed integration of cutting-edge neural data analysis methods for EEG and neuroimaging data, basic human memory theory, and neural network modeling make possible the ability to non-invasively track individual memories in the human brain as they compete with each other and are modified during sleep. The potential advances from this work could impact education, training situations, and public health by facilitating the development of new strategies for ensuring that important memories survive after initial learning.

Research suggests that memories compete for neural space such that reactivating one particular memory can exert "collateral damage" on other related memories. In other words, accessing one memory can come at the expense of later being able to access other nearby memories in the network space. The proposed studies test the hypothesis that importance shapes neural dynamics during sleep by selectively boosting memory reactivation; this boost ensures that important memories out-compete related memories during sleep, resulting in strengthening of important memories and weakening of less-important memories. To test this hypothesis, competition between memories will be elicited during sleep by playing sound cues, each of which was linked (during wake) to two different picture-location memories. Multiple interlocking approaches will track how memory competition during sleep shapes a memory's persistence versus fading. Neural network models will be used to generate predictions about how reward responses during encoding shape competitive dynamics during sleep, and how these competitive dynamics determine the eventual fates of competing memories. Predictions will be tested by using fMRI to measure neural activity associated with reward processing during encoding, EEG to measure brain activity during sleep, and pattern classifiers to decode memory activation from the sleep EEG data. Observations of competitive dynamics during sleep will then be related to later memory performance and to multivariate fMRI measures of memory change. The project has the potential to provide, for the first time, a comprehensive look "under the hood" at the life of a memory as it is acquired, processed during sleep, and eventually recalled. Pivotal knowledge will be gained about how variance in reward processing at encoding influences sleep replay dynamics, and about how sleep replay dynamics affect subsequent memory performance and the structure of neural representations.

Sleep is crucial to the body's ability to function but scientists have yet to uncover all the reasons for why we sleep. Recent evidence shows that sleep contributes to memory, although how that happens is not yet well understood. This project explores the idea that the ability to remember depends on sleep-based changes in how memories are stored. The investigators have the special opportunity to obtain recordings of brain activity from sensors placed inside the human brain. In some patients with frequent epileptic seizures, when pharmacological treatments are ineffective, a surgical treatment can greatly enhance quality-of-life. Sensors placed in selected brain locations can isolate the source of abnormal activity, after which this brain tissue can be removed. These patients are usually willing and able to help the investigators understand what happens during sleep to preserve memories. The investigators will explore how the reactivation of new memories during sleep improves the ability to remember the information. This type of memory processing during sleep could constitute an essential ingredient for learning. Results from this project may have implications for educational advances and for approaches to improving memory across the lifespan. The investigator plans activities to increase public understanding of science and to bolster recognition of the vital benefits of sleep.

A key hypothesis in the field is that a brain region known as the hippocampus functions during sleep to help consolidate memories. This can entail linking anatomically disparate components of a memory, gradually integrating new knowledge with prior knowledge, securing memory storage, and potentially transforming what is stored. The investigators will analyze recordings of brain activity from functional brain tissue to determine how memory storage is altered when audio input during sleep promotes memory reactivation. Brain oscillations identified previously (so-called slow oscillations, sleep spindles, and hippocampal ripples), and their precise temporal interrelationships, are hypothesized to be critical. Analyses of these signals obtained during memory reactivation are thus positioned to provide new knowledge about neural mechanisms of sleep-based memory improvement, thus advancing fundamental understanding of what allows a small subset of the massive number of memories people acquire each day to survive for the long term.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Stroke is the largest cause of major disability. This disability most often results from persistent arm impairment. A significant portion of arm impairment is caused not by weakness or spasticity, but by abnormal co-activation among arm muscles. This coordination dysfunction is pervasive in the most severely impaired patients, who are most in need of new therapies. To treat abnormal muscle co-activation, we developed a myoelectric-computer interface (MyoCI). In addition, we pioneered the use of targeted memory reactivation (TMR) to enhance memory consolidation during sleep. The long-term goal of this research is to develop an affordable, non- invasive, and easy-to-use combination of MyoCI and TMR that improves control of arm movements by reducing abnormal co-activation. Our preliminary studies show that TMR enhances consolidation of MyoCI learning in a single nap in a group of healthy individuals, and across several nights in three stroke survivors. Accordingly, we propose to determine whether this training-plus-sleep combination will generalize to improve motor function over an extended training protocol in stroke survivors. The objectives of this proposal are 1) to determine whether TMR can augment motor learning after stroke, and 2) to determine optimal parameters for the MyoCI+TMR paradigm to enhance motor function in stroke survivors. Our central hypothesis is that supplementing MyoCI training with TMR will augment learning considerably and thereby improve arm movement. We will test this hypothesis via the following specific aims: 1) Test the extent to which TMR during SWS enhances MyoCI learning after stroke, 2) Assess the ability of TMR across all sleep stages to enhance MyoCI learning after stroke, and 3) Assess the influence of TMR dose and stroke location on MyoCI learning. This proposal?s innovative combination of wearable, inexpensive, and noninvasive MyoCI+TMR will enable us to test the effects of TMR on motor learning after stroke. Achieving our objectives will be significant because it will facilitate the development of an enhanced rehabilitative therapy to improve function after stroke that could be used widely and could help the most severely impaired stroke survivors. We expect that the paradigm will be synergistic with other therapies, given its novel mechanism of action (reducing co-activation using myoelectric signals). The research will also provide basic information about what brain areas are critical for consolidating motor learning. We further expect that TMR could be applied to other types of stroke rehabilitation in addition to MyoCI. Finally, this project will provide critical information needed to plan larger clinical trials to assess efficacy of this and related approaches.