Daphna Shohamy - US grants

DESCRIPTION (provided by applicant): The long-term goal of this project is to understand the interaction between brain systems subserving different aspects of memory function. Decades of neuropsychological and behavioral studies have emphasized the dissociation between distinct forms of memory (such as non-declarative vs. declarative memory), each supported by distinct anatomical and functional systems. The studies proposed here will use functional imaging (fMRI) to begin exploring how and when such systems may interact in the healthy brain. The studies will focus on the interaction between incremental stimulus-response learning (a form of non-declarative learning) and episodic memory (a form of declarative memory). Incremental learning is thought to depend critically on the basal ganglia (BG), while episodic memory is supported by the medial temporal lobes (MTL). The proposed research program aims to examine how the BG and MTL interact during memory formation, and the implication of this interaction for memory function. The resulting knowledge is expected to enhance our understanding of memory processes in the intact brain, as well as for understanding the nature of memory impairments in various neurological diseases, including Parkinson's and Alzheimer's disease.

DESCRIPTION (provided by applicant): Habits are powerful in driving actions. In substance abuse, habits formed by repeated hedonic experience with a drug are more powerful than explicit knowledge of the drug's detrimental effects. Recent research in the cognitive neuroscience of learning and memory has greatly advanced our understanding of these two forms of learning, suggesting they depend on distinct neural systems. Yet there has been remarkably little progress in extending this line of research to the study of addiction. Here, we propose to bridge this gap to understand the neural mechanisms contributing to different forms of learning and motivation, and their disruption in addiction. Research demonstrates that habit learning depends on the striatum and its dopaminergic inputs, while explicit memory for facts and events, often referred to as declarative memory, depends on the medial temporal lobe (MTL;hippocampus and surrounding cortices). Recent evidence suggests these systems may interact competitively under some circumstances. The core hypothesis motivating this proposal is that substance abuse may be related to a disrupted balance between these two learning systems and the extent to which each of them guides behavior. The proposed research will use functional imaging (fMRI) and behavioral analyses to investigate how different forms of learning guide decisions and actions. The central aim is to understand how to modulate the contribution of these two learning systems to choice behavior, at both the neural and behavioral levels. In the first study, we will examine the effects of modulating striatal contributions to learning and choice. To that end, we will manipulate reinforcement, and will determine the effects on the cognitive and neural systems driving subsequent choice behavior (Expt 1). In the second study, we will examine the effects of modulating MTL contributions to learning and choice. To that end, we will manipulate stimulus and associative novelty, and will determine the effects on the cognitive and neural systems driving subsequent choice behavior (Expt 2). Determining factors that modulate the extent to which learning and choice depend on one system or the other will lay the foundation for future translational work on potential treatment interventions that can reduce the dependence of behavior on the striatal habit system in favor of the MTL declarative system. PUBLIC HEALTH RELEVANCE: Substance abuse is a serious problem of public health. Disruption to the brain and cognitive mechanisms underlying learning are central to the pathology of substance abuse and addiction. In substance abuse, individuals'behavior is driven by habits that are learned as a result of repeated experiences with the drug's hedonic, reinforcing effects. These learned habits override any other forms of learning, such as explicit knowledge or experiences of the drugs'detrimental effects. The proposed studies aim to understand the neural and cognitive mechanisms underlying the balance between habitual and explicit forms of learning, and how this balance can be modified. The resulting knowledge will inform future work on potential cognitive and pharmacological treatment strategies that can reduce the dependence of behavior on habits in favor of explicit, goal-directed mechanisms.

Learning is essential to behavior, enabling organisms to draw on past experience to improve choices. A fundamental question is how different brain systems involved in learning interact to support adaptive decisions. With funding from the National Science Foundation, Daphne Shohamy is addressing this fundamental question. One way in which people gradually learn the relationship between actions and outcomes is by using trial-by-error feedback. This type of learning has traditionally been defined as implicit or habitual, and it is thought to depend on a neural structure called the striatum. A distinct and independent learning system is thought to uniquely support explicit memory for facts and events and to depend on the hippocampus. Emerging data suggest that this dual-system view of memory is over-simplified. This research program explores how different brain systems for learning interact and jointly guide behavior. The project adopts an integrative approach that draws on functional magnetic resonance imaging (fMRI) studies in healthy individuals, combined with studies of learning in patients with isolated damage to specific brain systems. Imaging studies provide insight into the spatial and temporal characteristics of brain and cognitive mechanisms. Patient studies augment evidence of the necessity of a system for specific learning processes. By integrating functional imaging and patient research, results from this project will contribute to a deeper understanding of the role of the striatum and the hippocampus in learning, and of how these learning systems interact to guide adaptive behavior.

This research program provides unique opportunities for training and education at both the undergraduate and graduate levels. Integrating multiple core tools of cognitive neuroscience (functional imaging, patient studies, behavioral analyses), students are trained in different approaches to a unified research question. Through this training, they learn to value the principle of converging methods, a theme that is further emphasized in a program of regular joint meetings with students and faculty in laboratories with complementary expertise. Determining the brain and cognitive mechanisms underlying different forms of learning will lay the foundation for potential future interventions to improve education and treat learning deficits, both in healthy individuals and in individuals with brain disorders. To facilitate this translational process, the research program forms the basis of a program of public outreach about the relevance of research on learning and the brain for education and public health. Public lectures and an undergraduate course on cognitive neuroscience in the media focus on broad communication of the relevance of cognitive neuroscience research for many aspects of society, including education, law and medicine.

DESCRIPTION (provided by applicant): Habits can exert powerful control over behavior. Sometimes habits are maladaptive, such as in substance abuse. In many cases, however, habits can be adaptive, enabling organisms to engage in healthy choices rapidly and efficiently. In recent years there has been substantial progress in understanding the neural mechanisms that support the transition from effortful, goal-directed to more automatic, habitual behavior through research in rodents. This work has emphasized the role of dopamine and its striatal targets in habits and has begun to suggest the principles by which their strength is controlled. Surprisingly, however, the rodent research has led to only limited progress in understanding the balance between goal-directed vs. habitual behaviors in humans, preventing the scientific community from being able to apply this knowledge to clinical settings and everyday life. We hypothesize that the main reason for this gap is the lack of rich and precise behavioral markers for habitual behavior in laboratory settings in humans. The proposed research program aims to address this gap. We propose a series of functional imaging (fMRI) and neuropsychological studies that focus on computational characterization of distinct types of learning and the factors that impact their relative strength. Because both habits and goal-directed behaviors depend on past experience, we can leverage recent advances in characterizing brain systems for learning and memory in humans that have already led to a rich, quantitative characterization of multiple aspects of behavior and neural signaling. Using these graded and dynamic signatures of habit formation - specifically, by examining choices and choice-related fMRI signals during trial-and-error learning, and during subsequent probes of the memories formed - we propose to ask a series of questions collectively aimed at uncovering the mechanisms by which the brain balances goal-directed and habitual behaviors. Our specific aims are: (1) To determine how the capacity to form flexible memory representations modulates goal-directed and habitual systems (2) To understand how the timing of feedback modulates the balance between the systems; (3) To determine how the reliability of feedback modulates goal-directed and habitual systems. For each aim, we test healthy subjects with fMRI to determine the dynamic contribution of multiple brain regions to different aspects of behavior. Parallel studies with Parkinson's patients complement the fMRI studies and provide evidence about the causal role of dopaminergic inputs to the striatum in habit learning. Results from this research will advance understanding of the behavioral and neurobiological mechanisms that control the emergence of habits and the implications for decisions and actions. Determining how multiple brain systems for learning support the transition of behavior from goal-directed to habitual control will lay the foundation for future translational work on potential treatment interventions that can adaptively shift behavior towards the formation of healthy habits in both health and disease.

DESCRIPTION (provided by applicant): What aspects of previous experiences guide decisions? Much research concerns how the brain computes the average, over many experiences, of rewards received for an option. But such a summary - produced by prominent models of dopaminergic incremental learning- is chiefly useful for repetitive tasks. Much less is understood about how the brain can flexibly evaluate new or changing options in more realistic tasks, which must rely on less aggregated information. This application argues that this is fundamentally a function of memory, so this project looks to the brain's memories for the most individuated experiences - episodes - to seek new computational, cognitive and neural mechanisms that could support more flexible decisions. The overarching hypothesis is that episodic memory, supported by the hippocampus, plays a central role in guiding flexible decision making and complements the wellknown role of dopaminergic and striatal systems in incremental learning of value. What is the intellectual merit of the proposed activity? By connecting the computational neuroscience of decision making with the cognitive neuroscience of memory, and bringing together collaborators from each area, this project promises to shed light on both areas. This is because the neural mechanisms supporting episodic memory are well studied, but less so their contribution to adaptive behavior. Computationally, episodic memories can support a family of learning algorithms that draw on sparse, individual experiences, such as Monte Carlo and kernel methods. These suggest novel, plausible hypotheses for how the brain solves more realistic decision problems, and in particular how it implements goal-directed or model-based choices. The proposed studies aim to differentiate the contributions of incremental and episodic learning to value-based decisions, and test to what extent episodic memories contribute to decisions previously identified as model-based. Our hypotheses are tested fitting computational models to neural activity from functional MRI experiments in humans, and also to choice behavior in healthy individuals compared to patients with isolated damage to specific neural systems. This combination of computational, neuroimaging and neuropsychological approaches permits finely tracing the trial-by-trial dynamics of learning as reflected both in brain activity nd behavior, and also testing the causal role of particular brain regions in these same processes. What are the broader impacts of the proposed activity? A striking range of psychiatric and neurological disorders, including Parkinson's disease, schizophrenia and eating disorders, are accompanied by aberrant decision-making and by dysfunction in circuitry central to this proposal, such as striatal and fronto-temporal mechanisms. But understanding such dysfunction requires a better understanding of how each of these circuits separately influences decisions. A focus on untangling multiple decision systems is particularly pertinent to disorders such as drug abuse, which is hypothesized to center on the compromise of incremental reinforcement mechanisms that may support more habitual actions and underlie the compulsive nature of such diseases. At the same time, drugs may also weaken or compromise more deliberative or goal-directed choice systems that might otherwise be able to support more advantageous decisions. Formally understanding the roles played by both of these influences, and how they interact, promises to improve the conceptualization, diagnosis, and treatment of these and other disorders. The proposed program also provides unique opportunities for training and education. By integrating multiple core tools of systems and cognitive neuroscience (computational modeling, functional imaging, patient studies, behavioral analyses), students in the labs of both PIs are trained in different approaches to a unified research question, preparing them to be effective scientists in a more interdisciplinary future. Components of this training will also be extended to undergraduate and high school student populations through existing programs at both NYU and at Columbia and through outreach to New York area schools. This project will also help promote broader representation of minorities in science, including women. As a female neuroscientist with many women trainees in her laboratory, PI Shohamy serves as a role model and the collaborative project facilitates training for women in computational neuroscience, an area in which women are particularly underrepresented. Protections for Human Subjects: Acceptable Vertebrate Animals: Not applicable Resource Sharing: Acceptable. Data management plan is reasonable. Published data will be shared upon request when practically and ethically possible. Budget and Period of Support: Recommend as Requested

The Directorate of Social, Behavioral and Economic Sciences offers postdoctoral research fellowships to provide opportunities for recent doctoral graduates to obtain additional training, to gain research experience under the sponsorship of established scientists, and to broaden their scientific horizons beyond their undergraduate and graduate training. Postdoctoral fellowships are further designed to assist new scientists to direct their research efforts across traditional disciplinary lines and to avail themselves of unique research resources, sites, and facilities, including at foreign locations. This postdoctoral fellowship award supports a rising interdisciplinary scholar at the intersection of psychology, neuroscience and economics. In this project, the goal is to explore how decision-making is influenced by episodic memory, by using tools and theories from the above-mentioned fields, with the addition of computational modeling. Memory is essential to adaptive behavior, enabling organisms to draw on past experience to improve choices. Yet, the neural and cognitive mechanisms by which memory guides decision making are poorly understood. Despite substantial advances in understanding neural mechanisms of memory, on one hand, and those of decision making, on the other, remarkably little is known about a central adaptive aspect of memory function: how memory for the past is used to guide decisions. The proposed research aims to address this gap by bringing together three fields: psychology, neuroscience and economics. This NSF Fellow proposes a novel framework for beginning to understand how memory for specific episodes ("episodic memory") is used to guide value-based decisions. Our overarching hypothesis is that many value-based decisions involve sampling evidence from memory to inform the decision. This team will test their hypothesis by integrating computational modeling with eyetracking and functional imaging (fMRI) in humans to investigate the neural mechanisms by which episodic memory contributes to decision making. Determining the brain and cognitive mechanisms by which memory guides decisions will lay the foundation for potential future interventions which could radically shape policy. Poor decision making has been linked to poverty and aging with cascading effects on society more generally. The proposed results could help improve individual and collective decision making with clear implications for improving education and decision making across a diverse population.

Although it may seem obvious that many decisions are guided by memory, most studies on value-based decisions have focused on how repeated rewards incrementally form habitual decisions, which are distinct from pervasive more flexible and deliberative decisions that rely on episodic memory. This team's overall approach draws on advances in the neurobiological mechanisms of perceptual decision making. In perceptual decisions, such as deciding the direction of moving random dots, visual motion information is accumulated and when enough information is accumulated, a decision is made. This accumulation process is reflected in the firing rates of neurons in association and premotor cortices. Furthermore, the speed and accuracy of the decision are explained by a threshold (or bound) applied to the accumulation of information from the visual cortex. This team hypothesizes that a similar process accounts for how memories guide value-based decisions; in particular, we propose that sequential memory retrieval enters value based decisions in the same way that visual motion information is accumulated towards a perceptual decision. By linking memory, value and choice, this knowledge is expected to have important implications for multiple fields, including psychology, economics and neuroscience.

The decisions we make are shaped by memories of our previous experiences. Indeed, what decision you make may ultimately depend on which memories your brain accesses, and which ones you neglect, in contemplating a potential action's outcome. This project aims to measure memory access in support of choices, using functional neuroimaging, so as to study which memories are accessed when, and how retrieving these memories affects the choices people make, either immediately or later on. Understanding these processes will lay the foundation for a better, more unified understanding of many diverse phenomena affecting choices - planning, when habits arise, the role of dreams, and the impacts of advertising. This could also improve our understanding of maladaptive choice in various disorders, such as rumination, compulsion, and craving. The experiments also aim to examine how manipulating the structure of previous experience affects these memory-access patterns, and ultimately choices. In addition to its scientific aims, the project aims to train young scientists in an interdisciplinary range of techniques, combining computational and cognitive neuroscience, and to serve diversity especially by facilitating training of women in these areas.

Actions can be separated from their consequences by many steps in space and time. Anticipating these consequences so as to choose the best actions requires integrating memories of multiple intermediate events, which often were not originally experienced together. But so far there has been a lack of a principled and unified account of which memories are accessed, when, and which are neglected, to support value-based decisions. This project aims to test a recent computational theory that formalizes the ways in which particular memories are accessed and integrated to evaluate options, and the consequences for choice. The overarching hypothesis is that the brain sequentially integrates multiple memories for separate experiences either retroactively or prospectively, prioritizing the most valuable ones depending on the statistics of previous experience. The project will test this hypothesis using functional Magnetic Resonance Imaging in humans that will be engaged in solving several reinforcement-learning tasks. The project will take advantage of category-specific visual activity to measure memory access at different points during acquisition and deliberation, compare these patterns to subsequent choices, and test whether manipulations of the statistical structure of experience affects both memory access and the resulting choices.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Project Summary The differentiation between positive and negative valence is central to psychiatry. A seemingly categorical distinction between the drive toward rewards vs. the effort to avoid punishment appears central to many symptoms of psychiatric dysfunction and is evident in both how diagnostic categories are delineated and in the definition of cross-diagnostic constructs in RDoC. However, while there has been major progress in understanding how reward drives learning and actions and the underlying neural mechanisms, there has been much less progress in understanding the mechanisms by which loss and punishment affect behavior. Indeed, there has been continued controversy about whether the neural mechanisms of reward and loss are dissociable at all. Studies of the neural bases of reward seeking vs. loss avoidance have yielded mixed results, manifested both in inconsistent findings about shared vs. separate neural circuitry, and in surprising results in psychiatric populations, for instance showing reward processing abnormalities in psychiatric conditions that appear at face value to be driven by avoidance (e.g. OCD and anxiety). This has made it virtually impossible to address the critical question of defining valid measurements for reward seeking vs. loss avoidance separately, let alone for understanding the balance between them and their relation to other dimensional constructs and psychopathology. Here we address this challenge. We build on a computational framework that resolves the inconsistency in existing results by formalizing how avoiding a loss can ? in certain circumstances and in some people ? be reframed as a reward. Here we advance the hypothesis that using computational methods for quantifying and isolating this subjective reframing will allow us better to disentangle the relative, covert contributions of reward seeking vs. loss avoidance, and clarify their neural underpinnings. We propose to test this hypothesis by rigorously assessing the validity of the resulting measures (compared to simpler measures of overt reward and loss behavior) across tasks, measures, and test-retest replications. In particular, we address two specific aims. First, we seek to compare neural and behavioral measures of reward seeking and loss avoidance across tasks and participants using computational models and functional MRI in a large sample of participants. Second, we seek to examine individual differences in reward seeking and loss avoidance learning and their relationship to dimensions of psychiatric symptomatology using a large online sample. Both aims make use of two parallel and complementary experimental tasks which each test reward seeking, loss avoidance, and the extent to which the balance between the two is affected by differences in baseline expectations of reward or loss. Together, these studies offer an integrative computational framework to test the construct validity of measures of reward seeking and loss avoidance, the relationship between them, the new construct of their relative reframing, and how individual differences in these constructs are manifest across the population in brain and behavior.

This project proposal seeks to build and strengthen an existing program for high school students to perform mentored research during the summer, Brain Research Apprenticeships in New York at Columbia (BRAINYAC). Since 2013, BRAINYAC has trained and prepared 108 high school students from low-resourced neighborhoods in New York City for summer research experiences in neuroscience laboratories at Columbia University in the City of New York. We will provide high school students from underprivileged communities and/or who are under-represented minorities the necessary support for a competitive college application and insight into science research as a career choice. We propose to recruit students through partnerships with youth-serving programs and an inclusive application process. We aim to improve students? scientific knowledge and confidence in key scientific skills through directed training, mentoring and research experience. We will achieve this via the two main parts of the program: (A) A comprehensive training program during the spring that incorporates parental involvement, training sessions focused on science content, critical thinking skills, and communication skills, and orientation sessions to assist students in choosing a mentor and laboratory. These sessions are co-developed and run by a scientist and a science educator. (B) Summer laboratory experiences combined with weekly advisory sessions, a final poster presentation and field trips including a social event and a professional development event at the American Museum of Natural History (AMNH). We aim to support our participants? career development and college applications in STEM fields. We will achieve this through comprehensive support structure to work toward our goal of encouraging students to pursue studies and careers in STEM. We will track our success using a mixed-methods evaluation plan and an alumni engagement program including additional opportunities for alumni. In this way, we will reinforce our participants? intent to pursue studies or careers in science, technology, engineering and math (STEM).