Daphna Shohamy - US grants

DESCRIPTION (provided by applicant): The long-term goal of this project is to understand the interaction between brain systems subserving different aspects of memory function. Decades of neuropsychological and behavioral studies have emphasized the dissociation between distinct forms of memory (such as non-declarative vs. declarative memory), each supported by distinct anatomical and functional systems. The studies proposed here will use functional imaging (fMRI) to begin exploring how and when such systems may interact in the healthy brain. The studies will focus on the interaction between incremental stimulus-response learning (a form of non-declarative learning) and episodic memory (a form of declarative memory). Incremental learning is thought to depend critically on the basal ganglia (BG), while episodic memory is supported by the medial temporal lobes (MTL). The proposed research program aims to examine how the BG and MTL interact during memory formation, and the implication of this interaction for memory function. The resulting knowledge is expected to enhance our understanding of memory processes in the intact brain, as well as for understanding the nature of memory impairments in various neurological diseases, including Parkinson's and Alzheimer's disease.

DESCRIPTION (provided by applicant): Habits are powerful in driving actions. In substance abuse, habits formed by repeated hedonic experience with a drug are more powerful than explicit knowledge of the drug's detrimental effects. Recent research in the cognitive neuroscience of learning and memory has greatly advanced our understanding of these two forms of learning, suggesting they depend on distinct neural systems. Yet there has been remarkably little progress in extending this line of research to the study of addiction. Here, we propose to bridge this gap to understand the neural mechanisms contributing to different forms of learning and motivation, and their disruption in addiction. Research demonstrates that habit learning depends on the striatum and its dopaminergic inputs, while explicit memory for facts and events, often referred to as declarative memory, depends on the medial temporal lobe (MTL;hippocampus and surrounding cortices). Recent evidence suggests these systems may interact competitively under some circumstances. The core hypothesis motivating this proposal is that substance abuse may be related to a disrupted balance between these two learning systems and the extent to which each of them guides behavior. The proposed research will use functional imaging (fMRI) and behavioral analyses to investigate how different forms of learning guide decisions and actions. The central aim is to understand how to modulate the contribution of these two learning systems to choice behavior, at both the neural and behavioral levels. In the first study, we will examine the effects of modulating striatal contributions to learning and choice. To that end, we will manipulate reinforcement, and will determine the effects on the cognitive and neural systems driving subsequent choice behavior (Expt 1). In the second study, we will examine the effects of modulating MTL contributions to learning and choice. To that end, we will manipulate stimulus and associative novelty, and will determine the effects on the cognitive and neural systems driving subsequent choice behavior (Expt 2). Determining factors that modulate the extent to which learning and choice depend on one system or the other will lay the foundation for future translational work on potential treatment interventions that can reduce the dependence of behavior on the striatal habit system in favor of the MTL declarative system. PUBLIC HEALTH RELEVANCE: Substance abuse is a serious problem of public health. Disruption to the brain and cognitive mechanisms underlying learning are central to the pathology of substance abuse and addiction. In substance abuse, individuals'behavior is driven by habits that are learned as a result of repeated experiences with the drug's hedonic, reinforcing effects. These learned habits override any other forms of learning, such as explicit knowledge or experiences of the drugs'detrimental effects. The proposed studies aim to understand the neural and cognitive mechanisms underlying the balance between habitual and explicit forms of learning, and how this balance can be modified. The resulting knowledge will inform future work on potential cognitive and pharmacological treatment strategies that can reduce the dependence of behavior on habits in favor of explicit, goal-directed mechanisms.

DESCRIPTION (provided by applicant): Habits can exert powerful control over behavior. Sometimes habits are maladaptive, such as in substance abuse. In many cases, however, habits can be adaptive, enabling organisms to engage in healthy choices rapidly and efficiently. In recent years there has been substantial progress in understanding the neural mechanisms that support the transition from effortful, goal-directed to more automatic, habitual behavior through research in rodents. This work has emphasized the role of dopamine and its striatal targets in habits and has begun to suggest the principles by which their strength is controlled. Surprisingly, however, the rodent research has led to only limited progress in understanding the balance between goal-directed vs. habitual behaviors in humans, preventing the scientific community from being able to apply this knowledge to clinical settings and everyday life. We hypothesize that the main reason for this gap is the lack of rich and precise behavioral markers for habitual behavior in laboratory settings in humans. The proposed research program aims to address this gap. We propose a series of functional imaging (fMRI) and neuropsychological studies that focus on computational characterization of distinct types of learning and the factors that impact their relative strength. Because both habits and goal-directed behaviors depend on past experience, we can leverage recent advances in characterizing brain systems for learning and memory in humans that have already led to a rich, quantitative characterization of multiple aspects of behavior and neural signaling. Using these graded and dynamic signatures of habit formation - specifically, by examining choices and choice-related fMRI signals during trial-and-error learning, and during subsequent probes of the memories formed - we propose to ask a series of questions collectively aimed at uncovering the mechanisms by which the brain balances goal-directed and habitual behaviors. Our specific aims are: (1) To determine how the capacity to form flexible memory representations modulates goal-directed and habitual systems (2) To understand how the timing of feedback modulates the balance between the systems; (3) To determine how the reliability of feedback modulates goal-directed and habitual systems. For each aim, we test healthy subjects with fMRI to determine the dynamic contribution of multiple brain regions to different aspects of behavior. Parallel studies with Parkinson's patients complement the fMRI studies and provide evidence about the causal role of dopaminergic inputs to the striatum in habit learning. Results from this research will advance understanding of the behavioral and neurobiological mechanisms that control the emergence of habits and the implications for decisions and actions. Determining how multiple brain systems for learning support the transition of behavior from goal-directed to habitual control will lay the foundation for future translational work on potential treatment interventions that can adaptively shift behavior towards the formation of healthy habits in both health and disease.

The Directorate of Social, Behavioral and Economic Sciences offers postdoctoral research fellowships to provide opportunities for recent doctoral graduates to obtain additional training, to gain research experience under the sponsorship of established scientists, and to broaden their scientific horizons beyond their undergraduate and graduate training. Postdoctoral fellowships are further designed to assist new scientists to direct their research efforts across traditional disciplinary lines and to avail themselves of unique research resources, sites, and facilities, including at foreign locations. This postdoctoral fellowship award supports a rising interdisciplinary scholar at the intersection of psychology, neuroscience and economics. In this project, the goal is to explore how decision-making is influenced by episodic memory, by using tools and theories from the above-mentioned fields, with the addition of computational modeling. Memory is essential to adaptive behavior, enabling organisms to draw on past experience to improve choices. Yet, the neural and cognitive mechanisms by which memory guides decision making are poorly understood. Despite substantial advances in understanding neural mechanisms of memory, on one hand, and those of decision making, on the other, remarkably little is known about a central adaptive aspect of memory function: how memory for the past is used to guide decisions. The proposed research aims to address this gap by bringing together three fields: psychology, neuroscience and economics. This NSF Fellow proposes a novel framework for beginning to understand how memory for specific episodes ("episodic memory") is used to guide value-based decisions. Our overarching hypothesis is that many value-based decisions involve sampling evidence from memory to inform the decision. This team will test their hypothesis by integrating computational modeling with eyetracking and functional imaging (fMRI) in humans to investigate the neural mechanisms by which episodic memory contributes to decision making. Determining the brain and cognitive mechanisms by which memory guides decisions will lay the foundation for potential future interventions which could radically shape policy. Poor decision making has been linked to poverty and aging with cascading effects on society more generally. The proposed results could help improve individual and collective decision making with clear implications for improving education and decision making across a diverse population.

Although it may seem obvious that many decisions are guided by memory, most studies on value-based decisions have focused on how repeated rewards incrementally form habitual decisions, which are distinct from pervasive more flexible and deliberative decisions that rely on episodic memory. This team's overall approach draws on advances in the neurobiological mechanisms of perceptual decision making. In perceptual decisions, such as deciding the direction of moving random dots, visual motion information is accumulated and when enough information is accumulated, a decision is made. This accumulation process is reflected in the firing rates of neurons in association and premotor cortices. Furthermore, the speed and accuracy of the decision are explained by a threshold (or bound) applied to the accumulation of information from the visual cortex. This team hypothesizes that a similar process accounts for how memories guide value-based decisions; in particular, we propose that sequential memory retrieval enters value based decisions in the same way that visual motion information is accumulated towards a perceptual decision. By linking memory, value and choice, this knowledge is expected to have important implications for multiple fields, including psychology, economics and neuroscience.

Project Summary The differentiation between positive and negative valence is central to psychiatry. A seemingly categorical distinction between the drive toward rewards vs. the effort to avoid punishment appears central to many symptoms of psychiatric dysfunction and is evident in both how diagnostic categories are delineated and in the definition of cross-diagnostic constructs in RDoC. However, while there has been major progress in understanding how reward drives learning and actions and the underlying neural mechanisms, there has been much less progress in understanding the mechanisms by which loss and punishment affect behavior. Indeed, there has been continued controversy about whether the neural mechanisms of reward and loss are dissociable at all. Studies of the neural bases of reward seeking vs. loss avoidance have yielded mixed results, manifested both in inconsistent findings about shared vs. separate neural circuitry, and in surprising results in psychiatric populations, for instance showing reward processing abnormalities in psychiatric conditions that appear at face value to be driven by avoidance (e.g. OCD and anxiety). This has made it virtually impossible to address the critical question of defining valid measurements for reward seeking vs. loss avoidance separately, let alone for understanding the balance between them and their relation to other dimensional constructs and psychopathology. Here we address this challenge. We build on a computational framework that resolves the inconsistency in existing results by formalizing how avoiding a loss can ? in certain circumstances and in some people ? be reframed as a reward. Here we advance the hypothesis that using computational methods for quantifying and isolating this subjective reframing will allow us better to disentangle the relative, covert contributions of reward seeking vs. loss avoidance, and clarify their neural underpinnings. We propose to test this hypothesis by rigorously assessing the validity of the resulting measures (compared to simpler measures of overt reward and loss behavior) across tasks, measures, and test-retest replications. In particular, we address two specific aims. First, we seek to compare neural and behavioral measures of reward seeking and loss avoidance across tasks and participants using computational models and functional MRI in a large sample of participants. Second, we seek to examine individual differences in reward seeking and loss avoidance learning and their relationship to dimensions of psychiatric symptomatology using a large online sample. Both aims make use of two parallel and complementary experimental tasks which each test reward seeking, loss avoidance, and the extent to which the balance between the two is affected by differences in baseline expectations of reward or loss. Together, these studies offer an integrative computational framework to test the construct validity of measures of reward seeking and loss avoidance, the relationship between them, the new construct of their relative reframing, and how individual differences in these constructs are manifest across the population in brain and behavior.