Alicia Izquierdo, Ph.D. - US grants

DESCRIPTION (provided by applicant): Methamphetamine (mAMPH) is a highly addictive psychostimulant. In many regions of the United States, it is second only to alcohol and marijuana as the most frequently used drug, often outpacing both heroin and cocaine abuse. Despite an increased number of studies recently devoting attention to measuring mAMPH effects on learning and memory, no systematic investigation has been conducted on mAMPH's impact on cognitive flexibility in either an animal model or in humans. Cognitive flexibility is of vital importance to the success of an organism and refers to the ability to shift strategy given a new set of circumstances. If mAMPH impacts functioning of the orbitofrontal cortex (OFC), as is currently posited, then mAMPH likely has an impact on flexible cognition. In addition, there is now evidence for serotonergic modulation of this flexible cognition and given that neuronal serotonin integrity is compromised in mAMPH abusers (and mAMPH-treated animals), this is a timely issue. We propose to study the effects of mAMPH on flexible cognition using a touch screen-based operant procedure sensitive to pharmacological manipulation in rodents. Our measures of flexible cognition are reversal learning and extinction, two separate tasks sensitive to both OFC recruitment and serotonergic modulation and that are well-equipped with a host of auxiliary measures of motivation, attention, and inhibitory control. Our long-term goal is to reconcile our understanding of the neural circuitry that subserves flexible cognition with the pathological effects of drugs of abuse. To begin to accomplish our long-term goal, two specific aims are proposed: 1) to establish baseline cognitive flexibility performance in untreated rats and 2) to compare the effects of a binge regimen and an escalating, neuroprotective dose regimen of mAMPH on cognitive flexibility and the OFC. Results arising from this project will enhance our understanding of the impact of mAMPH on flexible cognitive processes and increase our ability to identify therapeutic targets to ameliorate the poor decision making arising from mAMPH abuse. At best, a rescued cognitive flexibility could aid mAMPH addicts in remaining abstinent. Public Health Relevance: Methamphetamine is a highly addictive psychostimulant and a growing public health concern, with a record of nearly 1.4 million people over the age of 12 using methamphetamine in the United States in 2005. Those addicted to this substance make poor, disadvantageous choices and relapse often. The long-term goal of this project is to provide knowledge about the effects of methamphetamine on decision making and the brain.

PROJECT DESCRIPTION 1. BACKGROUND AND SIGNIFICANCE Learning from feedback in the real w'orld is limited by constant fluctuations in reward outcomes associated with choosing certain options or actions. Some of these fluctuations are caused by fundamental changes in the reward values of those options/actions that necessitate dramatic adjustments to the current learning strategies, like in epiphany learning or one-shot learning [Chen & Krajbich, 2017; Lee et al. 2015]. Other changes represent inherent stochasticity in an otherwise stable environment and should be tolerated and ignored to maintain stable choice preferences. In other words, learning in dynamic environments is bounded by a tradeoff between being adaptable (i.e. respond quickly to changes in the environment) and being precise (i.e. update slowly after each feedback to be more accurate), which we refer to as the adaptability-precision tradeoff [Farashahi et al., 2017; Khorsand & Soltani, 2017]. Therefore, distinguishing meaningful changes in the environment from natural fluctuations can greatly enhance adaptive learning, indicating that adaptive learning depends on interactions between multiple brain areas. To date, most computational models of learning under uncertainty are very high-level and/or descriptive [Behrens et al., 2007; Costa et al., 2015; ligaya, 2016; Jang et al., 2015; Nassar et al., 201 O; Payzan-LeNestour & Bossaerts, 2011] and therefore, do not provide specific testable predictions. On the other hand, neural mechanisms of uncertainty monitoring for adaptive learning have been predominantly investigated in humans, and in a few cases monkeys, both of which are limited in terms of circuit-level manipulations. However, interactions between brain areas unfold on short timescales and can be specific to certain cell types. These properties have severely limited the ability of functional MRI [Logothetis, 2003] or MEG [Dale et al., 2000; Mostert et al., 2015] to reveal the microcircuit mechanisms within brain regions and fine-grained contributions between brain regions. To overcome these limitations and reveal neural mechanisms underlying adaptive learning under uncertainty, we propose a combination of detailed computational modeling, imaging of stable neuronal ensembles, and precise system-level manipulation of interactions between multiple brain areas in rodents. The latter is possible in part due to powerful circuit- dissection techniques in rodents that allow manipulations of genetically-tractable cell types and thus, specific projections between brain regions. Combined with decoding of neuronal activity in cortex and guided by mechanistic computational modeling, this approach enables us to investigate both microcircuit and system-level mechanisms of adaptive learning under uncertainty. We have recently proposed a mechanistic model for adaptive learning under uncertainty [Farashahi et al., 2017]. This model, which we refer to as reward-dependent metaplasticity (ROMP) model, provides a synaptic mechanism for how learning can be self-adjusted to reward statistics in the environment. The model predicts as more time spent in a given environment with a certain reward schedule, the organisms should become less sensitive to feedback that does not support what is learned. This and other predictions of the model were confirmed using a large set of behavioral data in monkeys during a probabilistic reversal learning task [Farashahi et al., 2017]. Although the proposed metaplasticity mechanism enables the model to become more robust against random fluctuations, it also causes the model to not respond quickly to actual changes in the environment. This limitation can be partially mitigated by allowing synapses to become unstable in response to changes in the environment [ligaya, 2016]. Interestingly, in our model, the changes in the activity of neurons that encode reward values can be used by another system to compute volatility in the environment. This signal can be used subsequently to increase the speed of learning when volatility is high, that is, when there is a higher chance of real changes in the environment. We hypothesize that such interactions between value-encoding and uncertainty-monitoring systems can enhance adaptability required in dynamic environments. In addition to this modeling study, we recently have shown that both basolateral amygdala (BLA) and orbitofrontal cortex (OFC) have complementary roles in adaptive value learning under uncertainty in rodents [Stolyarova & Izquierdo, 2017]. In this experiment, rats learned the variance in delays for food rewards associated with different visual stimuli upon selecting between them. We found that OFC is necessary to accurately learn such stimulus-outcome association (in terms of 1 21

Various neuropsychiatric conditions lead to failures in generating accurate models of the reward environment or inabilities in using those models to guide flexible behavior, very often manifesting as impaired reversal learning. The anterior cingulate cortex (ACC) and the orbitofrontal cortex (OFC) are frontocortical regions important for flexible reinforcement learning, and have been theorized to work in a hierarchy of parallel processes for reward- based choice. In OFC, there is priority encoding of lower-level attributes like reward-predictive value of sensory cues, the palatability of specific rewards, and the current stimulus-reward mappings relevant to behavior. In ACC, these variables are thought to be multiplexed for higher-level computations of reward prediction error (RPE) and confidence/uncertainty of predictions, which are used to monitor performance and update behavioral strategies when necessary (particularly overall trial strategy following positive feedback, i.e., WinStay). These computations may depend upon propagation of spikes from OFC to ACC. However, it remains poorly understood how flexible reward learning is mediated by interactions between OFC and ACC. Here we will investigate this question using a robust animal model of adaptive learning under uncertainty: stimulus-based probabilistic reversal learning (PRL). In freely behaving rats, we will use a combination of in vivo 1-photon calcium imaging and electrophysiology, chemogenetics, and closed-loop neural control of reward delivery to examine how OFC and ACC regulate PRL. Using new technology that we have recently developed for online decoding of calcium activity we will use a novel strategy of regulating reward delivery based upon neural activity in ACC and OFC to test whether flexible reward learning depends upon accurate neural representations in these frontocortical areas. To date, we have: demonstrated effective DREADDs manipulation in vivo and in transduced cortical slices; designed and tested custom electrode arrays to perform chronic in vivo electrophysiological recordings in these areas simultaneously; and imaged ensemble activity time-locked to behavior, which has proven stable over multiple sessions, ideal to study learning. Leveraging these technical advances and using this capacity as a platform, we propose to identify the precise cortico-cortical mechanisms of encoding variables in flexible reinforcement learning across two Aims. Collectively, these experiments will: 1) shed new light on the signaling signatures of cortical regions and their respective roles in flexible reinforcement learning, 2) accelerate groundbreaking experiments as they would be performed in closed-loop: control of reversal learning in real-time using decoded neural expectation, and 3) these signals would eventually be compared in animal models of psychopathology because of their known failures in reversal learning. These novel and unconventional approaches make the R21 mechanism ideal for the proposed work.