Joshua D. Berke - US grants

The long-term objective of the proposed research is to better understand human brain disorders that involve specific, learned, compulsive, behaviors. These include drug-taking in addiction, and (for example) repetitive hand-washing in obsessive compulsive disorder. These conditions have been linked to abnormal function of brain circuits that include the striatum. The proposed project will first examine normal learning processes in the striatum, that are thought to be responsible for the progressive automatization of frequently repeated behaviors. It is known that rats will initially perform certain maze tasks using a hippocampus-dependent spatial strategy, but then progressively switch to using a 'response' strategy that involves the striatum. During this transition, multiple electrodes will be used to record from neuronal ensembles in both hippocampus and striatum. The specific aim is to examine how the behavioral switch is reflected is altered neural representations of task components. In the second phase of the project, the effect of psychostimulant drugs on the alteration of these representations will be examined. Post-training injections of amphetamine have been shown to enhance learning of striatum-based behavioral strategies; the proposed project will address whether this facilitation electrophysiologically resembles normal learning.

The proposed research program examines how multiple neural circuits involved in learning contribute to making a decision, and how this process may be subverted by addictive drugs. Several current models suggest that the abnormal engagement of normal learning mechanisms contributes to drug addiction. In particular, it has been argued that repeated drug-enhanced release of dopamine in the striatum produces unusually strong learned habits of drug-seeking and drug- taking. Such habits are hard to suppress, resulting in a progressive narrowing of behavioral repertoire, and greatly diminished control over drug intake. To better understand this process, we shall examine neural coding mechanisms involved in habit formation and inhibition, in striatum, hippocampus and medial frontal cortex. We shall also investigate how neural representations are altered when habits are artificially strengthened by the psychomotor stimulant drug amphetamine. Our approach has two essential features. Firstly we apply electrophysiological methods to tasks whose behavioral and neuroanatomical characteristics are relatively well understood. Secondly we perform comparisons between closely related situations, aiming to isolate aspects of neural representations that are specifically associated with distinct cognitive demands. Rats will perform two radial maze tasks that are identical in the stimuli presented to the animal, differing only in the strategies required to obtain rewards. In the 'win-stay' task (visual stimulus- response) the rat has to choose the arm that is illuminated, regardless of its recent history of choices. Learning this task has been shown to require the striatum, and can be enhanced by intra-striatal injections of amphetamine. In the other task ('win-shift'), the rat has to avoid the most- recently-visited arm, and the visual cue is irrelevant. This is a spatial working-memory task, that requires intact hippocampal function. Shifting between the visually-cued and spatial strategies requires suppression of the learned habit, and has been shown to involve the rat medial frontal cortex. By examining neural representations associated with acquisition of a visual stimulus-response habit, with drug enhancement of a habit, and with suppression of a habit, we aim to gain convergent data on how habits are encoded, and how excessively strong habits may contribute to addiction. At the same time we aim to provide a behavioral model of drug-induced loss of behavioral flexibility, that could be used by investigators testing novel drug abuse therapies. A fuller understanding of neural representations in frontal- striatal circuits, and how they are affected by dopamine, would also greatly contribute to our understanding of schizophrenia, obsessive-compulsive disorder, Tourette's syndrome, and Parkinson's Disease, as well as drug abuse.

DESCRIPTION (provided by applicant): This proposal is to test a key hypothesis about the neural organization of motivation, using newly- developed techniques for the identification and control of neurons in freely-moving animals. Maladaptive motivation plays a central role in a range of common and debilitating human disorders, including drug addictions, compulsions, depression and obesity. An accurate description of the roles of particular brain circuits in motivational subprocesses would be a major advance towards the understanding of these disorders. Lesion studies, drug manipulations and neuroimaging have all implicated the nucleus accumbens as a critical node in motivational information processing. However, the fundamental calculations being performed by this structure remain unclear. A major obstacle to the analysis of accumbens function has been the inability to distinguish the activity patterns of distinct cellular components. In particular, separate sets of accumbens neurons express distinct dopamine receptors, and project to distinct targets. These subpopulations are hypothesized to have dissociable functional roles, respectively enabling and suppressing the translation of motivationally salient signals into action. Prior investigations of neural coding in accumbens have been forced to consider these subpopulations together, but this obstacle can now be overcome using a combination of electrophysiological and optogenetic techniques in freely moving mice. The light-sensitive cation channel channelrhodopsin-2 will be selectively expressed in subpopulations of accumbens neurons that express either dopamine D1, or D2, receptors. These cells will be monitored using microelectrodes, and distinguished via their response to brief pulses of laser light. We will test the hypothesis that accumbens core neurons that increase firing to reward-predictive cues predominantly express D1 receptors, while those that decrease firing predominantly express D2 receptors. We will also compare the response of D1- and D2- expressing neurons to amphetamine and to the D2 antagonist eticlopride, to further examine the connection between the firing of specific accumbens neurons and psychomotor activation. In the proposed R21 period we expect to complete the first phase of our research program integrating optogenetics tools into behavioral electrophysiology, while testing simple yet critical hypotheses about the organization of limbic circuits. If funded, in subsequent phases we would make full use of these tools to progressively explore fundamental mechanisms of motivational information processing, and how this goes awry in key human disorders. PUBLIC HEALTH RELEVANCE: This project aims for a better understanding of the neural mechanisms underlying motivation. Success in this project may assist the design of novel therapies for common human disorders that are characterized by inappropriate or inadequate motivation, including drug addiction, depression and obesity.

DESCRIPTION (provided by applicant): Drugs of abuse can provoke long-lasting behavioral change, in part by abnormally engaging neural plasticity mechanisms that underlie normal reinforcement-driven learning. In particular, critical features of drug addiction may arise from altered function of dorsal striatal circuits that handle the progressive automatization of behavioral responses. Striatal circuits are widely thought to operate using essentially similar principles as artificial reinforcement learning (RL) algorithms for adaptive decision-making. However, how specific components of these circuits map onto specific computational/behavioral functions remains controversial. Within dorsal striatum there are remarkable subregions called patches (or striosomes) that have very different connectivity and neurochemistry to the surrounding matrix. It has long been hypothesized that patches have a special role in reinforcement learning, helping to control evaluative feedback about the consequences of actions. However, testing such ideas has not been possible, largely due to technical limitations in distinguishing patch and matrix neurons in behaving animals. We are now in a position to overcome this critical barrier to progress, and greatly advance understanding of just how rewarding experiences lead to altered behavior. The goals of this application are a) to complete development of a new generation of electrophysiological probes for high-density recording from identified striatal locations, and b) to use these devices to compare the activity patterns of patch and matrix neurons during reinforcement learning tasks. In this way we will test the specific hypothesis that patch neurons encode signals related to reward prediction. An accurate description of the roles of striatal compartments in reinforcement learning would greatly assist investigations into how abused drugs hijack normal decision-making. In addition, the new devices would be of broad application for investigating neural coding in both striatum and other neural circuits.

DESCRIPTION (provided by applicant): This is a Multiple-PI R01 proposal for coordinated investigation of the striatum, a brain structure critically involved in normal movement and motivation. Altered striatal function underlies a range of serious, common neurological disorders, including Parkinson's Disease and dystonia. Yet the mechanisms by which this structure normally processes information, and how this can go awry, are not well understood. One particular cell type, the fast-spiking interneuron (FSI), is rare but has a disproportionate influence over other striatal neurons. Loss of FSIs has been observed in animal models of dystonia and in human Tourette syndrome. In recent studies we have observed activation of FSIs as highly trained yet unwanted choices need to be suppressed, and that selective suppression of FSIs results in dystonia-like symptoms. FSIs thus appear to have a key coordinating role within striatal networks, and there is a pressing need to better understand their physiological and behavioral functions. The proposed complementary experiments in brain slices and awake behaving animals make full use of advanced electrophysiological, pharmacological and optogenetic methods. Aim 1 examines how distinct inputs from cortex, thalamus, and globus pallidus influence FSI firing patterns, both spontaneously and at critical moments of choice task performance. Aim 2 examines the conditions under which FSIs control striatal projection cells of the two major output pathways, and how FSI suppression affects network dynamics and behavior. Finally, Aim 3 investigates the consequences of dopamine loss on striatal microcircuits, examining changes in local connectivity and firing patterns that may underlie core movement difficulties in Parkinson's Disease. The long-term goals of this research program are to determine the fundamental operational principles of striatal circuits from sub-cellular to network levels. This knowledge would be of immense value in designing improved therapies for Parkinson's Disease, dystonia, Tourette Syndrome and other serious brain disorders.

DESCRIPTION (provided by applicant): Behavioral inhibition is central to self-control. Daily life is made immeasurably easier by a repertoire of learned responses to stimuli, yet we need to interrupt and override such responses as circumstances and goals change. Problems with inhibitory function characterize a range of psychiatric disorders including drug addiction, attention-deficit hyperactivity disorder, and Tourette Syndrome. Despite the importance of behavioral inhibition, our understanding of the neural mechanisms involved remains very limited. A standard tool to probe behavioral inhibition is the Stop-signal task. Subjects are signaled to make quick actions, and in a subset of trials are later instructed to cancel those movements before they begin. It has long been hypothesized that Stop-signal performance reflects a race between Go and Stop processes, but how this race corresponds to brain activity is not clear. Although there is a great deal of evidence that deep brain structures called the basal ganglia are involved in stopping, there has been little corresponding investigation of the basal ganglia using the method with the best temporal resolution - electrophysiology of single neurons. We have recently found evidence for a neural race between distinct basal ganglia pathways. Activity in sensorimotor striatum (STR) appeared to correspond to a Go process, while Stop cues instead provoked very fast responses in the subthalamic nucleus (STN). Both of these areas project to the substantia nigra pars reticulata (SNr), which can operate as a gateway to motor output. The relative timing of STR and STN firing determined whether SNr cells responded to the Stop cue (observed when inhibition was successful), or not (when inhibition failed). However, our data also suggest that the STN-SNr pathway actually provides a fast yet transient movement pause, with complete cancellation requiring a separate suppression of STR output. We hypothesize that these two mechanisms serve complementary functions, allowing behavioral inhibition to be both fast and selective. To investigate these processes further, we propose a series of experiments using state-of-the-art techniques for monitoring and manipulating the basal ganglia. For Aim 1 we will compare Stop-related activity in distinct subregions within STR, STN and SNr, to better define how information flows through motor and cognitive circuits. For Aim 2 we will investigate whether STN signals are specific to stopping, and whether they are driven by the intralaminar thalamus, an area involved in fast orienting reactions. For Aim 3 we will use selective optogenetic suppression and stimulation of the STN-SNr pathway to confirm that it provides a fast motor pause. Finally, for Aim 4 we will explore how the key neuromodulators acetylcholine and dopamine contribute to the suppression of STR output during successfully cancelled actions. Overall, this project would break new ground in determining with unprecedented precision how we are able to rapidly suppress unwanted or inappropriate actions, in the service of adaptive, flexible behavior.

DESCRIPTION (provided by applicant): A major goal in neuroscience is to understand the computations performed by local brain circuits. A large obstacle to achieving this goal is that - at least in mammals - we currently cannot observe the spiking activity of most neurons within a circuit. A key reason is that standard electrodes are just too big, and provoke too much damage to brain tissue. If placed with high enough density to sample a majority of neurons, they would destroy the very circuit they are intended to monitor. Another important obstacle to understanding local brain computations is that circuit dynamics are rapidly and dramatically altered by chemical neuromodulators, which normally go unobserved. Real-time monitoring of critical modulators such as dopamine can be achieved using fast-scan cyclic voltammetry, but this method has not yet been effectively combined with large-scale circuit recordings. The proposed work would make important progress towards overcoming these obstacles, using ultra-dense arrays of 8µm carbon thread electrodes. These are stiff enough to insert deep into the brain, yet small enough to avoid a destructive immune response. By using an 80µm distance between electrodes, the great majority of neurons within a cortical layer would be within recording range. Furthermore, carbon thread electrodes are well-suited for chemical sensing using voltammetry. This proposal is to construct advanced new tools for neuroscientific investigation in a series of modular steps, culminating in 1024-channel, combined electrophysiological and electrochemical recording in freely-behaving rats. Aim 1 involves the development and testing of silicon frameworks that allow assembly of ultra-dense arrays, together with updated headstages that allow hundreds of channels to be monitored simultaneously. Aim 2 will exploit the ability of carbon thread electrodes to be sliced in situ during histological processing. This greatly facilitates the ability to localize individual recordig sites within microcircuit architecture, and to identify individual recorded neurons. Aim 3 involves further optimization of carbon thread electrodes for chemical sensing, and joint single-unit recording and fast-scan cyclic voltammetry across many electrodes simultaneously. Overall this project combines expertise in electrical engineering, neurophysiology, and neurochemistry to create innovative, powerful devices that will be widely disseminated and may have transformational impact for our understanding of how our brains work.

DESCRIPTION (provided by applicant): A major goal in neuroscience is to understand the computations performed by local brain circuits. A large obstacle to achieving this goal is that - at least in mammals - we currently cannot observe the spiking activity of most neurons within a circuit. A key reason is that standard electrodes are just too big, and provoke too much damage to brain tissue. If placed with high enough density to sample a majority of neurons, they would destroy the very circuit they are intended to monitor. Another important obstacle to understanding local brain computations is that circuit dynamics are rapidly and dramatically altered by chemical neuromodulators, which normally go unobserved. Real-time monitoring of critical modulators such as dopamine can be achieved using fast-scan cyclic voltammetry, but this method has not yet been effectively combined with large-scale circuit recordings. The proposed work would make important progress towards overcoming these obstacles, using ultra-dense arrays of 8µm carbon thread electrodes. These are stiff enough to insert deep into the brain, yet small enough to avoid a destructive immune response. By using an 80µm distance between electrodes, the great majority of neurons within a cortical layer would be within recording range. Furthermore, carbon thread electrodes are well-suited for chemical sensing using voltammetry. This proposal is to construct advanced new tools for neuroscientific investigation in a series of modular steps, culminating in 1024-channel, combined electrophysiological and electrochemical recording in freely-behaving rats. Aim 1 involves the development and testing of silicon frameworks that allow assembly of ultra-dense arrays, together with updated headstages that allow hundreds of channels to be monitored simultaneously. Aim 2 will exploit the ability of carbon thread electrodes to be sliced in situ during histological processing. This greatly facilitates the ability to localize individual recordig sites within microcircuit architecture, and to identify individual recorded neurons. Aim 3 involves further optimization of carbon thread electrodes for chemical sensing, and joint single-unit recording and fast-scan cyclic voltammetry across many electrodes simultaneously. Overall this project combines expertise in electrical engineering, neurophysiology, and neurochemistry to create innovative, powerful devices that will be widely disseminated and may have transformational impact for our understanding of how our brains work.

PROJECT SUMMARY/ABSTRACT Dopamine is a key modulator of motivated behavior. Dopamine is also a key modulator of reinforcement- driven learning. Yet the relationship between these critical functions is unclear. Based on seminal recordings of dopamine cells in head-fixed animals, the dominant theory is that dopamine signals reward prediction errors - i.e. a learning signal. However, the actual release of dopamine has been repeatedly found to escalate as freely-moving animals approach rewards, in a manner more consistent with reward prediction than reward prediction errors. Furthermore, optogenetic stimulation of dopamine immediately invigorates behavior, as if boosting reward predictions. This project seeks to resolve this apparent discrepancy, and gain a new understanding of dopamine signaling and regulation. Prior studies in brain slices have established that dopamine release is strongly influenced by local mechanisms, especially nicotinic acetylcholine receptors on dopamine terminals. Aim 1 will directly test whether there is a dissociation between dopamine firing and dopamine release. VTA dopamine cell body activity will be assessed using both optogenetic identification of single neurons, and fiber photometry, and dopamine terminal activity in accumbens core and shell will be measured using both fast-scan cyclic voltammetry and fiber photometry. These measures will be taken as rats perform multiple behavioral tasks, including a trial-and-error reinforcement learning task and a more passive Pavlovian task for better comparison to prior studies. Aim 2 will monitor and manipulate accumbens cholinergic interneurons during the same behavioral tasks, while examining dopamine terminal activity. The hypothesis is that these interneurons can both shape the motivational message conveyed by dopamine release, and rapidly switch this message to a reinforcement learning signal. Finally, Aim 3 will use variably-timed local optogenetic manipulations of dopamine and accumbens spiny neuron subpopulations (direct vs indirect) to investigate the exact timing requirements for dopamine to serve as a reinforcement learning signal. The long-term goal of this research program is to understand circuit mechanisms of adaptive decision- making, and how drugs such as nicotine perturb these mechanisms to produce addictive behavior. By using state-of-the-art techniques and carefully-designed behavioral tasks to test novel hypotheses, this project may transform our understanding of the neurobiology of motivated behavior.

Abstract Behavior is motivated by reward, and the most powerful rewards are those that satisfy a physiologic need. For decades, neuroscientists have studied the midbrain dopamine system to understand reward and hypothalamic circuits to understand sensing of internal needs. But how these two neural systems are interact to give rise to behaviors like eating and drinking remains poorly understood. Recently, we have used approaches for simultaneous neural recording and manipulation to observe directly the communication between these two systems. We have also mapped the signals they each receive from the gut in response to ingestion of food and fluids. This has revealed that hunger and thirst powerfully modulate the dopamine system, but do so in different ways and likely involve distinct circuit mechanisms. We propose here to build on these findings to systematically delineate how these neural circuits for need and reward interact in the brain. In Aim 1, we investigate how these circuits represent internal needs, by recording their dynamics at multiple levels of analysis under different physiologic states, and further measuring how those dynamics are influenced by targeted circuit manipulations. In Aim 2, we investigate how these circuits use information about bodily needs to drive learning about food, by monitoring and manipulating their activity during the learning process. In Aim 3, we investigate how these circuits use information about internal state to drive motivation, by monitoring and manipulating their activity during tasks where animals must evaluate competing needs and rewards. These studies will provide fundamental insight into the mechanisms by which information about body needs is utilized by the brain to generate learning and motivation.

Summary / Abstract The dorsal striatum (DS) is an important brain structure for normal sensorimotor control, including decisions about how vigorously to move. As one example, loss of the dopamine input to DS is responsible for bradykinesia in Parkinson's Disease. Yet how DS circuits processes information, and how this information processing is modulated by dopamine, are not well understood. DS circuits include sparse populations of interneurons - most commonly expressing either parvalbumin (PV+), somatostatin (SST+) or acetylcholine (ChAT+). Interneurons appear to coordinate the activity of striatal spiny projection neurons (SPNs), and alterations in striatal interneurons are found in human Tourette Syndrome and rodent models of dystonia. Studies in brain slices have found many ways in which striatal interneurons can affect SPNs and each other, via direct connections and by modulation of dopamine release. However it has been challenging to connect these various results together into a coherent vision of DS microcircuit function. Progress has been hampered by the lack of critical data about the joint activity patterns of DS interneurons, SPNs, and local dopamine fluctuations, at precise moments during well-controlled behavioral tasks. To overcome this obstacle, this proposal uses an innovative, technically-advanced combination of behavioral electrophysiology, optogenetics and optical dopamine sensors. We will perform real-time measurements and manipulations of DS interneurons and dopamine, as freely-moving rats respond to cues. The response times depend on rats' reward expectation for the selected action. Taking advantage of the computational framework of reinforcement learning to derive trial-by-trial estimates of internal decision- variables, we will test specific hypotheses about how the activity of distinct interneuron types is shaped by recent choice and reward history. Aim 1 will characterize the activity of DS PV+, SST+ and ChAT+ interneurons as actions are initiated. In both dorsolateral and dorsomedial striatum we will record bulk calcium signals from each subpopulation, or the spiking of identified interneurons, simultaneously with dopamine signals. Aim 2 will examine how, and when, transient suppression of interneurons affects movement initiation and the activity of nearby SPN ensembles. Aim 3 will determine how loss of DS dopamine jointly affects interneuron activity and behavior. The long-term goals of this research program are to determine the fundamental operational principles of striatal circuits. This knowledge would be of immense value in designing improved therapies for a wide range of human neurological disorders.