Matthew Roesch - US grants

DESCRIPTION (provided by applicant): The experiments in this proposal are designed to investigate the neurophysiology and neurochemistry underlying impulsivity in normal animals and in animal models of schizophrenia and drug abuse. Like humans, animals impulsively choose a small immediate reward over a larger delayed reward (i.e. time discounting). We will test whether impulsive choice is governed by dopaminergic modulation of time discounted reward signals to nucleus accumbens (NA) from the orbitofrontal cortex (OFC) and whether disrupted function in this circuit due to changes in encoding in OFC and dopaminergic tone in NA gives rise to abnormal levels of impulsivity in schizophrenia and drug abuse. Such complementary changes would explain the high incidence of drug abuse in schizophrenic patients. Through the Mentored Research Scientist Development Award I can acquire the necessary techniques and didactic training in new disciplines in order to achieve my immediate goal of testing these hypotheses and my long-term career goal of becoming a successful independent faculty member, competent in a variety of disciplines including neurophysiology, neuropharmacology and the study of schizophrenia and drug abuse. My training and specific research aims can be broken down into three main components: (1) To characterize neural correlates of impulsive choice in behaving animals. For this, I will continue my training under Dr. Schoenbaum, who is an expert in behavioral electrophysiology and learning theory. (2) To characterize the role of dopamine on interactions between OFC and NA. For this, I will learn how to record intracellularly in anesthetized rats during pharmacological manipulations under the mentoring of Dr. O'Donnell. (3) To characterize the impact of schizophrenia and cocaine on this circuit. For this, I will acquire new skills from both Dr. Schoenbaum and Dr. O'Donnell. In addition to technical and intellectual support from my mentors I will receive didactic training through courses, journal clubs and seminars offered at the University of Maryland and the MPRC. This research will increase our understanding of the neurobiology of impulsivity common to many psychiatric illnesses. Furthermore, it will serve as a platform to investigate impulsive choice in animal models of schizophrenia and drug abuse, which has relevance to understanding the high incidence of drug abuse comorbidity observed in the schizophrenic population.

DESCRIPTION (provided by applicant): The Actor/Critic model has been suggested to be the computational solution to optimizing long term gains. In this model, decisions made by the Actor are updated by the Critic when outcomes deviate from what is expected. Past research shows that midbrain dopamine (DA) neurons signal errors in reward prediction;however, it is unknown how these signals are generated or how they impact decision policies in downstream brains areas. According to the Actor/Critic model, midbrain DA neurons compute prediction errors by comparing the predicted value of reward, signaled by ventral striatum (VS), to the actual value of reward received, but this has not been directly tested. Subsequently, prediction errors are thought to modify behavior by updating the action policies of the Actor, dorsal striatum (DS). Neural correlates in DS include policies related to stimuli, responses and outcomes, but how these correlates are modulated by the DA system during learning remains unknown. Here, these issues will be addressed by recording from single neurons in DS and midbrain DA neurons after DA and VS inactivation, respectively. The importance of these interactions will be verified by inactivation techniques. Importantly, this circuit has been shown to be abnormal in addiction, which makes sense, considering that addicts cannot optimize choice behavior in the face of changing consequences. A final experiment will examine neural correlates of reward predictions, prediction errors and decision policies in rats that have chronically self-administered cocaine;the results will help determine how these neural representations are disrupted after long-term drug exposure. PUBLIC HEALTH RELEVANCE: Optimal decision-making is thought to depend on a circuit involving striatum and midbrain dopamine neurons;areas affected by long-term exposure to drugs of abuse. Together, these brain regions are thought to compute errors in reward prediction, which are subsequently used to update policies that guide future decisions. Improving our understanding of the neural mechanisms underlying the decision making process will provide a better working knowledge of how we learn normally and how this circuit is affected by chronic drug use.

? DESCRIPTION (provided by applicant): The objective of this proposal is to provide powerful insight into the computational and neurobiological processes underlying learning during Pavlovian conditioning, and to elucidate the origin of differences between individuals in their response to a conditioned stimulus (CS), namely sign-trackers (ST) attracted by the CS and goal-trackers (GT) directly attracted by the reward. One of the project's partners recently proposed a computational model which accounts for a large set of studies examining ST/GT behaviors. More importantly, this model has led to a series of new experimental predictions which, if tested experimentally, could help further validate or refute the computational mechanisms that underlie everyday learning. Here we propose a unique series of model-driven experiments to precisely test those predictions on both the computational and neurobiological levels using rigorous behavioral protocols and state-of-the art optogenetic and pharmacogenetic methods. This will enable us to assess and refine the proposed computational theory, and thus to provide a detailed description of the mechanisms underlying inter-individual differences during learning. Intellectual Merit (provided by applicant): Understanding how the brain integrates predictive information is a fundamental issue and has major implications at both theoretical and applied levels. In both ecological and artificial situations, these processes enable animals, humans and even robots to flexibly adapt their performances according to changes in the environment. Particularly poorly understood here are the mechanisms underlying inter-individual differences in learning, which may explain why some individuals fail in learning in particular situations while others succeed. Understanding these individual differences can help us better characterize why some individuals are more prone to drug addiction and craving in front of a CS associated to a drug-taking context, and has implications about individualized treatment. The research herein draws on complementary expertise from Biology, Psychology, Medicine, Applied Mathematics and Engineering in order to elucidate the combination of computational processes and behavioral traits that underlie these differences. To this end, we will systematically manipulate parameters that the model identifies as crucial and evaluate the dynamics and role of dopaminergic error signaling. Our work involves a unique combination of correlative and selective interventional approaches that directly test the fundamental assumptions of the model. Our results will thus provide definitive evidence regarding the competition of model-free and model-based processes in conditioning. The computational model that is at the heart of this proposal may thus represent a major step in the approach of individual differences. Broader Impact (provided by applicant): The broader impacts of this proposal will occur through the integration of the proposed research with teaching and training as follows. 1) Outreach. The PI will broaden the participation of underrepresented groups through research opportunities provided to high school students and underrepresented undergraduates through an ongoing partnership with Eleanor Roosevelt High School. The latter program will consist of a 3-week scientific boot camp during the summer for 1-2 students followed by a year of research in the PI's lab 2) Professional development. This project will train one postdoc, three graduate students, at least three undergraduates, and several high school students in collaborative research. Tiered peer mentoring will allow training of new personnel by more senior lab members, all carried out under the guidance of the PI. 3) Teaching. The boot camp, developed by the PI, will provide students with a foundation in scientific methods and will use evidence-based approaches to introduce them to how scientists investigate research questions using techniques performed in the PI's lab. The French co-PIs will on their side continue their development of introductions to science classes for high-school students and courses at various university levels. 4) Dissemination of research findings. All trainees will present their research t lab meetings, journal clubs, and conferences and will participate in manuscript preparation in order to share results with the scientific community. All publications generated by the project wil be made open-access (via PI's faculty pages and HAL in France).

DESCRIPTION (provided by applicant): Recognizing receipt of reward in others guides our daily behavior. For example, children that observe classmates receive reinforcement for good behavior recognize the benefits of such actions. In the work place, observation of colleagues receiving a promotion lets us know that our work has potential payoff. These are all positive associations that alter our own behavior based on receipt of reward in others. It is unknown what brain regions represent this information. One likely candidate is the dopamine (DA) system. We know that DA is released in ventral striatum (VS) when reward is unexpectedly delivered and is critical for reinforcement learning. It might also be critical for recognizing rewards delivered to others, yet this hypothesis has never been tested. Here, in AIM 1, we ask if subsecond DA release is elevated in rats when reward is delivered to a conspecific. However, social observation of reward does not always lead to positive affect. For example, observing your colleague get promoted or receive a bonus instead of you, might lead to jealousy, frustration, and other negative affective states. This emotion must reflect a discrepancy between the reward that you expect for yourself and what you actually received. Such signals are referred to as negative prediction errors and are encoded by midbrain DA neurons. It is unknown if this signal is modulated by observation of reward delivered to others. Here, in AIM 2, we will ask if subsecond DA release related to negative predictions errors are modulated by conspecific reward. These are not just interesting questions that would advance our basic understanding of the DA system, but they are clinically relevant because the ability to recognize reward in a conspecific is disrupted in several psychiatric disorders (e.g., autism, psychopathy). To date, we know very little about the neurobiological substrates that control these functions because detailed work in animals at the single-unit and neurotransmitter level has not yet occurred. Here we propose a first step to addressing this issue. We will record subsecond DA release using fast-scan cyclic voltammetry (FSCV) in VS, while rats observe reward delivery to a conspecific in cases when they do or do not expect reward for themselves. We will examine differences between cagemates and non-cagemates because 'empathy' studies have suggested that cagemates are more adept at recognizing social cues compared to rats that are unacquainted. If successful, these studies will lead to a host of experiments that would test observational learning and underlying circuits, but as a first step, we must determine if DA signals are necessary and sufficient for behavioral output.

PROJECT SUMMARY Recognizing the emotions of others guides our daily decisions. We perform actions that benefit others and suppress those that might cause harm or distress, even when it requires personal sacrifice. The recognition of a conspecific's distress and ability to alter ones behavior in light of that distress is disrupted in several psychiatric disorders (e.g., autism, psychopathy). Unfortunately, we know very little about the neurobiological substrates that control these functions because detailed work in animals at the single-unit and neurotransmitter level has not yet occurred. Recently, there have been a number of behavioral studies demonstrating that rodents can recognize conspecific distress and choose to alter behavior to alleviate that distress. Here, we propose to use cutting edge neuroscience techniques ? Designer Receptors Exclusively Activated by Designer Drugs (DREADDS), single-unit recording across multiple brain areas simultaneously, fast-scan cyclic voltammetry (FSCV), optogenetics, and calcium imaging ? to elucidate the neural mechanisms related to modification of reward-guided behavior during conspecific distress in multiple social contexts and time scales as contingencies are learned and social relationships change with experience. We propose a circuit by which behaviors are modulated by predicted social distress via interactions between basolateral amygdala (ABL), anterior cingulate cortex (ACC), nucleus accumbens core (NAc), and accumbal dopamine (DA) release. The dynamic relationship between areas within this circuit will be uncovered with precise spatial and temporal resolution during learning and long-term social interaction by recording from multiple brain areas simultaneously and determining if altered communication (DREADDS) between areas impacts behavior. Calcium imaging will allow us to monitor activity across single neurons and large groups of neurons over multiple days. We will jointly analyze images at different time instances and determine what is common across time points versus what has changed, and statistically determine how components correlate with behavior to determine how areas process social information at an internal network level. We predict that the ABL-ACC circuit is important for pairing recognition of conspecific distress with predictive stimuli and is necessary for correlates related to motivated behavior in NAc to be modified in social contexts. Furthermore, ACC will be more heavily involved in co-registering information pertaining to oneself and the conspecific, but is dependent on ABL during learning. We also theorize that DA release modulates predictive value signals in downstream targets such as NAc by reporting negative and positive prediction errors when rewards are accompanied by conspecific distress and shock avoidance. Finally, we propose experiments that will attempt to modulate pro- and anti-social behavior via optogenetic stimulation and inhibition of the DA system and by oxytocin administration, a novel therapeutic treatment for mental disorders characterized by social dysfunction.

PROJECT SUMMARY Reward-guided decision-making and impulse control are disrupted after chronic cocaine use. These changes have been attributed to altered function in brain circuits critical for computations of reward predictions and action policies. `Reward prediction' signals reflect the reward the animal expects to receive as a result of behavior, thus reflecting goals associated with decisions. `Action policies' are rules that govern behavior that at triggered by external stimuli or context, and are thought to underlie habits. Both reward predictions and action policies are modified when there are violations in predictions known as `reward prediction errors'. `Signed' reward prediction errors reflect the valence associated with an error, strengthening or weakening the associability between cues and outcomes/responses. `Unsigned' prediction errors reflect the surprise induced by errors which lead to increases in attention so that learning can occur. We have uncovered neural correlates of these constructs and the relationship between them by recording from multiple brain areas as rats perform an odor guided decision-making task in which we unexpectedly varied the delay to and size of reward across several trial blocks. We have shown that nucleus accumbens core (NAc) encodes reward predictions, firing strongly for cues that predict more valued reward, whereas firing in dorsal lateral striatum (DLS) is highly associative, encoding action policies such as stimulus-response associations and contextual bias signals (e.g., in this context bias choices to the right). We have also shown that midbrain dopamine (DA) neurons increase firing to unexpected reward and decrease firing to unexpected reward omission. During learning these signed prediction errors transfer to cues, with cues predicting more valued reward eliciting stronger firing. Unlike firing of DA neurons, our work has shown that firing in anterior cingulate cortex (ACC) better reflects an integrated unsigned reward prediction error signal, increasing during unexpected up- and down-shifts in value at the time of the error and during cue sampling on subsequent trials. This work suggests a model by which DA reward prediction errors modify reward prediction signals in NAc and action policy signals in DLS, while ACC increases attention toward stimuli after violations in reward prediction (signaled by DA) so that learning can occur. Cocaine exposure impairs reward prediction signals and prediction error signals in NAc and DA neurons, while increasing the prevalence of contextual action policies in DLS. In Aim 1 we propose to restore the cocaine induced imbalance of processing between NAc and DLS by repairing DA signals via optogenetics. In Aim 2 we will determine if attention and error correlates in ACC are altered after cocaine exposure. Finally, in Aim 3, we will determine how ACC and DA neurons interact during the computation of errors and the development of cue selectivity. By performing these experiments we will gain further insight into how the brain functions normally, how it is disrupted after chronic cocaine use, and determine if repairing neural signals might restore behavior and neural constructs in downstream regions to normal levels.