P Read Montague - US grants

In the vertebrate nervous system, activation of the N-methyl-D-aspartate (NMDA) glutamate receptor on neurons plays an important role in excitatory synaptic transmission, developmental and synaptic plasticity, and neurotoxicity. NMDA receptor activation can also lead to the production of the free radical nitric oxide (NO). Recent experiments suggest that NO production is necessary for synaptic plasticity in the mammalian cerebellum, hippocampus, and cortex. In the hippocampus, the conjunction of presynaptic activity and elevated NO or carbon monoxide (CO) levels potentiates transmission at active presynaptic terminals and leaves inactive terminals unaffected. NO production is also implicated in neurotoxicity, synchronization of neural activity leading to kindling, the control of general cerebrovascular tone, the coupling of local changes in neural activity to local changes in blood flow, and changes in neurotransmitter release. Importantly, NO is a potent vasodilatory substance, hence, mechanisms of synaptic plasticity that employ signals like NO link events associated with learning and synaptic plasticity to pathological states involving blood flow and seizures. Therefore, it is important to understand the computational properties of learning mechanisms that utilize rapidly diffusible signals like NO or CO. Since both CO and NO are membrane permeant gases, they are not restricted to their synapse of origin, rather, they act as volume signals that potentially influence synaptic function throughout a local volume of neural tissue. This fact opens exciting possibilities for the function of the vertebrate brain since standard models of neural transmission and plasticity will have to change to incorporate the ability of one synapse to pass information to another synapse whether or not they innervate the same postsynaptic cell. Additionally, a plasticity mechanism that utilizes membrane-permeant, rapidly diffusible substances would permit associations between afferent inputs to develop in local volumes of neural tissue, hence, this kind of learning mechanism has been called volume learning. There are two long-term goals of this project: 1) To elucidate the theoretical underpinnings of these new forms of neural communication and plasticity, and 2) to understand how such mechanisms would act during both activity-dependent development and learning. We will investigate the theoretical and computational consequences of using membrane permeant, rapidly diffusible signals to modulate synaptic plasticity and transmission. A particular emphasis will be placed on how the exact three dimensional relationships among synaptic contacts influences the production and action of the rapidly diffusible signals described above. This emphasis places a premium on the three dimensional dendritic morphology of postsynaptic neurons since this structure provides the scaffold on which the diffusive signals originate.

DESCRIPTION (provided by applicant): Drugs of abuse operate in part by subverting the adaptive mechanisms (algorithms) that are normally used to value actions and make decisions. It is well known that many if not most drugs of abuse affect midbrain dopamine systems, and in recent years detailed computational models of dopaminergic function (called Reinforcement Learning or RL models) have advanced to the point that they are now used profitably to interpret functional imaging experiments on reward learning in human subjects. These same models can also account for important features of the addicted state. RL models predict the existence and behavioral impact of a range of learning signals including expectation errors (ongoing differences between expectations and actual outcomes) and fictive errors (ongoing differences between `what might have happened'and actual outcomes). Consequently, the connection of RL models to dopamine systems immediately recommends their use as quantitative probes of learning and decision-making in addicted populations. Despite the intimate connection between RL models, midbrain dopamine systems, and reward-guided choice, no model-driven imaging approaches have been used to probe any addicted populations. In this proposal, we seek to fill this gap, and will pursue a rigorous, model-based approach to reward-dependent learning signals, their generation, and their mathematical character in humans undergoing functional magnetic resonance imaging while they execute sequential choice tasks. This effort will be carried out in healthy human subjects and smokers, and we have developed a substantial body of preliminary data to support the specific goals of this project. We have chosen to apply a model-based approach to smokers because they use a legal drug, there is less prevalence of poly-substance abuse, smoking is generally thought to be a gateway drug for other drugs of abuse, and smokers represent a large health burden on society especially in their later years. By using RL models to guide the design, analysis, and interpretation of a range of reward-harvesting experiments, this proposal will yield new insights into the computational underpinnings of reward- dependent choice and its pathological hijacking by a common drug of abuse. PUBLIC HEALTH RELEVANCE Our understanding of the information distributed by midbrain dopamine systems has grown dramatically in recent years to the point that computer models of drug addiction are now usefully employed in the design and interpretation of reward-dependent learning experiments in humans. Drugs of abuse operate in part by subverting these learning mechanisms, which are normally used to value actions and make decisions. In this proposal, we plan to use computational model-based imaging studies to understand how a `gateway'drug (nicotine) perturbs experiential and `fictive'learning signals that guide human decision-making. We have chosen to use apply a model-based approach to smokers because they use a legal drug, there is less prevalence poly-substance abuse, smoking is generally thought to be a gateway drug for other drugs of abuse, and smokers represent a large health burden on society especially in their later years.

DESCRIPTION: (Adapted from the Investigator's Abstract) In the past decade, our conception of neural communication has been broadened by the evidence that certain molecules may act as rapid volume signals that diffuse throughout local regions of neural tissue. One such volume signal, nitric oxide (NO), has been implicated in behavioral learning, synaptic plasticity, blood flow control, neurotoxicity, and the ongoing control of synaptic transmission. One of the novelties of using a volume signal like NO is the idea of cross talk through neural tissue in the absence of direct synaptic contacts. We have termed this cross-talk volume signaling and the long-term changes resulting from it volume learning. The long-term goal of this project is to uncover the computational properties of learning and processing mechanisms in the brain that use volume signals. Our previous work on focused on NO and the computational consequences of learning mechanisms that operate in a diffusion-defined domain. We propose to extend our work on nitric oxide signaling to include two other volume signals: (1) the catecholamine dopamine and (2) changes in external calcium concentrations. The movement of dopamine through the interstices of the extracellular space is important because changes in dopamine delivery appear to encode prediction errors about the time and magnitude of future rewarding events. Moreover, dopamine systems are targets of drugs of abuse, so an understanding of computations carried out by fluctuating dopamine delivery is paramount. For external calcium, experiments have long shown that dramatic changes in its levels attend normal neural activity. We have developed a computational framework in which changes in external calcium levels represent information-bearing signals. This new framework is in its infancy, but current results suggest that a new style of computing is being implemented by external calcium fluctuations. Furthermore, the ubiquitous dependence of synaptic transmission on external calcium levels makes crucial an understanding of how and why information processing in the brain would employ fluctuations in this limited, necessary resource.

DESCRIPTION (provided by applicant): Ongoing economic evaluation is a central function for any system that must operate with limited, finite resources, that is, all mobile creatures. The need for neural valuation mechanisms arises from the sheer breadth and variety of information available to a mobile creature's nervous system, and the fact that sensory stimuli and possible behavioral output must be prioritized. This presents a fundamental information-processing problem: vastly different stimuli and behavioral output must be placed on some common valuation scale. Without internal valuation systems in the nervous system, a creature would be unable to assess the relative value of intrinsically different events like drinking water, smelling food, scanning for predators, sitting quietly in the sun, and so forth. To decide on an appropriate behavior, the nervous system must estimate the value of each of these potential actions or stimuli, convert it to a common scale (currency), and use this scale to determine a course of action. This issue has long been appreciated by behavioral psychologists and economists; however, only recently have the underlying neural substrates been addressed experimentally in humans. Midbrain dopamine systems and the target neural structures to which they project have now been identified as participating in the valuation of future rewarding events. In particular, computational work has shown that a subset of these dopamine neurons encode and distribute a prediction error signal representing the ongoing difference between actual reward and predicted reward. This prediction error model of dopaminergic function has now led to behavioral models that predict human behavior on sequential decision-making tasks. These tasks ask the question: "How do humans value ongoing changes in rewarding stimuli?", a question with important implications for drug abuse. However, there has been no biological measure to correlate with these model-based behavioral predictions. The long-term goal of this proposal is to use functional magnetic resonance imaging (fMRI) during the execution of sequential decision tasks to probe the brain responses that correlate with human performance. This work will provide fundamental insights into valuations mechanisms present in human brains.

[unreadable] DESCRIPTION (provided by applicant): The convergence of 4 separate innovations has exposed the possibility of gaining unprecedented insights into the function of dopamine systems in humans. The many aspects of cognition, learning, decision-making, and action selection that depend on intact dopaminergic function make this opportunity important, timely, and exciting. The four innovations are (1) the invention, implantation, and successful chronic use of deep brain stimulation electrodes (DBS) in late stage Parkinson's patients, (2) the development of computational models of dopaminergic function that have now been validated at the level of single unit activity in dopamine neurons, dopamine release at target structures, and functional imaging experiments (fMRI) in human subjects, (3) the demonstration that learning signals thought to be encoded by dopamine delivery can be tracked in Parkinson's patients using fMRI and simple conditioning and decision-making tasks, and (4) the development of biocompatible electrodes capable of making sub-second measurements of extracellular dopamine during learning and decision-making tasks. This project will yield unprecedented insight into the function of dopamine systems in the human brain and should prove invaluable to a range of problems involving dopaminergic function including addiction and various forms of mental illness. The primary physical deficit in Parkinson's disease (PD) is the loss of 80-90% of dopamine neurons and so this proposal seeks to use the acute phase of DBS electrode implantation to gain new insights into exactly the pathology that tragically afflicts sufferers of the disease. [unreadable] [unreadable] [unreadable]

DESCRIPTION (provided by applicant): Cocaine use and its associated risky sexual behaviors (e.g., sex with multiple partners, inconsistent condom use, trading sex for drugs or money) represent a significant contributor to the ongoing spread of HIV. However, little is known about how sex, drugs, and money are valued in this population, nor about the perturbed neural processes mediating these tradeoffs. In this application, we directly address the objective of PAS-07-324 to increase understanding of processes of cocaine addiction that influence decisions about high-risk sexual behavior. We propose to explore this via the convergence of behavioral and neural underpinnings of the pathological decisions made by Chronic Cocaine Users (CCUs) not in treatment using a model-based approach, behavioral decision tasks, and functional magnetic resonance imaging (fMRI). Our overall hypothesis is that the processes of addiction result in a dysfunctional decision system that underlies the risky sexual behavior engaged in by CCUs; in other words, CCUs engage in risky sexual behavior because they discount future outcomes as a result of a hypoactive executive system (in prefrontal cortex) and a hyperactive impulsive system (in limbic brain circuits). To improve our understanding of cocaine addiction processes that influence decisions about risky sexual behavior, we will obtain critically needed information about the CCU's valuation of relevant commodities (sex, drugs, money), recognizing that these commodities serve multiple functions and may interact with one another in novel ways. We will study valuation and inter-temporal choice within and across different commodities to gain new insights into the decisions made by CCUs, including decisions closely tied to the high-risk behavior of trading sex for drugs or money, and how they differ from Recreational Cocaine Users (RCUs) and Community Control Participants (CCPs). We hypothesize that different commodities will show different profiles of effect depending on availability of other commodities (same or different commodities), their temporal location (immediate or later), and the subject group (CCUs, RCUs, CCPs). Additionally, given the existing data, we anticipate systematically replicating that the discounting of money (money now vs. later) will be predictive of HIV risk behavior in a new population (CCUs). The inclusion of neural correlates will permit us to identify for the first time the role of different neurobehavioral decision systems in decision making predictive of HIV risk behavior. By comparing money discounting to discounting of drug and sexual activity within and across commodities, we will determine whether novel discounting procedures and associated neural processes are more predictive of risky behavior than money discounting. Completion of this project will provide substantial new information about neural valuation systems that are altered by addiction and lead to risky sexual behavior. Understanding how the commodities of interest interact with one another and the neural systems that participate in that valuation may suggest new approaches to alter the pathologic valuation and impact risky behavior associated with the spread of HIV.

DESCRIPTION (provided by applicant): One of the fundamental cognitive components of all social exchange is reciprocity - the concept that a social gesture by one agent should be answered by a roughly equivalent gesture from the other agent. Despite the importance of reciprocity as a fundamental cognitive mechanism, our neural and behavioral understanding of reciprocity instincts in humans remains rudimentary. The overall goal of this proposal is to probe reciprocity across a variety of two- agent settings, expose its influence on behavioral responses during such staged social exchange, and generate mathematical data-driven descriptions of its computational components. In all cases, we seek to relate behavioral variables involved in reciprocal interaction between two agents to underlying neural correlates as measured by functional magnetic resonance imaging (fMRI). We make liberal use of economic exchange games commonly employed in behavioral economics. PUBLIC HEALTH RELEVANCE: The fundamental brain and cognitive mechanisms underlying social interactions among humans are almost completely unknown. This tremendous gap in our knowledge is made worse by the fact that many forms of mental illness and brain injury severely impair our capacity to sustain normal cooperative interactions with other humans. This proposal will use functional magnetic resonance imaging, staged social interactions, and computational theory to probe the way that humans carry out reciprocal social exchange.

DESCRIPTION (provided by applicant): Little is known about the basic mechanisms of medical decision-making, or in fact any decision making in similar sorts of high-dimensional environment. We propose that these environments, paired with the social conditions around expertise lend themselves to significant decision biases, particularly confirmation-bias and anchoring, which may in part contribute to slow CER adoption. We feel that studying decision-making processes in doctors under controlled experimental conditions, paired with the collection of neural data will allow us to understand the fundamental mechanisms fueling these biases and lead to better training and behavioral interventions to correct them. We will be using both behavioral and fMRI analysis to examine the computational and mechanistic underpinnings of valuation in medical professionals. In addition we will be using quantitative models to explore the computational mechanisms involved in decision making in this population. Further, we will be applying new techniques using real-time fMRI feedback for behavioral modification. Confirmation-bias, success chasing, and anchoring are among the potential causes for faulty updating in physicians. Our preliminary data show that our low-performing subject physicians had a tendency to come to a conclusion quickly and then ignore treatment failures that might invalidate their beliefs. This confirmation-bias led to our subjects forming suboptimal treatment algorithms. In this aim, we will: 1) estimate models of physician learning in a controlled experimental setting in both medical and non-medical contexts, and 2) estimate models of belief formation and information search in physicians given extra external prior information, such as a previous diagnosis;identify the behavioral and neural markers of physicians who are particularly adept at this process;and assess the efficacy of a simple behavioral intervention on outcomes. Our preliminary work showed that high-performers in our clinical decision-making task showed distinctly different neural activations. We believe that by giving physicians feedback on their neural responses as well as their treatment outcomes we will be able to improve their ability to correctly learn the optimal treatment algorithm. PUBLIC HEALTH RELEVANCE: Little is known about the basic mechanisms of medical decision-making. We propose that such multidimensional environments, paired with the social conditions around expertise, lend themselves to significant decision biases, which in part may explain slow CER adoption. We will study decision-making processes in physicians under controlled experimental conditions, paired with the neuroimaging data, to allow for better understanding of the fundamental mechanisms fueling these biases.

? DESCRIPTION (provided by applicant): Diseases and disorders directly affected by dopamine systems (e.g., drug addiction and Parkinson's disease) highlight the importance of these systems in motivated human behavior and cognition. The dopamine system is known to be a critical component of normal learning, reward processing, and decision-making (reviewed in Montague et al., 2004). Unfortunately, our present knowledge of dopamine systems in human brains is relatively sparse compared to the wealth of experimentation and computational modeling on these systems in rodents and non-human primates. Previously, technological constraints have limited direct experimentation in human brains. This proposal capitalizes on our group's recent technological innovation, which was supported by the NIH R21 mechanism - CEBRA: R21DA024140 - and resulted in the successful completion of the first sub-second measurements of dopamine release in a human brain. Furthermore, these measurements took place during an active decision-making task that was framed by computational models of learning and reward processing (Kishida et al., 2011 and Kishida et al., under review). We propose to pursue three specific aims, which combine our technological advance with active learning tasks designed to probe the role of dopamine in human behavior. Our aims incorporate three learning signals, where actual and counterfactual experience will each be examined in human striatal responses. The proposed work will inform on the controversial role for dopamine in reward/movement interactions. The experiments proposed will yield unprecedented insight into the function of the dopamine system in the humans afflicted with Parkinson's disease and Essential Tremor. With the support of the NIH (R21DA024140), our team successfully developed a complete prototype system for making electrochemical measurements of dopamine delivery in the human brain. Feasibility has been demonstrated by obtaining the first dopamine measurements in the striata of subjects with Parkinson's during a decision-making task. This substantial preliminary work is now ready for a larger scale with specific hypothesis testing about the role of dopamine systems in Parkinson's disease, Essential tremor, and human decision-making and behavior.

SUMMARY Dopamine and serotonin systems in the human brain represent two key neuromodulatory signalling systems that impact mood, value-based decision-making, learning, and a host of other cognitive functions. Despite their importance, there is no consensus view from a psychological or computational perspective concerning their exact information-processing functions. Consequently, there is a scarceness of strategies to treat disorders that afflict these systems. We believe that this gap exists primarily because of methodological limitations. While there are many fast methods for recording action potential activity and local field potentials, similar progress for tracking `the other end of the problem' - neurochemical dynamics - has lagged far behind. This is unfortunate since just considering the catecholamines dopamine, serotonin, and norepinephrine, the worldwide health burden of dysfunction in these neuromodulatory systems is immense approaching 400 million people worldwide if one includes just Major Depression and Attention-deficit hyperactivity disorders (WHO, 2017). The overall goal of this project is to investigate the computational and neuromodulatory substrates of interactive social processes hypothesized to be trans-diagnostic RDoC constructs. We will utilize two levels of analysis (molecules/circuits and behavioral/computational) across two RDoC constructs (systems for social processes: perception and understanding of others, subconstruct understanding mental states, and positive valence systems: reward learning, subconstruct reward prediction error) to make inroads into understanding the computational and neural underpinnings of social interaction in humans. Crucially, the interactive social tasks used in this proposal generate an important class of learning signal ? a reward prediction error signal ? but expressed in the context of an inter-personal interaction. Our two levels of analysis employ (1) computational models of human (two-agent) social exchange estimated from detailed, observed behavior and (2) unique sub- second measurements of dopamine and serotonin from human striatum (three separate sites: caudate, putamen, ventral striatum) during the window-of-opportunity afforded by deep brain stimulating (DBS) electrode implantation surgery for Parkinson's Disease, Essential Tremors, Obsessive Compulsive Disorder.

PROJECT SUMMARY Much work in decision neuroscience has been predicated on the hypothesis that dopaminergic signaling plays a critical role in value representations and their updating, and recent work has linked abnormalities in such value representations to specific features of psychiatric illness (e.g., anhedonia in major depression). However, despite the fundamental role that risk plays in evaluating choice options as well as the role sensitivity to risk plays in psychiatric illness, including anxiety disorders at one extreme and health risk behaviors like substance use at another other, an understanding of the neurobiology of risk remains elusive. Indirect evidence from pharmacological studies have suggested both serotoninergic and noradrengergic signaling may play roles in representations of risk; however, direct measurement of serotonin or norepinephrine signaling during risky choice has yet to be examined in humans. Recent advances by MPI Montague?s group allow the unprecedented ability to track neuromodulator responses with high temporal resolution and chemical specificity. Specifically, MPI Montague?s team is able to directly and simultaneously measure dopamine and serotonin responses in awake humans with the temporal resolution (~ 1 ms) required to examine the relationship of neuromodulator release with decision-making processes. For signal identification and extraction, the recording method uses machine-learning algorithms (elastic net regression) combined with electrochemistry using only off-the shelf hardware and software. The product of this ?elastic net electrochemistry? is recordings of in vivo neuromodulator fluctuations at sub-second resolution. This application merges the decision neuroscience expertise of MPI King-Casas with these advances of MPI Montague to directly examine serotonergic, noradrenergic, and dopaminergic functioning during risky choice. To achieve this goal, we will record neuromodulator responses in participants with medication-resistant epilepsy who already have intracranial depth electrodes in place for phase-II monitoring. Depth electrodes will be implanted by our neurosurgery colleagues at Virginia Tech?s medical affiliate Carilion Clinic (Carilion Clinic PI: Witcher). During recording, participants will perform i) a risk elicitation task (Holt & Laury type task) and ii) a reward learning task (multi-arm bandit task) that have been shown by our group and others both to reliably evoke neural responses associated with risk and representations as they are monitored in a standard (i.e., non-surgical) hospital suite. Depth recordings will be made using a standard montage that includes multiple contacts along the dorsal-rostral axis of the medial prefrontal cortex.

PROJECT SUMMARY This proposal develops the next generation magnetoencephalography (MEG) for human social neuroscience by combining the latest available technology in optically-pumped magnetometers (OPMs) and magnetically shielded rooms (MSRs). Successful completion of the proposed research and development will enhance MEG's ease of use to enable the first ever 2-person face-to-face MEG recordings of social interactions. Motivated by a pressing need to improve the relevance of human neuroimaging - which includes upright, social movement - scalp-based sensors represent the most promising set of technologies that are both available now and are also expected to enjoy major improvements over the next several decades. While electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) are currently capable of such recordings, both suffer from limitations that would be complimented or ameliorated by ?untethered? MEG. Within this context, the research objectives of this proposal are threefold. First, Aim 1 integrates the latest generation of OPM sensors and MSR technology to deliver a next generation platform for OPM-MEG experiments. These data, then provide a positive control to use this system for multi-person, movement tolerant neuroimaging in Aim 2. Finally, in Aim 3 we will evaluate sensor mounting strategies and source reconstructions strategies that will avoid obscuring parts of the face and could reduce cost and improve experimental ease. The work proposed will conducted by an assembled team of the world's leading academic and industry experts in OPM-MEG, magnetic shielding, social neuroimaging, and neuroimaging data analysis.