Jacqueline Gottlieb - US grants

DESCRIPTION (provided by applicant): To make adaptive decisions, animals require sophisticated mechanisms for tracking the desirability, magnitude and probability of an outcome. In addition, animals must integrate outcome information over multiple time scales, and must have the capacity to orient not only toward immediate but also toward delayed rewards. For humans, nearly every decision requires some form of advance planning, on time scales that can range from relatively short (going to the store in order to get an ice cream) to the exceptionally long (investing money now in order to retire in several decades). The capacity for planning is critical for normal behavior, and impairments in this faculty can have devastating consequences. Two major psychiatric disorders - attention deficit disorder and addiction - have been associated with planning deficits and with dysfunction in frontal and parietal cortical areas. However, despite the importance of long-term planning, the underlying neural mechanisms are poorly understood. Here we address this question using a sequential decision making task in conjunction with single- neuron recording in monkey lateral intra-parietal area (LIP). The task requires monkeys to choose between two alternative strategies - a suboptimal strategy oriented toward an immediate reward, and an optimal strategy oriented toward a final (delayed) reward. While prior studies have focused on ordering and timing mechanisms during sequential actions, here we focus on the underlying decisional - action evaluation - computations. How does the brain assign value to an action, based on a future (projected) reward that only arrives after multiple intervening sensory and motor events? How does the brain distinguish between actions that are and those that are not critical for obtaining a delayed reward? And finally, how does the brain resolve inter-temporal conflict, foregoing an immediate reward in order to remain focused on the final goal? While most prior studies have focused on the frontal cortex, here we record from the parietal lobe which, together with frontal areas, is likely to be critical for advance planning. Thus the proposed experiments extend our knowledge of the decision mechanisms required for long-term planning in a broader cortical network. These studies are necessary for a mechanistic understanding of long-term planning and its breakdown during mental disorders. 1 PUBLIC HEALTH RELEVANCE: The capacity for long-term planning is critical for normal decision making. Impairments in planning are thought to be part of the etiology of psychiatric disorders, including attention deficit disorders and addiction. While studies of long-term planning have been carried out in human subjects, the underlying neuronal mechanisms remain largely unknown. The present experiment probes the mechanisms by which the brain assigns action value based on delayed, rather than immediate rewards during a sequential decision making task. The goal is to provide a detailed understanding of planning mechanisms during normal behavior and mental illness. 1

DESCRIPTION (provided by applicant): Drug abuse and addiction are severe disorders characterized by excessive and compulsive drug consumption, affecting millions of individuals in the Western world. Converging evidence shows that long-term exposure to drugs affects subcortically mediated hedonic and emotional learning processes, as well as cortically dependent decisions. An important source of decision bias in drug abuse comes from Pavlovian stimuli and contexts which, by virtue of their association with drugs of abuse gain incentive salience and the power to influence actions. In addicted individuals, the mere exposure to drug-associated cues can motivate further drug consumption or even trigger relapse after a period of abstinence. Despite the importance of these Pavlovian effects on cognitive decisions, their neural mechanisms remain unknown. Recently we discovered that Pavlovian learning strongly affects activity related to attention and eye movements in a cortical association area, the lateral intraparietal area (LIP) of the monkey. Conditioned stimuli predicting reward ("good news") attract attention and produce local excitation in the spatial attention map in LIP;stimuli conditioned to predict no reward ("bad news") repel attention from their location and produce local inhibition in LIP. Like the effects described in rodents and humans, the effects in our study arose automatically and persisted even though they interfered with performance;moreover, after prolonged training the effects increased and became habitual, partly resistant to contextual control. We aim to extend these findings by testing whether, in addition to biasing simple orienting, Pavlovian stimuli also bias more complex decisions that are not spatially related to the cues. These studies are innovative because they establish a new link between two disparate research traditions - the study of Pavlovian learning (traditionally conducted in rodents using pharmacological and behavioral methods) and the study of decision formation (traditionally conduced in monkeys using single-neuron recordings). Thus, they will provide insight into interactions between emotional learning and decision formation in normal behavior and in relation to drug abuse, and may suggest cognitive strategies for overcoming the power of emotional learning when this learning has undesirable, maladaptive effects. 1 PUBLIC HEALTH RELEVANCE: Abundant evidence shows that Pavlovian learning plays an important role in drug abuse. Cues or environments associated with drugs of abuse gain incentive salience and the power to elicit cravings, potentially contributing to the maintenance or relapse of drug consumption. To understand the mechanisms of these effects we test the influence of Pavlovian learning on attention using single-neuron recording in a parietal area related to attention and decision formation in rhesus monkeys. The long-term goal is to understand interactions between emotional learning and cognitive decisions in normal behavior and drug addiction. 1

DESCRIPTION (provided by applicant): To act adaptively in complex worlds, animals require the ability identify relevant information, which helps improve their predictions and reduce the uncertainty of future actions. The selection of relevant information is accomplished by systems of attention, which have been intensively investigated in the visual domain. These studies have focused on attentional modulations of sensory perception and on mechanisms of oculomotor target selection. However, less is known about the cognitive mechanisms of attentional control: how does the brain identify stimuli that are relevant or attention worthy in a given task? Evidence from computational and behavioral studies suggests that relevance depends both on the reward associations and on the informational properties of a sensory cue. However, informational properties (uncertainty, new information) are investigated mostly in studies of learning, while studies of oculomotor control focus on the role of reward. Thus it is unknown how the brain computes relevance based on both the reward and informational constraints of a task. In the oculomotor system, two areas implicated in attentional control are the frontal eye field (FEF) and the lateral intraparietal area (LIP). However, studies of these areas have focused on their role in target selection rather than computing relevance per se. In addition, these studies reveal very similar responses in FEF and LIP, raising questions regarding their specific roles. Here we aim to answer these questions by using a combination of single neuron recordings and reversible inactivation and explicitly comparing FEF and LIP. We seek to understand (1) how is target selection in FEF and LIP influenced by reliability/uncertainty and redundancy/new information, (2) how are these variables related to expected reward and (3) whether LIP/FEF have distinct contributions to discounting redundant information and attending to reliable versus uncertain cues. The studies are innovative in that they link two topics that have been pursued in isolation, the study of attention on one hand and that of learning and statistical inference on the other, and directly compare the functions of FEF and LIP. The studies have high clinical significance since attentional disturbances are strong in many psychiatric disorders including attention deficit disorder, depression and addiction. Thus, the studies will lead to an integrated theory of attentional control based on reward and uncertainty reduction, processes that are core for decision making in both health and disease.

? DESCRIPTION (provided by applicant): Saccades are rapid eye movements through which foveate animals examine visual scenes. Converging evidence from humans and monkeys show that a key role of saccades is to samplie visual information to guide subsequent actions. Consistent with these behavioral observations, neurons throughout the monkey visual system respond selectively for task-relevant stimuli relative to distractors, and this enhancement is thought to arise in part from two interconnected areas, the lateral intraparietal area (LIP) and th frontal eye field (FEF). However, we have little understanding about how this selection is made: what are the computations by which FEF and LIP neurons identify relevant items? Here we examine a dual-control hypothesis that is suggested by computational modeling of natural behavior and states that top-down gaze control is based on two interacting factors: the gains in reward and gains in information that a saccade is expected to bring. We use a novel two- step task where monkeys make two saccades on each trial - a first saccade to sample visual information and a second saccade to report a decision based on that information. We manipulate the reward and informational demands (uncertainty) of the final decision, and examine how this affects target selection for the first, information sampling action. Using this paradigm we obtained preliminary evidence that supports the dual control hypothesis and shows that (1) target selective responses in LIP show a strong modulation by both reward and uncertainty (or equivalently, expected information gains), and (2) reversible inactivation of this area impairs target selection based on cue validity (expected information gains) but not simple reward associations. We propose to extend these findings to systematically explore how reward and information gains modulate saccade selection responses in FEF and LIP (Aim 1), how the two factors modulate responses related to covert selection (Aim 2), and how reversible inactivation of each area impacts reward and information-based selection (Aim 3). These studies are highly significant and innovative for oculomotor research because they (1) provide new and much needed empirical evidence to support computational modeling of top-down gaze control, (2) clarify functional distinctions between FEF and LIP, which so far appear to perform very similar functions, (3) clarify how oculomotor decisions depend on information gains in addition to primary rewards, thereby resolving longstanding confounds between reward and attention and linking neural studies with studies of natural gaze control. The studies are also significant and innovative on a broader scope because they are among the first to investigate the relation between attention and decision making and to probe the neural basis of decisions based on the value of information. Thus, the studies are expected to have a strong impact on our understanding of attention and decision making, two core cognitive functions that are critical in complex behaviors and psychiatric diseases including attention deficit disorder, drug addiction, anxiety and depression. 1

Saccades are rapid eye movements through which foveate animals examine visual scenes. However, while in natural behavior saccades implement an active sensing strategy for sampling task-relevant cues, these strategies have been seldom investigated and very little is known about their neural mechanisms. In the vast majority of laboratory paradigms, participants (humans or non-human primates) make saccades in a purely exploitative fashion - to harvest rewards, as the primary goal of the task, rather than gather information about future task states. The use of saccades for information sampling raises two major questions beyond those arising in traditional paradigms. First and foremost, how are saccades motivated by uncertainty ? or expected gains in information ? in addition to reward expectation? Second, how is the selection of an informative target (before the saccade) coordinated with the processing of the sampled information (after the saccade)? We examine these questions in the monkey lateral intraparietal area (LIP), a cortical area implicated in top-down oculomotor control, using neural recordings and reversible inactivation combined with a novel information sampling paradigm. In this paradigm, monkeys are free to deploy gaze to obtain information about a reward that accrues after a delay, but receive no rewards or punishments for a specific gaze pattern. Preliminary evidence shows that saccades depend on the monkeys? prior beliefs about reward probability and uncertainty, consistent with computational models of gaze allocation in natural conditions; in addition, target selective LIP cells (1) are modulated by reward and uncertainty, (2) show a trans-saccadic integration of the effects in their pre- and post-saccadic responses, and (3) respond specifically in relation to information sampling rather than global changes in arousal/motivation. Starting from these observations, we aim to understand the functional contribution of area LIP to information sampling based on reward probability and uncertainty. The results are significant because (1) they represent the first investigation of top-down deployment of gaze as an active sensing policy, (2) they will clarify the role of LIP in reward decisions, which has been the topic of protracted debate and (3) will help integrate research on attention and decision making, two cognitive functions that are naturally coordinated but have been so far studied separately. Thus, the studies are expected to have a strong impact on our understanding of attention and information constrained decisions, which are critical in complex behaviors and psychiatric diseases including attention deficit disorder, drug addiction, anxiety and depression.