Marshall Hussain Shuler - US grants

Description (provided by applicant): In recently published findings, I provided evidence that pairing visual stimuli with subsequent reward leads to the emergence of reward-timing activity in the primary visual cortex. Therefore, neural activity in the primary visual cortex is not simply a re- presentation of a visual cue, but rather relates the processing of its behavioral significance. These findings have implications for understanding how our brains imbue sensory experience with behavioral meaning, and forms the basis of my long-term career goal: to investigate, at a leading research university, the interaction between sensory and reward systems in the formation of adaptive behaviors. The properties of this reward timing activity in the primary visual cortex suggest that it is generated locally. If so, V1 is privy to the acquisition of reward by the animal. With attributes ideal for mediating plasticity in V1, the cholinergic system is the most likely system to convey such a reward signal. Therefore, the proposed research is directed towards testing the hypothesis that reward timing activity is generated within the visual cortex by the interaction of cholinergic inputs signifying reward and thalamic inputs signifying the stimuli that predict reward. To address this question, the research program proposed consists of multi-site extracellular recordings combined with 1) mimicking the action of the cholinergic system in the formation of reward timing by a novel application of photolytically uncaged acetylcholine agonist in vivo, 2) local pharmacological blockade to establish the impact of acetylcholine on the formation of reward timing in the visual cortex, and 3) hijacking the cholinergic axonal terminals within V1, to demonstrate causality between their activity and the emergence of learned reward timing. The consequences of mimicking, blocking, and hijacking the cholinergic system on the emergence of reward timing activity in V1 will be compared to that which emerges in normal rats. If the hypothesis is correct, the primary visual cortex could be a powerful model system for dissecting mechanisms of reward-based learning. The insight gained from these experiments will inform upon the role brain reward systems have on shaping sensory systems, of which very little is known, yet which impact directly our understanding of human pathologies such as Alzheimer's, schizophrenia, and drug abuse. PUBLIC HEALTH RELEVANCE The mechanism by which the brain comes to attribute behavioral meaning to environmental stimuli is unknown, though it is hypothesized that neuromodulatory "reward" systems relate the outcome of behavior with preceding neural activity. Here, I test the hypothesis that one such system mediates the learning of behavioral meaning in the primary visual cortex by mimicking, blocking, and hijacking the brain's cholinergic neuromodulatory system. Success in this study would establish experimental evidence for theories of reinforcement learning, furthering our understanding of neural mechanisms of learning and memory and of cognitive impairment due to disease and aging.

DESCRIPTION (provided by applicant): This is a resubmission of a research proposal to uncover the basis of timing estimation in the brain based on recent experimental and theoretical advances. This is a combined experimental and computational proposal. A fundamental task accomplished by the brain is the formation of adaptive behaviors generated in response to learned contingencies between environmental stimuli. Yet, the neural process by which we learn the behavioral relevance of environmental cues, specifically visual cues, and the mechanism by which brains generate temporal expectancies based on such visual evidence, is unknown. Exemplifying this process and motivating the work proposed here is the unexpected finding by Shuler and Bear[1] that pairing visual stimuli with delayed reward leads to the emergence of reward-timing activity in the primary visual cortex (V1). This finding suggests that V1 does not act simply as a passive filter bank surveying the visual world, but instead contains complex internal programs that signal the behavioral relevance of visual events, and participates in computing the animals' behavioral response. Based on these recent findings, we developed a network-based theory of timing computation in cortex. This theory can account both for how times are computed and how they are learned. In this proposal we develop and test this model for understanding this issue, proffering a series of experimentally tractable predictions that test a radical notion: Learning visually-cued expectancies occurs locally within the primary visual cortex (V1) as a result of an interaction between an impinging reinforcement signal conveying the outcome of behavior with prior synaptic activity. We will test using optogenetic techniques, the identity and nature of the reward signal in V1, and consequently also test if the computation is local to V1. Using a combination of behavioral electrophysiology and computational techniques we will establish if there is a relationship between the observed dynamics in V1 and behavior and manipulate that relationship ontogenetically, and also account for the experimentally observed scalar property in time perception on the basis of the observed physiology and the computational model, all hallmarks of our model. The neural mechanism by which V1 - and more broadly, cortex - comes to express cue-reward intervals is unknown and is addressed in this study. Consequently, the results of this study will bear greatly on neural processes of memory and learning, forming a basis of understanding for cognitive dysfunction. PUBLIC HEALTH RELEVANCE: The proposal will address the neural mechanism by which the brain can learn the behavioral significance of environmental cues, specifically visual cues, and generate neural and behavioral temporal expectancies based on such visual evidence. These discoveries will provide new insight into our current understanding of reinforcement learning. Consequently, the results of this study will bear greatly on neural processes of memory, learning, and reward, forming a basis of understanding for cognitive dysfunction.

Project Summary Learning and decision-making are driven by expectations of future reward. Two key parameters determining the valuation of future rewards are 1) ?how much? reward to expect, and 2) ?when? to expect it (ie, Reward Prediction). However, how reward prediction is generated by the brain in response to predictive cues is poorly understood. Exemplifying the when of reward-prediction is so-called ?reward timing? activity in the primary visual cortex (V1), which emerges in V1 when visual stimuli are behaviorally conditioned with delayed water reward. Previously, we have demonstrated that this timing activity is generated within V1 itself, requires basal forebrain cholinergic innervation to be formed, and informs on the timing of visually-cued actions. More recently we have conceived, computationally, how the harder problem of reward prediction signaling could be learned within a network by the action of a reinforcement signal. Together, these observations make V1 a powerful system to address how reward prediction can be learned and reported neurally. Combined with our theory of intertemporal decision-making, these observations well motivate our research into how V1 circuitry produces reward prediction signals, how cholinergic innervation teaches that circuitry to learn reward prediction, and whether reward prediction signaling in V1 informs decision-making. Whether behavioral conditioning leads to V1 learning to produce reward prediction signals in V1 is unknown, though pilot data indicates it is (Aim1a). Testing predictions from our formal model, reward prediction responses will be mapped onto opto-identified inhibitory cell types (Aim1b). Optogenetic perturbation of inhibitory subtypes will specifically test predictions on the expression of reward prediction (Aim1c). Pilot Ca2+ imaging of cholinergic fibers within V1 indicates that reward is indeed reported to V1 by this input as predicted (Aim2a). Therefore, the degree of cholinergic activation within V1 (as controlled optogenetically) may serve to teach V1 to express reward prediction signaling (Aim2b). This ability to optogenetically mimic reward signaling affords a means to test whether learned reward prediction signaling in V1 informs decision-making: By instilling fictive reward expectancies atop behaviorally conditioned reward expectancies of otherwise equal value, reward prediction signaling in V1 can be shown to impact future decision making (Aim3a&b). Observations made here will advance an understanding of the mechanisms? impaired in many cognitive diseases?of how the behavioral meaning of sensory information is learned in order to remember past experiences and inform decision making.

Project Summary Alzheimer's disease (AD) degrades the ability to learn and make appropriate decisions. As the basal forebrain cholinergic system is highly susceptible to amyloidosis, one neuromodulator that is especially implicated in cognitive decline caused by AD is acetylcholine, which is essential in many forms of learning and memory. While higher-order brain areas associated with decision-making have been intensively investigated as they relate to AD, recently there has been a call for greater focus on the consequences of AD in sensory- and motor-related areas. As we have previously characterized a cholinergic-dependent form of learning and memory in the visual cortex, one sensory- and motor-related system that is particularly attractive in this regard is the visual corticostriatal pathway. A rudimentary, yet fundamental, form of decision-making that epitomizes a sensorimotor transformation carried out by the corticostriatal pathway is when to time reward-seeking actions in response to reward-predicting stimuli. Observation of neural activity from the visual cortex and the dorsal striatum reveal visually-cued timing activity to expected reward, providing a window into the process of transforming visual cues into reward-seeking motor action. By combining a mouse model of AD that develops amyloidosis with a line that affords a means to control and functionally image cholinergic axons, the effects of amyloidosis on cholinergic-dependent interval timing activity can be assessed, neurally and behaviorally, compared to that caused by the normal course of aging, and rescued by augmenting cholinergic signaling of reward using a novel optogenetic intervention. We hypothesize that aging and amyloidosis disrupts the ability of the visual corticostriatal system to learn and produce visually-cued interval timing activity (Aim1-Impairment), which degrades the ability to produce appropriately timed reward-seeking behaviors, and that the proximal cause is a functional impairment of cholinergic signaling of reward (Aim2-Proximal Cause). We further hypothesize that optogenetic augmentation of intact cholinergic fibers' report of reward will rescue visually-cued interval timing activity in the corticostriatal system, thereby re-establishing appropriately timed reward-seeking behavior (Aim3- Intervention). These aims will 1) reveal new neurophysiological and behavioral biomarkers foretelling future onset of the disease (Aim1), 2) lead to a greater understanding of the causative processes underlying cognitive decline (Aim2), and 3) point to new interventions for mitigating dysfunction caused by AD pathology (Aim3).

Project Summary Learning and decision-making are driven by expectations of future outcomes. Three key parameters determining the valuation of future outcomes are 1) ?how much? to expect, 2) ?when? to expect it, and 3) ?what? to expect (ie, Outcome Prediction). However, how outcome prediction is generated by the brain in response to predictive cues is poorly understood. Exemplifying the when of outcome-prediction is so-called ?reward timing? activity in the primary visual cortex (VC), which emerges in VC when visual stimuli are behaviorally conditioned with delayed water reward. Previously, we have demonstrated that this timing activity is generated within VC itself and requires basal forebrain cholinergic innervation to be formed. We have also demonstrated that this activity informs on the timing of visually-cued actions. Indeed, the dorsal striatum (DS) is VC's direct downstream motor-related target, and it is also observed in pilot data to expresses this activity. Together, these observations make the visual corticostriatal circuit (VC»DS) a powerful system to address how outcome prediction can be learned and reported neurally. Combined with our computational model of how outcome prediction signaling could be learned by reinforcement signaling within VC»DS, these observations well motivate our research into how VC»DS circuitry produces outcome prediction signals, how cholinergic signaling teaches this circuit to learn outcome predictive signaling, and whether predictive signaling in VC»DS informs decision-making behavior. Whether appetitive (Aim1a) and aversive (Aim1b) conditioning leads to the visual corticostriatal circuit learning to produce outcome prediction signals is unknown, though pilot data indicates it is. Testing predictions from our formal model, selectively perturbing inhibitory circuit elements will assess whether VC is a site sourcing predictive signaling to DS (Aim1c). Pilot Ca2+ imaging of cholinergic fibers within VC indicates that reward, as well as punishment is reported to VC (Aim2a) in keeping with its purported role as a teaching signal, but raising the possibility that outcome valence is learned downstream in DS (Aim2b). Therefore, the degree of cholinergic activation within VC may serve to teach VC to express and source to DS signals predicting the time and magnitude of expected outcomes (Aim2c), while DS may serve as a site associating those predictive signals with their appropriate reward-seeking/punishment avoiding behaviors. The ability to optogenetically mimic outcome signaling affords a means to test whether learned outcome prediction signaling in VC»DS informs decision-making: By instilling fictive reward expectancies atop behaviorally conditioned reward expectancies of otherwise equal value, outcome prediction signaling in VC»DS can be shown to impact future decision making (Aim3a&b). Observations made here will advance an understanding of the mechanisms?impaired in many cognitive diseases?of how the behavioral meaning of sensory information is learned in order to remember past experiences and inform decision making.