2017 — 2022 |
Liljeholm, Mimi |
N/AActivity Code Description: No activity code was retrieved: click on the grant title for more information |
Career: Instrumental Divergence and Goal-Directed Choice @ University of California-Irvine
Theories of instrumental behavior distinguish between goal-directed decisions, motivated by a deliberate consideration of the probability and current utility of their consequences, and habits, which are rigidly and automatically elicited by the stimulus environment based on reinforcement history. In spite of the far-reaching implications of this distinction, ranging from the structuring of economic policies to the diagnosis and treatment of behavioral pathology, much is still unknown about what factors shape goal-directed decisions and what conditions prompt a transition from goal-directed to habitual action selection. Generally, while computationally expensive, a goal-directed strategy offers greater levels of flexible instrumental control. Since subjective utilities often change from one moment to the next, such flexibility is essential for reward maximization and thus may have intrinsic value, potentially serving to motivate and reinforce specific decisions, as well as to justify the general processing cost of goal-directed computations. A critical requirement for flexible instrumental control, however, is that available action alternatives yield distinct outcome states. With the support of this NSF Career award, Dr. Mimi Liljeholm is investigating the novel hypotheses that instrumental divergence? the difference between outcome probability distributions associated with alternative actions? can shape choice preferences, induce conditioned reinforcement, and arbitrate between goal-directed and habitual decision strategies. The objective of this research is to address important gaps in current knowledge about the nature and limits of goal-directed behavior, using a combination of innovative experimental designs, computational modeling and functional magnetic resonance imaging (fMRI). The educational component of the award provides hands-on training in neuroimaging methods, and in the computational and neural bases of learning and decision-making, at undergraduate and graduate levels.
All studies use a simple gambling task in which alternative actions yield different colored tokens, each worth a particular amount of money, with various probabilities. In studies assessing a preference for flexible instrumental control, the relevant choice is between pairs of actions with different levels of instrumental divergence. Expected monetary pay-offs vary independently of instrumental divergence across options, dissociating the relative contribution of each factor to behavioral choice performance. Studies investigating the capacity of high instrumental divergence to induce conditioned reinforcement measure changes in the affective valence of visual stimuli based on their association with high versus low instrumental divergence. Finally, following extended exposure to high versus low instrumental divergence, the degree to which behavior is goal-directed or habitual is assessed using a standardly employed outcome devaluation procedure, in which the monetary amount associated with a particular token color is altered: Goal-directed, but not habitual, decisions are modulated by such changes in the utility of sensory-specific outcomes states. Neuroimaging data is acquired by scanning participants with fMRI as they perform the task, and a reinforcement learning framework is used to model the intrinsic value of flexible instrumental control (by treating instrumental divergence as a surrogate reward) at behavioral and neural levels. Since many psychiatric disorders are characterized by an abnormal sense of agency, and addiction associated with a rapid transition from goal-directed to habitual action-selection, broader impacts of this project include the potential development of pre-clinical diagnostic assays for early detection of cognitive, affective and behavioral pathology. The concepts advanced under this project may also help improve the performance of reinforcement learning algorithms, for example by using instrumental divergence to specify new optimization criteria, potentially benefiting medical, industrial and commercial applications of artificial intelligence.
|
0.981 |