Matthew M. Botvinick - US grants

DESCRIPTION (provided by applicant): A Mentored Research Scientist Development Award (K0l) is requested, to support the establishment of an interdisciplinary research program examining the cognitive mechanisms underlying routine sequential behavior. Routine, goal-oriented action on objects -- the kind of action involved in everyday tasks such as making a cup of coffee -- is fundamental to independent functioning in daily life. When the ability to perform such actions is impaired, as frequently seen in stroke, head injury and neurodegenerative disorders, the impact is typically devastating. Understanding the mechanisms underlying routine sequential behavior, including those involved in representing goals, sequencing actions, and selecting objects, thus represents an important public health objective. The training and research contained in the present proposal pursue this goal by drawing on three important developments in recent research: (1) the application of recurrent neural network models to routine sequential action, (2) detailed tracking of eye and hand movements during the performance of naturalistic tasks, and (3) the analysis of performance in disorders affecting routine sequential action, e.g., action disorganization syndrome (ADS). Recurrent neural networks provide a framework for understanding routine behavior that differs strongly from traditional, schema-based accounts, and which appears to overcome several of their basic problems. In the proposed work, a series of computer simulations will evaluate recurrent networks as models of sequential action on objects, with an initial focus on two theoretically important issues: how objects are selected to become targets of action, and how established procedural knowledge is extended to partially novel task circumstances. Concurrent behavioral experimentation will serve to test predictions of the modeling work, and to provide empirical constraints for the developing theory. Four specific studies are proposed, two using error analyses and chronometric techniques to test predictions about naturalistic task performance in normal subjects and patients with ADS, and two using eye- and hand-tracking techniques to test detailed predictions about object selection and behavior in partially novel settings, again involving both normal and apraxic patients. In support of these research activities, the proposal includes coursework, mentored training activities, and external laboratory rotations, designed to facilitate the acquisition of new skills relating both to computational modeling and empirical research.

[unreadable] DESCRIPTION (provided by applicant): We aim to apply an important new functional neuroimaging technique, known as multi-voxel pattern analysis, to the study of phonological processing. Multi-voxel pattern analysis focuses on distributed patterns of brain activation, as measured with functional magnetic resonance imaging (fMRI), applying pattern analytic techniques to determine the information these patterns may carry. Applications of the technique to visual processing have produced dramatic results, allowing surprisingly fine-grained discriminations among cortical states, and accessing the similarity structure of distributed neural representations. Our preliminary research indicates that multi-voxel pattern analysis may also represent a powerful tool for studying the neural basis of language, affording access to item-specific phonological representations and their similarity relations. The objective of the proposed work is to apply our established approach to several key issues concerning phonological processing. First, we aim to address the question of where phonemes are represented, seeking to confirm the role of several cortical regions in representing phonological information. Within these areas, we propose to investigate how phonemes are represented, examining the similarity structure of phonological representations and relating this to the acoustic and articulatory features of individual phonemes. This portion of the work will also investigate the neural correlates of categorical structure and voice-face integration in phoneme perception, and test for context-sensitivity in phoneme representation. A second major goal of the project is to use multi-voxel pattern analysis to investigate the neural representation of phonology in populations displaying deficits in phonological processing, and in particular among individuals with developmental dyslexia. A final aim is to track experience-induced changes in cortical representations of phonology, in both normal and dyslexic individuals. The proposed research is intended to shed light on basic aspects of normal and disordered language processing, as well as to pioneer the application of new methods for studying the neural basis of language. [unreadable] [unreadable] [unreadable]

Research on human behavior has long emphasized its hierarchical structure: Simple actions group together into subtask sequences, and these in turn cohere to bring about higher-level goals. This hierarchical structure is critical to humans' unique ability to tackle complex, large-scale tasks, since it allows such tasks to be decomposed or broken down into more manageable parts. While some progress has been made toward understanding the origins and mechanisms of hierarchical behavior, key questions remain: How are task-subtask-action hierarchies initially assembled through learning? How does learning operate within such hierarchies, allowing adaptive hierarchical behavior to take shape? How do the relevant learning and action-selection processes play out in neural hardware?

To pursue these questions, the present proposal will leverage ideas emerging from the computational framework of Hierarchical Reinforcement Learning (HRL). HRL builds on a highly successful machine-learning paradigm known as reinforcement learning (RL), extending it to include task-subtask-action hierarchies. Recent neuroscience and behavioral research has suggested that standard RL mechanisms may be directly relevant to reward-based learning in humans and animals. The present proposal hypothesizes that the mechanisms introduced in computational HRL may be similarly relevant, providing insight into the cognitive and neural underpinnings of hierarchical behavior.

The project brings together two computational cognitive neuroscientists and a computer scientist with expertise in machine learning. The proposed research, which includes both computational modeling and human functional neuroimaging and behavioral studies, pursues a set of hypotheses drawn directly from HRL research. A first set of hypotheses relates to the question of how complex tasks are decomposed into manageable subtasks. Here, fMRI and computational work will leverage the idea, drawn from HRL research, that useful decompositions "carve" tasks at points identifiable through graph-theoretic measures of centrality. A second set of hypotheses relates to the question of how learning occurs within hierarchies. Here, fMRI and modeling work will pursue the idea that hierarchical learning may be driven by reward prediction errors akin to those arising within the HRL framework. The project as a whole aims to construct a biologically constrained neural network model, translating computational HRL into an account of how the brain supports hierarchically structured behavior.

DESCRIPTION (provided by applicant): It is a longstanding idea that cognitive processing is subjectively costly: All else being equal, people will choose strategies or actions that minimize cognitive demands. This principle has been presumed to explain a wide array of decision-making phenomena, but has been subjected to surprisingly little direct experimental scrutiny. In recent work, we have gone some distance toward rectifying this, providing the first direct evidence for cognitive demand avoidance. Our initial behavioral findings link cognitive costs to demands on executive function or controlled information processing. Consistent with this, we have obtained neuroimaging results that tie cognitive costs to cortical areas centrally involved in executive control. In the present application, our objective is to build on this initial foundation constructing a fuller account of the role that control costs play in decision making, and of the relevant neural processes. The proposed experiments employ behavioral, neuroimaging and genetic techniques to pursue three specific aims: 1) We aim to evaluate the broader decision-making ramifications of control costs. Using an individual-differences approach, we propose to test for the involvement of control costs in two particularly important areas: A. Research on intertemporal choice has suggested a role for 'self-control' in resisting the temptation of immediate rewards. We aim to test the hypothesis that such forbearance carries control costs, potentially explaining why self-control in intertemporal choice is prone to occasional failure. B. Heuristic use in decision making has been proposed to arise from an effort-minimization principle. We propose to evaluate this hypothesis, by testing for a correlation between heuristic use and cognitive demand avoidance. 2) We aim to further probe the neural foundations of control costs. In particular, we will investigate the relationship between control costs and two key findings from previous work: A. Previous research has revealed a central role for dopamine in determining willingness to exert physical effort. Using a behavioral-genetic approach, we aim to investigate whether dopaminergic function plays a related role in the case of cognitive effort. B. Neuroimaging research has revealed that cortical areas supporting cognitive control operate alongside a 'default mode network.' Having linked control-cost computations with the control-related areas, we propose new neuroimaging work aimed at investigating the potential involvement of the default network in shaping cost-based decision making. 3) We aim to test a novel economic model of cost-sensitive decision making based on economic labor supply theory, a formal framework originally developed to account for effort allocation in labor markets. Initial experiments suggest that labor supply theory may provide a powerful tool for understanding the role of control costs in reward-based decision making. We propose to extend this initial work to address two outstanding questions: A. Does cognitive demand avoidance arise from a purely reactive mechanism, or does decision making actively anticipate demands for control? B. Can the labor-supply model account for shifts in control allocation over time, including so-called 'ego-depletion' effects? PUBLIC HEALTH RELEVANCE: The proposed work addresses a fundamental aspect of the cost-benefit analyses that underlie human decision making: cognitive demand avoidance. A tendency to avoid demands for mental effort has been proposed to explain a very wide array of decision-making phenomena, ranging from strategy selection in arithmetic to racial prejudice, and exaggerations of this bias have been hypothesized to underlie mental fatigue in depression and other illnesses. The proposed work aims to shed new light on the behavioral principles, the genetic determinants, and neural mechanisms underlying cognitive demand avoidance in decision making.

DESCRIPTION (provided by applicant): It is a longstanding idea that cognitive processing is subjectively costly: All else being equal, people will choose strategies or actions that minimize cognitive demands. This principle has been presumed to explain a wide array of decision-making phenomena, but has been subjected to surprisingly little direct experimental scrutiny. In recent work, we have gone some distance toward rectifying this, providing the first direct evidence for cognitive demand avoidance. Our initial behavioral findings link cognitive costs to demands on executive function or controlled information processing. Consistent with this, we have obtained neuroimaging results that tie cognitive costs to cortical areas centrally involved in executive control. In the present application, our objective is to build on this initial foundation constructing a fuller account of the role that control costs play in decision making, and of the relevant neural processes. The proposed experiments employ behavioral, neuroimaging and genetic techniques to pursue three specific aims: 1) We aim to evaluate the broader decision-making ramifications of control costs. Using an individual-differences approach, we propose to test for the involvement of control costs in two particularly important areas: A. Research on intertemporal choice has suggested a role for 'self-control' in resisting the temptation of immediate rewards. We aim to test the hypothesis that such forbearance carries control costs, potentially explaining why self-control in intertemporal choice is prone to occasional failure. B. Heuristic use in decision making has been proposed to arise from an effort-minimization principle. We propose to evaluate this hypothesis, by testing for a correlation between heuristic use and cognitive demand avoidance. 2) We aim to further probe the neural foundations of control costs. In particular, we will investigate the relationship between control costs and two key findings from previous work: A. Previous research has revealed a central role for dopamine in determining willingness to exert physical effort. Using a behavioral-genetic approach, we aim to investigate whether dopaminergic function plays a related role in the case of cognitive effort. B. Neuroimaging research has revealed that cortical areas supporting cognitive control operate alongside a 'default mode network.' Having linked control-cost computations with the control-related areas, we propose new neuroimaging work aimed at investigating the potential involvement of the default network in shaping cost-based decision making. 3) We aim to test a novel economic model of cost-sensitive decision making based on economic labor supply theory, a formal framework originally developed to account for effort allocation in labor markets. Initial experiments suggest that labor supply theory may provide a powerful tool for understanding the role of control costs in reward-based decision making. We propose to extend this initial work to address two outstanding questions: A. Does cognitive demand avoidance arise from a purely reactive mechanism, or does decision making actively anticipate demands for control? B. Can the labor-supply model account for shifts in control allocation over time, including so-called 'ego-depletion' effects?