Thomas J. Palmeri - US grants

Abstract

Palmieri, Thomas
BCS-9910756
Perceptual Categorization and Memory



Any time we decide that some visually presented object is a terrier rather
than a collie, a bottle rather than a jar, or a tree rather than a shrub, we are making categorization decisions by comparing the perceptual attributes of an object with information about categories that have been acquired previously through experience. Developing psychological theories of perceptual categorization requires an understanding of what information is provided by the perceptual system, how that information is compared with category information that has been previously acquired, what kinds of representations are stored in memory, how category representations change with experience, and how classification decisions are made on the basis of evidence for various categories. Perceptual categorization forms a fundamental interface between basic perceptual processes and higher-level cognition. By comparing the relative abilities of various formal models to account for qualitative and quantitative aspects of observed data, this theoretical work will test well-specified hypotheses about the fundamental mechanisms of perceptual categorization. The present work focuses on the role of specific remembered category instances (referred to exemplars) in perceptual categorization as formalized by a proposed exemplar-based diffusion model (EBDM). This model combines elements of Nosofsky's generalized context model of categorization, Nosofsky and Palmeri's exemplar-based random walk model of categorization and automaticity, Logan's instance theory of automaticity, and Ratcliff's diffusion model under a single theoretical framework. According to the proposed model, categories are represented in terms of stored exemplars, evidence for a particular category is a function of the relative summed similarity of a presented item to stored exemplars, and category responses are determined by a continuous-time diffusion process driven by retrieved exemplar information. Preliminary work shows the model able to qualitatively and quantitatively account for both categorization accuracy and categorization response times under a variety of conditions with relatively few free parameters. Several empirical studies are planned to contrast the predictions of the EBDM with other competing frameworks centered around category representations based on prototypes, rules, and decision boundaries. New theoretical advancements are also outlined that specify how perceptual information might evolve overtime within a particular categorization episode.

Experiments and theoretical work investigate basic mechanisms of human perceptual categorization. The working hypothesis is that both the application of rules and the retrieval of instances from memory underlie the human ability to classify objects into different categories. Rule- based processes are assumed to compete against instance-based processes. Early in category learning, rule-based processes dominate (if rules have been provided or can be induced). With more experience classifying items, instance-based processes come to dominate as more information about specific instances have been stored in memory. When rules are simply unavailable, instance-based processes are used entirely throughout learning. This theoretical framework motivates a series of proposed studies. This framework will be contrasted with other proposed theories of perceptual categorization. The first study investigates the relationship between perceptual categorization and recognition memory in normal and memory-impaired individuals (both simulated amnesiacs and Alzheimer's Disease patients will be tested). Instance-based models assume an empirical relationship between categorization and recognition, while multiple memory-system theories do not. The second study specifically tests for shifts from rule-based to instance-based processes in categorization as a function of learning using stimuli and category structures that allow the formation of rules. Subjects will be trained to classify items into categories and will be tested on their generalizations to new items at various points in learning to gauge the types of strategies they are employing. Both empirical studies and theoretical modeling work are proposed to provide converging evidence for categorization strategy shifts. The third study develops and tests a new model of perceptual categorization that combines an instance-based memory-retrieval mechanism with a diffusion process to make classification decisions. Theoretical extensions of this new model are also proposed. Long-term mental health implications of this research stem from a broadened understanding of basic mechanisms of perceptual categorization, a fundamental cognitive process. It can be argued that understanding causes of cognitive deficits in mental disorders requires a complete understanding of normal cognition. Proposed studies with AD patients should lead to important insights into the cognitive deficits surrounding this debilitating and widespread disease.

Stochastic Models of Executive Control in Monkeys and Humans

Abstract

With National Science Foundation support, Drs. Logan, Palmeri, and Schall will conduct a three-year investigation of the executive control processes that underlie flexible responding in monkeys and humans. It is the hallmark of primate intelligence to be able to respond flexibly, focusing on different aspects of the same situation to produce arbitrary responses that are appropriate to current goals. The goal of this project is to specify executive processes computationally and neurally, focusing on the control of attention, categorization, and response preparation. To accomplish this goal, monkeys and humans will perform tasks that require them to make saccadic eye movements toward or away from targets that appear in displays of distractors. Experimental variables will be manipulated to selectively influence attention to the targets, categorization of targets and distractors, and preparation of eye movement responses. The timing and accuracy of eye movements will be recorded in both humans and monkeys performing the task, and the activity of ensembles of neurons in the frontal lobes of monkeys will be recorded while they are performing the task. The overt eye movement behavior of humans and monkeys and the neural activity of monkeys will be described in terms of a mathematically precise computational theory with three distinct components, as follows. (1) An attention component that selects behaviorally-relevant targets from a field of distractors; (2) a categorization component that selects goal-relevant interpretations of target stimuli; and (3) a response preparation component that selects responses necessary to accomplish the goals. The theory provides a common language that makes it possible to relate the overt behavior of humans to the overt behavior of monkeys and to relate the overt behavior of monkeys to the neural activity that underlies it.

The research is significant in three respects. First, it will advance understanding of executive control processes by specifying them concretely in terms of computational and neural processes. Executive control processes are critical in a variety of contexts in the workplace, educational settings, and mental health settings that require people to deal with competing goals and switch between various activities, including the workplace, education, and mental health. The research will have implications for human factors, ergonomics, design of training programs in education and industry, and diagnosis and treatment of mental disorders. Second, the research will advance understanding of neural processes by providing linking propositions that relate single-cell behavior to psychological states of the cognitive processes that the single cells implement. Single-cell behavior makes sense only in the context of the behavior it underlies, and the research will provide that relation. Third, the research will advance understanding of the computations that underlie cognitive processes of attention, categorization, and response preparation. Computational models of these processes are limited by an inability to "open the black box" and observe the inner brain processes that underlie them. Computational models with very different internal processes often predict the same overt behavior. The research will identify cognitive processes with neural behavior, allowing distinctions between these computational models of human and monkey cognition.

HSD-DHBS05 Understanding Conceptual and Cultural Change: The Role of Expertise and Flexibility in Folk Medicine (Norbert O. Ross, Thomas J. Palmeri, David Noelle)
This project explores cultural and expertise differences in conceptual knowledge, conceptual learning, and conceptual change in individuals, and explores the computational mechanisms that underlie conceptual knowledge. We bring together tools, techniques, and insights from cognitive psychology, anthropology, and computer science to investigate the dynamics of conceptual knowledge. Intellectual Merit. At the center of the project is an exploration of the multifaceted dynamics of conceptual knowledge about folk medicine. The primary research site is the Highlands of Chiapas, where the researchers will study within- and cross-cultural differences in conceptual knowledge among Tzotzil Maya and Ladinos. This study examines how folkmedical concepts are represented and used by novices and various experts and relate patterns of conceptual agreement to the structure of social and expert networks. Given the isolation of their community, cultural differences are expected between folkmedical concepts of Maya and Ladinos. The researchers will explore short-term dynamics of conceptual change by sponsoring a medical workshop provided by a local NGO and a Vanderbilt physician, consultants on this proposal. Long-term dynamics of conceptual change are explored by examining changes in folkmedical knowledge by expert and novice groups in Pichataro, a Purepecha community that has witnessed significant change over the thirty years since Garro's original research. Thre researchers also explore conceptual knowledge of folkmedicine for Hispanics in the Nashville area, providing a key comparison group to Pitchataro (an area where many US immigrants originate). The project aims to better understand how conceptual knowledge is represented, how it differs among experts and novices, how it is acquired from instruction, observation, and intervention, and how it changes with new experiences. Computational modeling grounds psychological mechanisms in mathematical and computational formalisms, adding rigor to our theories and allowing the complex dynamics of conceptual change to be explored in simulation. New advances in modeling investigate how conceptual knowledge can be incrementally adjusted from new experiences, how causal knowledge is integrated with rule-based and statistical knowledge, and how conceptual models and agent-based models can be integrated. Ultimately,this project will bridge multiple levels of analysis in order to develop an understanding of how cultural processes constrain individual cognition, how cognitive mechanisms contribute to cultural change, and how these mechanisms can be formally characterized in computational models.
Broader Impacts. This collaboration will enhances the interdisciplinary perspective of investigators from anthropology, psychology, and computer science, influencing future research, teaching, and public outreach. It trains a new generation of scientists to combine methods, perspectives, and theoretical approaches from different fields. It fosters international ties with researchers in Mexico.While the focus is on the dynamics of conceptual knowledge, a specific understanding of conceptual knowledge about folk medicine could contribute toward educating the public - especially our growing immigrant population - about scientific medical treatments. Furthermore, the project includes Vanderbilt Medical School personnel and will help train a new generation of medical staff to deal with the challenges of attending to an increasing number of patients from different cultural settings.

This program aims to revitalize undergraduate computing education through the development of a computational science minor targeted to undergraduate majors in science and engineering. These majors represent a broad community of learners for whom computation is an increasingly critical tool. Modern scientific and engineering applications of significant complexity require high-performance computing solutions, and scientists and engineers require computational thinking competencies to achieve such solutions.

The project introduces concurrent, parallel, and distributed computing concepts, techniques, and patterns early in the curriculum. The advent of multi-core processors at the commodity level, necessitated by the efforts to prolong Moore's law, have made understanding these topics a critical learning outcome. The overall effect of this project will thus be to teach computational thinking competencies, modern software design methods, high-performance computing, and scientific computing to a broad community of learners sorely in need of them. By renovating the curriculum with non-computer science majors in mind, computer science majors will also benefit significantly because concurrent, parallel, and distributed computational methods will be infused into the curriculum earlier than they are normally encountered. The computer science curriculum will also be revitalized by introducing real-world examples from science and engineering that have computational interest.

The demographics of science and engineering are different enough from traditional computer science that underrepresented groups in computing will receive significant exposure to core ideas of computational thinking and computing. The diffusion of computational techniques throughout a variety of disciplines will also change the way computational thinking and computation are perceived and taught within the core computer science discipline. A rigorous evaluation plan throughout all phases of the project will measure quantitatively the changes in the preparation of undergraduates for scientific computing by the proposed computational sciences minor, leading to a greater likelihood of being adopted or adapted by other institutions. By improving the computational skills of scientists and engineers, at Vanderbilt and elsewhere, the project will achieve the broader impact of improving science education in the United States, making students far better prepared for the work force and advanced graduate training.

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).

Graphics Processors (GPUs) are potentially a cost effective and low power vehicle for science and engineering research that requires high performance computation. The primary challenge to the use of GPUs more broadly is the difficulty in programming. Dr. Walker and a team of colleagues representing five different scientific and engineering disciplines propose to pursue research topics in each of the disciplines. By selecting important research topics which require a fundamentally similar computational algorithm for a class of problems labelled "n-body problems", the project offers opportunity for meaningful interdisciplinary collaboration across scientific domains that are normally quite distinct. Since, solutions to this class of problem are particularly well suited to GPUs, there is likelihood of advances in multiple areas of scientific interest at a fraction of alternative costs and power. Therefore NSF's Office of Cyberinfrastructure (OCI) is supporting the acquisition of the instrument.

DESCRIPTION (provided by applicant): The long-term goal of our research is to understand how computational models of performance of visual tasks like locating and shifting gaze to a target a visual array map onto specific neural processes producing that performance. Elucidating this mapping provides converging constraints for discriminating between competing model architectures and provides functional explanations of neural circuit function. The aims of this proposal test, extend, refine, and integrate two major new computational models of target selection during visual search that we have recently developed. Data will consist of performance of monkeys and human participants searching for a target in a visual array in which target location can change unpredictably supplemented by neurophysiological data from FEF that was collected previously. The models provide quantitative accounts of detailed patterns of correct and error saccade behavior during visual search and also provide explanations for the temporal modulation of neurons in frontal eye field (FEF). Unlike previous models of visual search, ours account for the entire range of correct and error response probabilities and response time distributions during efficient and inefficient search, even when the target changes location unexpectedly. Aim 1 will develop, refine, and extend an INTERACTIVE RACE model of saccade target selection. We will test competing model architectures consisting of multiple stochastic accumulators (GO units) that govern when and where a saccade is made, where the nature of the interactions between GO units and the potential inclusion of a STOP unit for exerting cognitive control is manipulated across model variants. Successful models predict response probabilities and response time distributions in monkeys and humans and neural activity observed previously in monkeys. Aim 2 will test, refine, and extend a GATED ACCUMULATOR model of how visual salience is translated into a saccade command. The visual salience representation provided by FEF neurons will be the input to a neural network of stochastic GO units with alternative architectures that implement competing hypotheses about the role of feed forward, lateral and gating inhibition. Aim 3 will integrate these two models. This integration will be guided by new data from human participants performing visual search tasks in which key variables are manipulated to obtain new measures to test competing architectures.

People with perceptual expertise are skilled at making rapid identifications of specialized objects at a glance, often in poor light and camouflage. Forensic experts can accurately match exemplars to latent fingerprints that may be small, distorted, or smudged. Expert radiologists can quickly categorize medical images as normal or cancerous. This project examines perception, categorization, and identification along the continuum from novice to expert performance in two real-world perceptual domains. The overall aim is to understand how fundamental perceptual and cognitive mechanisms are tuned and modified by experience and expertise. The models arising from this project will enable us to understand the development of real-world perceptual expertise and to validate theoretically-grounded measures of expert performance.

Why study perceptual expertise? Just as gifted athletes push the limits of their bodies, or prize-winning mathematicians push the limits of their minds, perceptual experts push the limits of their perceptual systems. Perhaps with better markers of perceptual expertise and a better understanding of how people become perceptual experts, we could identify potential perceptual experts more effectively, train new perceptual experts more efficiently, and evaluate existing perceptual experts more thoroughly. Studying perceptual expertise can also help inform our understanding of the kinds of everyday expertise that we all have, such as recognizing faces or reading words. This can yield new insights into education and workforce training along with new insights into how the ravages of brain damage or disease might lead to perceptual and learning deficits and potentially inform future breakthroughs in evaluation, intervention, or treatment.

DESCRIPTION (provided by applicant): Support is requested to continue a productive collaboration aimed to develop, test, and extend computational models of eye movement control in visual decision making and visual search. Our research program is guided by converging constraints from computational, behavioral, and neurophysiological perspectives that link detailed patterns of behavior in humans and monkeys performing visual saccade tasks with patterns of modulation in neurons recorded in monkeys through the use of computational models that predict behavioral and neural dynamics. We propose new computational modeling of existing monkey behavioral and neurophysiological experiments and new computational modeling of new human experiments that mirror and significantly extend experiments previously conducted with monkeys. Our theoretical foundation is a class of stochastic accumulation of evidence models that mathematical psychologists and systems neuroscientists have converged upon as a general theoretical framework to understand and explain the time course of visual decision making; these include an interactive race model and a gated accumulator model we proposed previously. Unlike most approaches, (1) we quantitatively test alternative model architectures (including race, diffusion, competitive, gated accumulators) on detailed behavioral data in both humans and monkeys, including response probabilities and distributions of correct and error response times for saccades, (2) we constrain model mechanisms and model parameters based on neurophysiological recordings, specifically neurons in frontal eye field (FEF) hypothesized to represent the evolving time-course of task-relevant visual evidence, (3) we quantitatively test model architectures on how well they predict the recorded dynamics of neurons involved in make a visual decision, specifically neurons in FEF that determine when and where the eyes move. Aim 1 will develop and test the gated accumulator model against alternative models of countermanding and control of saccadic eye movements. Aim 2 will develop and test the gated accumulator model against alternative models of speed-accuracy control of saccadic eye movements in visual search. Aim 3 will investigate how to scale the broad class of stochastic accumulator models, including gated accumulator, from a single accumulator associated with each response to ensembles of thousands of accumulator neurons associated with each response. To understand normal behavior as well as illness, disability, and disease, abstract computational models, like stochastic accumulation of evidence models, can be a just right theoretical level in that best-fitting parameters of these models can characterize well individual differences in behavior and provide theoretical markers for understanding brain measures - our models provide that just right theoretical level. Yet to the extent that certain neurological conditions have a biophysical basis at the level of individual neurons and neural circuits, we also need to understand how these abstract computational models map onto neural circuits - making this mapping is also core to our proposed work.

This Science of Learning Collaborative Network brings together researchers from Vanderbilt University, Carnegie-Mellon University, and University of California-San Diego to investigate how and why people differ in their ability to recognize, remember, and categorize faces and objects. Many important real-world problems, such as forensics, medical imaging, and homeland security demand precise visual understanding from human experts. Understanding individual differences in high-level visual cognition has received little attention compared to other aspects of human performance. Recent studies indicate that there likely is far greater variability than commonly acknowledged in the ability to learn high-level visual skills and that such ability is poorly predicted by general intelligence. This project supports a collaborative interdisciplinary research network that aims to develop measures of individual differences in visual recognition, relate behavioral and neural markers of individual differences, develop models that explain individual differences, and relate models with neural data. Because outcomes in many real-world domains depend on decisions based on visual information, developing measures, markers, and models of individual differences can have broader impacts on identifying real-world visual talent and improving visual performance and training. Students and fellows conducting research as part of this collaborative network, including female scientists and underrepresented minorities, will be mentored by scientists from multiple disciplines, providing them with an understanding far deeper than that achievable by a single discipline.

The project will support the activities of a collaborative research network on the study of individual differences in visual recognition. The scientists involved in these interdisciplinary efforts include experts in brain imaging at ultra-high field strength, cutting-edge methods in the development of psychological tests, and cognitive and "deep" convolutional neural network models of high-level vision. The project will investigate how functional brain activity and anatomical brain structure can predict the quality and time-course of visual performance and visual learning. The team will develop and validate tests of visual ability that can be used to make precise predictions about brain activity and behavioral performance. These brain measures and behavioral tests will be related to deep convolutional neural network models; such models are the most successful computer vision models to date, and higher layers of these hierarchical networks provide outstanding models of brain areas critical to object recognition. So far these models have not been used to understand individual differences. Instead of the typical approach seeking to achieve the best performance possible, the collaborative team will seek models that can mirror human variability, making errors when people make errors, being slow when people are slow, and displaying a range of visual abilities and learning as observed in humans.

The award is from the Science of Learning-Collaborative Networks (SL-CN) Program, with funding from the SBE Division of Behavioral and Cognitive Sciences (BCS), the SBE Office of Multidisciplinary Activities (SMA), and the CISE Division of Computer and Network Systems (CNS).