Stephen Kosslyn - US grants

Behavioral studies of pigeons probe the higher levels of biological vision. Preliminary studies with complex visual stimuli (including natural scenes) have shown great similarities between pigeon vision and early human vision, despite an evolutionary gap of 100 million years. This suggests the existence of extraordinarily robust biological solutions to visual analysis. Further experiments on visual shape categorization are to be performed on human and pigeon subjects. The goals are 1) to infer underlying algorithms in the two species; 2) to examine adaptation of these algorithms to the circumstances of their use; 3) to formalize the algorithms and their adaptive capacities mathematically; 4) to synthesize biological visual categorization on a computer; and 5) to develop experimental procedures with the long-term goal of creating artificial visual machines.

For many years, most researchers believed the left cerebral hemisphere to be specialized for language and the right for perception and mental imagery. In the early 1980's, it became clear, however, that the left hemisphere is actually better than the right at some imagery tasks, but it is worse than the right at other imagery tasks and roughly as good as the right at still other imagery tasks. It is now clear that complex tasks such as imagery are not performed by a single part of the brain, but rather arise from the joint action of many components working in concert. Some of these actions apparently are carried out by the left hemisphere, and some are carried out by the right. This research will characterize the nature of these component processes and will determine how they are lateralized to the left or right hemisphere. For example, experiments will examine the processes that allow one to activate a pattern into a visual image, to rotate patterns in images, and to "inspect" patterns in images. Patients who have suffered damage to the left or right cerebral hemisphere will be tested on a set of imagery tasks, and their deficits compared. The theory underlying the research holds that some tasks will be impaired more following damage to the left hemisphere than the right and others will be impaired more following damage to the right hemisphere. Examination of which tasks are always lateralized in the same way as certain other tasks will provide evidence for the existence of distinct processes. Furthermore, these patterns of lateralization will allow for the test of computer models of how experience affects cerebral lateralization. The increased understanding of imagery gained from this research is relevant to training in various branches of science and engineering, because of the critical role of imagery in the spatial thinking so necessary in those areas. If we understand the component processes of imagery and which processes are used in which kinds of tasks, we may be able to train people to use imagery more effectively in their thinking. Such training might allow students who would otherwise drop out of visually oriented technical disciplines to succeed in such work.

Visual mental imagery helps people to perform a host of tasks, including remembering events, spatial reasoning, and language comprehension. The images produced in the service of these activities are the result of a complex information processing system. This system can be conceptualized as being composed of a set of distinct "processing subsystems," each of which performs a specific cognitive operation (e.g., shifting the orientation of an imaged object, activating stored visual memories to create an image, encoding the relative location of part of the imaged object). The research described here makes use of a theory of these processing subsystems that draws on concepts from artificial intelligence and facts about the neurological substrate of high.level vision. The research will characterize the relative efficacy of eleven of these subsystems in the elderly, compared to younger adults. The primary aim of the research is to discover whether certain subsystems are, relative to other subsystems, selectively more effective in senescence. The research aims to disprove the idea that cognitive aging can be understood solely in terms of "generalized slowing," showing that slowing with age differs for different subsystems. If so, then strategies that make use of relatively effective subsystems should be more useful for elderly people than strategies that rely on ineffective subsystems. The experiments designed here are a step toward using contemporary theory from cognitive neuroscience and techniques of task analysis from cognitive science to design new tests for cognitive efficiency in the senescence.

The goal of this research is to characterize the nature of the internal representations that underlie the experience of visual mental imagery. By characterizing the brain areas that underlie imagery, the proposed research will provide insights into the possible consequences of stroke on a large variety of mental functions. The research is specifically designed to resolve contradictory results that have been reported by the Orsay positron emission tomography (PET) group and the Harvard/ Massachusetts General Hospital group. The Harvard group (as well as five others) has found that low-level visual areas 17 and/or 18 are activated during visual imagery, whereas the Orsay PET group (as well as one other) has not found such activation. In the proposed research, they will conduct two fMRI experiments to test three hypotheses regarding the disparity in findings. They will investigate whether the disparity is due to a difference in the resolution of the stimuli that are visualized, the difference between performing spatial task and visualizing the surface properties of objects, or simply individual difference.

imagery; neural information processing; visual perception; sequential perception; visual cortex; transcranial magnetic stimulation; human subject; functional magnetic resonance imaging; behavioral /social science research tag; clinical research;

Proposal number: 0106760
PI: Stephen Kosslyn
Institution: Harvard University
Title: Cognitive Styles and Individual Differences in Imagery

Award Abstract


This is a study of cognitive styles (such as the visualizer-verbalizer distinction) people (both students and professionals) use in thinking through and learning science and mathematics (specifically in the topic area of kinematics). The project has two goals: (1) To investigate and revise the traditional verbalizer-visualizer cognitive style dimension, including the possibility that more than these two basic styles exist (such as object and spatial visual styles). This approach is based on findings and methods from cognitive psychology and cognitive neuroscience to conceptualize the nature of cognitive style and its neural underpinnings. (2) To examine the instructional implications of these cognitive styles and their relevance to educational practice. The research will proceed along three lines: behavioral studies, classroom-based studies, and functional magnetic resonance imaging (fMRI). Based on evidence from this research, the investigators will develop theoretically-based guidelines for teaching students to process visually/spatially and materials which build on the specific strengths of each type of thinker.

DESCRIPTION (provided by applicant): Visual mental imagery is used in memory, reasoning, and learning. A deeper understanding of the nature of imagery has practical implications for the diagnosis and treatment of a variety of neuropsychological conditions. The general goal of this research is to conduct functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS) studies to investigate the representations and processes that underlie visual mental imagery. FMRI will be used to identify the neural structures that underlie different aspects of visual imagery, and TMS will be used to investigate the causal role of specific activated areas in task performance. The specific goals are to perform six studies addressing the following issues: (1) Do differences in the orientation of imaged objects map directly onto differences in the pattern of activation in Area V1, and is the pattern of activation sufficient to identify the orientation at which an image was formed? (2) Is V1 more likely to be activated when participants must use high resolution to examine details of imaged objects? (3) Is V1 activated during tasks that require visualizing shape but not during tasks that rely on visualizing locations or spatial relations? (4) The mechanisms underlying image maintenance over time have received virtually no attention in the literature. We directly compare the mechanisms involved in generating an image and maintaining it over time, and examine the effects of stimulus complexity on the two functions. (5) We also study how images of locations or spatial relations are maintained, and we compare the processes underlying such image maintenance with those underlying "spatial working memory." (6) Finally, we directly compare the brain areas that are activated during imagery for shapes, imagery for locations, and image rotation. Mental rotation is often considered a "spatial task," but we hypothesize that it also draws on mechanisms that specify shape. We not only compare these tasks with respect to a baseline, but also compare the brain areas in which variations in activation predict variations in performance on each task.

The proposed research program will explore a series of questions concerning spatial imagery abilities in the context of cognitive styles. A cognitive style is a psychological dimension that specifies consistencies in how an individual acquires and processes information. Although research suggests that cognitive styles have important implications for educational theory and practice, many previous studies were not motivated by a theory or general framework that species the dimensions along which cognitive processing may vary, and as a consequence suffered from arbitrary distinctions and overlapping dimensions. For instance, although a number of studies have found that the verbal subscale of standard visualizer/verbalizer questionnaires does indeed correlate with verbal ability, the visual subscale is generally only weakly correlated with results on visual/spatial aptitude tests. However, current research on visual information processing suggests that there are actually two kinds of visualizers-those who construct vivid, concrete shape-based images of individual objects (object visualizers), and those who construct images that represent spatial relations among objects and transformations of objects (spatial visualizers). These two types of visualizers display very different patterns of performance on visual-spatial tasks, including tests of practical knowledge, such as the ability to interpret graphs or solve geometry pr9blems. This distinction is rooted in the brain: Neuropsychological findings have revealed that higher-level visual areas of the brain are divided into two functionally and anatomically distinct pathways, the object and spatial relations pathways. This proposal. has three major objectives. First, the investigators plan to examine the development of imagery skills as children age. They will explore the possibility that spatial and object imagery have different courses and rates of development. Second, they will conduct behavioral and fMRI studies to examine how practice using imagery changes performance as well as neural activity in the brain. Finally, they will examine how people in different professions differ in their mental imagery abilities and cognitive styles.

This project has three main goals. The first is to investigate the neural basis of deception. This goal will be achieved by using established neuroimaging technologies (fMRI), thus going beyond the traditional approaches based on the measurement of peripheral variables such as skin conductance, which have not proven highly valid. The second goal is to validate a novel taxonomy of lies based on prior research conducted in the PI's laboratory. The idea of subdividing lies into different categories and studying these categories separately is a major conceptual advance in the study of deception because the traditional approach has treated deception as a unitary phenomenon: either one is lying or one is telling the truth. In the proposed taxonomy, lies are divided according to two dimensions: 1) whether they are spontaneous or memorized in advance, and 2) whether they are isolated (i.e., unrelated to each other) or fit into a coherent story. The different types of lies are conceptualized in terms of the cognitive processes they are likely to engage. One of the methodological strengths of the proposed approach is that the brain regions supporting the hypothesized cognitive processes engaged during the different types of lies will be independently identified for each participant (by using a set of "neurocognitive localizer" tasks). These additional data will provide important constraints for the interpretation of the results in the deception conditions. The third goal is to use the resulting knowledge to develop a new technology for detecting the neural processes associated with different types of lies. This technology is known as diffuse optical imaging (DOI), and relies on measuring the absorption of near-infrared light in the cerebral cortex to track cortical processes. Although in its infancy, DOI has the advantage of being relatively inexpensive and portable.

A better understanding of the brain processes that underlie deception can potentially benefit society in several ways. First, it is likely to advance our knowledge (and consequently collective awareness) of the limitations and potential of lie detection methods. Second, the understanding gained in this work about the neural processes underlying deception may lead to more effective lie detection methods; this could be very important in forensic settings. Third, the development of an affordable technology (DOI) that can be used to study the neural correlates of deception could eventually revolutionize the field of lie detection.

This project will strengthen the collaboration between researchers whose main area of expertise is cognitive neuroscience and researchers whose expertise is in physics and engineering. Such collaborations weaken traditional barriers between disciplines and allow scientists in training (students, assistants, postdocs) to acquire a greater range of skills and experience. The PI's lab has a long history of training scientists who have gone on to make independent contributions, and that practice will be continued. This project offers educational opportunities for students and junior researchers. On the technical side, they will not only acquire training in fMRI, but also have the unique opportunity to learn the novel DOI technology and participate in its development at various levels; on the content side, these studies cut across several lines of research that are often not integrated. In doing so, they provide students and junior scientists with an unusual breadth as well as an opportunity to learn to integrate what appear to be parallel research domains. In order to disseminate the results of the research as broadly as possible, significant findings will be reported primarily as journal articles, and preliminary data will be discussed with other investigators at scientific meetings.

DESCRIPTION (provided by applicant): This plan for Graduate Training in Psychology and Neuroimaging rests on multidisciplinary collaborations between 23 faculty members in the Psychology Department at Harvard University and the Athinoula A. Martinos Center for Biomedical Imaging at the Massachusetts General Hospital. The training program aims to prepare a new generation of scientists whose graduate training concentrates on linking human brain function to cognitive processing through neuroimaging research. Functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), electroencephalography (EEG), positron emission tomography (PET), and near-infrared spectrography (NIRS) have emerged as central tools in cognitive neuroscience. By training students to use these neuroimaging methods effectively, our goal is to equip them with the means to answer questions about the neural bases of mental processing with technological competence. A demanding 5-year program of graduate training is proposed to meet this goal. Although the emphasis is on traditional cognitive psychology, it will be readily possible for trainees to address issues of cognitive development, social cognition and psychopathology. Each student will be admitted into a program in the psychology department and will be expected to fulfill the requirements of that specific program, including first-year, second-year and Ph.D. thesis research projects. In addition, every trainee will take two team-taught courses in their first two years: a course in Cognition, Brain and Behavior (taught by all the training faculty in psychology and faculty in cognitive psychology at the Martinos Center) and an intensive introduction to functional brain imaging (taught by training and other faculty from the Martinos Center). In subsequent years, trainees will take a research seminar jointly taught by all faculty. Students will explore particular areas of cognitive and brain function in depth, develop skills for crafting experiments that dissect specific cognitive functions, present their research, and interact with other trainees in the program. The goal of these requirements is to impart knowledge that is broad and deep, both about the human brain and about the cognitive functions that it performs. Throughout their training, students will be co-advised by one Harvard Psychology and one Martinos Center faculty.