Stephen A. Engel - US grants

The goal of this project is to understand the nature of color opponent processing in human visual areas. Color perception results from a sequence of neural transformations of a signal originally encoded by the photoreceptors. The first stages of this sequence are fairly well understood, but the neural basis for the second perceptually identified stage of processing, opponent color mechanisms, remains unclear. A long history of perceptual experiments indicate that a second stage encodes color as three combinations of cone signals, encoding the relative local amounts of red and green, blue and yellow, and light and dark. While parvocellular neurons in the lateral geniculate nucleus (LGN) receive opponent inputs from cones, these cells differ from perceptual mechanisms in important ways. LGN neurons appear to jointly encode red-green and light-dark, they respond well to high temporal frequencies, and they over-represent red-green relative to blue-yellow. The work proposed here attempts to localize cortical areas whose response properties more closely match perceptual mechanisms. We will measure the responsiveness of human visual areas to stimuli of many colors. Functional MRI (fMRI) data will be acquired while subjects view large colored patterns. Responses to each stimulus will be averaged within the LGN and visual areas VI-V4. Color tuning is a measure of responsiveness as a function of color. A series of experiments will measure the color tuning of visual areas under conditions that reveal identifying features of perceptual mechanisms. First, the linearity of color tuning will be tested. Next, color tuning will be measured at a variety of temporal and spatial frequencies. Then it will be measured in the presence of adapting stimuli. Finally, color tuning will be measured in the visual areas of dichromatic observers. All of these conditions effect perceptual opponent color mechanisms in distinctive ways. Visual areas whose response properties resemble perceptual mechanisms are likely to be important stages in the neural pathways that give rise to color perception. Differences in the response properties of areas will help reveal the sequence of neural transformations that results in perceptual color opponency.

The goal of this project is to advance understanding of how color is represented in cortex. While the first stages of color perception are well understood, much less is known about subsequent processing. The general aim of this proposal is to identify neural populations in cortex that support a second stage of processing, the opponent colors representation. Perceptual studies reveal that this representation encodes color as three combinations of cone signals, encoding relative amounts of red and green, blue and yellow, and light and dark. These perceptual mechanisms: 1) signal specific, linear combinations of cone signals; 2) maintain their color properties even as spatial pattern changes; 3) change their responsiveness independently of each other; 4) are selective for spatial frequency and orientation; 5) pass signals to other, non-cardinal color mechanisms. But the neural bases of opponent colors mechanisms remain unkown. While parvocellular neurons in the lateral geniculate nucleus receive opposing cone inputs, these cells fail to show properties 2 and 3, listed above. Some neurons in areas V2-V4 are tuned for non-cardinal colors, and are selective for spatial frequency, but the crucial experiments linking these neurons to perceptual results have not been performed. The work proposed here is designed to identify neural populations that can account for the opponent colors mechanisms. The responsiveness of human visual areas and LGN to stimuli of many colors will be measured using functional magnetic resonance imaging. A series of experiments will test for the 5 key properties of opponent color mechanisms. First, the linearity of color responses will be tested. Next, color responses will be measured at several spatial frequencies and in the presence of adapting stimuli, testing for properties 2, 3, and 5. Finally, measurements of transfer of adaptation will test for property 4. Visual areas whose responses show these properties are likely to contain populations of neurons that support color perception. Populations showing the different properties will most likely be found in different visual areas, helping to reveal the specific role of each area in the computations that underly color perception, and testing computational theories of color perception.

This subproject is one of many research subprojects utilizing the resources provided by a Center grant funded by NIH/NCRR. The subproject and investigator (PI) may have received primary funding from another NIH source, and thus could be represented in other CRISP entries. The institution listed is for the Center, which is not necessarily the institution for the investigator. Data sets we have obtained from previous studies suggest a large (>200 msec) difference in BOLD onset timing due to differences in stimulus strength: stronger responses may begin earlier than weaker ones. Although stimulus strength (such as contrast amplitude) is thought to result in small onset timing differences in the neural response in the LGN (Hartveit and Heggelund 1992), V1 (Reich et al 2001), and other visual areas (Gawne et al 1996), these small timing differences (10's of msec) are an order of magnitude smaller than what we have seen in the BOLD fMRI response. We propose to make fine measurements of the timing of the BOLD response using an fMRI sequence with a short TR, gathering one volume every 250 milliseconds. Should timing differences as a function of response strength exist, they would have a substantial impact in the field, as an increasing number of studies are investigating timing of neural response using fMRI. The timing differences may also impact work using dynamic causal modeling (DCM), in which timing is used to infer causal links between cortical areas (Stephan et al 2008, Mechelli et al 2003). The data obtained here will be key for several grant proposals which propose to use fMRI and joint fMRI/EEG to study timing of perceptual processes. Engel, Olman, Kersten, and He all have interests or proposals under development in which such measurements of timing may play an important role.

This subproject is one of many research subprojects utilizing the resources provided by a Center grant funded by NIH/NCRR. The subproject and investigator (PI) may have received primary funding from another NIH source, and thus could be represented in other CRISP entries. The institution listed is for the Center, which is not necessarily the institution for the investigator. The aim of this proposal is to allow researchers to monitor subjects'eyes while simultaneously acquiring MRI data. First, eye tracking is a necessary experimental control for many studies. By monitoring participants'eyes, the researcher can make sure that eye movement differences are not causing differences in the fMRI signal that are attributed to perception and cognition. A second, perhaps stronger reason for combining eye tracking and fMRI are the new research questions it allows investigators to address. Currently, the eye tracker in the 3T scanner room at CMRR is not used much. The main reason being that very few people know how to use it, and the time investments for learning are high.

The brain has a remarkable ability to change its own function. In the visual system, such plasticity is evident when children learn to read, when the elderly adjust to impaired vision, and when new drivers master navigating traffic. The mechanisms that produce long-lasting plasticity in the visual brain remain unclear, however. With support from the National Science Foundation, Dr. Stephen Engel of the University of Minnesota is carrying out research to elucidate mechanisms of long-term visual plasticity. The work builds upon effects of short-term plasticity that are relatively well-understood. For example, when observers view a bright pattern of stripes for a few minutes, their visual systems adapt, and fairly dramatic consequences can result: Faint patterns of similar stripes that were previously apparent can be briefly rendered completely invisible. But few laboratory studies have examined perceptual adaptation over the long term, primarily due to methodological limitations. This project uses recently developed technology to overcome those roadblocks. Participants view video on a head-mounted display that originates in a head-mounted camera and is image-processed by a lap-top computer. The system allows participants to live in a visual world that has been digitally altered. Four experiments use this system to test hypotheses about long-lasting plasticity in the visual system. The first hypothesis is that longer duration adaptation will produce longer lasting plasticity. Participants are placed for up to five days in environments similar to those that produce short-term perceptual adaptation effects. The persistence of adaptation is measured using traditional tests of perceptual abilities. The second hypothesis is that the visual system adapts to discount noisy, uninformative visual input, as suggested by preliminary results. Experiments in this project measure the strength of this novel form of adaptation over the long term. The third hypothesis is that both kinds of adaptation effects arise in early visual cerebral cortex. To test this, the neural bases of long-term adaptation is measured with functional magnetic resonance imaging (fMRI). Together, these experiments can strongly constrain both empirical and theoretical accounts of long-lasting visual plasticity.

This research is advancing understanding of how the human visual system can modify its own operation. This ability, visual plasticity, underlies the acquisition a wide array of human skills, from learning to read, to learning to hit a baseball, to learning to see patterns in satellite imagery. The project uses novel technology to place observers in a digitally altered world for up to 5 days, and measures how their visual perception and their visual cortex adapt to this challenge. Results can help identify specific factors, for example the length and kind of visual stimulation, which lead to long-lasting changes in visual performance. These in turn should have important applications in a diverse array of fields where visual plasticity is critical, from education to the military to public health.

The brain has a remarkable ability to modify its own function. In the visual system such neuroplasticity occurs routinely, for example when we move from indoors to outside. Changing environments affects visual input dramatically, and cortical neurons alter how they respond in order to keep us seeing well. This type of neuroplasticity, called visual adaptation, occurs readily in the adult brain. Its effects tend to be short-lasting, however, limiting its potential applications in education, training, and health. To understand and overcome this roadblock, Dr. Stephen Engel and colleagues at The University of Minnesota will test the limits of adaptation, by extending laboratory studies an order of magnitude in duration. Participants will adapt for up to a week continuously, by viewing the world through virtual reality goggles that display modified output of a head-mounted camera. The experiments will measure how long-lasting the effects of adaptation can become, and test several factors that could be limiting the growth of this type of neuroplasticity. These include competition between the parts of the brain that benefit from neuroplasticity and the parts that could be hurt by it, since they depend upon stability of the brain regions that provide their input. Understanding adaptation should allow it to produce beneficial neuroplasticity, which has many applications. It could become part of training visual experts such as baggage scanner operators or imagery analysts. It could become part of efforts to rewire the brains of patients suffering from visual diseases, such as amblyopia ("lazy eye"). The work will also test theories of neuroplasticity that are applicable in many domains. It will produce educational materials for teaching of cognitive neuroscience, and will directly support the training of a diverse set of graduate and undergraduate students in cutting edge methods.


Adult visual cortex can alter its function dramatically in response to changes in the environment, allowing it to function optimally, despite large changes in input. Critically, the longer vision experiences an environment, the stronger and longer lasting adaptation becomes. This suggests that long-term exposure to altered visual input may produce long-lasting changes in cortex. Technical issues have prevented the study of long adapting durations under controlled conditions, however leaving it unknown whether some factors 'put on the brakes', restricting its strength and durability. The PI developed methods to overcome this roadblock, and examined effects of a 4-days of continuous adaptation, revealing significant limits. The general goal of the grant is to understand the brakes producing those limits. For vision as a whole, adaptation can produce costs, as well as benefits, and this proposal will test whether observed limits are due to them. A first set of experiments will test whether costs limit adaptation in early visual areas, examining specific costs from the literature, removing them, and observing whether limits on adaptation remain present. A second set of experiments will examine longer-term adaptation. They will test the hypothesis that long-term adaptation can overcome limits, reducing costs by correcting for a "coding catastrophe" in later visual areas. The work will distinguish between major theories of adaptation, using quantitative analysis of behavioral and functional MRI (fMRI) data. It also represents a major empirical step forward; dozens of studies have examined adaptation in visual cortex, but no previous work has measured or tested adaptation's limits. Results of this research could influence work in a diverse array of fields where visual plasticity is important, from education to the military to public health. Results of the work will be incorporated into educational materials (an introductory fMRI course and text) that will be freely distributed. The grant will also provide training opportunities for a diverse group of students.

Project Summary Age-related macular degeneration (AMD) typically produces extensive central vision loss, leading to difficulty in reading, navigating, and other critical tasks. Current assistive devices for vision with central scotomas, while promising, are limited in effectiveness. This proposal aims to fill this gap by advancing understanding of remapping, a method that improves the effectiveness of patients? residual vision by shifting information from inside the central blind spot (scotoma) to intact locations in the visual field. Only a handful of past studies have examined remapping, including just a single study of patients. Our group recently demonstrated the method?s promise, showing that it can improve reading substantially in observers with simulated field loss. Critically, patients differ widely in the shape of their scotomas, and the quality of their vision across the visual field. This variability imposes severe limitations on the ?one size fits all? remapping approaches used in prior work, that simply shift the image away from the scotoma center. If, for example, a patient?s preferred retinal location (used for high acuity tasks) is near the lower left edge of their scotoma, shifting text upward or rightward may not aid reading as much as shifting it downward, as the latter will place the text in regions of best visual acuity. This proposal will test the value of personalized remappings, which shift the image in a way that is optimized for each patient?s residual vision. The remappings are constructed using a novel letter recognition perimetry task, which measures performance across the visual field: The personalized remappings are constructed to shift text to maximize observers? total letter recognition ability. Proposed work will first test the hypothesis that personalized remapping can improve single word recognition. Single word reading will be measured with a variety of scotoma sizes, shapes, and PRL locations. Performance with personalized remapping will be compared to that with traditional remapping and no remapping in both control observers with simulated scotomas and people with macular degeneration. Proposed work will also test the hypothesis that personalized remappings can improve free reading. Reading speed, error rates, and eye movement patterns will be measured in a sentence reading task, and in free reading of natural images containing text. Preliminary data support the value of remapping generally, and the potential of the letter recognition perimetry task for building personalized remappings. Results of this proposal will provide the first thorough testing of remapping, and will also inform models of peripheral reading in patients and controls. The proposed studies will develop and evaluate a novel method for assisting people with low-visions, tailored to their individual residual performance. Personlaized remapping may also be incorporated into practical visual aids to improve daily visual function and quality of life.