Takeo Watanabe - US grants

Visual perception is thought of as a passive and simple event. This view may stand on the assumption that visual information goes from the retina to higher-level areas in the brain only in a one-way direction. Processing that flows in this direction is called "bottom-up processing." However, recent anatomical and physiological findings indicate that there are massive feedback pathways in the brain along which information is carried from these higher-level brain areas back to low-level areas, including those involved in processing visual information. Processing that occurs along these pathways is called "top-down processing." Unfortunately, it is poorly known what role top-down processing play in visual information processing in the brain. In the meantime, recent physiological studies have pointed out that even the parts of the adult brain which process primitive visual information have much higher plasticity than had been previously thought. The behavioral manifestation of this plasticity is that very primitive visual functions such as discrimination of motion direction and orientation of lines can be improved as a result of repetitive trials. Thus, an interesting question arises. Does top-down processing influence perceptual learning in the early stages of motion processing? If so, what is the role of top-down processing? The proposed study will examine the nature and role of top-down processing and its interaction with bottom-up processing during perceptual learning.
A series of experiments will be conducted using several human subjects. They will watch a stimulus called random-dot cinematogram which consists of small dots moving randomly within a certain range of directions. In each stimulus, a global flow is perceived to move in one direction just like a flow of a river. This indicates that there are at least two stages in early motion processing; the first, local motion stage and the second, global motion stages. We will first examine the extent to which top-down signals from the stage of global motion processing affect lower stages. The results of preliminary experiments suggest that during the learning of global motion lower stages may be strongly affected by top-down signals. We will conduct experiments to examine this more systematically. Second, we will examine the role of the interaction between top-down and bottom-up processing in perceptual learning. Essential to our experiments is a "bottom-up-absent" random-dot cinematogram which does not include any dots moving in directions close to the global motion direction such that activation of the intended global motion direction can not be attributed to bottom-up signals. Using this bottom-up-absent stimulus, we will examine location specificity as well as the stages involved in learning and compare the results with those using the conventional "bottom-up-present" stimulus. A difference in results between the two types of stimuli would suggest roles for bottom-up activation and the interaction between bottom-up and top-down processing.

Optical flow motion processing is fundamental to everyday life. In order to obtain important environmental information to the viewer that will allow her/him to react appropriately, the visual system needs to extract and differentiate different patterns of optical flow motion signals from the environment. For example, systematic transformation of moving objects is one of the most powerful sources for reconstructing the 3-dimensional surfaces of the objects. With support from the National Science Foundation, Dr. Takeo Watanabe's research aims at understanding how different types of motion signals are related to each other in the brain from a global framework of motion processing. The research will be conducted using psychophysics and fMRI in which Dr. Watanabe will systematically measure brain activity with different types of motion and can learn how they are related to each other.
Broader impacts of the project include educational and medical advances, improved navigational skills of drivers and pilots, and enhanced artificial intelligence/robotics. Educational benefits will include the training of graduate and undergraduate students in use of the methodology and technology. The research also has the potential to be directly applied to clinical research in neurosurgery by contributing to scientific knowledge leading to development of medical tools for improved diagnosis and treatment of brain disorders or lesions. The research can also provide important knowledge to enhance the navigation skills of drivers and pilots. Finally, construction of robots with the artificial intellectual power to effectively detect and utilize motion information would be of benefit to both the civilian/medical and military fields.

DESCRIPTION (provided by applicant): Perceptual learning (PL) is defined as long-term enhancement in a visual task or the process for the enhancement as a result of repeated visual experience and is regarded as manifestation of visual plasticity. Thus, studying PL will lead to increased clarification of visual plasticity, which is one of the most important goals in the visual sciences. Recently, there have been two important developments in studies of PL. First, reinforcement signals play an important role in PL. Second, sleep is fundamental for consolidation of PL. However, the neural mechanisms for these aspects of PL are not well clarified. In addition, these aspects have been studied totally separately (i.e., without relating to each other). Furthermore, although there are (1) task- relevant PL (TRPL) resulting from repetitive performance of a task and (2) task-irrelevant PL (TIPL) resulting from exposure to a visual feature, the roles of reinforcement signals and sleep consolidation have yet to be examined taking these two different types of PL into consideration. Using a decoding method applied to fMRI signals, we have observed how the tuning curve in each of several brain areas changes (global tuning curve changes) under various conditions in association with PL. In this proposal, by taking advantage of using this new technique, we will investigate what the neural mechanisms are for reinforcement signal and sleep consolidation in TRPL and TIPL from the same viewpoint and aim to clarify how these mechanisms relate to each other. The goal of Aim 1 of current proposal is to clarify the neural mechanism for reinforcement signals. While it has been pointed out that reward-driven reinforcement signal plays an important role in PL, it remains unclear how the reward influences neural mechanisms in PL. We will address this question by examining how training of PL with reward leads to global tuning curve changes in brain areas. The goal of Aim 2 is to clarify neural mechanism changes during sleep consolidation. While it has been shown that sleep is fundamental for PL consolidation, it remains unclear how the neural mechanisms change during consolidation in different types of sleep (Non-REM and REM), different types of training (task-relevant and task-irrelevant PL), and with and without reward. We will address these questions by examining global tuning curve changes in these different conditions. Also, whether common neural mechanisms are involved in reward and sleep consolidation in PL will be examined by comparing global tuning curves obtained in these two aims. To date have examined reinforcement signals and sleep consolidation in PL using different paradigms and stimuli. Systematic investigation of the neural mechanisms for reinforcement signal and sleep consolidation and their interactions in each of TRPL and TIPL within the same framework will lead to significantly greater understanding of the underlying, different researchers neural mechanisms for PL.

[unreadable] DESCRIPTION (provided by applicant): The goal of the present proposal is to understand effects of rewards signals on plasticity of low-level visual areas. Recently, it has been found that sensitivity enhancement occurs not only to features on the basis of which subjects perform tasks (task-relevant learning), but also to presented features that are irrelevant to the task (task-irrelevant learning). A on a series of psychophysical work (Watanabe et al, 2001, 2002; Seitz & Watanabe, 2003; Seitz et al, 2004, 2005abc; Seitz and Watanabe, 2005) indicates that visual learning occurs as a result of interactions between diffusive reinforcement signals triggered by internal factors including reward and bottom-up stimulus signals from presented visual features, irrespective of whether these features are task-relevant or irrelevant. In the present proposal, we will test the hypothesis that task-irrelevant learning is mediated by such reward-driven reinforcement signals. First, to test the validity of the model, we will address the following questions. (1), Does processing of a task work as an internal reward? (2), Does this reinforcement signal modulate activity in low-level visual areas? (3), Does task-irrelevant learning occur for any feature that is predictive of trial-outcome? (4), After learning forms, is activity with prediction errors observed in the same way as reinforcement learning? Second, we will use external liquid rewards to directly test whether reward leads to perceptual learning. If that is the case, we will vary the timing and probability of reward-delivery to test whether perceptual learning follows rules found in reinforcement learning. To address these questions, we will measure behavioral performance by psychophysics and activity in the human brain by means of fMRI. To our knowledge, no research has directly examined effects of reward on low-level visual processing and plasticity or relations between low-level visual learning and reinforcement learning. Thus, the present research is highly novel. At the same time, the predictions of the results are solid since they are made on the basis on consistent results of a series of psychophysical studies. We believe that R21 is an appropriate track. The proposed research has the potential for clinical applications by contributing to scientific knowledge leading to development of medical tools for improved diagnosis of, and rehabilitative therapies for, brain disorders and lesions. Moreover, the research results may help reveal sources of, and treatments for, learning deficits. [unreadable] [unreadable] [unreadable]

Perceptual learning refers to improvements in one's sensory abilities that occur through experience with a particular environment, and it is thought to help us to become more efficient in processing sensory information. Recently Drs. Aaron Seitz and Takeo Watanabe have found that viewers can improve their perceptual abilities through sub-threshold exposure to visual stimuli. These improvements in perception have been shown to involve the primary visual areas of the brain and are thought to be an indicator of basic mechanisms of learning that are common throughout the nervous system. With support from the National Science Foundation, Drs. Watanabe and Seitz will conduct research that will clarify two important aspects of perceptual learning. In particular, their work will identify neural circuits that are modified during perceptual learning, and it will help to better characterize the processes that lead to this phenomenon. This research will be conducted by combining innovative psychophysical methods with functional Magnetic Resonance Imaging (fMRI) in order to systematically measure activity in different areas of the brain before during, and after such learning has occurred.

This research will have broad impacts. The conduct of the studies will help to train graduate and undergraduate students in research methodology and the use of brain imaging technology. The results from the studies will help illuminate the processes by which perceptual skills can be improved; such knowledge could lead to enhanced training techniques for those afflicted with disorders that affect sensory processing, as well as to improvements in training methods for experts who perform work that requires highly acute sensory abilities, such as pilots and radiologists. This work also has important implications for clinical and rehabilitative medicine as it has the potential to lead to improvements in medical prostheses and tools for the diagnosis and treatment of sensory disorders. Finally, improvements in the understanding of human sensory and learning processes could help to guide the development of technologies for automating such abilities in the context of a wide variety of industrial applications.

DESCRIPTION (provided by applicant): Repetitive training of a perceptual task leads to performance enhancement on that task. This so-called perceptual learning is regarded as a manifestation of neural plasticity in the sensory/perceptual system. It has been found that after training completion the brain continues a process of strengthening learning and memory for long term retention of the learned task. This process is called consolidation. Recent research results have led some researchers to suggest that consolidation occurs during sleep after training. However, that sleep plays a role at all in consolidation is doubted by a considerable number of researchers, and among studies that advocate a role of consolidation in sleep there is controversy regarding during which stage of sleep consolidation occurs. In this proposal, specifically, we will conduct experiments in which fMRI brain activity will be measured during sleep after subjects perform a visual training task that leads to learning of the task. It has been found that some types of visual tasks involve a highly local neural circuit in a low-level stage of visual processing. If fMRI activity changes are observed specifically in the trained region during sleep after training of a visual task and performance is higher after the sleep, this would be strong direct evidence that consolidation of PL occurs during sleep. In the proposed research, a new technology will be used that allows us to simultaneously measure polysomnogram (PSG) while using functional magnetic resonance imaging (fMRI), so that we can objectively determine the onset/offset of sleep and sleep stages using the most standard method in sleep studies. This will enable us to indicate during which stage of sleep, if any, consolidation occurs. The proposed study is highly novel and successful results would not only allow us to reveal the role of sleep in consolidation of PL but also to resolve some serious controversies about the role of sleep in learning and memory that has attracted attention of researchers in the fields of memory, learning and sleep in general. At the same time, the proposal contains highly risky and exploratory natures. This proposal has potential for generalization to applied clinical fields. Recovery and rehabilitation from damages of visual function depend on plasticity and reorganization of remaining neural structures. However, the degree of plasticity and reorganization that take place in the brain is yet unknown. Positive findings from our studies may be applied to the development of better rehabilitation programs and better evaluation of the role of sleep in the recovery and rehabilitation process for a number of neurological conditions. PUBLIC HEALTH RELEVANCE The primary goal of the proposed research is to test whether sleep plays an important role in consolidation of visual learning. For this purpose, it will be examined whether or not activity in the region of a visual area that is involved in the learning is changed during sleep after training, by means of functional magnetic resonance imaging (fMRI). Successful research results will provide a base of knowledge from which new visual functions and recovery from damages of a visual function are effectively obtained.

DESCRIPTION (provided by applicant): A well documented finding in the literature is that visual perception declines with age. These declines include the perception of target orientation, low contrast stimuli, motion, shape and form. Age related declines in visual processing have been implicated as a leading cause of falls among the elderly as well as the increased risk of accidents for older drivers. The purpose of this research is to examine whether perceptual learning with sub threshold stimuli can be used to recover age-related declines in visual perception. The proposed research will include behavioral studies and fMRI studies. The proposed behavioral experiments will determine whether the improved performance for older observers via perceptual learning is due to changes in early stage visual processing, later stages of visual processing, or non-sensory processing. The effects of perceptual learning and the locus of improvement will be assessed by examining location and interocular transfer and by assessing transfer of training from perceptual tasks associated at the same stage of processing (e.g., orientation and contrast) as well as different stages of processing (e.g., global motion and orientation). The impact of perceptual learning on non-sensory processing will be assessed by examining changes in response criteria and improvements in divided attention. fMRI studies will examine whether the improved behavioral performance from perceptual learning for older observers is due to functional plasticity, neural recruitment, or changes in hemispheric asymmetry of activation. The fMRI studies will examine the same perceptual learning tasks that will be examined in the behavioral studies and will examine changes in activation in several regions in visual cortex as well as prefrontal cortex. In addition, contingent on the results of the behavioral and fMRI studies we plan on conducting structural imaging studies to determine whether perceptual learning results in physiological changes in neuronal systems. Finally, we also plan on conducting intervention studies to determine whether improved performance for older observers via perceptual learning can improve performance in driving related tasks. The goal of this line of research is to examine whether recovery of visual function using perceptual learning can be applied to situations known to be of importance for the safety, well being and quality of life of older populations. PUBLIC HEALTH RELEVANCE: The proposed research will examine how training with visual displays can be used to improve visual perception for older individuals. The studies will include behavioral research and brain imaging research to determine how perception improves and how the improvement is related to changes in brain activity.

? DESCRIPTION (provided by applicant): This is a renewal proposal for the currently funded one-year R01 grant (EY019466). Visual perceptual learning (VPL) is defined as a long-term enhancement of visual task performance as a result of visual experience, is regarded as a manifestation of visual and brain plasticity and is seen as a promising tool with which to improve vision that has been degraded due to visual disorders. Over the one-year funding period, we have made significant progress in clarifying specificity of VPL. However, we have yet to address an important question: How can the visual system be sufficiently plastic to adapt to environmental changes, but stable enough to protect existing visual information from being replaced with new information? Our long-term goal is to resolve this dilemma in VPL. Developmental studies of vision have resolved the dilemma by setting a window (critical period) during which the visual system is highly plastic in one's early life. This inspires us to investigae the mechanisms of plasticity periods in VPL with adults. We have found two plasticity periods in VPL. The short-term plasticity period may begin during each visual training session and last less than one hour after the end of the session. The long-term plasticity period may last for at least several days, during which a reorganization of visual processing gradually occurs until performance levels for the trained task saturates. Thus, we need to examine the mechanisms of these two periods separately. In Aim 1, we will conduct systematic psychophysical investigations to better understand the two plasticity periods in VPL. We have found that 3-hour monocular deprivation (MD) enhances plasticity of monocular processing possibly by abolishing inter-ocular inhibition, whereas reward priming enhances plasticity of binocular processing in a descending way. We will examine how 3-hour MD and reward priming influence the plasticity and stabilization of these two plasticity periods. In Aim 2, we will examine roles of excitatory and inhibitory processing in the two plasticity periods. The start and end of a visual critical period are associated with changes in the ratio of excitatory to inhibitory signals (E/I ratio). We will address the question of whether and how the two plasticity periods in VPL with adults also relate to E/I ratio changes by means of magnetic resonance spectroscopy. Strong preliminary results have led us to the perspective that the results from Aims 1 and 2 will be consistently and constructively integrated so that functional and neural mechanisms of the two plasticity periods will be better clarified. The results would also be used to develop effective interventions for visual disorders. Although VPL is likely to be associated with changes both within and beyond the visual areas, as our initial approach we will focus on examining visual areas in this proposal.

DESCRIPTION (provided by applicant): A growing body of evidence suggests that sleep facilitates and is beneficial to perceptual learning. However, the underlying mechanism of this facilitatory action is largely unknown. There are two possible types of processing during sleep that may account for the facilitatory action: use-dependent processing and learning- consolidation processing. The use-dependent processing occurs during sleep in the brain mechanisms that are generally used during wakefulness prior to sleep. This processing leads to general changes of neural processing and does not occur specifically for the sake of learning. On the other hand, the learning- consolidation processing works specifically for learning. It is highly controversial concerning whether the use- dependent processing is sufficient for the facilitatory action or whether learning-consolidation processing is necessary for the facilitatory action. It is fundamentally important to know which type of processing occurs during sleep to clarify the mechanism of sleep facilitating perceptual learning, since it has not been directly tested which model is valid. We address this question in the present proposal. Sleep consists of different dynamics, such as those reflected by a multitude of frequency bands in spontaneous brain oscillations in each rapid eye movement (REM) and non-REM (NREM) sleep. This raises the possibility that only one type of processing does not necessarily occur consistently throughout the whole sleep period: the learning-consolidation processing might occur for some frequency bands in some cortical areas, while use-dependent processing might occur for others. Thus, we will systematically examine whether learning-consolidation processing or use-dependent processing occurs in each band in each REM and NREM sleep, and in different brain area(s). To test whether the learning-consolidation processing is necessary for facilitating perceptual learning during sleep, we will compare the spatio-temporal brain activation patterns during sleep that follows task performance that causes learning (learning paradigm) with those during sleep that follows task performance that does not cause learning (interference paradigm). For this purpose, we must obtain highly localized spatio- temporal information about brain activation during sleep;we will use a cutting-edge neuroimaging technique that combines fine temporal information from magnetoencephalography (MEG) and electroencephalography (EEG) with fine spatial information from magnetic resonance imaging (MRI) as well as individual retinotopic mapping in the early visual areas, to estimate the power and phase information of spontaneous oscillatory activities in the precisely localized cortical regions. Imaging will be conducted with concurrent polysomnography measurement to objectively identify sleep stages. Successful research results would provide significant knowledge to clarify how improvement of perceptual learning of a visual task and visual plasticity occurs during sleep after training of the task. PUBLIC HEALTH RELEVANCE: The primary goal of the proposed research is to investigate how learning is strengthened during sleep in young adults using advanced neuroimaging techniques. Successful research results may be used to improve our vision and enhance our learning ability.

DESCRIPTION (provided by applicant): This is a proposal for competitive renewal of the currently funded R01 grant entitled Systematic psychophysical investigation of visual learning. Visual perceptual learning (VPL) is regarded as a promising tool that can help clarify important aspects of visual and brain plasticity. Our long-term goal is to delineate general rules that govern VPL, and to gain a better understanding of the mechanisms of visual and brain plasticity. During the currently supported period, we have attained great success in clarifying the characteristics of perceptual and behavioral changes under VPL, often by examining how performance at or around a trained feature value is changed as a result of visual training. However, new serious controversies about the mechanisms of VPL have emerged. One controversy concerns the locus of VPL, or the visual and brain information processing stage that is changed in association with VPL. The other concerns how VPL with location specificity or transfer is developed during training. We will conduct systematic psychophysical research with the aim of building two separate but related models, each of which concerns where or how VPL is developed to resolve each of these controversies, so that these two models will be integrated to a unified model of VPL. In Aim 1, we will attempt to resolve the controversy about the locus of VPL by developing a two-stage model that can account for both the findings that support low-level changes and those that support higher-level stages, without denying either evidence for the low-level or higher-level models. By conducting a variety of experiments with different types of tasks, stimuli and procedures, we will psychophysically test the validity of this two-stage model as well as existing models. In Aim 2, we will attempt to resolve the second controversy. The reduction of the degree of location specificity by double training has exposed serious limitations in how existing models explain location specificity. Thus far, no theoretical model has been proposed that can clearly explain how location specificity occurs by single training, as well as how location specificity is weakened/eliminated by double training. Based on our recent finding of elimination of VPL followed by reactivation and our influential reinforcement model, we will build the reinforcement reactivation model that assumes that the elimination of location specificity after double training is due to a failure to reconsolidate location specific learning tat was reactivated by reinforcement signals. In a series of experiments, we will test the validity of this reinforcement reactivation model and other existing models. In both aims, results of experiments will also be used to further refine the proposed models to most suitably account for where and how VPL is developed. Examinations of these aims will help to achieve our long-term goal of clarifying general functional rules that govern VPL.

Every one needs to sleep. Many people may think that the purpose of sleep is a quiescent status with minimum level of brain activation to merely take a rest and passively remove fatigue accumulated during wakefulness. However, a growing body of evidence in neuroscience indicates that sleep plays much more active roles. One of the most important roles of sleep is to augment, or consolidate, learning and memory obtained during wakefulness. A better understanding of how sleep consolidates learning and memory would provide important knowledge of how students and adults can effectively learn and memorize. In the proposed research, we will specifically use visual perceptual learning (VPL), which refers to a long-term improvement of visual task performance by repetitive training on a particular visual task. By taking advantages of neural and behavioral characteristics of VPL and advanced non-invasive neuroimaging techniques, Dr. Sasaki and her team at Brown University will investigate detailed neural activation of the visual cortex during sleep. Investigations on how sleep functions to consolidate VPL may lead to better understanding of visual and brain plasticity (learning and memory) in human adults. Thus, the proposed activity will promote the progress of science in sleep and learning and advance the national health.

There has been a controversy whether consolidation of learning and memory during sleep occurs by reactivating what was learned or memorized during wakefulness (reactivation), or by removing unimportant information (downscaling). Based on our preliminary findings, we propose a two-process model of the sleep consolidation of VPL, in which both reactivation and downscaling occur in a serial and integrative fashion. In the first reactivation process, neural circuits only for the trained visual feature are reactivated for enhancement of learning where sigma activity is mainly involved. This reactivation process is followed by the downscaling process that occurs both with the circuits for the trained and an exposed but unlearned, therefore unimportant, feature. The validity of two-process model will be tested using psychophysics, and advanced neuroimaging techniques including concurrent measurements of polysomnography and magnetic resonance imaging, magnetic resonance spectroscopy and a decoding technique on blood-oxygen-level dependent signals as well as retinotopic mapping. Successful research results would verify the two-process model and provide invaluable information with which the quality of learning and memory will be enhanced.

Visual perceptual learning (VPL) is defined as a long-term increase in visual performance as a result of visual experiences. The purpose of the proposed project entitled ?Comprehensive frameworks of perceptual learning? is to advance the uncovering of the mechanisms underlying VPL and the basic mechanisms underlying visual and brain plasticity. This may, in turn, lead to interventions that are able to ameliorate diseases affecting vision and other pathological or age-related visual declines. We have identified three issues that must be resolved. First, there are contradictory interpretations of studies as to whether or not task-relevant VPL (R-VPL), defined as VPL of a feature relevant to a given task, is associated with a low- or higher-level stage in visual processing. A second issue involves whether R-VPL and task-irrelevant VPL (I-VPL), defined as VPL of a feature not relevant to a given task, are related to each other and, if they are, what the mechanisms are that connect them. A third issue is that there is no systematic research to test whether VPLs of different features (e.g., motion and shape) have the same underlying mechanism(s). To investigate these two issues, We will test the following three specific aims examining how local and global features are learned, using psychophysics and the online fMRI decoded neurofeedback. Specific Aim 1 will address the question as to which stage in the visual/brain processing and what mechanisms underlie R-VPL of motion. Hypothesis (H)1a: R-VPL of motion consists of both active plasticity developed with conscious effort and passive plasticity developed without conscious effort. H1b: Different types of plasticity in VPL of motion occur at different stages in visual processing as well as in different phases during the time course of learning. Specific Aim 2 will address the similarities between I-VPL and R-VPL of motion. H2a: The plasticity underlying I-VPL of motion is similar to passive plasticity in R-VPL of motion. H2b: I-VPL of motion occurs at the same phase in the timecourse of learning as passive plasticity in R-VPL of motion. Specific Aim 3 will examine how much the mechanisms involved in the VPL of motion can be generalized to VPL of shape of the target stimulus. H3a: There are passive and active plasticity in VPL of shape. H3b: Passive and active plasticity in VPL of shape occurs in different brain areas at different phases in the timecourse of learning. Through these experiments, we aim to provide important information toward a comprehensive framework to integrate and compromise controversies over VPL. !