Tony Ro - US grants

touch; perception; neural information processing; visual stimulus; human subject;

In our everyday lives, we are frequently confronted with information from multiple sensory modalities. Recently, there has been increasing interest in the circumstances under which stimuli presented in one sensory modality influence sensations in a different modality. For instance, the sound of a mosquito buzzing seemingly enhances sensitivity to touch (tactile stimulation) on our skin; seeing an insect crawling on someone else's arm seems to affect our own tactile perception. Despite several recent studies examining the influence of audition and vision on touch, the brain mechanisms responsible for these interactions are poorly understood. An NSF-funded collaborative effort of Tony Ro (Rice University) and Michael Beauchamp (University of Texas Health Science Center, Houston) will use a combination of converging methods to examine tactile processing in isolation and the influence of vision and audition on touch in the human brain. Psychophysical studies will be conducted to determine the optimal stimulus parameters that demonstrate an influence of vision and audition on tactile perception. Functional magnetic resonance imaging (fMRI), transcranial magnetic stimulation (TMS), and magnetoencephalography (MEG) will be used to localize the brain regions involved with integrating multisensory information. While most of these experiments will be conducted using normal controls, an additional series of experiments will be conducted in a unique patient who acquired auditory-tactile synesthesia following a stroke. Tactile sensitivity on the patient's left hand and arm was impaired, but he now feels tactile sensations in that area in response to sounds. Psychophysical and imaging experiment will be completed on this patient to determine the neural mechanisms responsible for the synesthesia, especially whether plastic neural changes have reconstituted the patient's somatosensory cortex so that it is now responds to sounds.

These studies will not only better our understanding of multisensory integration, but will provide a deeper appreciation of general information processing mechanisms of the human brain. Such knowledge will contribute towards the development of better rehabilitative tools for patients with congenital or acquired sensory deficits to one or more of the sensory systems. Additionally, this research will provide a better understanding of the mechanisms of natural and brain-damaged induced changes that take place in the adult human brain. The funding will be used to support research training opportunities for undergraduate, graduate, and post-doctoral trainees in cognitive neuroscience and brain imaging in the Houston area. In addition to training the next generation of brain scientists, the findings of this research will be disseminated through scientific and lay publications, as well as other media outlets, allowing for a deeper understanding and appreciation of the human brain in society.

Under most circumstances, our visual systems are bombarded with much more information than they can handle. Because of this information overload, the majority of visual information is not subjectively experienced and seemingly goes unnoticed as well as unprocessed within the brain. Despite this impression, however, an increasing body of evidence suggests that unconscious visual information is indeed processed within our visual systems even though we remain unaware of it. With support from the National Science Foundation, Dr. Ro and his colleagues will examine the cognitive and neural mechanisms for conscious and unconscious vision. One set of experiments will use procedures that will render some visual events unconscious to participants in order to determine whether the representations of unconscious visual stimuli are encoded at processing levels that are distinct from consciously perceived ones. Another set of experiments will use transcranial magnetic stimulation of the human primary visual cortex to manipulate neural processing and hence awareness of visual events. These transcranial magnetic stimulation experiments will investigate differences in information coding, neutral structures, and cortical pathways for conscious and unconscious vision.

This research will provide a better understanding of visual information processing in the human brain and will lend some clues for developing useful compensatory strategies in patients with visual deficits. This work could also contribute towards the development of better visual assistive devices or prostheses for patients with congenital or acquired visual deficits. In addition, this project will provide training opportunities for undergraduate and graduate students and postdoctoral fellows at The City College of the City University of New York. The results of this funded research will be broadly disseminated to both scientific and lay audiences and should contribute towards enhancing scientific understanding by the public.

The objective of this EFRI-M3C project is to develop models of sensorimotor control in order to establish a set of design criteria for developing improved assistive technologies for visually impaired people. Multimodal sensory information obtained by wearable machine sensors will substitute for and/or augment impaired vision with somatosensory and other novel stimulation (?transducing?) methods. The project has the following three research threads: (1) Determination of the information needed by visually impaired people to perform wayfinding and arm reaching tasks and the impact on task performance when transducing information through visual, auditory and vibrotactile modalities. (2) Understanding of motor skill acquisition and control through in vivo measurements of brain activity and movement performance. (3) Development of a sensorimotor model through man-machine co-learning, which can be used for improving sensor/display designs and applied to machines.

The intellectual merit of this research is that it will generate a theoretical foundation for a deeper understanding of the neural mechanisms of sensorimotor integration and motor learning, shedding new light on these functions in humans and machines. This will lead to new design concepts of alternative perception, formulation of required information necessary for successful orientation and wayfinding, and development of cost-effective and revolutionary mechatronic devices to assist visually impaired people in achieving mobility functions comparable to people with normal vision. The interdisciplinary team includes experts in engineering, computer science, psychology, and applied physiology from the City College of New York and Georgia Tech, tackling challenging problems on the boundaries of sensing, cognition, and action. Advisory board members, including experts in (neuro-)ophthalmology and human vision research, as well as counselors at the NYS Commission for the Blind and Visually Handicapped, will provide guidance for the study.

The broader impacts of this research will be assistive technologies for individuals with sensory impairments, whose numbers have been rising due to the increasing population of older adults in the US and around the world. The project will also create new and expand existing academic programs in assistive technologies, brain computer interfaces and sensorimotor integration. It will increase cross-campus education opportunities for students particularly in under-represented groups of the two campuses.

This travel grant supports students and domain experts to attend the workshop on "Multimodal and Alternative Perception for Visually Impaired People" in July, 2013 in conjunction with the International Conference on Multimedia & Expo (ICME). The workshop brings researchers and practitioners from multiple disciplines (computer vision, neuroscience, multimedia computing, sensor technologies and assistive technology applications) to discuss fundamental issues in visual perception, computational intelligence, neuroscience and visual prosthesis for helping blind and visually impaired people and people working in visually challenged environments. The workshop papers are included along with the IEEE ICME 2013 proceedings. The workshop homepage is used to disseminate results of the panel discussions and paper presentations as well as the report.

When we open our eyes, we seem to effortlessly perceive the external world and have the impression that our representations of the external world are highly accurate. Yet the reality is that our perceptions of an identical event can vary from one moment to the next and this variation is at least in part due to the timing of stimulus arrival relative to the on-going brain waves, i.e., the oscillation in the electrical activity of the brain. With support from the National Science Foundation, Dr. Ro and colleagues of the City University of New York (CUNY) will conduct a systematic series of experiments to assess the timing of visual information processing between different regions of the brain and the joint influence of feedforward and feedback information on visual perception. Using converging methodological approaches that include structural and functional magnetic resonance imaging (MRI), electroencephalography (EEG), fast signal optical imaging, transcranial magnetic stimulation (TMS), and psychophysical methods, this project introduces an innovative approach to the understanding of variations in visual processing.

This research will advance our knowledge of visual perception, which in turn will provide a solid foundation for developing better visual substitution or rehabilitation techniques for the nearly 300 million individuals worldwide with congenital or acquired visual processing deficits. This research will also provide cutting-edge training opportunities in neuroscience for postdoctoral fellows and undergraduate and graduate students, including women and underrepresented minorities. Finally, these studies will yield a large amount of TMS, MRI, EEG, and optical imaging data that will be available for sharing with other researchers.

At any given instance, there is typically far more information in the world than our visual systems can handle. For example, while reading this sentence, only a small proportion of the letters and words are processed at any given time. Although seemingly simple, the process of selectively attending to some information while ignoring other information requires the orchestration of many different brain areas, each of which contributes differentially towards this feat. This workshop will bring together international experts to discuss how the human brain focuses attention. In addition to covering historical and contemporary perspectives on the neural basis of attention, speakers will pay special tribute to the substantial contributions and influences of Professor Robert Rafal, a cognitive and behavioral neurologist whose career focused on better understanding the neural underpinnings of attention, consciousness, eye movements, and perception. This award will primarily fund travel for deserving students and early career stage researchers in the field and publications costs associated with producing a special edited volume from this meeting.

The meeting will take place on March 22-23, 2018 in the Segal Theater of The Graduate Center of the City University of New York. The talks and edited volume will highlight some of the important mainstay principles of attention research, as well as ones that have been substantially revised or overturned. Furthermore, unlike most other conferences and workshops, which tend to schedule more established researchers as speakers, this workshop will include speakers of varying experiences and ages, ranging from current and recent students to established senior researchers. Speakers will include several prominent female speakers, which should be especially inspiring for the younger female researchers and attendees. Additionally, specific efforts will be made to include a diverse group of student attendees, including invitations to underrepresented minority high school students and undergraduates in the STEM fields at CUNY, a highly diverse public institution in New York City. The student attendees should greatly benefit from the knowledge to be gained from attending this workshop and may decide to pursue a research or academic career focusing on better understanding the neural basis of attention.