Paul Bach-y-Rita - US grants

This research will develop an electrotactile haptic display to present computer graphics to blind persons. The increasing use of computer graphics and pictorial displays (such as the Macintosh and Microsoft Windows environments) presents a major problem for blind computer users. While text output is available through Braille, Optacon (optical-to- tactile converter), or speech synthesis, none of these techniques are suitable for presentation of complex pictorial or spatial information (organizational charts, computer program or process flowcharts, electrical, architectural, or mechanical drawings, three-dimensional graphs, pictures, visual artwork). The proposed investigations will create a flat, smooth tablet with embedded electrodes and external driver circuitry that can electrically display pictorial information from a computer directly to the skin. A blind user will then be able to interpret the properly-formatted percepts that s/he acquires by scanning the display with the fingertips. Exploratory research during Year 1 will extend present electrotactile knowledge to electrical stimulation of touch on the fingertip. Studies will determine the stimulation mode (direct afferent nerve stimulation or indirect electrostatic/mechanical stimulation), electrode geometry, and waveform characteristics that produce strong, repeatable, and comfortable single-point stimulation. Year 2 efforts will develop a small two-dimensional fingertip-scanned (haptic) display. Basic perceptual studies with this two-dimensional display will likely suggest further improvements in electrode geometry or driving waveform. During Year 3, this display (with associated electronics and control software) will grow at least ten times larger so that complex geometric and pictorial information from a computer screen can be displayed. Extensive perceptual studies during Year 4 will test the hypothesis that complex pictorial information is more readily perceived by haptic exploration with the fingertips than through a fixed-location 384-point electrotactile display on the abdomen (currently-available technology). Research during Year 5 will explore the feasibility of other rehabilitation applications, including retraining the visual perception of individuals with stroke-induced visual agnosia (intact eye and optic nerve), and dyslexia. We hypothesize that simultaneous display of "visual" information tactually and visually (on a computer screen), in conjunction with the user's properly-functioning tactile perceptual system, can cause functional reorganization of the damaged visual processing areas of the brain, effecting improved visual perception.

This research will develop an electrotactile haptic display to present computer graphics to blind persons. The increasing use of computer graphics and pictorial displays (such as the Macintosh and Microsoft Windows environments) presents a major problem for blind computer users. While text output is available through Braille, Optacon (optical-to- tactile converter), or speech synthesis, none of these techniques are suitable for presentation of complex pictorial or spatial information (organizational charts, computer program or process flowcharts, electrical, architectural, or mechanical drawings, three-dimensional graphs, pictures, visual artwork). The proposed investigations will create a flat, smooth tablet with embedded electrodes and external driver circuitry that can electrically display pictorial information from a computer directly to the skin. A blind user will then be able to interpret the properly-formatted percepts that s/he acquires by scanning the display with the fingertips. Exploratory research during Year 1 will extend present electrotactile knowledge to electrical stimulation of touch on the fingertip. Studies will determine the stimulation mode (direct afferent nerve stimulation or indirect electrostatic/mechanical stimulation), electrode geometry, and waveform characteristics that produce strong, repeatable, and comfortable single-point stimulation. Year 2 efforts will develop a small two-dimensional fingertip-scanned (haptic) display. Basic perceptual studies with this two-dimensional display will likely suggest further improvements in electrode geometry or driving waveform. During Year 3, this display (with associated electronics and control software) will grow at least ten times larger so that complex geometric and pictorial information from a computer screen can be displayed. Extensive perceptual studies during Year 4 will test the hypothesis that complex pictorial information is more readily perceived by haptic exploration with the fingertips than through a fixed-location 384-point electrotactile display on the abdomen (currently-available technology). Research during Year 5 will explore the feasibility of other rehabilitation applications, including retraining the visual perception of individuals with stroke-induced visual agnosia (intact eye and optic nerve), and dyslexia. We hypothesize that simultaneous display of "visual" information tactually and visually (on a computer screen), in conjunction with the user's properly-functioning tactile perceptual system, can cause functional reorganization of the damaged visual processing areas of the brain, effecting improved visual perception.

The PI and his team will explore solutions to the problem of access to computer graphics for blind users by combining a limited number of sensors and stimulators with the very high human capacity for fine movement control and spatial localization. He will conduct experiments to determine performance with a large number of skin stimulation points (by means of linked photosensor-to-vibratory stimulator units) on the hand, thus permitting individual finger motion (e.g., finger spreading) as well as total hand motion. Both the PI's own prior work and sensorimotor studies by others, along with concepts of the role of motor outflow brain signals in determining precise spatial localization, have led him to the hypothesis that a haptic system - as opposed to one which is strictly tactile, because the fingertips actively scan the display - can convey important spatial concepts from 2D and 3D images on a computer monitor. The PI's approach is based on concepts of brain plasticity underlying his prior sensory substitution studies, which have demonstrated that blind persons can obtain complex visual information in real-time through TV cameras leading to tactile interfaces. The current project will result in a tool for perceptual, electrophysiological and brain imaging (e.g., transcranial electromagnetic stimulation) studies on sensory substitution and late brain plasticity, and to practical applications for science education of blind students as well as for increasing their vocational opportunities in information technology.