Erik Viirre, M.D, Ph.D. - US grants

The Virtual Retinal Display (VRD) is a new form of display device developed at the Human Interface Technology Laboratory in Seattle, Washington. This device scans light directly into the eye to create images at the retina. It is anticipated that the VRD will become a ubiquitous display device as it will be miniaturized to wear on a spectacles frame. The VRD is bright, yet safe and uses lasers as highly color saturated sources of light. The research from this proposal will be studies on the fundamentals of light perception of scanned images. These studies will allow us to optimize the scan characteristics for best color, resolution and contrast. We intend to understand the features of temporal and spatial integration by the retinal processing elements of the scanned light and how that integration affects the image perception. We will then be able to produce images with excellent color matching and image sharpness. By optimizing the VRD in this research we will allow its use where image quality is essential, such as in remote surgery, or surgical training. Further, bright, high quality images with the VRD will make it excellent for use in augmented vision applications, where images are superimposed on the real world.

This project will develop engineering designs for an appliance that will help people with certain types of low, or impaired, vision to see. The designs will be based on the principle that engineering design is a decision-making process, not merely a problem-solving one. Thus, as an aid for the decision maker, a model will be built that defines concepts such as value, attributes, demand, time, price, and other factors, and utilizes a rational for optimization. The project will have two main parts: (1) the development of the decision-making model and its application to the design of the Low-Vision Aid (LVA) and (2) the development of LVA demonstrations that will illustrate the feasibility and importance of such a product for people with low vision. Vision enhancement will be achieved by exploiting the unique attributes of the Virtual Retinal Display (VRD), a means of safely scanning image data directly onto the retina without intermediate image formation surfaces. The VRD projects an image on the retina that is very bright, has high contrast, has highly saturated colors, and is projected into the eye via a very small exit pupil. If successful, this research will lead to the development of easy-to-use vision enhancement aids, devices that will enable the visually impaired to read printed material (e.g., books, magazines, newspapers, etc.), to watch TV, to interact with a computer monitor, and to navigate in the physical world. The impairments that this instrument will help overcome include corneal scars, keratoconus, presbyopia, and cataracts. The optical characteristics of the VRD may also prove to be an aid for macular degeneration and amblyopia. The development of engineering designs is the first step in the process of making the LVA available to people subject to these diseases. The project will also provide an example application of decision-based engineering design.

Lesions of the vestibular organ lead to complaints of unstable vision, dizziness and imbalance. Such lesions are also accompanied by abnormalities on vestibular function testing: specifically, the vestibulo-ocular reflex (VOR) is altered, resulting in abnormal caloric responses and rotary chair testing. The VOR is a vision stabilizing reflex. In it, as the head moves, signals from the vestibular apparatus drive movements of the eyes, to keep the visual world stable. We know that small lesions of the vestibular apparatus lead to changes in the VOR. Specifically, the gain of the VOR (defined as the magnitude of the eye velocity output divided by the head velocity stimulus) may be lowered. There is a neural mechanism that adapts to changes in VOR gain. Small defects in VOR gain, such as when a spectacle prescription is worn, are rapidly corrected. However, with large changes in VOR gain, such as when the vestibular apparatus in one ear is surgically removed, the adaptive mechanism appears to fail. These patients usually have chronically low VOR gains and persistent sensations of vertigo and imbalance. There are two theoretic possibilities: the lesion could have ablated the adaptive mechanism as the VOR was disrupted, or the adaptive mechanism may have been overwhelmed by the magnitude of the lesion. The latter appears to be the case. We have tested 6 patients with persistently low VOR gains and have found that it is possible to slightly increase their VOR gains temporarily with a new method of adaptation. We have developed an immersive computer graphics environment designed for visual-vestibular interaction research. A subject wears a head mounted display that provides a wide image and blocks the view of the outside world. As the subject moves, a head position and orientation tracker measures the position of the head. The computer rendering system then shifts the scene to correspond to the new point of view. In the graphics environment the magnitude of visual scene movement relative to head movement and rate of optic flow is under software control. Our protocol for adaptation uses the computer control and uses the observation that small required changes in VOR gain are rapidly adapted to. Thus is a subject had a VOR gain of 0.4, we demagnified the scene to 0.44, thus requiring an adaptation of 10% instead of 250%. 30 minutes of exposure to successively larger increments in this paradigm proceed significant VOR gain increases at 0.32 and 0.64 Hz measured immediately after the exposure. In the proposed work here, we will test subjects after repeated exposures to successive increments in required VOR gain and determine if we can induce a persistent VOR gain change and a reduction in symptoms.

The Virtual Retinal Display (VRD) developed by the PIs and their colleagues creates an image by scanning non-coherent or coherent light across the retina in a raster pattern, rather than using a conventional image plane screen. The VRD has full color and a broad luminance range for high contrast, even in bright ambient daylight; the mechanism will eventually be miniaturized to fit on a spectacle frame, making wearable displays feasible. Because of these attributes, and the fact that VRD parameters can be configured in various combinations to alter characteristics of the perceived image such as field of view, resolution and the size of the scanning aperture, the VRD holds great promise for enhancing vision in the partially sighted. The goal of this research is to investigate how best to realize this promise. The small exit pupil of the VRD allows clear images to be seen by users with corneal distortions and/or partial opacities of the lens such as cataracts. For central retinal diseases such as macular degeneration, the VRD can be configured to direct high contrast, large feature images to the peripheral retina. For peripheral disorders, where only a small central area may remain functional, the display can be configured with a high-resolution narrow field of view. Pilot tests of subjects with partial loss of vision indicate that even non-optimized images from the VRD can usually be seen better than CRT images. The PIs will conduct psychophysical visual testing with partially sighted users, and systematic comparison with standard computer displays (CRTs and LCDs), to determine for a variety of visual disorders the appropriate VRD scanning and modulation characteristics for optimal image resolution, contrast sensitivity, color perception and absence of flicker. The ability of users to view text, see video images and use common computer applications such as word processors, spreadsheets, and web browsers will also be assessed. The expectation is that this research will ultimately lead to new visual display products and techniques to enhance computer access for people with low vision, including the growing elderly population

visual tracking; psychomotor function; fatigue; eye movements; performance; sleep deprivation; sensorimotor system; rest; cognition; clinical research; human subject;

The project proposes to create a distributed, multi-user social computing environment that will develop the capabilities of human Electroencephalography (EEG) to analyze users engagement with digitally based experiences. For this project, users will wear non-invasive, EEG headsets while navigating a shared virtual world. Beginning with a handful of EEG systems, the team will scale up over the course of the project to gather signals from dozens of users, providing a basis for larger scale studies. By comparing the EEG signals with each participants activities in virtual world, and with the brain activity and the activities of other users, a model of human brain activity will be developed for different types of behavior profiles and subjective states. This will allow significant improvement for the development of neural markers of human perceptual, cognitive and affective states, the parsing of EEG signals, the applicability of EEG interfaces to new types of experiences, all of which can enhance distance learning, collaborative distributed work, improved mobile computing interfaces and health care applications. The project will advance the capabilities for determining an individuals cognitive state by the creation of new computing methods utilizing comparative EEG analysis and data analysis of event states in a digital simulation. Bringing methods of large scale data analysis to articulate patterns across many users in the situated milieu of the online virtual world will create a new method to utilize EEG analysis to infer human subjective experience. The necessity of conducting this analysis in real-time, with data gathered from distributed, wireless EEG instruments will provide the impetus for utilizing accelerated hybrid multi-core techniques to bear on this domain.

The results from the project will be applicable for a variety of digital environments including computer aided learning and training, digitally mediated collaborative work environments, visualization of complex data sets, and digitally based entertainment experiences such as virtual worlds and computer games. The project will improve the functionality and outcomes of these digital media environments, better adjusting them to the cognitive states of the users. The PIs will train and employ a diverse body of participants to be involved in these activities. As part of an internship program at the Preuss School, a nationally recognized middle and high school on the campus of UCSD, UCSD faculty and staff mentor high school seniors to provide these students valuable experiences in a research laboratory. In addition, UMBC operates the very successful Meyerhoff program for minority students, primarily African-American, for which over 90% of a yearly entrance number of more than fifty go on to graduate school in science or engineering. Many of these students are computer science majors that have had the elective graduate course of Service Oriented Computing for Scientists and Engineers.

This project will explore ways to increase bandwidth to the brain though optimal spatial-temporal simulation of the far peripheral retina. There are many questions related to the functionality of the far periphery and how it engenders of a sense of place and presence within a virtual environment. To address these questions this project will develop: (1) a state-of-the-art retinal image projection system that can generate and position spatial patterns of light encompassing the complete retina; (2) methods for measuring the effect of that stimulation on the brain wave activity, balance and task performance; and (3) new approaches for organizing and portraying information over the complete retina that increase throughput of data to the brain. The research from this project can lead to development of better virtual reality components that unlock the power of human intelligence and link minds globally. This line of research is essential as the industry presses for optimizing the utility of 5G cellular bandwidths in servicing wireless and ubiquitous virtual and augmented displays via WebVR and related systems.

This project will investigate how to input the autonomic nervous system to enhance the cognitive behavior of users. A key component in the research agenda is to design and build an experimental system that generates and positions patterns of light stimuli onto the various regions and eccentricities of the entire visual field of the participant. This system must allow for the precise generation of the spatial, temporal and spectral properties of these stimuli. These factors constitute the independent variables in the research. Similarly, this research apparatus must also collect the voluntary and involuntary behavioral response of the participant to these stimuli using direct methods that include event-related cortical potentials, oculomotor behavior and involuntary movement of the body (e.g., posture platform). Voluntary or performance-based tracking and manipulation of the virtual world experience will also be included in the experimental apparatus.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

This project will study brain aging and inflammation in microgravity. Long-term space exposure creates a series of physiological alterations including cognitive decline. Studying astronauts' brains before and after their mission can tell what goes wrong but provides little insight on the mechanisms responsible for the observed alterations. This, this project will grow stem cell-derived human brain tissues at the International Space Station to accelerate our understanding of the mechanisms involved in brain aging. These studies will have profound implications for improved neurological pre-clinical models for applications on Earth. Educational benefits from this investigation include incorporating the results into the training of graduate students and high school students. The work will also be disseminated to the broader community through a video channel and podcast episodes.

Microgravity has been shown to evoke alterations in cell growth, differentiation, cell communication, aging, and epigenetic/gene expression alterations in various cell types, including brain cells. Preliminary data, from a previous spaceflight, showed exposure to microgravity induced a series of molecular changes that mimic aging in human cells. Such alterations might accelerate or enhance cellular (telomerase dynamics) and molecular processes (activation of endogenous retroelements) that are important for neurological disorders. Here, this study will mechanistically investigate this aging phenotype on human brain organoids infused with microglia. First, the brain organoids will be optimized for spaceflight compatibility. Second, the impact of re-activation of endogenous retroelements will be characterized. And third, brain organoids will be generated with inducible TERT and modulate their activity at different developmental stages to determine how telomerase activity affects the development of brain organoids. The proposed work will advance understanding of 3D brain organoids enabling improved models of neurological disorders that represent significant health burdens.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.