Robert Kenyon - US grants

9213822 Defanti This is the third year funding of a three year continuation award. Virtual reality includes 3D display of views which track the user's perspective viewpoint in real time. Two major existing modes are head-mounted displays (HMD) and boom-mounted displays. This work is on an alternative environment: a room ("CAVE") constructed from 7'x7' screens on which graphics are projected. This work will add engineering enhancements to improve the performance and sonic feedback of this room. Quantitative assessment of the benefits of this type of display for a number of tasks will be done. Collaborative effort will take place with discipline scientists working on grand challenge problems in computational science.

If one is to posit that virtual reality will be used to train humans in critical performance tasks, the key becomes finding what sort of training is transferable between the virtual and physical worlds. Experiments performed thus far in the CAVE Virtual Environment showed significant performance improvement in virtual reality-trained subjects compared to the untrained population for the same task. Further experiments will quantitatively assess what may be perhaps the most important aspect of virtual reality research--the relationship between the technology and the training task which would result in a maximum transfer of training. Early results indicate that latency is a severe problem and that restricting virtual reality solely to the visual domain limits its training application. This work will extend the study of latency measurement to wide area networks using CAVE-to-CAVE experiments over the vBNS national network as the model. The results are expected to have broad applicability in the National Challenge areas, particularly in manufacturing and education If one is to posit that virtual reality will be used to train humans in critical performance tasks, the key becomes finding what sort of training is transferable between the virtual and physical worlds. Experiments performed thus far in the CAVE Virtual Environment showed significant performance improvement in virtual reality-trained subjects compared to the untrained population for the same task. Further experiments will quantitatively assess what may be perhaps the most important aspect of virtual reality research--the relationship between the technology and the training task which would result in a maximum transfer of training. Early results indicate that latency is a severe problem and that restricting virtual reality solely to the visual domain limits its training application. This work will extend the study of latency measurement to wide area networks using CAVE-to-CAVE experiments over the vBNS national network as the model. The results are expected to have broad applicability in the National Challenge areas, particularly in manufacturing and education.

EIA-9802090
Defanti, Thomas A.

University of Illinois

CISE Research Infrastructure: CAVERN: The CAVE Research

UIC's research goal is to move Tele-Immersion and Data Mining from the laboratory to the Next Generation Internet. Scientists are just now beginning to get access to high-speed networks to retrieve information from remote datasets (whether large disk farms, scientific instrumentation, or satellites), analyze that information using remote supercomputers, and use virtual reality (VR) to collaborate with distant colleagues. UIC has promoted VR and data mining in select communities of users, and now wishes to create the teams, tools, hardware, system software, and human-interface models to enable national-scale, multi-site collaborations to facilitate solutions to National Challenge and Grand Challenge problems.

Planned CAVE Research Network (CAVERN) activities are twofold: enabling technology development and application driver support. This proposal lays out several application disciplines- - computational science and engineering, education and training, and every citizen interfaces (ECI) with emphasis on rehabilitation for the disabled - - to drive the development of the tools and techniques for tele-immersion and data mining. Enabling technologies for tele-immersion and data mining include CAVERNsoft, ECI interfaces, data mining tools, video, and performance optimization.

0216550
Mussa-Ivaldi
The goal of this project is to design and develop a specialized robot manipulator with a state-of-the-art augmented-reality display. This development project is a collaboration of world-leaders in engineering, robotics, virtual-reality, motor system neuroscience, and rehabilitation; affiliated with the Rehabilitation Institute of Chicago (RIC), Northwestern University, the University of Illinois in Chicago and Barrett Technology Inc, an advanced robotics company based in Cambridge, MA. This system will be a natural extension of technologies that are currently employed at RIC's Robotics Lab.

This project, developing SENSEI (SENSor Environment Imaging), an instrument that targets a broad range of big data science and engineering challenges, promises a unique sensor platform for 2D and 3D recording within dynamic environments. SENSEI contributes to a broad range of data-driven application domains. These range from fundamental research in instrument design and development to data processing, fusion and synthesis, enabling not only the creation of the instrument, but also its use as a resource for multi-domain science and engineering on the ground, in the air, and underwater.

Specifically, SENSEI is a spherical, (ultra) high-resolution (9-times-IMAX resolution and ~terapixel/minute flood of imagery), vision-based capture system capable of video-rate data-acquisition. The instrument will address domain challenges in science, engineering, medicine, and beyond by enabling investigation of big data acquisition, streaming, processing, archiving and access, visualization, and analytics.

The broader significance of this project will be felt in a variety of image-intensive scientific disciplines. The areas of environmental monitoring, remote sensing, situational awareness, homeland security, and mechanical and structural engineering can greatly benefit from the proposed instrument. The instrument will be designed for replication by the global community of researchers and the graduate students. The technology will be communicated through classes, projects, theses, publications, as well as museum exhibits and conferences. Special attention has been paid to broadening participation through the Minority Serving Institutions Cyber-Infrastructure Empowerment Coalition (MSI-CIEC).

Massive amounts of brain imaging data open an unprecedented window into the structure and function of the brain, yet the tools to aid the understanding of the data are lagging. The main goal of this research project is to understand the dynamics of brain function, particularly in the auditory system, through global yet very high spatial and temporal resolution imaging techniques and using the state-of-the-art analytical tools developed for analysis of dynamic social networks. The interdisciplinary neuroscience and computational team, building on promising initial results, will work to adapt these tools developed for understanding human and animal behavior to the context of brain networks and the processes that happen over them. Using this innovative approach, the team will study a particular brain pathway that connects two brain regions that are critical for normal hearing. The project will not only lead to a greater understanding of brain function, but will also bring a new technique to the neuroscience toolbox which may help other investigators to study network properties of the brain. Graduate students and postdocs in computer science and neuroscience will collaborate across disciplinary boundaries, building new scientific approaches and insights.

Top-down projections are ubiquitous in sensory systems and are poorly understood. In the current proposal, a model descending system, the auditory corticothalamic projection in the mouse, will be examined. The research team will take advantage of recent methodological developments in the study of this system and ask: What is the impact of corticothalamic projections on network interactions across populations of thalamic neurons? To answer this, a novel dynamic network analysis method known as Community Dynamic Analysis, or CommDy, will be used to analyze imaging data from a brain slice preparation that retains connectivity between the auditory cortex, auditory thalamus, and other related structures in the mouse. Both calcium imaging data and flavoprotein autofluoresence imaging data will be used for this analysis. Since this study represents the first use of CommDy in neuroscience, validation studies will be done in a simplified brain slice preparation containing bilateral motor cortices and the corpus callosum.

This project, developing The Continuum, a cyberinfrastructure instrument that unifies high-resolution computer-enhanced group collaboration workspaces with arrays of ambient sensors, aims to enable users to collaborate with local and remote colleagues and their data more effectively utilizing a War room to benefit the collaboration. The room itself is given the ability to anticipate needs, improve interaction, and focus the time on discovery rather than the technology. Continuum serves as a digital assistant providing a foundation for new computer science research, benefits global scientific collaboratories. War rooms benefit collaboration because they support continuous interaction and awareness of team members, enable persistent information sharing, and encourage mobility among team members and information. A computer enhanced war room, such as The Continuum, which 1. Connects distributed teams with their data (whether stored in the cloud or streamed from major computational resources over high-speed networks), 2. Supports persistent digital artifacts, 3. Enables ultra-high-resolution 2D and 3D stereoscopic digital media to be simultaneously viewed, and 4. Supports ubiquitous and intuitive interaction, creates a powerful and easy-to-use information-rich environment in support of scientific discovery. Just as NSF cyberinfrastructure produces greater volumes and varieties of data from data storage systems, online instrumentation and major computational resources like XSEDE and Blue Waters, advanced visualization instruments serve as the 'lenses' of these data generators (similar to the telescopes and microscopes of yore), enabling researchers to peer into cyberspace to manage, access and analyze big data.

Continuum is the culmination of a quarter century of experience at the institution developing interactive instruments, from the original CAVE in 1992, to the ultra-high-resolution LambdaVision tiled LCD wall in 2004, and the hybrid-reality CAVE2 in 2012. Each new generation of visualization instrumentation provided scientific communities with more advanced features. Continuum development focuses on three areas: 1. Transforming passive display walls that respond to user commands into attentive display walls that monitor the users in a room and anticipate their needs to improve their experience; 2. Providing an alternative approach to constructing high-resolution visualization display walls using thin-border 4K passive stereo Organic Light Emitting Diode (OLED) displays that have impressive brightness, contrast, uniformity, off-axis viewing, and low reflectivity in brightly lit rooms; and 4. Utilizing cloud computing to prototype future Continuum spaces that do not require local high-end compute and display resources. Twelve funded research projects from local institutions, in Astronomy, Astrophysics, Biology, Combustion, Geophysics, Neuroscience, Pathology, Physics, Psychiatry, and High Performance Computing, are poised to use Continuum for their data visualization needs. The Continuum will open up new opportunities in computer science research at the intersection of large-scale data visualization, human-computer interaction, and high-speed networking. Continuum will also directly support 10 classes taught in the UIC Computer Science, Art and Design, and Biomedical Science departments.

This project is about acquiring a much-in-demand Graphics Processing Unit (GPU)-based instrument, to develop a Service Instrument for Deep Learning and Visualization called "COMPaaS DLV: COMposable Platform". The project aims to complement available campus computing resources via the campus's research network. It will be able to access local and remote computing and storage facilities funded, including XSEDE, Blue Waters, Deep Learning Instrument and Chameleon. This critical instrumentation will provide a platform to pursue fundamental science and engineering research training in deep learning (data mining and data analytics, computer vision, natural language processing, artificial intelligence), visualization (simulation, rendering, visual analytics, video steaming, image processing), and a combination of deep learning and visualization (e.g., when data is so large that it cannot be easily visualized, then deep learning is used to extract features of interest to be visualized). The instrumentation also enables investigation and contribution to societal issues in disciplines such as anthropology, biology, cybersecurity, data-literacy, fraud detection, healthcare, manufacturing, urban sustainability, and cyber-physical systems (e.g., autonomous cars).

Its design utilizes state-of-the-art computer architecture, known as composable architecture, in which the computer's components (traditional processor, GPU, storage, and networking) form a fluid pool of resources, such that different applications with different workflows can be run simultaneously, with each configuring the resources it requires almost instantaneously, at any time. Given composable infrastructure scalability and agility, it is more beneficial than traditional clouds and clusters that are rigid, overprovisioned, and expensive.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.