Thomas A. Defanti - US grants

On February 9-10, 1987, an advisory panel met at NSF to represent the views of the scientific community on issues arising from graphics, image processing, and workstations. The panel consisted of representative, prominent scientists who build or use sophisticated visualization tools in the conduct of their research. As a result of that meeting, the panel now wants to generate a report defining the field of visualization and detailing its uses in the discipline sciences (e.g. biology, physics, economics). The proposal seeks to support the generation of that report. The report will consist of two pieces: a written report and a videotape. The report will define the field of Visualization in Scientific Computing, its domain, personnel, and key canonical problems. The body of the report will be written by researchers from certain discipline sciences that need advances in the underlying sciences of visualization. The videotape will present exemplary computer-generated graphics of scientific simulations. The report brings together a sampling of hard scientific problems that await advances in visualization in scientific computing, from NSF-supported researchers and other agencies of government; and , it suggest some programs, policies, and technologies to meet those needs. The investigators hope to change the way that science gets done on advanced scientific computers, with implications for scientific productivity and national competitiveness

9213822 Defanti This is the third year funding of a three year continuation award. Virtual reality includes 3D display of views which track the user's perspective viewpoint in real time. Two major existing modes are head-mounted displays (HMD) and boom-mounted displays. This work is on an alternative environment: a room ("CAVE") constructed from 7'x7' screens on which graphics are projected. This work will add engineering enhancements to improve the performance and sonic feedback of this room. Quantitative assessment of the benefits of this type of display for a number of tasks will be done. Collaborative effort will take place with discipline scientists working on grand challenge problems in computational science.

This award supports a project under the direction of Dr. Thomas DeFanti, U of Illinois at Chicago. Faculty, staff and graduate students at the National Center for Supercomputing Applications in Champaign-Urbana and the Electronic Visualization Laboratory in Chicago have been actively working with discipline scientists to develope prototype software and user interfaces for doing remote computational science and visualization. The High Performance Computing and Communication Visualization Project (HPCCV) is a combined exhibit and theater area designed for demonstrating computing, communications and visualization. The project is a human resources and technology transfer project, one that transfers computer science methods to the field of Computational science and one that aims to aggressively drive the acceptance of this methodology by showcasing real applications. HPCCV will demonstrate what it is like for scientists and engineers to have gigaflop-per-second computing brought to the desktop on light fiber. The exhibit will be displayed at "SIGGRAPH '92 at McCormick Place in Chicago on July 28-30, 1992. High speed networking within the exhibit and to the NSFnet will tie vector and parallel supercomputers all over the nation to the SIGGRAPH 92 exhibits floor. FDDI, HiPPI and TC technology will be extensively explored and utilized. The exhibit will also be displayed at Supercomputing 92 in the fall of this year.

9224605 Grossman SDB: Computing with Persistent Stores of Scientific Objects This is the first year funding of a three-year continuing award. The objective of this project is to develop technology to enable scientists to make computationally intensive queries on very large amounts of data in an object store. The project consists of three closely related parts. The first part of the project consists of developing algorithms to store, access, and analyze objects distributed in a hierarchical storage system. The second part of the project consists of developing a suite of software tools for working with objects distributed in a hierarchical storage system. The third part of the project consists of using these tools to create and analyze several stores, including: an EventStore -- a store of proton collisions (events) generated by counter rotating proton beams in a particle collider and registered by a variety of sophisticated electronic equipment; a TrajectoryStore -- a store of trajectory segments describing possible flight paths of aircraft modeled by nonlinear systems; and a VortexStore -- a store of vortex solutions of modified Navier-Stokes equations. As a result of this project, scientists will be able to analyze data with object stores, even when the size of the data requires that the majority of the data reside in tertiary storage. ***

9303433 DeFanti This award is for the establishment of a laboratory for research on the design, implementation, dissemination, and use of highly interactive computing technologies for the benefit of computational, biomedical and engineering sciences. The application areas share the need for very high-speed data capture and presentation facilities, very high bandwidth communication, and very large information stores. The research topics to be explored by this laboratory include the design and implementation of virtual reality environments; the storage, retrieval, and navigation of very large information stores; the design and evaluation of user- centered domain-specific, multi-media applications; and the remote, shared access to specialized instrumentation resources. The researchers come from the Computer Science Department, the School of Art and Design, the Department of Electrical Engineering, the Department of Bioengineering, and the Laboratory for Biomedical visualization. Extensive collaborations are in place with the National Laboratories. The award provides for the acquisition of instrumentation to match this research. The laboratory will consist of the CAVE Virtual Reality Theater, a Database Computing Facility, an Interactive Multimedia laboratory, and a Networked Remote User Facility. The instrumentation provided consists of high performance visualization computers, mass storage devices, and high speed communications equipment.

9418068 DeFanti This Metacenter Regional Alliance will tie together three complementary organizations: the Metacenter, the Software Technologies Research Center (STRC) at the University of Illinois at Chicago (UIC), and the Chicago Manufacturing Center (CMC). The organizing principle is the provision of a high-end testbed for virtual reality applied to manufacturing product design and rapid prototyping, to both large companies and consortia of small ones. The proposal will create and transfer the technology of virtual reality within the National Information Infrastructure. The software and advanced human interface technology for remote virtual prototyping and its supercomputing support will be developed with regional metacenter partners and small businesses. The proposed link to the vBNS is a vital part of this proposal. The STRC has demonstrated over a twenty year history that it can apply visualization technology to many areas, e.g., education, manufacturing, and training. It has provided the expertise that culminated in the CAVE virtual reality theater at the SIGGRAPH and Supercomputing conferences. CMC manages one of the newest NIST regional manufacturing Centers whose aim is to assist small and medium sized businesses modernize operations and improve competitiveness. Through its partnership with STRC, CMC, and NCSA this Alliance has the opportunity to provide leadership in areas of HPCC technology deployment to the Chicago region, and create a paradigm that can be replicated by other urban research institutions.

9407777 DeFanti The CAVE is a virtual reality System which allows users to surround themselves with artificially generated reality. One of its unique features is the ability for a group of persons to immerse themselves at the same time. This makes the CAVE a natural for demonstrations. This project will choose, by peer review, around 40 science projects and will work with the chosen disciplinary scientists to translate their work into the CAVE environment. A CAVE system will be set up at SIGGRAPH (the foremost exhibition in high-end visualization equipment, software, and underlying research) to show the results of the project.

If one is to posit that virtual reality will be used to train humans in critical performance tasks, the key becomes finding what sort of training is transferable between the virtual and physical worlds. Experiments performed thus far in the CAVE Virtual Environment showed significant performance improvement in virtual reality-trained subjects compared to the untrained population for the same task. Further experiments will quantitatively assess what may be perhaps the most important aspect of virtual reality research--the relationship between the technology and the training task which would result in a maximum transfer of training. Early results indicate that latency is a severe problem and that restricting virtual reality solely to the visual domain limits its training application. This work will extend the study of latency measurement to wide area networks using CAVE-to-CAVE experiments over the vBNS national network as the model. The results are expected to have broad applicability in the National Challenge areas, particularly in manufacturing and education If one is to posit that virtual reality will be used to train humans in critical performance tasks, the key becomes finding what sort of training is transferable between the virtual and physical worlds. Experiments performed thus far in the CAVE Virtual Environment showed significant performance improvement in virtual reality-trained subjects compared to the untrained population for the same task. Further experiments will quantitatively assess what may be perhaps the most important aspect of virtual reality research--the relationship between the technology and the training task which would result in a maximum transfer of training. Early results indicate that latency is a severe problem and that restricting virtual reality solely to the visual domain limits its training application. This work will extend the study of latency measurement to wide area networks using CAVE-to-CAVE experiments over the vBNS national network as the model. The results are expected to have broad applicability in the National Challenge areas, particularly in manufacturing and education.

9500396 Banerjee The use of virtual environment could provide a major advancement for manufacturing information management and decision support technologies. In this research virtual reality will be used in facilities and product layout information management and optimization systems. In the layout domain, the research will address three important problems: (1) three-dimensional (3D) general layout design, (2) layout of manufacturing cells and material flow paths in factories, and (3) layout of multi-module assemblies. The short-term objective is to demonstrate the feasibility of using virtual reality in reducing the manufacturing cycle time. The long-term objective is to develop raw technology for a real-time-virtually-real management control and decision support tool. There is also a major software development component. In addition, the research involves collaborations with Motorola and five other industrial firms, the Chicago Manufacturing Center, and National Institute of Standards and Technology, and industry collaborators will co-fund the work. The research aims to enable a shorter lead time in manufacturing. It will enhance the fundamental understanding of manufacturing decision support in light of integration of new technological developments in virtual reality. It will provide a new generation of tools for training, develop alternatives to physical prototyping and provide avenues for sustaining employee interest in better design of manufacturing systems through continuous improvement. The research will be introduced through lectures and laboratory experiments to graduate and undergraduate students. Concepts will be incorporated into courses in computer-aided design, virtual environments, production control, intelligent manufacturing systems and facilities layout. Seminars will also be conducted for industrial partners and other companies.

9512272 DeFanti The main focus of the proposed project is to create virtual-reality and virtual-prototyping instrumentation for viewing scientific and engineering data. The University has been involved in an ongoing development of the CAVE virtual-reality theater, a room-sized, high-resolution, projection-based system that enables users to experience good immersion in full 3D imagery. The goal of this proposed project is to broaden the scope for making the CAVE both smaller and larger. A smaller, software compatible, drafting-table format version, called ImmersaDesk, is being designed. It uses a high-end graphics system and projector for each ImmersaDesk, and a ATM gear to allow remote ImmersaDesk to ImmersaDesk collaborative design sessions. A larger high-resolution (1600 x 2048) stereo screen, called NII/Wall is being developed for large audiences. The NII/Wall will serve as a portal into networked scalable computers located around the country so scientists and engineers can interact with simulations running on distributed clusters of shared-memory processors. ***

9720351 Thomas Defanti University of Illinois Deep Learning And Visualization Techniques This award is to study the use of visualization techniques in encouraging deep learning of concept. The notion was initially developed on a virtual reality CAVE environment to illustrate the concept of gravity. The PIs are studying other educational applications where a compelling case for virtual reality can be made.

The University of Illinois at Chicago and its partners, the National Center for Supercomputer Applications (NCSA), the Argonne National Laboratory (ANL) and Ameritech will provide an operational interconnection point at the Ameritech Network Access Point (NAP) that will enable international connectivity and interoperability of the cutting edge US research networks (e.g., vBNS, Internet 2, Next Generation Internet components) and high performance networks of other nations. This project is a key piece of infrastructure for high performance international Internet connections to the vBNS. It will complement the forthcoming NSF solicitation for High Performance International Internet Services (HPIIS), and it will support the G7 Global Information Infrastructure project "Global Interoperability of Broadband Networking" (GIBN) by providing a common interconnection point with staff support and performance monitoring. The Canadian Network for the Advancement of Research, Industry and Education (CANARIE) has already connected to the vBNS through the Ameritech NAP in an initial proof-of-concept test.

EIA-9802090
Defanti, Thomas A.

University of Illinois

CISE Research Infrastructure: CAVERN: The CAVE Research

UIC's research goal is to move Tele-Immersion and Data Mining from the laboratory to the Next Generation Internet. Scientists are just now beginning to get access to high-speed networks to retrieve information from remote datasets (whether large disk farms, scientific instrumentation, or satellites), analyze that information using remote supercomputers, and use virtual reality (VR) to collaborate with distant colleagues. UIC has promoted VR and data mining in select communities of users, and now wishes to create the teams, tools, hardware, system software, and human-interface models to enable national-scale, multi-site collaborations to facilitate solutions to National Challenge and Grand Challenge problems.

Planned CAVE Research Network (CAVERN) activities are twofold: enabling technology development and application driver support. This proposal lays out several application disciplines- - computational science and engineering, education and training, and every citizen interfaces (ECI) with emphasis on rehabilitation for the disabled - - to drive the development of the tools and techniques for tele-immersion and data mining. Enabling technologies for tele-immersion and data mining include CAVERNsoft, ECI interfaces, data mining tools, video, and performance optimization.

EIA-9871058 DeFanti, Thomas A. Brown, Maxine D. University of Illinois Chicago MRI: Instrumentation Development for Human-Centered Tele-Immersion The Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago has developed an aggressive program over the past decade to partner with dozens of computations scientists and engineers all over the Nation. The focus of this effort has been to create visualization and virtual reality (VR) devices and applications for collaborative exploration of scientific and engineering data. Since 1995, the EVL research and development activities have incorporated emerging high bandwidth networks like the vBNS, in an effort now called Tele-Immersion. As a result of past experience in building first and second generation VR devices to support these applications, the EVL now wishes to conduct research in third-generation VR devices to construct variable resolution and desktop/office-sized displays. Since no current technology is yet configurable with ideal resolution and size, there is a need to simulate these devices with available parts, and then build more advanced prototypes. This proposal requests significant equipment funds to buy, evaluate, and integrate a variety of emerging display devices, such as large color plasma displays, LCD projectors, LED panels, Feffo-electric Liquid Crystal displays and Digital Macro Mirror Displays. This proposal also requests funds to purchase new workstations for use by the EVL's 50+ students, and dozen faculty and staff. The proposal aims at long-term, basic display configuration research and development.

This award to the University of Illinois at Chicago, (UIC) entitled"EuroLink: High Performance International Internet Services between Research and Education Institutions in the United States and Europe/Israel" provides support for a High Performance International Internet Services connection between the advanced networks of the Nordic countries (Denmark, Finland, Iceland, Norway and Sweden), France, Netherlands, Israel and NSF's very high-performance Backbone Network Services (vBNS). The overall consortium (European networks plus UIC) is called EuroLink. EuroLink proposed to provide four links, each nominally at 45 Mbps, between the European NRNs and the vBNS via the NSF-funded international connection point for high-performance networks, STAR TAP , in Chicago. It is anticipated that other European countries may join EuroLink in the future. Round-the-clock operational support will be provided under subcontract to the Indiana University Network Operations Center (NOC). The Indiana NOC also serves under a companion HPIIS award to TransPAC for Asia-Pacific collaborations, and it also provides operational support to the Internet2/Abilene network.

The Science, Technology and Research Transit Access Point (STAR TAP) was funded by NSF ANI-9712283 for the period May 1,1997-April 30, 2000. The current award funds a three-year continuation of the pivotal infrastructure services provided by STAR TAP. STAR TAP services will be expanded in depth to take account of dynamic changes that have taken place in high performance networking since the inception of the Next Generation Internet (NGI) Program as well as related domestic (e.g., Internet 2) and foreign (e.g., Canada, Japan, Singapore, Netherlands, Israel, Nordic countries and others to follow) initiatives. The nature of the effort, however, is substantially unchanged, namely to support global collaborations on progressing high performance networking and advanced networked applications by providing persistent, reliable interconnections between domestic and international partner advanced academic networks.

EIA-0115809
Thomas A. DeFanti
University of Illinois at Chicago

MRI: Development of Instrumentation for AGAVE: the Access Grid Autostereo Virtual Environment

This is a proposal for instrumentation development under the Major Research Instrumentation (MRI) program to support research and student training in tele-immersion technologies for a networked, collaborative virtual-reality environment. The focus is on AGAVE, a tiled, high-resolution autostereo display that integrates well with very-high-speed networks.

At the end of this decade, national and international-scale scientific collaborative applications will need intelligent signaling and dynamic control of very-high-performance optical networks. The software proposed here will allow scientific applications to directly control an advanced, all-optical, IP-over-wavelength metropolitan-scale network, based on Dense Wave Division Multiplexing (DWDM) and photonic switching.
Optical networking technology is rapidly migrating from ultra-expensive long-haul implementations to regional- and metro-area networks. The trend is to provide a general infrastructure with a wide range of common services. However, the flexibility inherent in these new technologies provides the research community with an opportunity to move beyond general requirements and support large-scale e-science applications that require advanced networking capabilities. We propose a software development effort that will lead to 21st-century applications over 1000-fiber, 1000-wavelength photonic networks.
These evolving, extreme applications requiring optical networks include high-energy physics, astrophysics, climate modeling, oceanographic modeling, architectural design, molecular modeling, industrial design, advanced photon source experimentation, materials science, and industrial engineering. Underlying such applications are cross-cutting support technologies, such as advanced digital video, remote access to scientific instruments, specialized visualization displays, data-mining, cluster supercomputing and high-performance distributed computational systems. To enable the full potential of such applications, it is not sufficient to simply provide high-performance networks; these applications need intelligent, dynamic controls to adjust network resources. The proposed software development efforts will leverage the significant potential of a newly installed metro optical testbed for application-level dynamic control of resource discovery, allocation and adjustment. Efforts at many levels are required to make such flexibility available in service provisioning, infrastructure and service resource management:
Research into the behavior of advanced scientific applications, not just on extremely high-performance
optical network, but on one that can be dynamically adjusted at a granular level
Identify application-level networking requirements, investigate management techniques for optical
networks, and study new service provisioning models related to application needs
Research new methods for application signaling
Investigate interconnections between application signaling and IP-based control-plane methods, such as
through GMPLS Test deployment of those techniques on an advanced testbed and analyze results
Experiment with multiple-service provisioning to ensure gateways to traditional networks and protocols
Develop a system for performance metrics, monitoring and analysis
Create a testbed for StarLight, the next-generation, optically based STAR TAP, and for other advanced
research networks.
The testbed for this project is an a four-node optical network, OMNInet, initially linking a core node on
Northwestern University's Chicago campus with a node at the University of Illinois at Chicago, the Canadian
Network for the Advancement of Research, Industry and Education (CANARIE) CA*net4 node at its Chicago Point of Presence and a node at Northwestern's Evanston campus. The sites are separated by distances of 5 to 20 miles, connected by dedicated technology trial-fiber service provided by SBC/Ameritech. Each node includes a Nortel Networks WDM photonic switch, an Optical Fiber Amplifier (OFA) and high performance router/switches. These sites will also have access to Nortel and SBC/Ameritech testing personnel, expertise, and equipment. Participants in this project, led by the Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago, include the International Center for Advanced Internet Research at Northwestern University, CANARIE, Argonne National Laboratory, MREN (Metropolitan Area Research and Education Network), Nortel and Ameritech.

The Intellectual Merit of StarLight: StarLight is a research support facility planned by researchers for researchers that follows the mode of Research and Operations now established by the "STARTAP" project. These researchers are networking engineers, electrical engineers, computer engineers, computer scientists, e-Scientists and application programmers. StarLight will anchor future wavelength-rich LambdaGrids, with switching and routing at the highest experimental levels, laying the foundation for fully optical switching in three years. Following the tradition of the earlier "STARTAP" project, this project will enable the next cycle of network operations growth with researchers.

Optical networking technology is rapidly migrating from ultra-expensive long-haul carrier implementations to affordable regional- and metro-area community networks. The next step is to expand, integrate and tune these resources with specific large-scale scientific applications. The commercial trend is to provide a general infrastructure with a wide range of common services for a very broad client base. However, optical networking technologies provide networking researchers with an opportunity to move far beyond general requirements, and support large-scale e-Science applications that exploit very advanced networking to tackle complex problems.

Until recently, STAR TAP, an NSF-funded project providing interoperability and interconnectivity of advanced networks, adequately served the ATM connections in Chicago among the US Federal Networks (Fednets), Internet2.s Abilene, Chicago's Metropolitan Research and Education Network, and several international networks.
However, several months ago, as domestic and foreign networks began to request 1-to-10Gb switched connections in Chicago, STAR TAP quickly migrated its focus and connections to StarLight, a new colocation facility.

The Nation needs a persistent facility, staffed and equipped, to serve researchers using IP-over-lambda networks, addressing restoration issues, building LambdaGrids, optimizing DNS services, and testing novel protocols for long, fat connections. StarLight will be a persistent meeting point as a node on various LambdaGrids and will support advanced applications and middleware research, and aggressive advanced services. StarLight will grow into a:

(1) Multi-vendor, multi-lambda 1Gb, 2.5Gb, 10Gb (and perhaps higher) experimental exchange and LambdaGrid nodal point that provides the "other end" or switching point for National and international experiments; Starlight has 3-year plans to make available 60 fully-powered and fiber-optic-connected racks to researchers.
(2) Facility to support 3D protocol stack development (for optical management, control and data planes).
(3) Middleware research environment, in which application-oriented middleware, like Globus, and data transport protocols are implemented and tested by the middleware community with major experimental e-Science drivers.
(4) Host for national and global research activities in application-centric network measurement and monitoring.
(5) Inter-domain security testbed to research and deploy encryption at lambda speeds and to test getting data out to crisis management personnel involved in natural or human-caused disasters.
(6) Laboratory for developing/testing means for high-performance application provisioning on optical networks.
(7) Facilitator of broad outreach efforts to communities, using a cultural heritage project to bring in under-represented communities as content developers, providers and users of data and visualization LambdaGrids.
(8) Metro-scale laboratory for CISE students' access to the interdependent worlds of computing and networking.

EIA 02-24306
DeFanti, Thomas A.
Grossman, Robert L.; Leigh, Jason; Nelson, Peter C.; Yu, Oliver
University of Illinois Chicago

CISE RR: Matching Advanced Visualization and Intelligent Data Mining to High-Performance Experimental Networks

This project, developing techniques for advanced grid computing with focus on visualization and data-mining applications, aims at setting up a high-performance, high-bandwidth Grid matched with Visualization and Data Mining Research. The Grid is made up of Lambdas (clusters of PCs) that are connected by high-bandwidths connections to other clusters. Software and toolkits to enable high-performance applications will be built with the goal of attaining a high-speed optical metropolitan-area network to be used in the data mining and visualization applications. The project expands grid technology to infrastructure, protocols, network memory and distributed control, and applications in constraint logic programming. Since optical networking technology is rapidly migrating from ultra-expensive long-haul carrier implementations to affordable regional- and metro-area community networks, this project explores inherent flexibilities in these new technologies to support large-scale data, visualization, and collaboration-intensive applications with very advanced real-time demands. Thus, application-aware software and middleware will be created to help interconnect tomorrow's terascale-class machines with distributed petabyte data stores, remote sensors, instrumentation and visualization over gigabit/sec to terabit/sec networks. LambdaNodes, defined as PC clusters with storage and visualization coupled to like clusters by numerous wavelengths (called lambdas), will be used connected by lambda networks to create a prototype metropolitan-scale LambdaGrid. With the goal of optimally matching data mining and visualization to high-performance optical networks with e-Science and homeland security model applications as expected drivers (achieving a 10x or greater end-to-end improvement over today), the equipment and support enables the following:
1. Matching data-mining and visualization capabilities on clusters to emerging wavelength-rich networks,
2. Distributing parallel computation and rendering for high-resolution volume visualization,
3. Providing applications signaling and control of both electronically and optically switched lambdas,
4. Measuring and monitoring multi-gigabit circuits over multiple wavelengths,
5. Providing users with networks that have known and knowable bandwidth and latency,
6. Investigating high-availability/uninterrupted cluster computing to support time-critical collaborations,
7. Addressing real-time applications in security domain, and, eventually, security of the data as well,
8. Integrating metropolitan-scale LambdaGrids with the emerging Global Grid and the National TeraGrid, and
9. Incorporating distributed data mining and visualization into undergraduate African-American-centered coursework and research through the Virtual Harlem Project.

This project, supporting collaboration among co-located and remote experts requiring interactive ultra-high-resolution imagery, aims at developing a high-end visualization "LambdaVision" multipanel display as a means to advance both scientific research and public safety as validated by users in various disciplines and training exercises in disaster response. LambdaVision will target applications requiring extremely high data rates and very large, high-resolution images, such as earth surface imagery. Applications include scientific simulations and disaster response. Plans included the development of smaller versions for use in the classroom and in the field. The following research applications will use the system:
Geoscience Research ,The US Geological Survey, Integrated Ocean Drilling Program
US National Lacustrine Core Repository, Computer Science Research in Visualization, Advanced Networking Middleware, and Multi-User Collaboration ,Scalable Adaptive Graphics,Advanced Networking Middleware,Collaborative Methods for Display-Rich Environments
Research Instrumentation Development, LambdaVision and Lambda Table Architecture LambdaVision and Lambda Tabel Seamless Display Technology ,LambdaTable Tracking System

Broader Impact: PIs have a history of developing technology that domain scientists use. The work includes designing systems suitable for use in the field, classroom, and lab.

This proposal describes an engineered solution to providing high capacity, high performance, leading-edge network services between the U.S. and Europe. Specifically, connections are proposed connecting Chicago, New York, and Amsterdam. The proposal leverages existing partnerships with SURFnet in the Netherlands and expertise and experience from years of engineering and running the StarLight facility in providing in this proposal production-level international network connection services spanning the IP layer through "wavelength" services at Layer 1. The StarLight facility in Chicago is used, as is the new ManLan facility in New York City.

This collaborative project, developing and evaluating lifelike, natural computer interfaces as portals to intelligent programs in the context of Decision Support System (DSS), aims at providing a natural interface that supports realistic spoken dialog, non-verbal cues, and the capability of learning to maintain its knowledge current and correct. The research objectives focus around the development of an avatar-based interface with which the DDS user can interact. Communication with the avatar takes place in spoken natural language combined with gesture expressions or by pointing on the screen. The system supports speaker-independent continuous speech input as a spontaneous dialog within the specified DSS domain. A robust backend that can respond intelligently to the questions asked by the DDS user is expected to generate the responses spoken in reply by the avatar with realistic inflection and visual expressions.

The work develops, prototypes, and evaluates the desired user interface capabilities by using the model of a program officer to create a realistic avatar that can answer users' questions and respond in a humanly natural manner. The project extends a current sponsored project on information gathered related to a centers program where a program officer serves as subject matter expert. The recently-developed AlexDSS system that answers questions to users about the I/UCRC program provides the baseline intelligent system behind the avatar. The avatar interfaces are targeted for both general users as well as for experts responsible for updating/correcting the domain knowledge therein.

The work represents a collaborative project between the Intelligent Systems Laboratory (ISL) at UCF and the Electronic Visualization Laboratory (EVL) at UIC. The EVL team focuses on avatar development encompassing Visualization and Interaction with Realistic Avatars and Evaluation of System Naturalness and Usability. The ISL team concentrates on Natural Language Recognition and on Automated Knowledge Update and Refinement.