Ruzena Bajcsy - US grants

The research will investigate the integration of a number of sensors and actuation mechanisms to form a robust robot controller. Principal research issues are: (1) the subdivision of the task to allow for concurrent planning and execution by the sensor and action systems, or "agents", (2) the integration of a number of disparate partial sensor observations from these agents into a "best" statistical estimate of task state; and (3) the development of a real time programming system to support such a distributed agent system. The project will also investigate language primitives for expressing timing constraints within agents, distributed scheduling of agent execution, and inter-agent communication requirements.

The overall objective of this project is to develop techniques and ultimately a system which will allow the rapid, objective and quantitative analysis of brain scan data obtained from computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET). This entails relating the scans to a computer based atlas of the human brain for objective determination of anatomic information with which to relate the functional image. We plan to develop a system with "user friendly software" (e.g. simple to use by clinicians and basic researchers). We expect that the system will be highly interactive and will provide real-time analytic feedback abilities to the user. Techniques will be developed for the interactive display, manipulation, and image processing of 3-D data. A 3-D Voxel representation of a standard Atlas of the human brain as well as of clinical CT, MRI or PET data will be implemented. Image processing capabilities will include automatic thresholding, reslicing in any arbitrary plane, calculation of various statistics, and highlighting outlines or areas of interest. The display capabilities will include real-time translation, rotation, scaling, segmentation, and enhancement of 3-D medical objects. Interactive editing of such 3-D medical objects and the merging of multiple modalities (i.e., CT and MRI) for comparison and analysis will also be supported.

This grant will support three students for research in robotic exploratory perception, robust multisensor fusion, and motion- based visual determination of navigational obstacles. The studies will advance our understanding of complex sensor-system design, spatial knowledge representation, and exploratory procedures for learning unknown dynamic environments.

This award will provide infrastructure for research that is organized around five laboratories: 1. LINC - for research on artificial intelligence and natural language processing; 2. GRASP - for research on machine perception and robotics; 3. GRAPHICS - for research on graphic interfaces, movement description, and animation; 4. DSL - for research in computer architecture and computer communication; 5. LOGIC & COMPUTATION - for research in logic and computation, including theory of computation, database systems, and programming languages. Two new facets of the research, integration and upward scaling, require an enhanced experimental environment involving machines with massively parallel architectures. The award will help to develop this environment by providing funds for a SIMD machine for work in natural language processing, and active perception and real time manipulation; a MIMD machine for simulation and research involving extensive scientific calculations; as well as high speed workstations with rich environments for work in theoretical computer science.

This Software Capitalization Grant will support the free distribution of important image analysis software to all interested universities, colleges, and industrial sites. The research software, developed with support from a 1983 Center for Engineering Research award, includes routines for superquadric volumetric modeling, wavelet decomposition of signals, image segmentation, and tactile camera visualization. Polishing of the user interface (including documentation) and porting of the suite as necessary to Sun, Microvax, and IBM platforms will be a done by a team of undergraduate students, providing valuable research experience and career training. An effort will also be made to identify serious users and compile information useful for evaluating the success of this experimental program.

This award provides partial support for a workshop in which university, industry, and Federal government researchers will identify and assess the research needs for automated mobile robotic platforms with material handling capabilities. DARPA is sharing in the funding of the workshop, which is expected to produce recommendations for a common set of specifications and features for such mobile platforms suitable for a variety of research in vehicle navigation, noncontact sensors such as vision systems, contact position and force sensing, processing of sensory information for guiding mobility and manipulation of multiple manipulators and multiple sensors, and multi-modal systems integration.

Visual measurement of surface reflectance properties is an important issue both in basic and in applied computer vision research. Surface reflectance gives a reliable cue for image segmentation and object recognition. The goal of the proposed research is to detect and separate specularity from Lambertian reflectance based on physical models. In order to provide enough information to extract various properties of reflectance, we generally need more than one frame of images. One direction is to use color information and another is to use multiple images with different views. An algorithm for color image segmentation and detection of specularity and inter-reflection is proposed. In addition to the use of color, the use of different views to differentiate specularity from Lamertian reflectance and to obtain object structure is proposed. the proposed algorithm is based on the fact that the appearance of specularity varies depending on viewing direction while that of Lambertian does not. In order to build a more robust and reliable vision system, information by color and multiple views will be integrated. For efficient processing of a large amount of spectral (color), temporal (multiple views) and spatial (image) data, parallel algorithms using a Connection Machine of 4K processors will be designed and implemented. The high degree of computational parallelism will bring the algorithms into practical reality.

This award is to support a postdoctoral associate to work in experimental computer science. The associate, Marcos Salganicoff, will work with Dr. Ruzena Bajcsy on problems involving robot control and planning. The research will blend learning, data structures, and parallelism. Planning information will be stored in binary trees (such a k-D-trees or quadtrees). This information, usually massive amounts of it, is then used in learning algorithms. A hybrid approach using distributed workstations for some of the computation and a MIMD computer for some of the rest will be employed to implement these learning algorithms in a cost effective manner.

9355018 Bajcsy The proposed program is focused on research and education relating to the fundamental scientific issues underlying the coordination of multiple robotic agents for manipulating and transporting objects with limited assistance from human agents. We argue that the narrow specialization of the PhD does not always serve the needs of education and engineering today. We propose an integrated approach to research which fosters multi-faculty-student interaction. In this approach several faculty and students from different backgrounds, but with common goals, collaboratively pursue a research agenda which requires the collective expertise of the group. The interdisciplinary GRASP Laboratory with faculty and students from four different engineering departments is uniquely qualified to train PhD students in the focus area and provide them with the breadth and depth that is required in today's high technology society. We request five traineeships for the laboratory. ***

9307126 Bajcsy This is the first year of a three year continuing award. The research involves investigation of modeling, analysis and synthesis of visual behaviors of Active Observers. An Active Observer is a mobile agent whose task is to observe, and report/communicate information about the activity of another agent(s) whose task may be manipulation and/or navigation. Active Observers are part of a small team of agents who work together in a cooperative fashion. All agents in the team are equipped with visual navigation capabilities, but the tasks they are engaged in differ, depending on their role in the team and their goal. The agents could navigate independently or in a cooperative fashion (i.e. in the case of two manipulator agents carrying an object together). For manipulatory-agents navigating in a cooperative fashion, the issue of communication arises; hence, the problem of representing visual information for communication. The visual tasks of the observers-agents will differ depending on whether their observations serve for the navigation or manipulation purposes of the other agents. The main emphasis in this research is on the control structure that the observer must have in order to deliver the proper observations for the task at hand. For the purpose of modeling behaviors a formalism from the Discrete Events Systems (DES) theory has been adopted, suitable for investigating control-theoretic issues of a system. It is anticipated that the contribution of this research will be in: 1) modeling visually guided behaviors which tightly couple visual processing with control architecture; and 2) the identification of elementary behaviors needed for navigation and communication and their composition leading to more complex behaviors. Two kinds of elementary behaviors are distinguished, one where observations are directly connected with physical actions, and the other where observations and actions are either received or transmitted. The use of DES formalism allows the synthesis of complex behaviors in a systematic fashion and guarantees their controllability and observability

9303980 Bajcsy This is the first year of a three-year continuing award to support collaboration of researchers in four U.S. institutions (University of Pennsylvania, University of Rochester, University of Maryland, and University of Massachusetts) with their European colleagues engaged in the ESPRIT-funded research project entitled "Vision-As- Process ". The collaborative objective is to develop a practical theory of perception control in active vision, while pooling expertise and avoiding unnecessary duplication of effort. The broad areas of concentration are development of novel physical and computer hardware, conceptual frameworks, and software tools for integrated vision/robotic systems, which include an actively controlled binocular camera head and various actuators for affecting the external world. The collaboration covers four subareas: Control of perception in active vision, Interpretation of dynamic scenes, Robust image and scene measures, and Dynamic gaze control and image formation. For each subarea a mailing list for information exchange will be created, scientific meetings held, and techniques and associated experiments organized. Two to three week partner-to-partner visits will facilitate intense collaboration on the specific topics. US Ph.D. students will spend time in EEC laboratories as part of an exchange agreement. Both sides will benefit from a demonstrated real-time active vision system, and development of a common framework for integration and control experiments. Such a system will enlarge the variety of experiments within the same class of problems and the range of applications in different domains. The results will be made widely accessible, through planned publication of a book on the theoretical and experimental results, journal papers, and presentations at conferences. ***

This award, in the Small Grants for Exploratory Research mode, provides seed funding for initial exploration of a national virtual laboratory (NVL) in robotics research. The Laboratory will take advantage of the emerging National Information Infrastructure by connecting research facilities across the country to design joint experiments in robotics. These will be geographically distributed experiments, using different agents at separate sites, such that perception/action modules will interact physically with their local environments, but will communicate over the network to coordinate their accomplishment of a common task. Initially the NVL will develop an infrastructure for sharing resources among geographically distributed university and industry research laboratories. This infrastructure will consist of communication capabilities for video, voice and data links offering high-quality real-time excange and remote operation capabilities. This will enable the researchers to share and test all their software and hardware resources under diverse environmental conditions.

The methods of rapid prototyping are ideally suited to rehabilitation devices. Because each person requires unique performance and function in a rehabilitation device, devices specific to each person must be rapidly designed and produced. This project is investigating a completely integrated approach to the design and prototyping of passive mechanical rehabilitation devices. The approach involves: the quantitative assessment of the form and performance of human limbs; the design of the assistive device; evaluation of the device using virtual prototyping; feedback from the consumer and therapist; actual prototyping of the device; evaluation of the function and performance of the device; and redesign based on performance. The contributions of the product include: the development of new computer-based tools for the assessment of human performance; a manufacturing technique for a new class of hyperelastic materials; the integration of tools into a rapid prototyping system for rehabilitation devices; and development of mechanisms for systematic evaluation of the final product.

9507160 Bajcsy This U.S.-Czech workshop on "Computer Vision" to be held in Prague during September 3-4, 1995, will bring together experts from Slovakia as well as the United States and the Czech Republic to discuss topics pertaining to computer analysis of images and patterns. Dr. Ruzena Bajcsy of the University of Pennsylvania will serve as the principal organizer for U.S. participation. Dr. Vaclav Hlavac of the Czech Technical University will serve as her counterpart and the organizer for the Czech side. The objective of the workshop is to identify computer vision research topics for subsequent, long-term collaborative research projects that feature complementary resources and expertise in computer science and relevant mathematical foundations. ***

CDA-9703220 Bajcsy, Ruzena University of Pennsylvania Asymmetric Bandwidth Channels: Applications to Real-Time Computing and Robotics This award is for the acquisition of infrastructure to support research which is to investigate a cost-effective and broadly deployed communication model, Asymmetric Bandwidth Channels, for which few abstractions in computer science are available. Research into the application of systems characterized by low-bandwidth interactive channels between clients and server, and a high-bandwidth broadcast from server to clients will be conducted. The proposed work will develop communications abstractions that applications can effectively use for such an infrastructure, and computational models for these abstractions. Model performance will then be evaluated on a testbed of multiple semi-autonomous robotic agents. The challenging problems to be addressed in this project include: (1) selecting which path to take from server to client; (2) determining the degree of broadcast channel sharing possible in a computer communications environment; and (3) scheduling transmissions from the shared broadcast terminal. The research will also target three fundamental problems underlying coordination of robotic agents: (1) development of world models based on observations of individual agents and exploration of an unknown or a partially known environment to build a complete model; (2) task planning based on the world model while accounting for possible uncertainties and latencies; and (3) control of robotic agents based on visual and other sensory information.

The goal is to develop a computer system to support geographically separated researchers performing cooperative experimental research in robotics. This system uses vision techniques under the control of smart planners to acquire a visual model (combining 3-D data and texture maps) of the physical environment where the experiment takes place. It provides real-time graphic renderings of the experiments on the researchers' graphic workstations along individually selected viewpoints. Hence, each researcher is able to OBSERVE the experiments using his/her own controllable virtual camera. The initial exploration will focus on one major component of the envisioned system, the automatic construction of a 3-D model of an experimental robotic site. This will be used to analyze basic issues that will also be encountered in the other components of the envisioned system, in particular the effect of uncertainty on the interaction between a user and a remote experimental site. This is a collaborative effort between R. Bajcsy of the GRASP laboratory at the University of Pennsylvania and J. C. Latombe at Stanford University.

This project investigates the design, modeling, (re)configuration, and validation of trustworthy networked embedded systems (NES) applications that support the privacy of their users, the confidentiality and integrity of the data, the availability of the provided services, while implementing the required functionality and quality of service by developing the fundamental new science of secure network embedded systems and its implications for the emerging infrastructure. NES applications must be protected against malicious attacks that exploit specific vulnerabilities and characteristics of networked embedded systems. Such critical support applications that must have trusted data include equipment and process control (avionics, veitronics, communications, and SCADA/DCS systems) and environmental monitoring (pollution and chem/bio agent detection).

This project seeks the following three innovations: (1) A suite of mathematical models to support the development and validation of trustworthy NES applications, (2) high confidence middleware components to assure the adaptability and survivability of NES applications, and (3) a large-scale test-bed to validate the suitability of these models and methods in a realistic NES application environment.

The work is a cooperative effort of three organizations. The University of California Berkeley models distributed hybrid and embedded systems theory and platforms and addresses issues of privacy in trusted NES. UCB is building a large-scale testbed network of around 103 embedded network devices (Motes) to explore issues of NES application trustworthiness in a realistic environment and experimentally validate how the modeling and components developed by all groups in the project can protect mission-critical NES applications from potential abuses. Vanderbilt University brings their extensive experience in modeling functional capabilities of NES applications, which yields new systems theory and high confidence composable middleware frameworks with probabilistic elements to them. SRI conducts research in modeling the trustworthy aspects of NES applications and on developing methods and tools to support model-based co-design that can enable the systematic and predictable interweaving of trustworthiness with the functional applications and middleware.

This proposal creates an experimental infrastructure network to support the development and demonstration of next-generation information security technologies for cyber defense. This cyber Defense Technology Experimental Research Network (DETER Network) will provide the necessary infrastructure -- networks, tools, methodologies and supporting process -- to support national-scale experimentation on emerging security research and advanced development technologies. The DETER network will be designed and operated to ensure direct participation from government entities and their sponsored researchers in a wide and varied community. Early activities and deliverables will describe policies procedures for use of the experimental facility along with a users guide. The work will facilitate scientific experimentation and validation against established baselines of attack behavior and allow experimental approaches that involve breaking the network infrastructure. The DETER network will promote and catalyze expanded research and commercialization efforts in this vital area.

Intellectual Merit: The proposal will develop architectures for test bed networks that are representative of the Internet itself at a somewhat smaller scale. It will leverage work from ACIR at the International Computer Science Institute, Berkeley on traffic generation for use in the DETER network. The development of both of these is itself a significant research challenge. On versions of the DETER network the researcher will study cyber security solutions to Distributed Denial of Service, Worm Defense and other network attacks through a combination of experiments, emulation, and analytical solutions. The aim is to integrate analytic methods with experiments on networks of adequate scale and complexity so as to be able to have confidence in solutions and transfer them to industrial partners participating in the test bed through their equipment.

Broader impact: This proposal is expected to be the key building block for bringing network security initially against Distributed Denial of Service and then Worm attacks but in general all intrusions. The testbed network will be integrated into course work at both the upper division and graduate level Berkeley, USC and at partner institutions at UC Davis, Purdue, and Penn State.

Technological innovations have made it possible to complement our traditional (physical) museums and libraries with digital archives. Emerging digital technologies are quickly making possible new forms of access to artifacts, including virtual museums built on immersive visual technology. In such environments, it is likely viewers will be less passive; they will configure artifacts as they wish, and not be limited by the experts' views of what is good, useful, or significant. This raises concerns about historical and interpretative accuracy, both concerning replication of materials and with respect to setting them in relation to each other to produce meaningful interpretations. As a first step to exploring such issues, the PI will in this project investigate the technological difficulties in creating accurate 3D representations of physical artifacts. She will focus in particular on three fundamental unsolved problems: automatic 3D data acquisition of arbitrary complex shapes; representation of these complex shapes allowing search algorithms to easily access the right object or a part thereof; and the proper display/visualization of such objects as they serve the user needs depending on the user's questions.

To anchor this research, the PI will focus on works of Sikh art and cultural artifacts, an area chosen because it supports a large collaborative project to establish a Virtual Museum of 500 years of Sikh art and cultural artifacts, involving the Center for Information Technology Research in the Interest of Society (CITRIS) at UC Berkeley, the University of California Humanities Research Institute (UCHRI) at UC Irvine, and the Sikh Foundation of Palo Alto, which will provide financial and in-kind assistance in identifying and enabling access to virtually all major Sikh art and cultural collections in the West.

Broader Impacts: This project constitutes in many ways the cutting edge of the relation between IT and humanities research. If successful, the work will form the basis for collecting, collating, displaying, and distributing materials for the purposes of advancing art, historical, cultural, and social research, and education on particular cultures. We will be able to assess comparative studies of similar objects (their geometries, materials and functionalities), and their evolution over time. In turn, this will enable better explanations of the evolution of the societies that produced these artifacts.

Cyber-infrastructure refers to the ability to access and integrate today's hardware, software, and human information technology resources in order to facilitate science, engineering, research and education goals. In the next five years, the challenge for Cyber-infrastructure will be socio-cultural and behavioral dynamics, economics, and policy. The objective of this award is to hold a hold a workshop that will seek to identify for the SBE and CISE Directorates at NSF the key problems in the social sciences, economics, organizational, and policy studies for Cyber-infrastructure. The goal of the workshop will be to stimulate discussion within and between the communities served by the SBE and CISE Directorates about Cyber-infrastructure and to produce a report that outlines some of the key challenges and areas for fruitful research, development and experimentation by SBE and CISE. Specific goals for the workshop are
1) To produce a report which lays out a Cyber-infrastructure research and development roadmap for the SBE and CISE community and provide a framework for projects and efforts going forward;
2) To provide a venue for community building within the SBE and IT communities;
3) To lay out a program for research on the effects of IT on society and the
dynamics of IT-focused organizations and the cyber-culture

Broader Impacts:
SBE plays a key role in the development of Cyber-infrastructure. Not only is data and computational Cyber-infrastructure an enabler for key efforts for SBE communities, but SBE can play a critical role in the design, development and culture surrounding the deployment of Cyber-infrastructure itself. A strategic approach to the development of a body of research, development, and experimentation that addresses the socio-cultural, economic, and policy challenges of Cyber-infrastructure will be critical to ensure its success. In particular, social and behavioral scientists, humanists, organizational, policy and management researchers, etc. are critical as process builders for Cyber-infrastructure as well as end users of Cyber-infrastructure. Such communities have only sporadically been engaged in the development of Cyber-infrastructure models, social and organizational structures and have a wealth of experience and context to offer.

Tele-immersion is an emerging technology that enables interaction of users, engaged in activities such as training of tai-chi movements, dancing, or assistance in physical therapy, between geographically distributed sites. This is achieved through realistic video and sound reconstruction of the activities in three-dimensional (3D) space in real time. Several components of these tele-immersive environments need to be in place to achieve seamless immersive experience: (a) set of cameras covering the 360 degree space in which the activity is taking place, hence creating 3D data sets, (b) sound system recovering sound from 360 degrees without echo, (c) broadband networking technology that enables throughput of large data sets across geographically distributed sites (end-to-end) with minimal latency and synchronous delivery, and (d) display technologies to display data from different sites in a consistent fashion. Many components of the tele-immersive environments have been developed, and preliminary small scale experiments with individual and customized components have been performed.

What is missing is a holistic deployment and experimentation with the current existing tele- immersive components. In this one-year project, we aim to (1) integrate current COTS components that are available on the market for tele-immersive environments, such as the current 3D cameras, wireless sound system, Internet2 protocols, and existing plasma and projector-type displays, (2) deploy the integrated COTS components at two geographically distributed sites, UC Berkeley and University of Illinois at Urbana-Champaign (UIUC), and (3) test and execute preliminary experiments with our holistic integrated system, called Tele-immersive Environment for EVErybody (TEEVE). This integration and deployment will present very unique results because they will enable testing of the existing COTS components and their capabilities and limitations to deliver cutting-edge technology to broader audience. We expect also that the integration and deployment will reveal future research challenges since we plan to perform extensive experimental analysis of the TEEVE system. Our experimental analysis will span from network-based measurements where we plan to measure and evaluate accomplished bandwidth across Internet2 when sending large number of audio and video streams, accomplished latency, jitter, loss rate, response time, compression ratio on real-time 3D data, vision-based measurements where we plan to measure the correctness of existing 3D reconstruction algorithms in terms of semantics and real-time behavior, to user-based measurements where we plan to experiment with the tai-chi teacher at the UC Berkeley side and question students at the UIUC side about their experience versus using a standard video tape.

The intellectual Merit rests in (1) the extensive integration of COTS components which will consider and analyze the multiple configurations and synchronization mechanisms possible in our multi-tier system, (2) the deployment and experimentation with the integrated system, measuring the various metrics at different system/user levels, and (3) integration guidelines and setup methodologies how to put together such a holistic tele-immersive infrastructure like TEEVE. Experiences and experimental results will be shared with the research community in the form of documentation, software, and tutorial(s) at conferences such as ACM Multimedia, IEEE International Conference on Multimedia and Expo (ICME) and others.

Broader impacts will derive from the fact that will be the first holistic deployment of an affordable tele-immersive Internet-based infrastructure. TEEVE is the first step to allow sharing of activities that require full body visual and auditory information across geographically distributed sites. Examples exist in health care such as physical therapy to users in remote locations (e.g., Indians, Eskimos), in training of sport activities such as tai-chi, or in art training such as dancing.

Tele-immersive environments allow people separated by distance to physically interact and communicate in real time inside a shared 3D virtual environment through the use of large camera networks that enable the capture and reconstruction of 3D images and sound and the subsequent integration of this multimedia data from geographically distributed sites. This project develops and deploys a next generation tele-immersive environment built for the common user who does not have the luxury of expensive supercomputing facilities and dedicated networks. More particularly, the vision of this project is to create a geographically distributed and cost effective tele-immersive environment that facilitates ordinary people performing physical activities in their homes or schools or doctor''s offices or job training facilities under the supervision of a trainer, therapist, or teacher who is not co-located.

The broader impact of this project is in facilitating physical interaction and communication of the elderly and persons with disabilities in their homes and work places with relatives, health care providers, and other service providers, in particular whenever they need some interaction during a period of rehabilitation, recovery or training. This project will also examine multimedia distributed communication using common Internet connectivity, which does not require high bandwidth or extremely expensive equipment, and thus may promote wider use of tele-immersive environments.

Rice University, in an alliance with Boston University; the University of California at Berkeley; Southern Illinois University at Carbondale; the University of Colorado at Boulder; the University of Texas at Austin, and additional partners, proposes the Empowering Leadership: Computing Scholars of Tomorrow Alliance (EL Alliance). It will develop a national network of colleagues who share the common experience of being a minority student or faculty at a majority institution, and others deeply committed to the success of these students and faculty. The EL Alliance has enlisted many of the leaders in computing research to this network, including faculty, directors of national laboratories, leaders in industry, junior faculty, grad students, undergraduates, and professionals. The network will provide multiple, integrated, reinforcing programs and experiences to ensure that the few minority scholars in the computing disciplines at top departments matriculate and go on to make their mark in graduate education, research, industry, or national policy. These programs will include the national support network, local and online communities, and annual meetings, using technology to reach, inspire, and support isolated minority students. Furthermore the Alliance will be aligned with well-recognized programs such as AGEP Alliances, NSDL, CI-TEAM, the TeraGrid, and other Broadening Participation in Computing awardees. The EL Alliance will directly impact many of the finest underrepresented minority students in the country, encouraging, preparing, and retaining them as they pursue degrees in computing. In turn these students, as they take their places in the leadership of the community, will be in a position to impact additional students.

This project investigates (a) how immersive 3D technology will motivate, encourage, and stimulate dance collaborators in their creative processes through a model of dance, (b) identify new elements of the IT technology that support new choreography, e.g., new view management capabilities, new synchronization protocols, and (c) characterize two different interfaces for two very different groups of users: the dancers and the choreographers.

By studying dance we can learn about non-verbal communication among people in general. There is no other art form that comes as close as dance to everyday human activity and yet intensifies, amplifies, and transforms that everydayness into extra-ordinary activity. The significance of dance as a means of analysis of a creative process include the following: dance's coded and choreography's conventions can be understood as a means of communication that is practiced across racial, cultural, ethnic, and socio-economic boundaries; one can view dance as a shared human activity; dance is a form of expression that uses bodily movements that are rhythmic, patterned, spatial, dynamic, expressive, shaped, felt, improvised, etc. dance is often accompanied by music; one of the oldest art forms, dance is found in every culture and is performed for purposes ranging from ceremonial, liturgical and magical to the theoretical, social, and aesthetic; dance has also been used to effect change in bodily or social conditions for religious and/or political circumstances; in human time, dance has been used as an effective form of communication that went beyond boundaries of linguistic sign systems; dance has a potential shared by music and virtual arts to transform human behavior; dance lends itself to analysis because it exists in a triangle of communication: choreographer, dancer and viewer.

Intellectual Merit: This project will validate two major hypotheses: (1) Dancers will be able to learn faster new choreographies in the 3D tele-immersive spaces than via 2D video technologies, and (2) 3D tele-immersive spaces will allow choreographers to explore new choreographic elements when utilizing Digital Options not possible in co-located environments (e.g., different scales of dancers dancing together). The intellectual merit will be in understanding the creativity process in choreography domain, and in IT domain. These two hypotheses requires the development of IT tools for support of creative processes, i.e., flexible configurations of views for individual dancers, new synchronization protocols in spatial and temporal domain as the tele-immersive sides merge into one tele-immersive (TI space), and new interfaces for dancers and choreographers to access the new Digital Options in an easy manner.

Broader Impact: This project will impact developments in the 3D tele-immersive technology as well as the dancing area. From the IT perspective, the new algorithms, protocols, interfaces and corresponding tools will allow users interested in creativity processes to work with and manipulate different views, scales, numbers, backgrounds, shapes, and other digital options needed in visual arts creative processes. In the domain of dance, we will have a better understanding of the learning and the creative processes of dance choreographers. This project will also impact education. Teaching students the art of choreography requires that they expand their perceptions of the body in time and space. Often it takes the student years to learn how to do this. With the help of the 3D tele-immersive system, the student will be given tools to immediately start experimenting with bodies in space. This expansion of perception will have huge implications within the creative process itself, allowing students to see and experience new ways of being in the world, and recombining the human forms in infinitely new relationships to space and to other forms.

This project explores the impact of geographically distributed 3D tele-immersive environments on dancers' creativity and their perception of themselves and each other. More specifically, the project will study the impact of digital options, such as scale and multiply, on the dancers' creative expression and improvisation. A formal notation, called a creativity graph, will be derived from a Laban movement analysis of the dancers' movement. A dancer develops a creative dance, a new sequence of phrases, when he/she generates a new association between two movement states (e.g. a new association between two Laban positions) based on some feedback from the immersive environment due to either invoking digital options or due to some unexpected performance of the system. This project will have a fundamental impact on our understanding of dancers' creativity within tele-immersive dancing environments and computing and will contribute a new concept of transformational movement graphs as a computational representation of dance creativity in distributed multi-site 3D tele-immersive spaces.

The objective of this research is to study the formal design and verification of advanced vehicle dynamics control systems. The approach is to consider the vehicle-driver-road system as a cyber-physical system (CPS) by focusing on three critical components: (i) the tire-road interaction; (ii) the driver-vehicle interaction; and (iii) the controller design and validation.

Methods for quantifying and estimating the uncertainty of the road friction coefficient by using self-powered wireless sensors embedded in the tire are developed for considering tire-road interaction. Tools for real-time identification of nominal driver behavior and uncertainty bounds by using in-vehicle cameras and body wireless sensors are developed for considering driver-vehicle interaction. A predictive hybrid supervisory control scheme will guarantee that the vehicle performs safely for all possible uncertainty levels. In particular, for controller design and validation, the CPS autonomy level is continuously adapted as a function of human and environment conditions and their uncertainty bounds quantified by considering tire-road and driver-vehicle interaction.

High confidence is critical in all human operated and supervised cyber-physical systems. These include environmental monitoring, telesurgery, power networks, and any transportation CPS. When human and environment uncertainty bounds can be predicted, safety can be robustly guaranteed by a proper controller design and validation. This avoids lengthy and expensive trial and error design procedures and drastically increases their confidence level. Graduate, undergraduate and underrepresented engineering students benefit from this project through classroom instruction, involvement in the research and substantial interaction with industrial partners from the fields of tires, vehicle active safety, and wireless sensors.

Rice University leads a collaborative proposal to extend the impact of the Empowering Leadership: Computing Scholars of Tomorrow Alliance (ELA) through partnerships with new institutions and regional collaboratives that will adopt and expand ELA's successful models of engagement. The goal of ELA is to increase the number of students from underrepresented groups who major in computing at the nation's research universities. Additionally, ELA supports these students in securing positions in computing following graduation. The premise of the ELA is that minority students at research universities face challenges that can be mitigated by a supportive community that provides academic, social, and personal support. National in scope, the ELA is developing a network of computing faculty and leaders dedicated to providing this support. In addition to Rice University, ELA originally included Boston University, the University of California, Berkeley, and the University of Texas, Austin. With this proposal, the ELA will add the University of Georgia, Clemson University, Tufts University, and Stony Brook University as new lead institutions, along with several new partners including MentorNet, the Computer Science Teachers Association (CSTA), and the New England Computer Science Chairs (NECSC). The ELA uses three intersecting models of engagement, including the National, Regional and Local Models. The National Model seeks out and supports individuals across the country, building a virtual (and sometimes in-person) network; the Regional Model builds a support network among multiple universities within a region; and the Local Model builds a local support community within a single university. These three models serve to connect ELA students with each other and with tailored opportunities including internships, mentoring, conference participation, and summer research programs. The proposed extension will strengthen the three models at their current sites and test their scalability and transfer to the new lead institutions. It will also add programs developed under BPC demonstration projects to the suite of ELA offerings and test their scalability.

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).

The objective of this research is to develop bio-inspired algorithms for recognizing human movements and movement styles in videos of human activities. The approach is based on a new representation of static three-dimensional (3D) shape structure in the ventral visual pathway consisting of configurations of 3D structural fragments. This project uses neural experiments to validate an analogous four-dimensional (4D) representation of moving 3D shapes based on 4D (space and time) structure-in-motion (SiM) fragments. These SiM fragment models are used to develop algorithms for automatically extracting SiM fragments from videos of human activities. Hybrid system identification and clustering techniques are used to learn a dictionary of human movements used for recognition. This dictionary, in turn, influences the design of the neural experiments. The algorithms are evaluated using a real-time tele-immersion system.

The intellectual merit of this project is to substantially reduce the gap between human and machine perception of human movements through the interaction between computational and experimental analyses from neuroscience, computer vision, machine learning, and dynamical systems. This project also has the potential to generate advances in machine learning and dynamical systems, by extending classification, clustering and system identification methods to time-series data generated by collections of hybrid dynamical models.

Potential broader impacts include applications in surveillance, security, assisted home living, infant care, tele-immersion, and athlete motion analysis. The project also sponsors a competition where small robots are constructed by middle and high-school students from Baltimore, and the EL Alliance program for retaining underrepresented minorities at graduate schools.

This research project addresses the important problem of improving and maintaining peoples' healthy lifestyles by inventing smart technology based on fundamental scientific principles. The approaches are economically feasible and socially compelling approaches, with a focus on maintaining the health and independence of older adults in a home environment. The project uses a mix of networking and monitoring technologies to connect older adults with a remote health coach (real person facilitated by a semi-automated program) and remote family members. One of the key design issues is how best to preserve privacy and enable the participants to control the distribution and sharing of their data. The intervention is designed to provide coordinated and continuous health management.

The research for this project uses the integration of data from a variety of sensors in the home, yielding information for activity monitoring, sleep monitoring, gait and movement analysis, socialization measures, as well as a variety of cognitive metrics derived from computer interactions with adaptive games. Rigorous computational engineering models of the cognitive and physical functions of the patient, as well as context and environment, are used to infer patient state and provide feedback for the patient and the remote health coach. The modeling techniques include Partially Observable Markov Process and Hybrid Control Modes. User models that incorporate behavior change principles are then used to drive algorithms to optimize automated feedback and recommendations that serve as prompts for a health coach managing a large number of patients. These approaches to remote health management are evaluated by leveraging an existing prototype platform with the capability of collecting data from the homes of elderly participants.

This project focuses on the formal design of semi-autonomous automotive Cyber Physical Systems (CPS). Rather than disconnecting the driver from the vehicle, the goal is to obtain a vehicle where the degree of autonomy is continuously changed in real-time as a function of certified uncertainty ranges for driver behavior and environment reconstruction. The highly integrated research plan will advance the science and engineering for CPS by developing methods for (1) reconstructing 3D scenes which incorporate high-level topological and low-level metric information, (2) extracting driver behavioral models from large datasets using geometry, reasoning and inferences, (3) designing provably-safe control schemes which trade-off real-time feasibility and conservatism by using the evidence collected during actual driving.

Assisting humans in controlling complex and safety-critical systems is a global challenge. In order to improve the safety of human-operated CPS we need to provide guarantees in the reconstruction of the environment where the humans and the CPS operate, and to develop control systems that use predictive cognitive models of the human when interacting with the CPS. A successful and integrated research in both areas will impact not only the automotive sector but many other human-operated systems. These include telesurgery, homeland security, assisted rehabilitation, power networks, environmental monitoring, and all transportation CPS. Graduate, undergraduate and underrepresented engineering students will benefit through classroom instruction, involvement in the research and a continuous interaction with industrial partners who are leaders in the field of assisted driving.

The goal of this project is to vastly improve the way musculosketal modeling is performed and utilized by creating a data driven model that builds on the product of exponentials formulation for joints. By using parameter fitting, a kinematic chain of joints can be fitted the individual rather than scaling a generic template which does not take into account the large diversity in body shapes and sizes. The team will look at a structured, yet data driven approach to modeling a person, making it possible to compare joint ranges and muscular limitations both contralaterally, as well as against their peers. It is also possible to compare a patient and against their past history allowing for better understanding and diagnosis within a specified patient groups (scoliosis, the elderly, hip replacement recovery) as well as with the general population. The PI proposes a hybrid optimal control approach for determining muscular activation based on segmenting different dynamical modes. These modes can take into account changes in mass or geometric constraints such as assistive devices (e.g. crutches walkers or exoskeletons). They propose to apply these methods of musculoskeletal modeling to the upper limbs in the elderly group who experiences muscle weakness, joint damage and may have artificial prostheses.

The tools developed under this proposal can be used to analyze any number of biological creatures by modeling their joints in a similar manner. It expands to a wider robotic community where robotic kinematic chains can be directly compared to biological chains. This can be used for teleoperation of robotic devices where particular joints can be mapped between each other. It also adds to the tools that can be used to design assistive exoskeletons and prosthetic devices, as it allows biological and mechanical joints to be modeled in together- potentially improving the methods of controlling these devices. The project team consists of a PI from Computer Science and two consultants, one from Mechanical Engineering (UC Berkeley) and a MD from UCDMC who have extensive experience in workspace assessment techniques, human modeling, human-machine interaction, and control. Research findings will also be outreached to K-12 students and their parents though various official events at the University. The Center for Information Technology Research in the Interest of Society (CITRIS) at UC Berkeley provides a unique environment and opportunity for the investigators to interact and share research findings with other researchers, students, and broader public.

This proposal addresses modeling and control aspects of human-robot interaction by considering constraints imposed by an individual's physiology. The project is motivated by increasing demand for automation in unstructured environments that require high-level cognitive processing and complex decision-making which cannot yet be fully automated. By taking human-centric approach, data-driven musculoskeletal models are incorporated into the robot interaction model to account for differences of individuals.

Each cooperative activity is divided into action primitives requiring different control strategies while estimating human intent from various sensors. The framework is based on theory of hybrid systems that provides provable safety and stability criteria. The outcome of this research will facilitate methodology for safer and more reliable human-robot interaction and advance state-of-the-art in human movement analysis and control theory. The broader impacts of this research will be realized through new insights into understanding of human intent and haptic cooperation applicable to general human-machine interaction. With increasing interest in service robotics safe and reliable interaction will be the key to successful introduction of robots in human-occupied environments. The potential economic impact of robots engaged in services and manufacturing alongside humans are significant due to increased productivity and reduced costs. Another emerging area is rehabilitation and assistive robotics. The developed data-driven musculoskeletal models will also be applicable to quantification of physical impairments and estimation of muscular stress in healthcare and ergonomics. This interdisciplinary research provides excellent opportunities for undergraduate and graduate students to be engaged in analytical challenges, laboratory demonstrations of theoretical results, and experimental evaluations.

This NSF Cyber-Physical Systems (CPS) Frontier project "Verified Human Interfaces, Control, and Learning for Semi-Autonomous Systems (VeHICaL)" is developing the foundations of verified co-design of interfaces and control for human cyber-physical systems (h-CPS) --- cyber-physical systems that operate in concert with human operators. VeHICaL aims to bring a formal approach to designing both interfaces and control for h-CPS, with provable guarantees.

The VeHICaL project is grounded in a novel problem formulation that elucidates the unique requirements on h-CPS including not only traditional correctness properties on autonomous controllers but also quantitative requirements on the logic governing switching or sharing of control between human operator and autonomous controller, the user interface, privacy properties, etc. The project is making contributions along four thrusts: (1) formalisms for modeling h-CPS; (2) computational techniques for learning, verification, and control of h-CPS; (3) design and validation of sensor and human-machine interfaces, and (4) empirical evaluation in the domain of semi-autonomous vehicles. The VeHICaL approach is bringing a conceptual shift of focus away from separately addressing the design of control systems and human-machine interaction and towards the joint co-design of human interfaces and control using common modeling formalisms and requirements on the entire system. This co-design approach is making novel intellectual contributions to the areas of formal methods, control theory, sensing and perception, cognitive science, and human-machine interfaces.

Cyber-physical systems deployed in societal-scale applications almost always interact with humans. The foundational work being pursued in the VeHICaL project is being validated in two application domains: semi-autonomous ground vehicles that interact with human drivers, and semi-autonomous aerial vehicles (drones) that interact with human operators. A principled approach to h-CPS design --- one that obtains provable guarantees on system behavior with humans in the loop --- can have an enormous positive impact on the emerging national ``smart'' infrastructure. In addition, this project is pursuing a substantial educational and outreach program including: (i) integrating research into undergraduate and graduate coursework, especially capstone projects; (ii) extensive online course content leveraging existing work by the PIs; (iii) a strong undergraduate research program, and (iv) outreach and summer programs for school children with a focus on reaching under-represented groups.