Aaron Steinfeld - US grants

PUBLIC ABSTRACT

Proposal Number: CBET-0742350
Principal Investigator: Hansen, Jeffery
Affiliation: Carnegie-Mellon University
Proposal Title: Cyberinfrastructure for Human-Robot Interaction Research

This project addresses weaknesses in the field of human-robot interaction (HRI) by introducing an integrated cyberinfrastructure (CI) designed to facilitate research collaboration. Existing CI technologies and application domain code are available, but currently lack the tight integration and key features needed by the community. This Engineering Virtual Organization (EVO) project attempts to close the gap between what is currently available and the CI that is needed to enhance collaboration and data sharing to advance HRI research.

Early in the development of many research fields, researchers often utilize incomparable metrics due the lack of formal or informal standards. But, such common metrics are desirable as they can lead to common evaluation concepts and enhance clarity in the field's literature. Without pressure or facilitation, common metrics develop slowly, thus impeding research advances. Also, fields that have a moderate cost of entry for data collection tend not to develop collaborative CIs and many researchers are placed at a disadvantage. This project seeks to improve and integrate (i) high dimensional data analysis and visualization tools, (ii) off-the-shelf collaboration and data storage tools, and (iii) an existing simulation tool, USARSim, to establish a cyberinfrastructure for documentation of evaluation metrics, data collection, data sharing, and cross-study comparisons. The project is expected to develop and deploy a prototype CI to instantiate and anchor a collaborative research virtual organization for the field of human-robot interaction. The metrics and CI will be disseminated through an outreach effort involving metric and code dissemination, a data repository, and regular academic dissemination avenues. Learning opportunities will be provided to graduate and undergraduate students and the investigators will promote the use of the CI in related educational activities. This project has the potential to have direct value towards system design, validation, and evaluation of implementations where humans and robots interact. Also, the intent is that the collaboration and data repository components will be reusable by other virtual organizations, thus leveraging the work in this project to aid other researchers, businesses, and fields.

It is often assumed that the use of robots to help people execute tasks will result in better performance than if the person or robot were operating alone. However, research in automated systems suggests that the performance of a human-machine system depends on the extent to which the person trusts the machine and the extent to which this trust (or distrust) is justified. As robots are being developed to aid people with complex tasks, it is critical not only that we build systems which people can trust, but that these systems also foster an appropriate level of trust based on the capabilities of the systems. A user who does not have an appropriate level of trust in the robot may misuse or abuse the robot's autonomous capabilities or expose people to danger. This project proposes to develop quantitative metrics to measure a user's trust in a robot as well as a model to estimate the user's level of trust in real time. Using this information, the robot will be able to adjust its interaction accordingly.

Promoting appropriate levels of trust will be particularly beneficial in safety-critical domains such as urban search and rescue and assistive robotics, in which users risk harm to themselves, the robot, or the environment if users do not trust the robot enough to rely on its autonomous capabilities. The research has the potential for a large impact on the field of human-robot interaction as few studies have explicitly examined issues involving trust of robots. Being able to model trust and foster appropriate levels of trust will result in more effective use of robotic automation, safer interactions, and better task performance.

Despite the significant advances in robotics research and development over the years, there are still no pervasive intelligent mobile robots coexisting with humans in daily environments. Among the many possible reasons as to why this is the case, this project addresses the challenge of an effective concrete interaction of mobile robots with humans, focusing on tasks which enable joint human and robot performance and require spatial interaction. The PI's vision is that project outcomes will make it possible to have multiple robots in, say, an office building available for different navigational and informational tasks, including accompanying daylong visitors through their schedule of meetings, giving tours to occasional visitors, fetching objects for and taking them to people in offices, and delivering the daily mail. To achieve this goal, she plans to transform the state of the art in robot technology for social service robotics, by introducing a novel symbiotic human-robot and robot-robot interaction paradigm that allows robots to help and be helped by humans and each other. A robot will ask humans for assistance based on self awareness of its own limitations and a utility analysis of the estimated cost and benefits of the assistance. The PI and her team will develop and evaluate a robot platform-independent and building-independent problem environment representation, along with algorithms for incremental map learning, localization and navigation, and asynchronous (multi-robot) task partitioning and planning under uncertainty with a utility analysis that includes human availability for robot helping. They will explore effective spatial interaction between mobile robots in spaces with humans, utilizing social conventions, so that people are not just obstacles from the robot's perspective. The robot science and development research will be seamlessly integrated with educational and outreach activities, as well as with principled evaluation which will include fielding a team of robots in campus buildings.

Broader Impacts: Aside from dramatically advancing the state of the art in robot technology, enabling multiple mobile robots to be part of the workspace of an office building environment will have significant educational impact relating both to robot technology and interaction with robots. Continuous, openly available robot presence in the computer science and robotics research spaces will change the nature of the relationship between researchers and their classroom research projects, by triggering synergistic collaborations and new, higher-risk experiments with lower setup cost. C Campus outreach tours will be transformed from a narrow view of the future of technology in laboratory settings to a sweeping exposure to the reality and implications of humans and robots coexisting throughout the built environment, significantly broadening inquiry and discussion about the role of interactive technology in our lives. Disseminated curricula incorporating low-cost mobile robots in the secondary school classroom will lift the robot-classroom relationship from one of build kits for very low-capability robots to one of high-level interaction design, industrial design, and discussions of human-robot relationships.

PI: Dias, M. Bernadine; Steinfeld, Aaron
Proposal Number: 1317989

Summary: As robotics technology evolves to a stage where co-robots become a reality, we need to ensure they are equally capable of interacting with humans with disabilities. The proposed work addresses this challenge by exploring meaningful human-robot interaction in the context of assistive robots for blind travelers. For people with disabilities independent transportation remains a major barrier to employment and quality of life. Furthermore, emergency situations necessitating evacuation is one of the greatest fears they face. The key question we seek to answer in the proposed work is: what role can co-robots play in empowering people with disabilities to safely travel to and navigate unfamiliar environments? We hypothesize that co-robots can enhance the safety and independence of these travelers by assisting them to navigate unfamiliar urban environments effectively and providing support when evacuating. We begin our proposed work with a needs assessment to understand the preferences and challenges of blind travelers. The ultimate objective is to enhance the safety and independence of blind travelers.

Intellectual Merit: The proposed work explores three principal research areas applied to three scenarios relevant to assistive robots for blind travelers: (1) information exchange and object manipulation, (2) assistive localization, and (3) urban navigation and emergency building evacuation. The research areas we plan to explore are accessible interfaces, assistive interaction modalities, and effective cooperation mechanisms. Envisioned scenarios include robots assisting humans to localize within necessary resolution and context using a combination of perception, robot localization, and crowdsourcing, robots assisting humans to retrieve lost or fallen objects or locate objects or people of interest, robots assisting humans to interact with other aspects of the environment such as reading notices, and robots assisting humans during emergency evacuation of buildings. We will also explore means of these travelers "teaching" the robots to do tasks of interest to them. The robots will have to reason intelligently about task allocation among themselves and coordinating with humans when needed. Overall, the proposed research will significantly advance the knowledge of how assistive robots can meaningfully and effectively interact with travelers with disabilities. The uniqueness of the proposed research is captured in the accessibility of the interfaces, the richness of the interaction modalities, and the flexibility and range of the cooperation mechanisms. The combination of these contributions will significantly advance the state of the art in assistive technology as well as human-robot interaction.

Broader Impact: The team has a strong commitment to undergraduate research experience. Over 75% of the students mentored by the Principal Investigator and Co-Principal Investigator have been women, minorities, or people with disabilities. This commitment extends to the team's instructional activities. Team members regularly incorporate research findings into class presentations, guest lectures, and seminars. The team is also committed to community outreach. Both Dias and Steinfeld regularly speak to non-academics and will include aspects of this project in such talks. A final educational outcome will be several planned workshops at our partner organizations and the outcomes from the proposed work are expected to impact operations and methodologies used at these organizations. The assembled team of researchers and partner organizations further enhances the broader impact of this proposal. Principal Investigator Dias is one of the very few female robotics faculty members at the university and in the discipline. She is engaged in many mentoring and leadership activities to encourage and sustain the participation of women in computing and to address the needs of technologically underserved communities. The proposed team of undergraduate students, a graduate student, and a postdoctoral research assistant will also gain significant mentoring and education through their participation in this research. Industry interaction extends beyond regular contact due to faculty involvement in high profile centers. The research and evaluation program is specifically geared towards people with disabilities. Therefore, the contributions of the proposed work will make significant advancements towards realizing the vision of safe and independent travel for people with disabilities. The results of the proposed work will be disseminated broadly through a variety of avenues and all outcomes of the research will be made available in accessible formats to the community partners and their networks.

In this exploratory research, the PIs will seek to advance the state of the science on how best to convey a robot's imminent failure to a human (whether an operator, supervisor, or bystander), in a manner that could allow the human to intervene as effectively as possible to prevent the failure. This project has the potential to dramatically increase the safety of humans in and around autonomous robots and vehicles. Specific goals are to discover design principles for robot systems with respect to conveying failure, and to identify methods for expressing failure so that humans react appropriately. The research will focus on three use cases: remote operation, co-located operation, and bystander interaction. To these ends, the team will utilize a variety of robots in order to support different applications and movement scales. Robots available to the team include small and mid-size unmanned ground vehicles, human-scale torso robots, a robot wheelchair, a telepresence robot, and an autonomous Jeep. Project outcomes will impact the field of human-robot interaction and the future use of robots in many application domains, particularly those of mobile and manipulation robots, including autonomous vehicles, factory robots, and assistive technology, by enhancing productivity and task performance, increasing personal safety for those who work in hazardous occupations, and improving the lives of persons with disabilities.

The PIs' core research questions are informed by their substantial prior work with task-oriented robots. Based on that experience and other studies, they argue that the following three main factors strongly influence user actions during robot failure: perceived risk (e.g., a robot that crashes frequently is generally perceived as a high risk robot), perceived severity (e.g., the failure of a small robot made of soft materials is generally perceived as less severe than that of a full body humanoid robot), and role (e.g., is the user an operators or a bystander). Unexplored research questions about the manner in which these factors impact failure include. How do these factors, both independently and in combination, influence HRI during robot failures? How do humans utilize these factors during robot failure, and does this utilization have high variability or are humans very consistent? These factors will be used as independent variables during studies which will advance knowledge in three core areas: formulation and validation of generalizable quantitative and qualitative metrics for measuring a person's response to an imminent failure in a robot system; discovery of appropriate ways to communicate failure states to humans; and initial development of common design guidelines for handling failures. The primary goal is to make it easier for humans to rapidly understand failure events and to act or assist appropriately in a timely manner. The PIs are specifically focused on the human-robot interaction aspect of robot failures. As such, they will track literature and research on diagnosing failures, but will not develop new systems or concepts for this step. Instead, the team will seek appropriate and effective ways to convey failures to humans, appropriate human responses during failures, and appropriate failure states when human action is not possible or is insufficient.

This award supports research on the issue of intelligence re-embodiment in robots. The fundamental question to be addressed is whether robots should be designed so that different synthetic intelligences can take over. A smart phone can be repurposed to serve different users by swapping chips. Should robots with much higher functionality be designed so that they operate in a suitably similar way? The goal of this project is to answer this question along with a number of related questions including the following. Should robots be designed so different intelligences can take over? Where does the locus of intelligence sit? How does the user understand where it is and who or what is in control? How does re-embodiment impact issues of privacy? To answer these questions, this research effort will adopt a mixed-methods approach including surveys, fieldwork, simulations, and on-site testing of a robot operating system module. The findings of this work are expected to have direct value to robot developers and other researchers. Other impacts include interdisciplinary training of PhD students, and creating research opportunities for undergraduates. Team members plan to use the results of this research to enrich courses on human-robot interactions, and in outreach activities to engage non-academic audiences.

This research project uses a variety of methods to study the issue of intelligence re-embodiment in robots. Team members will conduct online surveys to systematically assess factors related to re-embodiment. They will engage in fieldwork to investigate current practices of people working and living in environments with several intelligent systems. These findings will in turn be used to inform a user enactment study that will create simulations of real-world contexts and test participants' reactions to different robot behaviors. Team members will also extract and evaluate a set of generalized interaction principles using a second online study. In addition, they will develop a robot operating system (ROS) module to enable re-embodiment of robots by a cloud-based intelligence; the module will be used to assess reactions of business owners and other stakeholders interacting with these systems. The results of these research components will advance knowledge on how people understand re-embodiment. They will serve to map out a set of design recommendations, design features, and the corresponding interaction patterns that do not yet exist around robot behavior in different contexts. They will also lead to the production of software components that support re-embodiment by remote intelligences. More generally, they will help achieve a vision of fully collaborative, ubiquitous, interconnected co-robot systems.

This project seeks to provide robots with the social intelligence to be aware of the mutual
dependency between their movements and the movements of humans around them. To this
end, the work will focus on (1) improving the way robots reason about spatial behavior, and (2)
developing navigation methods that lead to understandable and appropriate motion patterns in
social environments. This project will build upon prior work in robot perception and social
behavior in crowds and groups. This work will impact the future use of robots in many application domains,
especially for those where people untrained in robotics are present (e.g., delivery robots, guide robots, etc.).
Almost all robots that move near people will need to behave appropriately, so it is necessary to discover
socially intelligent navigation techniques, thereby increasing human acceptance and market
success. The team will also continue established and successful efforts in fostering diversity,
integrating education with research, disseminating new knowledge to the general public,
industry stakeholders, and other researchers.

Prior work has identified the importance of human-aware navigation, and has developed
methods to incorporate the social norms that govern human physical space into aspects of robot
path planning. Building on this foundational work, the team will address three main social
intelligence tasks: (1) enabling robots to reason jointly about nearby human spatial behavior and
their own, (2) enabling robots to communicate their intentions as they navigate so that their
motion is understandable by nearby humans, and (3) giving robots the ability to decide when it is
acceptable to violate pre-established social conventions. Research in these areas is
incomplete since most efforts do not include awareness or reasoning about mutual dependency.
This makes it difficult for a robot to reason intelligently on how to alter crowd motions in a
socially appropriate manner. Methods discovered by the team will also support the case where
multiple robots must mix with multiple humans.