Qin Zhu, Ph.D. - US grants

Robots may need to carefully decide when and how to reject commands given to them, if the actions required to carry out those commands are not morally permissible. Most previous work on this topic takes a norm-based ethical approach, where a robot would operate under a set of rules describing what states or actions are morally wrong, and use those rules to explain its actions. In contrast, this project explores a role-based perspective, in which the robot reasons about the relationships it holds with others, the roles it plays in those relationships, and whether the actions requested of it are benevolent with respect to those roles and relationships. Specifically, the researchers will develop a framework to allow robots to reason in this way and generate explanations of its actions based on this reasoning. The researchers will then explore how role-based and norm-based command rejections compare in terms of how they affect human-robot teamwork, and design algorithms to allow robots to automatically decide what type of rejection to generate based on their context. These algorithms and explanations will be evaluated in two very different contexts with different types of relationships, roles, and rules: with civilian undergraduates at the Colorado School of Mines, and with Air Force cadets at the US Air Force Academy. This work will not only increase robots' ability to behave ethically and act as good teammates, but will also advance moral philosophy by providing experimental evidence for the relative importance and effectiveness of different tenets of role-based moral philosophy.

More formally, the goals of this research are to investigate context-sensitive tradeoffs between rule-based and role-based responses, and the representations and mechanisms needed to facilitate role-based responses. The research team will do this by identifying metrics to assess response acceptability, quality, and effectiveness; modeling the generation of role-based responses and selection between role-based and rule-based responses; conducting experiments to validate those models and responses; and using the results to articulate novel moral and philosophical arguments. The team will start with exploratory studies at each experimental site contrasting the effectiveness of different command rejection phrasings formed by crossing different Speech-Act Theoretic communication strategies paired with different moral philosophical backgrounds, with respect to (a) field-standard survey measures of trust, likability, mindfulness, workload, and norm strength; (b) qualitative analysis of video data; and (c) statistical linguistic analyses from the multimodal interaction community. The researchers will then develop a framework for robots to generate these responses that will involve formal computational model development of inter-team relations, roles, and actions; machine learning-based modeling of norm violation response strategy selection using features such as task context characterization, role-theoretic proposed action benevolence, and expected responses to the responses; and integration into the DIARC robot cognitive architecture. Based on this work, the team will both advance traditional moral philosophical arguments and develop a novel framework in which moral philosophical arguments are justified based on the effects of computational models on moral psychology. Overall, the project will lead to foundational, interdisciplinary knowledge into norm violation response, consisting of algorithms for selecting between norm violation responses grounded in different ethical theories, guidelines for and insights into the design of morally competent language capable robots, novel computational accounts of role-based robot ethics, and novel empirically informed moral philosophical arguments.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

The NSF Convergence Accelerator supports team-based, multidisciplinary efforts that address challenges of national importance and show potential for deliverables in the near future.

The broader impact/potential benefit of this Convergence Accelerator Phase I project is multifaceted. Crowdsourcing has created a vast and rapidly growing online labor market. However, today's crowdsourcing platforms cannot well support crowdworkers, job requesters, and the healthy growth of this important online labor market due to four major problems: fairness, ethics, efficiency, and trustworthiness. This project is a convergence of the research and development from multiple intellectually distinct disciplines including Computer Science, Economics & Business, and Humanities & Social Sciences. By performing fundamental research with rapid development advances through partnerships with crowdsourcing platform providers, this project will deliver techniques that can be used to create fair, ethical, efficient, and trustworthy crowdsourcing platforms to support American crowdworkers. It will also enable job requesters including researchers, companies, and government or humanitarian aid organizations to receive high-quality and trustworthy task submissions for them to confidently conduct their important studies and make important decisions. This project will actively involve students from underrepresented groups including female and minority students. It will train students on research and on producing high-quality deliverables. It will widely disseminate its results via activities such as publishing research papers and promoting the wide use of the deliverables.

This Convergence Accelerator Phase I project has significant intellectual merit. It addresses the critical interdisciplinary challenges of creating a healthy crowdsourcing labor market that is crucial to the important studies, computations, and decisions of researchers, companies, as well as government and humanitarian aid organizations. This labor market is vast and rapidly growing, but has four major problems intertwined from the fairness, ethics, efficiency, and trustworthiness perspectives in a very complicated manner. This project addresses the four major problems by performing fundamental research with rapid development advances through partnerships with crowdsourcing platform providers. It will (1) design incentive structures based on economic theory to incentivize fairness in crowdsourcing, (2) design research, training, and assessment mechanisms to incorporate ethics into crowdsourcing, (3) design machine learning models to improve the efficiency of crowdworkers, and (4) design machine learning models to securely protect both crowdworkers and job requesters. It will integrate the designed techniques at the client-side into a web browser extension, and at the server-side into some industrial partner's crowdsourcing platform. Overall, it takes a convergence approach to advance the scientific knowledge and understanding of crowdsourcing and its closely related disciplines including economics, business, humanities, social sciences, and computer science.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Responsible Engineering Across Cultures: Investigating the Effects of Culture and Education on Ethical Reasoning and Dispositions of Engineering Students

Engineering is more cross-cultural and international than ever before, resulting in potential disagreements about (in)appropriate courses of action, which can impede engineering work. Despite high rates of international enrollment and an increased focus on global dimensions of engineering in US programs, ethical issues arising from global engineering have been insufficiently addressed. To address these issues, this project will assess the impact of culture and education on ethics among engineering students in North America, Europe, and Asia. Understanding if and how diverse cultural backgrounds and educational experiences affect professional decision-making and collaborations requires empirical investigation, to develop training that addresses the kinds of challenges engineering students, practitioners, programs, and organizations will increasingly encounter in the globalized world. This project will be beneficial for training the next generation of engineers who are competent in working professionally and ethically in the global context and are responsive to the value of diversity in quality and sustainable engineering work.

The goal of this project is to identify educational interventions with the greatest effects on ethical reasoning and dispositions of engineering students, whether these effects differ among cultural and national groups, and if/how to modify these interventions to respond effectively to cultural and national differences. To do so, researchers from Colorado School of Mines, University of Pittsburgh, Delft University of Technology, and Shanghai Jiao Tong University will implement mixed-method, quasi-experimental, longitudinal, and cross-sectional research to: (1) determine the effects of culture and foreign language on the ethical perspectives of first-year engineering students; (2) assess the relative effects of culture and education on these perspectives over four years; (3) use engineering ethics assessment tools across cultures and countries to examine their cross-cultural validity. Findings from this project will be essential to develop educational interventions that effectively respond to the globalized environments of contemporary engineering practice. They will also contribute to the development of more inclusive engineering education, by identifying perspectives potentially marginalized in the reigning paradigms. Finally, this project has implications for the development of responsible research education at the graduate level. Despite the fact graduate student bodies in STEM fields have become increasingly international, limited work has focused on developing culturally responsive ethics curricula for graduate students from diverse backgrounds.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.