Libby M. Hemphill, Ph.D. - US grants

This project reduces the complexity in protecting research data by developing and piloting an open badge research credential system (OBRCS). Open badges are visual tokens that signal achievement, affiliation, authorization, or another trust relationship and are shareable across the web. The challenges of managing and protecting restricted data mean that data providers are often wary of sharing sensitive data or that data ends up in the wrong hands, and potential gains to society and science from using those data go unrealized. OBRCS allows researchers to present their evolving credentials openly and to record their achievements and credentials publicly and enables more collaboration, facilitates data re-use, and supports replication efforts. OBRCS benefits the scientific community by ensuring the integrity, resilience, and reliability of research data.

Combining and analyzing collections of data enable scientific breakthroughs. Efficient, secure data sharing and reuse facilitates collaboration and replication, leading to better science. However, managing different access policies, authenticating, and authorizing access to sensitive data is a challenge faced by all data management organizations. Unauthorized access threatens the integrity of data and the privacy of study participants and these threats can impact the conclusions researchers draw. The project achieves these goals through three main activities: (a) develops an open badge system for managing researcher credentials, (b) articulates levels of data sensitivity and risk that indicate criteria for access, and (c) identifies the right balance between openness and privacy for data users in a restricted data access system.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Access to original research data supports innovative, interdisciplinary, and integrative research, and enables replication and review of prior work. Consequently, a growing number of funding agencies, journal publishers, and scientific societies now require that original research data must be shared and archived promptly after its collection or publication. However, there are still many unanswered questions about the best way to share and archive research data. For instance: how can data repositories best allocate their limited resources for different aspects of data archiving and processing? What is the most effective way of making data usable by the broadest audience? What data sharing policies most effectively achieve stakeholders? transparency and innovation goals? This project answers these questions by studying the impact of different "curatorial actions" (e.g., standardizing variables, improving documentation) on the reuse of data archived by the Inter-university Consortium for Political and Social Research (ICPSR). As one of the largest social science archives in the world and a leader in digital data curation practice, ICPSR is well-suited as a site for this project. ICPSR is also well-positioned to provide funding agencies and policy makers recommendations for data sharing policies that articulate the metrics needed in evaluating the appropriateness of data sharing and curation plans and their associated costs. This project achieves broader impacts by (1) recommending evidence-based data sharing policies to funders, repository staff,, and researchers and (2) improving research data curation practices.

To determine the impact of various curatorial activities on data reuse, the project first defines the different kinds of "curatorial actions" and "impact," and then explains the relationships among actions and impact. To identify curatorial actions and other features of datasets and ICPSR services that influence reuse, the project examines ICPSR's legacy curation logs and use records (such as downloads and citations). Curation logs contain data about specific data transformations or preservation steps. By connecting curation logs to data usage records, the actions are associated with higher rates of reuse or access will be identified. The project examines the utility of two measures of impact--secondary impact and diversity--by comparing use logs to the ICPSR Bibliography of Data-Related Literature. The ICPSR Bibliography links over 80,000 research publications to the ICPSR data on which they are based. "Secondary impact" is a measure of how many times the reuse publications have been cited and is constructed by gathering citation data for all items in the bibliography that are not the original PI's publications. "Diversity" measures the breadth of disciplines that use the data and can similarly be constructed from the bibliography. The project employs multivariate regression analysis and structural equation modeling to determine the relationships among curatorial actions, metadata, the dataset itself, ICPSR services, and reuse and impact. This analysis enables the development of cost models and metrics that allow repository managers to evaluate the return on investment of specific curatorial actions. The project will use these models to inform evidence-based data sharing and archiving policies.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Around the world, users of social media platforms generate millions of comments, videos, and photos per day. Within this content is dangerous material such as child pornography, sex trafficking, and terrorist propaganda. Though platforms leverage algorithmic systems to facilitate detection and removal of problematic content, decisions about whether to remove content, whether it's as benign as an off-topic comment or as dangerous as self-harm or abuse videos, are often made by humans. Companies are hiring moderators by the thousands and tens of thousands work as volunteer moderators. This work involves economic, emotional, and often physical safety risks. With social media content moderation as the focus of work and the content moderators as the workers, this project facilitates the human-technology partnership by designing new technologies to augment moderator performance. The project will improve moderators' quality of life, augment their capabilities, and help society understand how moderation decisions are made and how to support the workers who help keep the internet open and enjoyable. These advances will enable moderation efforts to keep pace with user-generated content and ensure that problematic content does not overwhelm internet users. The project includes outreach and engagement activities with academic, industry, policy-makers, and the public that ensure the project's findings and tools support broad stakeholders impacted by user-generated content and its moderation.

Specifically, the project involves five main research objectives that will be met through qualitative, historical, experimental, and computational research approaches. First, the project will improve understanding of human-in-the-loop decision making practices and mental models of moderation by conducting interviews and observations with moderators across different content domains. Second, it will assess the socioeconomic impact of technology-augmented moderation through industry personnel interviews. Third, the project will test interventions to decrease the emotional toll on human moderators and optimize their performance through a series of experiments utilizing theories of stress alleviation. Fourth, the project will design, develop, and test a suite of cognitive assistance tools for live streaming moderators. These tools will focus on removing easy decisions and helping moderators dynamically manage their emotional and cognitive capabilities. Finally, the project will employ a historical perspective to analyze companies' content moderation policies to inform legal and platform policies.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Social scientists are encouraged to share their data so that their work can be evaluated, replicated, and extended. Open science movements and public funding for science, especially, have increased the number and breadth of datasets that scientists share for reuse and inspection. However, sharing data does not guarantee that it will be found and reused. Search technologies have been enhanced by recommender systems, but they have not been widely applied to research data. A better understanding of how researchers search for existing data is needed in order to design systems to recommend relevant data to researchers. This study will determine if redesigning data search systems to include recommended results can help social scientists discover datasets to reuse effectively. The results of this project will help data archives ensure returns on our national investments in scientific data by increasing data reuse and will promote scientific progress by connecting researchers with relevant data.

The project will a) develop a model of human information behavior that explains how social scientists currently search for data, b) design a prototype data recommender system, and c) evaluate the model and system through field experiments. How data search compares to other information behaviors such as general search is not clear, and this project will explain how data search unfolds. The project also determines whether recommendation systems, popular in fields such as book and movie recommendations, can also work for data and increase their reuse.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.