Yue Yin, Ph.D. - US grants

Although there is substantial evidence about the positive effects of feedback, a major lack is systematic knowledge about what kinds of feedback are needed by different students, in what forms, and for which types of learning tasks. This project will define and synthesize effective feedback strategies that can be linked to specific features of daily classroom assessment practices.
Instead of providing general evidence about effectiveness, the project will summarize, integrate, and interpret a range of feedback studies that are conceptually comparable based on particular mediators and moderators involved in feedback practices. These mediators and moderators would include, for example, difficulty of instructional tasks, cognitive demands tapped by tasks (e.g., declarative, procedural, schematic, or strategic knowledge), or student characteristics (e.g., low or high achievers, low or high motivation.

The project will be guided by three research questions: (1) What constitutes the range of feedback strategies that have been studied in science and mathematics education?; (2) what constitutes the critical mediators and moderators of feedback practices (e.g., learning goals and student characteristics)in which such feedback strategies have been implemented?; and (3) what are the critical characteristics of feedback strategies that empirically have proved to have a positive impact on student learning?

The project will develop a framework, including a conceptual strand that will conceptualize feedback practice considering intrinsic and contextual dimensions and a methodological strand that will be used to describe and evaluate the feedback studies and findings to be synthesized. The proposed framework will provide a language that can be shared within and across multiple forms of research in various disciplines to portray feedback practices. The framework and research questions will also lead to criteria for inclusion and exclusion to be used to identify eligible articles and reports for the synthesis. Based on the framework, we will develop and apply a two-level coding system to specify and evaluate the findings from each eligible study. Finally, the coded information will be summarized to identify patterns and trends across the studies, which in turn will be used for a narrative review and a quantitative summary in the form of a meta-analysis.

Intellectual Merit: The project will result in a substantially more complete and detailed theoretical framework than what currently exists for characterizing effective feedback strategies in science education. This understanding will allow for (1) refinement of a theoretical framework that explains the feedback process and its quality, and (2) concrete, easy-to-apply recommendations for science teachers to effectively and formatively use assessments in their daily work.

Broader Impacts: The study will help professional developers and teachers understand what is required to effectively implement feedback practices that have a positive impact on student learning. The research will also produce clearer criteria for improving, evaluating, and monitoring teachers' feedback practices in daily teaching as well as for designing professional development for pre- and in-service teachers of science and mathematics. Results from the project will also fill in the research gap, contribute to the literature on formative assessment, and help define research and development agendas. The project will create a toolkit for practitioners as well as articles for publication.

Computing has become an integral part of everyday practice within modern fields of science, technology, engineering, and math (STEM). As a result, the STEM+Computing Partnerships (STEM+C) program seeks to advance new approaches to, and evidence-based understanding of, the integration of computing in STEM teaching and learning. 'Making' is an emerging movement in which kids use technological tools to create hardware and software that solves personally relevant problems, using things like programmable circuit boards, arts and craft materials, and computers. Building on a broad set of maker spaces in Chicago, the Assessing Computational Thinking in Making Activities (ACTMA) project will develop curricula and assessments to see how kids can learn physics and computation through making. Experts in educational measurement, in broadening STEM participation, and in instructing makers will work together with kids to design materials that are culturally responsive to girls, Hispanic-Americans, and African-Americans. By building and testing these curricula, the project will further our understanding of what kids are actually learning in informal maker spaces; how better to link maker activities to computer science and physics education; and will help provide actionable assessments that parents, informal educators, and researchers can use to help improve learning environments like maker spaces.

The intellectual merit of this project lies in two thrusts; first, the project will help to better understand existing student practices around physics and computation, in a culturally diverse, mature set of maker-oriented informal learning environments. Qualitative analysis based on Weintrop's computational thinking framework and Lewins' generalized qualitative analysis codes will identify target curricular goals through Charmaz's grounded theory process. The second thrust will provide both iterated curricular designs and learning assessments that are culturally responsive, based on the practices and learning goals identified in the first thrust. The curriculum will be iteratively designed using youth informants to ensure cultural responsiveness and validity. The assessments will also be iteratively developed, relying on practitioner observations which can target 'moments of notice' in which an educator-friendly rubric would allow instructors to observe evidence of learning (both in the subject domain knowledge, and in computational thinking practices, aligned with emerging standards from ISTE and CSTA). An external evaluator will gauge the use and cultural sensitivity of the curriculum and assessments. The broader impacts of the project will relate to the use of the curriculum and assessments in the Center for College Access and Success partners around Chicago, and potentially its uptake in makerspaces across the country. An additional potential broader impact is increasing our ability to design and assess the success of makerspaces for STEM learning, and even broadening participation in STEM by providing learning activities that are specifically designed to appeal to a number of underrepresented groups (girls, racial minorities, and ethnic minorities.)

The Developing Organizational Capacity to Improve K-8 Mathematics Teaching and Learning is a 4-year Implementation and Improvement project. The project will develop and test a leadership model to improve K-8 mathematics teaching and learning by involving stakeholders across the K-8 spectrum. The project will support teachers, teacher leaders, and administrators in collectively identifying and addressing problems of practice in the teaching and learning of mathematics, and in turn develop plans to improve school and district organizational capacities to support stronger mathematics teaching. At the heart of the project is the Elementary Mathematics Leadership (EML) model, which is designed to improve stakeholder understandings of effective math teaching practices. The EML model involves collaboratively identifying classroom-based problems of practice with school and district personnel, designing and implementing professional development aligned with the problems of practice, and iterating those cycles of development, implementation, and revision to assess the model's effectiveness.

The EML model operates at the teacher, school, and district level using a design-based implementation research approach. At the district level, leadership teams in conjunction with researchers will identify problems of practice for which work on those problems will lead to a more coherent mathematics instruction in the district. Following this, professional development and coaching at the teacher level will be designed and implemented to target the problem of practice, with a focus on big ideas within the Common Core State Standards for Mathematics. This phase of the model also includes professional development aimed at school leaders and district administrators to strengthen organizational capacity to support and lead change related to the problem of practice. The final phase of the model calls on researchers, district, and school personnel to engage in an annual redesign of the intervention, making use of data gathered during the school year and evidence about what is happening in classrooms. This design acknowledges the broader policy context in which schools and districts operate as they work towards instructional change. To evaluate the effectiveness of the overall EML model, the project will collect a wide variety of data, including student achievement outcomes using standardized tests; assessments of teachers' mathematical knowledge, instructional practices, and efficacy measures; and measures of leader, administrator, and organizational capacities to support high-quality mathematics instruction. Four districts will be recruited to participate, enacting the model in pairs with a staggered start for one pair of districts to be able to measure treatment effects, using a variation of a switching replications design. Classroom practice and teacher outcomes will be assessed using a variety of MKT assessments, the Mathematical Quality of Instruction (MQI), and the Instructional Quality Assessment (IQA). School level outcomes will be collected via a leadership assessment and interview data, and district level outcomes will be assessed through the use of interview and documentary data. Analysis will include a statistical analysis of the EML model using hierarchical linear modeling, MANOVA/ANOVA, and regression as appropriate at the level of students and teachers, and qualitative analysis and descriptive statistics will be used at the school and district level due to small sample size.

The Discovery Research preK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

This project will study the utility of a machine learning-based assessment system for supporting middle school science teachers in making instructional decisions based on automatically generated student reports (AutoRs). The assessments target three-dimensional (3D) science learning by requiring students to integrate scientific practices, crosscutting concepts, and disciplinary core ideas to make sense of phenomena or solve complex problems. Led by collaborators from University of Georgia, Michigan State University, University of Illinois at Chicago, and WestEd, the project team will develop computer scoring algorithms, a suite of AutoRs, and an array of pedagogical content knowledge supports (PCKSs). These products will assist middle school science teachers in the use of 3D assessments, making informative instructional changes, and improve students’ 3D learning. The project will generate knowledge about teachers’ uses of 3D assessments and examine the potential of automatically scored 3D assessments.

The project will achieve the research goals using a mixed-methods design in three phases. Phase I: Develop AutoRs. Machine scoring models for the 3D assessment tasks will be developed using existing data. To support teachers’ interpretation and use of automatic scores, the project team will develop AutoRs and examine how teachers make use of these initial reports. Based on observations and feedback from teachers, AutoRs will be refined using an iterative procedure so that teachers can use them with more efficiency and productivity. Phase II: Develop and test PCKSs. Findings from Phase I, the literature, and interviews with experienced teachers will be employed to develop PCKSs. The project will provide professional learning with teachers on how to use the AutoRs and PCKSs. The project will research how teachers use AutoRs and PCKSs to make instructional decisions. The findings will be used to refine the PCKSs. Phase III: Classroom implementation. In this phase a study will be conducted with a new group of teachers to explore the effectiveness and usability of AutoRs and PCKSs in terms of supporting teachers’ instructional decisions and students’ 3D learning. This project will create knowledge about and formulate a theory of how teachers interpret and attend to students’ performance on 3D assessments, providing critical information on how to support teachers’ responsive instructional decision making. The collaborative team will widely disseminate various products, such as 3D assessment scoring algorithms, AutoRs, PCKSs, and the corresponding professional development programs, and publications to facilitate 3D instruction and learning.


The Discovery Research preK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.