Jennifer Green - US grants

The purpose of this study is to develop statistical models to create a coherent model of the effects of teacher professional development in mathematics on student learning. The project works closely with two partnerships of the NSF Math and Science Partnership (MSP) program, the Math in the Middle Institute Partnership and NebraskaMATH, and with the Lincoln Public Schools. Utilizing data already collected through these MSP partnerships, the study builds newly developed models to help these and other MSP partnerships and their evaluators interpret student and teacher data in statistically productive and meaningful ways.

Specifically, the project focuses on layered value-added models to estimate teacher effects, and how those models can be adapted for use with messy data, for example when different tests are administered in different grade levels at different times of year. The project is investigating the use of Z-scores, parallel processing, and binning by quantile to address issues arising with available student achievement data. The scope of the project research includes: 1) methods to estimate student achievement trajectories over time; 2) methods for best connecting these trajectories to measures of teaching quality; 3) connecting measures of student and teacher attitudes to each other and to measures of student achievement and teaching quality; and 4) estimating the impact of MSP interventions on all of the above.

MSP projects, school districts and States are grappling with how to evaluate the effects of their professional development programs for teachers on student learning. Few studies have addressed how to use value-added models to analyze achievement data that are not on a single developmental scale, and even fewer have discussed how to use information from multiple instruments in a single year that are on different scales. In addition to this work, the research findings will be disseminated widely, through presentations at national conferences, articles published in peer-reviewed journals, and a dissemination conference targeting MSP project and evaluation personnel. Utilizing value-added models increases the potential to provide evidence of high quality teacher impact in high-need schools. By helping MSP partnerships use the methods developed in this study, the project is building build their capacity to inform the nation of how their MSPs impact teaching and learning.

Value-Added Models (VAMs) in education are increasingly being used to attempt to measure the impact of a teacher on student achievement in a way that takes into account students? and peers? prior performance and demographics. Data that typically inform a VAM rely largely on student achievement data from state testing systems. The RealVAMs project tests a multi-dimensional mechanism to include other measures of student success such as scores from different assessment instruments (SAT, ACT) and categorical responses such as whether or not a student graduated or persisted in a STEM field. The real world model using multi-dimensional data is being tested using a longitudinal database from a large school district. The project is creating an open source accessible program to facilitate the expanded measurement of VAMS across a larger scale.

This project has national significance as the use of VAMs for teacher evaluation is expanding rapidly across the country in schools and districts. A major objection to the use of VAMs in teacher evaluation has been the over-reliance on the use of state testing data. Opponents of the use of VAMs argue that this reliance contributes to a narrowing of curriculum and instruction to only what can be tested on state tests. Not all subjects are tested, so the use of VAMs does not support the evidence that can be used in the evaluation of teachers in non-tested subjects or grade levels. This project informs the educational practice community of schools and districts and provides for more nuanced and richer information about the impact of teachers for policy decision making.

Graduate education in the sciences, technology, engineering and mathematics (STEM) rigorously prepares students to innovate in their fields, yet it often does not include formal training in how to effectively communicate those innovations to others. To ensure we live in a just and vibrant society, it is vital that scientists are able to share their research findings with scientists in other fields, non-science experts, and the public. Using knowledge from the performing arts, this project will implement a novel pilot training program to teach oral communication to STEM graduate students, facilitating a deeper understanding of communication and providing practice in public speaking, improvisational techniques, and reading body language. This National Science Foundation Research Traineeship award in the Innovations in Graduate Education (IGE) track to Montana State University-Bozeman will modernize STEM graduate education by pinpointing aspects of oral communication training that positively impact students' abilities to communicate orally to a wide variety of audiences, and increase the public's understanding of and engagement with science. Three recruited cohorts of eight STEM graduate students will practice and refine their oral communication skills by creating podcasts and speaking at community events in which they explain their research to the public. The team working on this project includes domain experts in physics, biology, mathematics, and engineering, as well as experts in public speaking, acting, education research, and library sciences. This broad expertise will ensure that disciplinarily diverse STEM students are recruited into the program, providing interdisciplinary perspectives and instruction and facilitating the success of this pilot graduate training program.

This research will investigate whether targeted intervention in oral communication skills for STEM graduate students improves their ability to effectively convey their research to a broad audience. The project will use formative, summative, and longitudinal assessments to determine how various portions of the intervention are successful in developing students' oral communication skills in the following realms: avoiding the use of jargon, public speaking skills, and non-verbal delivery skills. Assessment data for year-long longitudinal studies of each cohort will include survey data related to three areas of oral communication, self-reports from students, and interviews with students and their mentors. To assess competencies such as reducing jargon for communicating with the public, this project will create, modify and use rubrics for effective public speaking. Pre- and post-intervention data will be compared among students and cohorts to determine the effects of each aspect of the training. Results and successes will be shared nationwide with other STEM experts at domain-specific conferences as well as at American Association for the Advancement of Sciences National Conferences.

The NSF Research Traineeship (NRT) Program is designed to encourage the development and implementation of bold, new, potentially transformative models for STEM graduate education training. The Innovations in Graduate Education Track is dedicated solely to piloting, testing, and evaluating novel, innovative, and potentially transformative approaches to graduate education.