Lei Bao - US grants

In real world learning situations, a wide variety of contextual cues become associated with gradually formed common sense knowledge. In contrast, school learning often emphasizes abstract symbolic manipulations, which can lead to students' difficulties in applying their knowledge to real world examples. To help students make intuitive sense of physics, it is necessary to address sensory stimuli that are explicitly or implicitly associated with students' common sense knowledge from experience. To do this, it is important to understand what contextual cues are involved in learning and how they affect the learning process.

For selected topics in introductory physics, we will conduct systematic qualitative studies to identify and investigate specific context cues that affect students' learning; we will develop methods to measure, evaluate and represent the involvement of context cues in different stages of learning; we will also develop physics activities in virtual environments to investigate experimentally how low-level sensory cues such as haptic feedback affect learning and how such cues may be addressed in instruction.

The resulting insights into the interplay between context and learning, and integration of contextual and sensory information into virtual environments will broaden the horizon of active engagement teaching methods and inform the development of teaching technologies.

This project is a partnership with the College of DuPage and Chicago State University to expand and improve on the use of innovative research-based educational techniques in order to maximize knowledge gains for the broadest student base.

In-class interactive engagement techniques, including electronic polling systems, also called clickers or voting machines (VM), are being used increasingly at colleges and universities throughout the world, and are offered by publishing companies as an adjunct to their textbooks. It has been demonstrated that such techniques engage students, but little has been done to develop and assess strategies for their use.

Intellectual Merit: This project is to develop question sequences that fully exploit VM technology and to conduct systematic research to evaluate their effectiveness. This project is based on the theoretical framework that learning is context dependent. Applying this idea in a series of preliminary studies, an effective method for formative use of VM in lectures has been identified. Each concept is incorporated into a sequence of VM questions with different surface features. The emphasis of the proposed research is to develop and assess question sequences using research-based methodologies and then establish a framework, making it easy for teaching faculty to use this new approach. The goal is to improve students' conceptual knowledge of physics by using VM to increase student interactivity and as a means of exposing students to concepts in a variety of contexts.

Outcomes are: (1) improved student learning in a lecture environment; (2) comparison results based on diverse student populations and class sizes; (3) approximately 135 VM single-concept question sequences that are tested and shown to be valid and reliable; (4) additional question sets developed to address specific misconceptions; and (5) a VM user's guide. Through workshops, interactive web sites, a "how to" VM manual, and working with significant numbers of instructors, the project is developing faculty expertise.

Broader Impacts: The project is developing research-based question sets and constructing a database of learning gains based on their usage that will advance discovery and understanding in the field of STEM education while promoting teaching, training, and learning. Collaboration among a community college, a Ph.D.-granting institution, and an undergraduate institution with approximately 96% minority students ensures student diversity. The project assumes a leading role in developing physics education research-based material for the modern electronic lecture classroom, and is disseminating results via non-technical literature and websites in order to reach the broadest educational audience. There are plans for commercial publication.

Physics (13). This project conducts systematic research to obtain convincing evidence about the effectiveness of using virtual experiment (VE) technologies in teaching physics and develops curriculum materials for implementation of VE in the introductory physics labs. In previous research, the PIs have found that using virtual experiments can help students understand difficult physics concepts at a more intuitive level and improve the effectiveness of laboratories on student learning. This project has Intellectual Merit as it brings cutting-edge computing technology to address the teaching laboratory that is a very important educational component of a STEM discipline. It advances technology and creates new strategies in the field of physics education. The Broad Impacts of the research can be considered in several areas. The VE technology itself has many advantages in helping improve students' interests, motivations, and attitudes toward physics and learning physics. Making a new technology affordable can have a significant impact in improving the education environment at a much larger scale. From a more general perspective, physical feedback is a novel component addition to a computer simulation. In medical applications, the attempt is to mimic the real thing. Animators have discovered that mimicking is best replaced by physics when possible and this project has significance in adding physics to the VE environment. The results of this research can also lead to full scale projects that develop VE technologies for a wide range of content areas in physical science.

Assessment/Research (91)
This project is establishing a proof-of-principle for a Database for Assessment of National STEM Education Research (DANSER), a crucial piece of the core infrastructure needed to support the research culture envisioned by national leaders in STEM education. The project is collecting high-quality, research-tested assessment instruments in seven STEM fields; creating the core of a secure, user-friendly environment to store data from these assessments, integrated with data on student, classroom, and institutional characteristics; developing methods for reporting results to teachers and researchers; field-testing the system in the physics and statistics education research communities; and holding cross-disciplinary meetings of STEM teachers and researchers to plan the translation of this proof-of-principle project into STEM-wide implementation. A long-term goal is the development of a culture in STEM education research where multiple disciplines are working efficiently in parallel, sharing tools, ideas, and results. DANSER will form the nucleus of this environment.

The intellectual merit of this proposal is the vast potential usefulness of a field-tested DANSER System. Pilot teaching of innovative STEM courses is occurring in many institutions of higher education. DANSER is facilitating the careful evaluation of these interventions under controlled experimental conditions to establish their impact on student learning or attitudes and their portability to other institutions and populations. This project is improving the ability of education researchers to compare student learning and attitudes in innovative courses to national norms and more readily evaluate how changes are affecting student outcomes. Broad impact is likely once STEM education researchers can draw on a consortium of institutions, already gathering consistent base-line data, to evaluate new ideas and instruments.

DESCRIPTION (provided by applicant): Science Learning and Scientific Reasoning This application addresses broad Challenge Area (12) Science, Technology, Engineering and Mathematics Education (STEM) and specific Challenge Topic, 12-OD-101: Efficacy of educational approaches toward promoting STEM competencies. In STEM education, widely accepted teaching goals include not only the development of solid content knowledge but also the development of general scientific reasoning abilities that will enable students to successfully handle open-ended real-world tasks in their future careers. The goal of this project is to develop a research infrastructure for conducting systematic studies and evaluation of the effectiveness of widely used interactive engagement and inquiry based education interventions on developing students'ability in scientific reasoning. To achieve the research objectives, we will conduct four areas of studies and developments: (1) Refine and complete a standardized assessment instrument on scientific reasoning to produce a valid and easy-to-use assessment tool suitable for large scale quantitative evaluations. (2) Evaluate the effectiveness of several education programs and innovations in their impacts on developing scientific reasoning ability. (3) Develop a large scale national and international quantitative assessment database on scientific reasoning for students from different age groups and backgrounds. (4) Develop a community users and researchers to sustain future research and development in education that promotes scientific reasoning. These new developments will provide educators and researchers with vital instruments, practical examples and resources, and a solid research knowledge base for developing and implementing curricula or interventions that are effective in helping students gain both content understanding and general scientific reasoning abilities in STEM areas. This project will make direct impact to hundreds of teachers and researchers and tens of thousands of students. The products of this project will make long-term sustained impact on the public's views and expectations of STEM education and how we teach future generations in science and technology. The outcomes will also help the transition of our education system from content-oriented teaching to ability fostering, which will produce a more competitive next generation workforce to sustain the prosperity of our society. Public Health Relevance: The goal of this project is to develop a research infrastructure for conducting systematic studies and evaluation of the effectiveness of widely used interactive engagement and inquiry based education interventions on developing students'ability in scientific reasoning. These new developments will provide educators and researchers with vital instruments, practical examples and resources, and a solid research knowledge base for developing and implementing curricula or interventions that are effective in helping students gain both content understanding and general scientific reasoning abilities in STEM areas.

Multidisciplinary (99)
To be effectively prepared for the workforce, students need to learn not only science content (facts) but also advanced transferable reasoning skills. Through development and practice of these skills students will be better prepared to successfully handle real-world tasks in future careers. Not only does inquiry-based instruction promote scientific reasoning (SR) abilities, but also the SR skills of teachers significantly impact their ability to effectively use inquiry methods. Effective STEM teachers need to be competent in both their own content knowledge as well as advanced reasoning skills.

Many teacher preparation programs include inquiry-based courses in order to develop both content knowledge and general abilities, such as scientific reasoning, while modeling best teaching practices. Innovative teacher education programs at many institutions now have many inquiry-based courses designed specifically for pre-service teachers. However, there is evidence (derived through pre- and post-tests of students' SR abilities and their understanding of the nature of science) that many pre-service teachers are not developing these critical skills. In response to this need, this project is developing SR-oriented training modules that can be readily transferred to teacher preparation programs at other institutions

In STEM education, widely accepted teaching goals include not only the development of solid content knowledge but also the development of general scientific reasoning abilities that will enable students to successfully handle open-ended real-world tasks in future careers and design their own experiments to solve scientific, engineering, and social problems. It is often expected by teachers that consistent and rigorous content learning will help develop students' general reasoning abilities. However, research has shown that the content-rich style of STEM education made little impact on the development of students' scientific reasoning abilities. Research also indicates that inquiry based science instruction can promote scientific reasoning abilities and that the scientific reasoning skills of instructors can also significantly affect their ability to use inquiry methods effectively in science courses.
In order to effectively implement inquiry based science curricula for students to develop science knowledge and scientific reasoning ability, good assessment tools are needed to evaluate the impact of the different inquiry based curricula. These assessment tools should be easily applied in large-scale applications and produce valid results for evaluating students' scientific reasoning abilities.
Existing assessment instruments do not provide enough coverage on scientific reasoning. This project will build on existing resources to develop an assessment instrument and a knowledge base on scientific reasoning. The proposed research and development will include the development of a set of standardized assessment instruments on scientific reasoning, a knowledge base ranging from senior elementary school to undergraduate college illuminating the development of students' scientific reasoning, and a community website to archive, share, and disseminate the resources and to sustain future research and development in education that promotes scientific reasoning.

Developing scientific reasoning skills is a major goal of contemporary education. The nature of jobs in the global, knowledge-based economy calls for a shift of educational goals from content drilling toward fostering higher-end skills in reasoning, creativity, and open problem solving. Unfortunately, the typical introductory science course does not significantly impact students in these areas. Although scientific reasoning has become a widely targeted domain for high-end skills in STEM learning, the knowledge base about the impact of curricula targeting scientific reasoning is still not very deep. This project will address the gap by producing a research-based curriculum specifically designed to develop scientific reasoning skills, as well as solid evidence about how the course activities and their specific strategies affect college students' development of scientific reasoning skills. Outcomes of the project will include (1) a complete lab curriculum for first-semester (introductory) physics, with lab modules designed such that they can be flexibly used in lab settings across institutions, and (2) a knowledge base with assessment outcomes on the effectiveness of different elements of the curriculum in helping students develop scientific reasoning skills, as well as best practices for implementing the curriculum in different settings to optimize learning. Because the scientific reasoning modules will be designed to readily fit into existing lab courses, this work has the potential to impact thousands of STEM majors (1,600 per year at the University of Cincinnati alone). Because scientific reasoning skills cross all areas of STEM, the project's research outcomes will be adaptable across many courses in the STEM disciplines.

This project combines the development of promising curricula with a rigorous assessment/evaluation study. The robust assessment plan will be conducted across six pre-selected test sites. A validated assessment tool will be used to measure students' scientific reasoning. The research design, which involves multiple repeated measures with random subgroups of a student population, will produce data that allows for the identification of patterns of class mean performance on scientific reasoning skills over time, as well as an understanding of how changes in scientific reasoning performance are connected with specific instructional events. Statistical analyses involving multiple t-tests will determine whether the evolution patterns and states are statistically significant and consistent with instructional events by comparing performance before and after relevant instruction. Survey data collected from instructors at test sites will provide evidence of successful implementation strategies, as well as challenges, such that the curriculum and related products (i.e., instructor guides) can be designed to optimize learning and promote widespread use of the end products.

The goal of this Exploratory Design and Development Teaching project is to develop and evaluate a module for use in a 7th grade classroom that promotes student development of 21st Century skills with a particular focus on student development of scientific reasoning. The technology-enhanced curriculum will be designed to engage learners in deep and meaningful investigations to promote student learning of content in parallel with 21st century skills. The module will be designed using principles of inquiry-based learning as well as the principles of universal design for learning (UDL). The motivation behind this project is that it will directly contribute to the limited research on the interventions that impact teachers' capacity to provide high quality 21st century STEM education to all students, with a specific focus on underrepresented minorities and those with disabilities. The classroom setting for which the curriculum will be delivered is within an urban district which includes a large number of minority students and over 20% students with specific learning disabilities. The project will catalyze students' deep understanding of content knowledge while developing 21st century skills in parallel; hence better preparing students for sustainable learning experiences into high school and beyond.

A study will be conducted to determine the effectiveness of the learning modules on classroom practices as well as student learning. A mixed methods design involving multiple measures will provide insights into changes in teachers' content knowledge, teaching practices that include a focus on 21st century learning, and fidelity of use of the TI21 framework for implementation of the learning activities. Pre- and post-testing of students using a scientific reasoning assessment and surveys on attitudes towards STEM, along with validated and widely used concept inventories, will provide further measures. As part of this exploratory project, the design and validity of instruments for use with the targeted population, which includes students with specific learning disabilities, will be further tested. This will include administering some of the assessments through web-based apps to meet the needs of these students. The learning modules, with embedded assessments and web-based apps, will provide an innovative approach in which transferable 21st century skills can be developed and measured. Outcomes of this project will be disseminated throughout the urban school system and therefore have the ability to impact thousands of other students (mostly minorities and many with disabilities) and their science, math, and technology teachers. Project outcomes will also inform the development of future science and/or modules for use in similar urban classroom settings.

The use of validated concept inventories, which are sets of questions designed and tested to accurately probe student understanding of a particular field or topic, is an accepted method of assessing student learning. However, the use of concept inventories is often avoided by instructors because they take considerable class time to complete. This project seeks to streamline and revise existing concept inventories into shortened versions that retain their statistical power for student assessment. The project will develop a standard method to create multiple shortened concept inventories from established inventories currently used in STEM education. Generating shortened but relevant concept inventories will decrease the class time needed to employ these inventories and will also mitigate memorization effects that can occur when the same concept inventory is used to measure learning before and after a lesson is conducted. Results from this project will provide a powerful new method of assessing student learning.

This three-year project to refine and apply short concept inventory creation methodology will result in shortened versions of four commonly used concept inventories - the Mechanics Baseline Test and the Brief Electricity and Magnetism Assessment (both from Physics), the Statics Concept Inventory (from Engineering), and the Biology Concept Inventory. Statistical analysis, item response theory, factor analysis, equating modeling, and discipline-specific experts will guide the creation process to help ensure equivalence to the original concept inventories and between the shortened versions. Randomized testing will produce data that allows for comparisons between new shortened concept inventories and the original full length tests to determine the equivalence of the multiple versions and the appropriate conversion models to translate scores between versions. Upon successful validation, the shortened concept inventories will be immediately ready to administer in relevant courses and the methodology will be applicable to other concept inventories by researchers across all STEM disciplines. This research will also produce baseline outcomes for implementation strategies and guide efforts to promote widespread use of shortened concept inventories. The methodology developed in this research will be carefully detailed, giving step by step instructions to apply to further concept inventories by other researchers. In addition, this research will help to inform future concept inventory development of short parallel concept inventories to make STEM learning assessment more accessible to all educators.