Marsha Lovett - US grants

9720354 Lovett This project is being funded by the Learning and Intelligent Systems (LIS) Initiative, including support from the Office of Multidisciplinary Activities of the Directorate for Mathematics and Physical Sciences. This project will develop the three core components of an innovative, intelligent learning environment for teaching statistical reasoning. It is aimed at directly facilitating students' ability to transfer what they have learned to situations outside the original learning context. The three components are (1) a computer interface that helps students develop a general understanding, (2) a detailed specification of the knowledge required to apply statistical reasoning effectively, and (3) new computational and statistical techniques for assessing the accuracy and generality of students' knowledge and then generating appropriate remediation. This project entails a unique collaboration among cognitive psychologists, statisticians, and computer scientists. This project will lead to fundamental advances on several fronts. First, the interface provides a new learning tool that will be used by every humanities and social sciences student at Carnegie Mellon University and will be disseminated to other colleges. Second, because the interface is designed to apply the principles revealed by recent cognitive psychology research, it offers a test of these principles' effectiveness in practice. Third, developing a detailed specification of the knowledge required for statistical reasoning will yield new insights that can inform statistics instruction and cognitive theories. Fourth, the techniques for assessing students' knowledge develop new ways of using the information recorded by computerized learning environments. Fifth, the rich data collected on students' transfer throughout this project will lead to a deeper understanding of how, when, and why transfer occurs. Statistical reasoning is the domain for this project because (a) effective transfer is critical here--stude nts must apply the skills they have learned across a wide range of issues and content areas, and (b) students often have great difficulty transferring these skills.

Lynne Reder

Abstract


Performance on cognitive tasks varies among individuals. This project is a of study two of the sources of these individual differences; namely, working memory capacity (people's limited resources for retrieving and maintaining information during cognitive processing) and strategy adaptivity (people's ability to change their approach to a task in order to achieve greater success). The first goal of this project is to develop, test, and refine a theory of how individual differences in working memory capacity impact performance across multiple tasks. This theory will be developed as a computational model that can make accurate predictions of individual subjects' performances across multiple tasks at a fine-grained, quantitative level. Specifically, the computational model will enable the estimation of an individual's working memory capacity from one task and then use that estimate to make predictions of performance on the second task. The parameters can be interpreted to represent stable differences between subjects and can be used to predict the same individual's performance on other tasks. Predicting individuals' performances in this way has not been achieved before now and will be a major contribution of this research. The second goal of this project is to explore how differences in strategy adaptivity can be understood in terms of differences in working memory capacity. In sum, the project will result in several unique achievements: 1) the development of a new way to understand individual differences in working memory capacity and strategy adaptivity; 2) provision of a mechanistic account of these differences; and 3) a determination of whether computational modeling can be used to predict performance in terms of zero parameter model fitting.

Failure to learn hidden skills is a persistent obstacle to students in science, math, and engineering domains. Hidden skills, which include problem categorization, feature detection, and planning, are critical to solving problems in a domain but do not have any immediate, external product for students to see. Unfortunately, it is unclear how best to identify and teach these difficult-to-learn skills. Instructional scaffolding is a popular and effective technique for providing targeted support and guidance while students learn to solve problems in a new domain. Scaffolding has great potential for improving hidden-skill learning. However, the reasons it works and how best to implement it are largely unknown.

The proposed research will explain the effectiveness of instructional scaffolding in terms of hidden skill learning. Several hypotheses about the relationship between scaffolding and hidden skills will be tested, and new scaffolding designs will be evaluated. This will lead to a systematic approach to teaching hidden skills that improves students' learning and transfer. The four specific aims of this project are:
(1) Develop a systematic, efficient method for identifying hidden skills. While methods currently exist for analyzing domain-specific knowledge, these methods are not robust for identifying hidden skills, and they tend to be difficult and slow. This project will develop and test an automated method that combines logistic regression models and heuristic search algorithms to infer where hidden skills lie.
(2) Develop a theoretical explanation for why scaffolding works. Although instructional scaffolds often lead to better learning, there has been little theoretical progress in explaining when and how scaffolding works. A sequence of experiments will be conducted to test three hypotheses that offer increasingly concrete levels of explanation for how scaffolding benefits learning and transfer.
(3) Develop practical guidelines for the design of effective instructional scaffolding. Three critical questions for scaffolding design will be examined: What level of scaffolding support is sufficient to achieve its main benefit? When and how should scaffolding support be built and faded? And how can human instructors (i.e., TA's) best complement a computerized scaffolded learning environment?
(4) Develop novel applications of our results on scaffolding hidden skills. There are at least two novel applications of this work, beyond the scope of learning theory and instructional design. First, the scaffolding designs from Specific Aim 3 will be used to develop new on-line assessments of students' understanding. Second, the results from Specific Aim 1 will be used to develop tools that train instructors to "see" the hidden skills in complex problems and thus better anticipate students' learning difficulties.

This grant will support participation of doctoral students selected to participate in the Sixth International Conference on Cognitive Modeling (ICCM-2004) Doctoral Consortium, which will be held on July 30 thru August 1, 2004 at the University of Pittsburgh and Carnegie Mellon University in Pittsburgh, Pennsylvania. Students will present results of their research, receive feedback from a panel of established researchers in their field, and receive guidance on future research directions.

The PI proposes to organize a multi-disciplinary symposium that will examine the roles that learning and instruction can play in how people reason with data: the kinds of representations they form, their understandings of measurement, of statistical noise, of graphical displays, and how they understand and use data to explain phenomena and make decisions. The proposed symposium will be the first of its kind to bring such a diverse group of junior and senior researchers together to forge common ground. It will allow them to exchange state-of-the-art work from their different disciplines, to seek converging results and common principles, and to discover research needs or gaps based on current work. At present, work in the disciplines of developmental psychology, statistics, decision analysis, and science and math education proceed with little interaction or sharing of insights. One of the focuses of the symposium is to bring current theoretical insights to bear on practice (in business and in the schools), as well as to make insights derived from the field available across field boundaries in order to allow it to advance the theoretical work.

[unreadable] DESCRIPTION (provided by applicant): We plan a three-day symposium to bring together leading researchers in the fields of developmental psychology, cognitive psychology, decision-making science, math education, statistics education, and science education to address basic and applied issues regarding how children and adults think with data. Thinking with data involves a variety of inter-related processes - generating appropriate data representations, interpreting and reasoning about those representations, making decisions or inferences from the data, and evaluating the validity of conclusions - that, until now, have largely been studied in isolation. The specific aims of the symposium are as follows: [unreadable] [unreadable] To communicate relevant, state-of-the-art research from these different disciplines to a diverse audience so that converging results and emergent principles can be explored and discussed. [unreadable] [unreadable] To discover the research needs or gaps based on current work so that future progress in both basic and applied research into how children and adults think with data can be effectively guided. [unreadable] [unreadable] To explore connections between cognitive, developmental, and instructional issues involved in how children learn to reason with data in formal and informal learning environments. [unreadable] [unreadable] To summarize the instructional implications that can be drawn from integrating the research on how children and adults think with data. [unreadable] [unreadable] To establish a newfound, multidisciplinary community of researchers that continues to communicate about and work toward these shared research goals. [unreadable] [unreadable] To encourage the development of new students and beginning scientists in joining and participating in this community of researchers. [unreadable] [unreadable] The work presented at the symposium will be published as the 33rd volume in the series of "Carnegie Symposium on Cognition" series. [unreadable] [unreadable]

Computer Science (31)

This project is extending the Tekkotsu open source robot programming framework developed in the PI's lab (see Tekkotsu.org) by creating new primitives for manipulation and for control of posture and balance, and by further enriching the existing repertoire of primitives for vision processing, mapping, and navigation. It also is providing the first systematic study of how a higher level approach to robot programming influences educational outcomes. This project is developing software and course materials that foster a new, higher-level approach to introductory robotics for undergraduates, called "cognitive robotics." Cognitive robotics courses are already offered at Carnegie Mellon, Spelman College, and several other schools with which the PI is collaborating. The project is promoting the wider adoption of cognitive robotics curricula by offering workshops at Carnegie Mellon for computer science educators, making presentations at conferences such as SIGCSE and AAAI, disseminating open source software and educational materials via the web, and creating a cognitive robotics textbook.

Until recently, undergraduate robotics courses have been limited by inexpensive platforms which provide only meager sensors and minimal processing power. Such courses have therefore tended to focus on mechanical construction activities and programming simple reactive behaviors such as wall following. While some platforms provide for an optional video camera, image processing support has typically been limited to crude blob detection, not true computer vision. In cognitive robotics, students use more sophisticated robots that can see and recognize objects, physically manipulate them, build a map of the environment, and navigate on that map. The Sony AIBO robot dog was the first platform suitable for this approach, but other capable platforms are now becoming available. Undergraduates can be taught to program these robots using high-level primitives that draw inspiration from ideas in cognitive science. This allows even beginning roboticists to explore interesting problems in perception and manipulation while the complexities of advanced image processing and motor control are taken care of for them.

This PI team aims to use artificial intelligence to exploit data collected from intelligent tutoring systems to provide feedback both to students and to teachers effectively and at the right times. The team is using a new analytic approach, which introduces hierarchical modeling to learning analytics, to investigate how to better understand students' learning states. Algorithms make valid interpretable and actionable inferences from student-learning data, drawing on cognitive theories and statistics to make it work. As in tutoring systems, analysis is at the level of component skills rather than looking at end performance on a task as a whole. Research is around construction of the algorithms for deducing student learning and student learning states and around learning ways of signaling both to learners and to their teachers what concepts and skills learners understand and are capable of and which they are having trouble with. A learning dashboard will allow teachers to visualize the learning needs of a whole class and adapt activities to student needs. Feedback aimed at learners themselves will help learners recognize activities they need to engage in next to better their skills or understanding. Evaluation will include the degree to which learners development of metacognitive skills when such tools are available.


The proposed work will contribute towards the next generation of intelligent tutoring systems as well as contribute to the data analytics needed to make use of large-scale educational data repositories. Because the Learning Dashboard will be independent of any particular domain, and because metacognition and self-assessment are foregrounded, the Learning Dashboard and what is learned about designing an effective learning dashboard should be applicable across disciplines and classes. The proposal brings together what is known about learning, metacognition, and intelligent tutoring systems to address timely learning analytics issues.

This project is implementing a unique curriculum that teaches leadership and innovation in an integrated, non-obtrusive way to mechanical engineering undergraduate students. Such skills are critical for the nation to remain competitive in a world market. The project's research component is helping identify effective ways of integrating these skills into an engineering curriculum without sacrificing technical content. The project team is well qualified and diverse, consisting of engineering faculty working with business faculty together with the University's Center for Teaching Excellence. This synergistic team is providing the engineering and leadership content knowledge with proper educational pedagogy and evaluation methods to make the project successful.

Assessment and evaluation activities consist of quantitative and qualitative approaches that address both student learning and attitudes. Qualitative assessment also includes focus group interviews of students. What researchers intend to explore is whether or not integrating leadership and innovation materials into a traditional technical engineering program provides sufficient depth to improve student skills. Results are also helping the education community better understand how to teach these important skills in a cost effective manner, by integrating them into existing curricula.