Roy D. Pea - US grants

This award supports the design and prototype development of a Web-based application for collaborative analysis of video records in the human sciences. This application, DIVER (Digital Interactive Video Exploration and Reflection), addresses a critical need for .collaboratory. functions in the human sciences, where common tools and a shared corpus of human interaction datasets can be used for investigating questions of theory and practice. The proposed effort will leverage an NSF-MRI Award (NSF-0216334) for related work and capitalize on an NSF-funded Center for Innovative Learning Technologies (CILT) workshop on Digital Video Inquiry in late November 2002. The CILT workshop will enable the introduction of the tool to a community of NSF project researchers, graduate students, and teacher-researchers, and to engage them in the further design and development of DIVER. This feedback is crucial for an application that is intended to help constitute and serve a community of users. The DIVER application will consist of a server holding analyses of video data, consisting of video sources and a series of annotated selections cropped in both time and space. Viewers of a DIVER analysis will be able to respond using threaded discussions attached to particular scenes from the original analysis, and they can return to the base video clip to create their own selections as their responses warrant. Several substantial technical issues, primary among them the manipulation of multimedia records in a web browser environment, and the storage, indexing, and retrieval of complex annotated multimedia documents need to be resolved.

With support from a NSF Major Research Instrumentation (MRI) award, Dr. Roy Pea will develop a new develop a new instrument for high-resolution digital panoramic audio-video recording, analysis and communication. The Digital Video Collaboratory (DVC) will provide an integrated suite of hardware technologies and Stanford's DIVER software system (Digital Interactive Video Exploration and Reflection) will allow researchers and educators to generate and share different points-of-view (POV) and analytic perspectives on the same richly recorded classroom events. Panoramic image acquisition is make possible by using a multi-camera digital video system coupled with mirrors to capture images over 360 degrees of horizontal arc. These camera images are then stitched together by software so that the captured scene may be displayed on a computer monitor as a rectangle that depicts the "unwrapped" 360-degree field of view.

To develop the DVC the team will integrate a state-of-the-art digital panoramic video capture system and a room-size microphone array in an experimental space with associated software and hardware tools for synchronizing, storing and managing the multimedia data streams. The DRIVER software system will be used to author and annotate POV-paths through these "Free-D" panoramic digital video records. Using a personal computer, the DIVER user pans and zooms a "virtual camera" window over a panoramic scene to focus on events of interest. The user may then mark interest points and record their POV-paths and associated textual annotations (for research or educational purposes). An annotated DIVER file can be shared through a computer network so others may re-experience and respond to the user's interpretive "tour" of the recorded activities. The user of DIVER is thus able to be a "virtual videographer", after the fact of panoramic video capture.

This project is important because researchers in the learning sciences increasingly rely on multimedia technologies to capture the complexities of learning environments and to repeatedly view recordings to deepen interoperation and analyses. Digital video and audio, integrated with other data types, promise far more complete records of learning activities than older methods, and enable multiple researchers to investigate common datasets, as in other disciplines. However, current approaches to multimedia recording are often ad hoc, yielding partial data and hindering the development of more powerful theory and practice. The Digital Video Collaboratory and the DIVER software system have potential for widespread use not only in learning sciences research and training and teacher education, but also in many fields for which panoramic audio-video recording and annotation can provide a useful function. An integral part of the project is the use of the DVC by many faculty at Stanford and elsewhere for their research, for graduate and undergraduate education in learning and cognitive sciences, and for teacher education.

The proposed effort will build upon prior work by participating institutions in audio and video capture, analysis and collaboration, allowing each institution to make progress more quickly by leveraging each other's respective strengths, capabilities and prior work. The project brings together NSF-funded efforts involving multimedia data in the human sciences: the DIVER project housed at Stanford University, and the TalkBank Project, housed at CMU and the University of Pennsylvania, as well as NSF Middleware and Grid projects. The proposed project will adopt a two-track research approach to designing and implementing the Collaboratory. The first project track is to develop a virtual video repository and video analysis community portal. The second track will produce an open community toolkit that greatly expands generality and capabilities for video analysis, video input and output. In addition to these goals of moving from local to distributed users, tools and data will evolve in an ongoing process of user testing, piloting, and software packaging, involving the community and refining the tools based on their experiences. Project results at each interim milestone will be disseminated broadly for enhancing scientific and technological understanding, especially to conferences and journals and complemented by an open website for sharing ongoing research, technical developments, technical reports and software releases to be downloaded to user communities. Video Collaboratory toolkits will be made available to participating institutions with efforts made to spur adoption that will facilitate multiple communities using collaborative commentary around video.

The initial focus of our project is the human sciences, with the aim of leveraging video records to enhance the learning sciences, education, teaching and training. We also expect that video application and infrastructure work can provide significant support to the physical sciences, since video is an important data type for scientific visualization and primary empirical data, and several Grid initiatives use video databases integrally. We anticipate producing open-source video software that can be utilized in the current Grid and Middleware platform initiatives. This could enable a new primary software layer, a set of standards for video analysis and collaboration middleware, and the potential for a substantial broader impact as an outcome of this work.

Media multitasking is defined in the literature as engaging in more than one media activity, such as watching TV, reading, playing video games, or instant messaging, over a specified period of time. Media multitasking is a growing phenomenon observed among 8-18 year olds, yet little is known regarding how this behavior influences key cognitive, developmental and behavioral processes which affect the way young people learn, reason, socialize, think creatively, and understand the world. Previous research in the adult population generally shows that media multitasking negatively impacts productivity. However, little research to date has examined the processes underlying children's media multitasking or how such processes may affect cognitive development and learning.

Clifford Nass of the Stanford CHIMe Lab and Roy Pea, Co-PI of the NSF LIFE (Learning in Informal and Formal Environments) Center, in partnership with the Joan Ganz Cooney Center at Sesame Workshop, will convene a symposium for an interdisciplinary group of leading scholars from neuroscience, child development, cognitive, communication, and education fields, and key policy leaders to advance our understanding of the potential implications of media multitasking for learning and cognitive development in children and youth. The primary goal of the symposium is to define a coherent agenda to stimulate needed theory-based research on the implications of media multitasking for learning and cognitive development. The symposium will consist of three panels of multi-disciplinary experts to examine: a) media multitasking and cognitive development, b) media multitasking, learning and productivity, and c) design principles for leveraging media multitasking in educational contexts. A meeting report will be distributed on the symposium website and in print form. The symposium website will also facilitate communication and knowledge sharing among a newly defined research community and serve as a clearinghouse for information on the implications of media multitasking for cognitive development and learning. Overall, the activities associated with this grant will help activate a new special interest academic community, and encourage and facilitate knowledge sharing and collaboration across institutional, intellectual and methodological disciplines. Through understanding how media multitasking may affect psychological processes underlying learning and cognition, practitioners will be better prepared to inform policy and to maximize educational impact in a broad range of learning contexts.

Stanford University proposes to develop and evaluate a proof-of-concept online middle school course (with a teacher version as well) that adapts concepts from the Exploring Computer Science (ECS) curriculum, specifically algorithmic thinking and introductory programming. The project will:
(1) Design and deploy an online six-week "Foundations for Advancing Computational Thinking" (FACT) curriculum on Stanford University's instance of the open edX online platform. Aimed at 12 to 15 year-old learners, the curriculum borrows from the ECS Programming and Problem Solving units. It will be driven by short video lessons with in-video and stand-alone quizzes for formative and summative assessments, and programming activities.
(2) Pilot FACT in a Bay Area public middle school class.
(3) Empirically examine the efficacy of: (a) the curriculum for the development of computational competencies, preparation for future learning of computing, and changes in student perceptions of CS via assessments designed for these purposes, and (b) student attitudes towards and experiences with online learning, including online course features such as in-video quizzes and discussion forums.
(4) Create and pilot an appropriately enhanced version of the curriculum for teachers to effectively prepare them to facilitate FACT/ECS use in their classrooms.

This Cyberlearning Capacity-Building project brings together learning scientists, experts in media creation, experts in child development, producers of public media assets, parent, and educators in an effort to build social infrastructure that will support bringing what is known about how people learn to the design of public media that can effectively connect school learning and out-of-school learning for young children. The team's theoretical framing and working hypothesis highlights the importance of media as a catalyst for collaboration and learning conversations; according to the theoretical base, these collaborations and learning conversations, when carried out across peers and in families, can play a powerful role in connecting children's school and outside-of-school experiences. Two workshops are being convened for the purpose of shedding light on the pragmatics of doing this -- the R&D partnerships needed, the methods that might be used, and the issues that need to be addressed for success. Through partnerships with children's educational media producers, the team is building capacity for interdisciplinary teams that include learning scientists and media producers to engage in research around how to use public media assets to promote the kinds of learning conversations in and out of school that will connect home and school settings into a distributed learning environment.

This project is laying the groundwork for new interdisciplinary research efforts addressing issues in early learning. The team's theoretical framework points to media as a catalyst for the kinds of collaborations and conversations that might promote learning and connect children's school and out-of-school experiences. Thus, this project is bringing together learning scientists and children's educational media producers (PBS, Sesame Workshop, the Joan Ganz Cooney Center) to seed future collaborations. The goals of this initial collaboration are to work together to establish new methods for studying learning with media and advance understanding of how public media assets can be leveraged to support the learning and interest development of young children and their families.