Thomas F. Shipley - US grants

Most objects perceived visually are partly occluded by other objects; yet human observers accurately detect the unity and boundaries of objects under most circumstances. There are other phenomena in which boundaries are perceived in the the absence of local visual information, such as illusory contours and some perceived transparency phenomena. In all of these cases, perceived object boundaries are interpolated between physically- specified edges. This project will address the various cases of boundary interpolation, testing and refining a recently proposed unifying theory, and more generally attempting to specify the conditions under which interpolation occurs. Specific aims include tests of edge interpolation in both two and three- dimensional displays, development of methods for generation of random displays, and development of a variety of measures for assessing boundary perception, including both perceptual report and non-verbal measures. This work will have implications both for understanding human perception of objects and for object and boundary detection in computer vision systems.

Events are the basic units of human experiences. People speak not only of individual objects like 'towels' and 'floors', but also of events like 'bending', and of goal-directed actions like 'picking up a towel from the floor.' The central aim of this research is to characterize how humans perceive and conceptualize information about the nuggets of human experience, the non-linguistic events that are often labeled by language. Using insights from psychology and cognitive linguistics, this work hypothesizes that infants and adults use path information as a wedge into the flux and flow of a dynamic world; that path is central to how people find the atoms of events. Thus, pushing becomes pulling when the path forward is reversed. Eight experiments with infants and adults test this hypothesis by posing three questions: 1) Can infants use geometric information about path trajectories to decompose their world into events?; 2) Can infants and adults recognize categories of events paths even when they appear in new locations and in different orientations?; 3) Are infants and adults sensitive to statistical probabilities within events, building event hierarchies that are based on path changes (e.g., 'kicking the ball' rather than just raising the leg, moving it forward, contact with ball, ball projects forward)?

Results from this foundational research inform both research and application. First, it forwards our knowledge of basic perception asking about the role that path plays in the interpretation of events. Second, given that events are the atoms of experience, this work speaks to how infants represent the world of events and whether they do so in ways that are compatible with adults. Third, exploration of how infants build a basis for event perception and representation positions researchers to ask how the children's burgeoning language knowledge interacts with their event perception. Relational terms like 'around, on, kick, throw' (prepositions and verbs) require that infants are aware not only of isolated objects or actions in their world, but of relations between these entities, relations through which grammar is born. Fourth, children with language deficits like those with autism often have difficulty learning relational terms. Some research suggests that the root of their problem lies not in language per se, but in the processing of event structures codified by language. Thus, diagnostic criteria relevant to both perception and language emerge in the context of this research.

Throughout our daily lives, we frequently observe other people carrying out various actions. How do we process these actions and come to understand the goals and intentions of the people that we are watching? Historically, the brain systems underlying the perception of an action and the actual carrying out of an action have often been considered to be quite separate, but recent research has suggested substantial overlap. One recent theory suggests that recognizing and understanding an action involves brain systems that would be involved in carrying out the same action--the ?mirror neuron system? (MNS). On that theory, in order to understand someone else's actions, our own brains actually covertly mirror that person's actions. The MNS was originally identified in monkeys, and recent work has suggested that a similar system also exists in humans. However, one key aspect of the MNS that is relatively unexplored is the development of the coupling between the perception and action systems during action observation. With support from the National Science Foundation, Drs. Peter Marshall and Thomas Shipley at Temple University are carrying out a series of studies designed to elucidate the development of the MNS. The studies involve infants, preschoolers and adults participating in different tasks which are designed to assess the overlap between the perception and action systems during action observation. These tasks include the passive viewing of human actions, the recognition of oneself performing an action, and the imitation of novel actions. During some of the tasks, the electroencephalograms (EEG) will be recorded. In combination with analyses of participants' overt behavior during these tasks, it will be determined whether motor areas of the brain are indeed active while infants and children observe human actions. The research will also investigate how this brain activity relates to other behavioral measurements of the coupling between perception and action, and how the patterns of brain activity relate to young children's abilities to understand and copy the actions of others.

The studies involve the active participation of undergraduate and graduate students at all stages of the research process. They have clear potential for being broadcast to professionals and laypeople interested in how children and adults process and understand other people's actions. The result may help us understand pervasive developmental disorders such as autism, which are characterized by deficits in imitation and social cognition and which may involve disruptions in MNS function.

The investigators propose to develop and test new instructional techniques aimed at teaching high-school science students how to use the diagrams that appear in their biology textbooks. Research from a range of disciplines has shown that people have difficulty making sense of diagrams. The investigators had previously found that inference and other high-level processes that are important for learning from text are even more important for learning from diagrams. They further showed that students show little gains in content understanding from textbook diagrams. The investigators will capitalize on commonalities among several theories of diagrammatic reasoning to develop four interventions, following an additive design, aimed at improving classroom instruction in how to use diagrams. In each iteration, one additional intervention feature is added to the initial intervention. In the first intervention, they will teach the components of diagrams (i.e., how to read captions, color keys, and other conventions of diagrams). In the second, they will also teach the coordinating of text and diagrams. In the third, they will also have the students engage in self-explanation. In the fourth, they will also have students construct their own drawings. In conjunction with these experimental studies, the investigators will collect eye tracking data on a subset of participants pre- and post-intervention in order to look for possible changes in gaze patterns.

Research on spatial skills in humans has been gaining momentum in recent years, both nationally and internationally. This momentum is partly driven by the importance of spatial skills to achievement in science, technology, engineering, and mathematics (STEM). It is also driven by the pervasive role of spatial cognition in human behavior, for example in tasks like navigation, tool usage, and even language comprehension and production. Two research centers that were recently formed to study spatial cognition are the NSF Spatial Intelligence and Learning Center (SILC) housed at Temple University, and the Transregional Collaborative Research Center Spatial Cognition (TCRCS) housed in Germany. With support of NSF, members of these two centers, along with other invited experts in the field, are holding a workshop with the goal of outlining a broad research agenda for the near and medium term future in the area of spatial cognition.

The workshop is conceived more specifically to consider the role of visualization in STEM education as it relates to navigation, robotics, and engineering design. The two centers bring complementary expertise and perspectives to the table: SILC focuses on a broad set of skills including spatial transformations, spatial language, and spatial analogies, whereas TCRCS focuses on navigation and robotics from a computational point of view, with emphasis on perception-action relations. Other experts were chosen to enhance the representation of neuroscientific and geoscientific perspectives at the workshop. Its format will offer attendees the opportunity to become familiar with each other?s research, discuss the research agenda for the field, and compose a set of recommendations for research directions. The results will be disseminated on a workshop web site using the networks of all participating centers and individuals.

This project is developing and evaluating curricular materials designed to strengthen penetrative thinking skills - the ability to visualize spatial relations within an object - among geoscience students. The curricular materials being developed by this project build on prior successful models that have improved and enhanced the mental rotation skills of engineering students through a variety of exercises. The investigators are producing and assessing two sets of exercises for students. First, online workbook exercises require students to visualize slices through a variety of objects, both geological and non-geological. These exercises build from simple to complex, thus scaffolding students' penetrative thinking skills. Second, classroom gesture exercises are also being developed via collaborations between geoscience instructors and cognitive scientists to study and enhance visualization abilities of students and to better learn how students use their hands and arms to represent various geological features. Collectively, these exercises are being field-tested in three different undergraduate geoscience classes (mineralogy, sedimentology, and structural geology) at three different institutions utilizing a well-designed formative assessment process that is increasing the overall effectiveness of the project. In addition, a parallel set of supplemental online resources for instructors, including a summary of research on spatial reasoning and learning and recommended strategies for using the workbook exercises and the gesture exercises with their classes, is being developed and disseminated.
Being able to successfully visualize the interior of objects at all scales is a core geoscience skill. However, the penetrative thinking abilities of undergraduate geoscience students are known to vary from excellent to poor, in part as a function of gender and socioeconomic status. Consequently, the curricular materials being developed by this project are expanding the pool of students who can succeed in geoscience by helping students with poor penetrative thinking skills overcome that barrier to success. Spatial thinking is known to be an essential component for success in a wide variety of fields, including a variety of STEM (science, technology, engineering and mathematics) disciplines. Thus, the project's focus on developing spatial skills, independent of content, in a way that transfers those skills to new settings, is enhancing the ability of students to persist and excel in various STEM fields. Consequently, the project is providing adoptable models for other disciplines and is enhancing the overall effectiveness of the STEM workforce.

This Fostering Interdisciplinary Research in Education (FIRE) project is a partnership between Columbia University and Temple University that seeks to understand the perceptual and cognitive processes by which undergraduate students and scientists transform earth science data into explanations and prediction.

The principal investigator of this FIRE award, a geoscientist, will be working with a mentor in spatial cognitive science to continue to develop her expertise in this area and eye tracking in particular. The researcher proposes two studies using eye-tracking and think aloud protocols. The first study will investigate students? perceptions and reasoning as they interpret a data set (from topography/bathymetry) that is known to be subject to misinterpretation. Study two is an intervention study, testing whether providing hypothesis templates will help students interpret a three-dimensional data volume in the area of ocean salinity. The research will be evaluated by experts in spatial cognition, epistemology of data, visualization, and earth science.
In the geosciences, many students have a difficult time learning the skills that are necessary to interpret maps and large amounts of spatial data. Such skills are central to what geoscientists do and how knowledge is rendered in this discipline. This study could lead to improvements in how students learn the skills necessary to visualize geoscience data and make claims based on data interpretation. The project clearly advances discovery and promote teaching and learning in the geosciences. The research may also have implications in the learning in other fields of science, technology, engineering, and mathematics that place high demands on a student's visualization abilities.

Scientists who work in the field have a common set of issues when it comes to documenting,storing, and representing data. Members from the different geological communities would benefitgreatly from the opportunity to discuss the types of data that they collect in the field with a group of cyberinfrastructure and software development professionals and researchers. By holding meetings in the field, the computer scientists will gain a better appreciation for the types of data that are typically collected in the field, common methods for collecting those data, the field tools/technology that are employed, data recording conventions, and the types of question typically addressed with these data. "The field" is an ideal location for appreciating geological concepts, and the very act of being in the field, together with other professionals, often foster personal connections that promote successful collaborations
and the exchange of ideas.

Work on this RCN will facilitate digitization of geological field data. The researchers will take steps to: 1) Document what exists currently for field data collection; 2) Assemble a community for discussing and exploring field data collection issues, specifically targeting young investigators; 3) Motivate distinct communities to work together on common issues associated with digitization; 4) Evaluate what is missing in the creation of open and accessible data. The objective of the RCN Proposal is to develop communication between cyberinfrastructure community and those involved in field-based, solid earth geoscience. In order to facilitate knowledge of the activities, they will conduct a series of both informal and formal meetings as national meetings - workshops at GSA and townhall meetings at AGU and AAPG). The goal of the initial meetings will be to: 1) foster community awareness of EarthCube-related activities, 2) discover and catalog the additional existing resources, 3) determine ways of giving publication ?credit? for recording and sharing digital data, 4) identify attributes of a clearinghouse website that would be most useful, and 5) find ways of motivating the community to move quickly toward digital data collection/conversion and data sharing.

This is a pilot REU Site associated with an NSF-funded Science of Learning Center (SLC) at Temple University: the Spatial Intelligence and Learning Center (SILC). Four faculty members from the developmental, cognitive and learning sciences run this program to mentor a diverse group of undergraduate summer interns. The students participate and get experience in an interdisciplinary team science approach to the science of learning. The PI-team aims to provide students with a research experience that informs them regarding a wide variety of synergistic activities using different methodologies and populations, all coordinated on achieving the overall goals of the SLC: improving spatial learning and, ultimately, STEM learning. This team approach goes beyond the boundaries of Temple, as SILC involves much cross-institutional collaboration. Thus, the REU participants reduce their "degrees of separation" for networking to pursue their own professional goals. Regular activities include lab meetings, social events, mentoring sessions, and ethics modules that are designed to draw the students from various labs together at least once a week. These cohort-building events bring together students from all the labs within the SILC, and involve a mix of social interaction, professional development, and research-oriented events. Because of the wealth of opportunities to connect with other undergraduates, graduate students, and faculty members from a variety of labs, the REU experience will be rewarding, both in terms of development of the interns' research knowledge and skills, and in discussing and sharing ideas, solving problems collaboratively, and getting a sense of what it is like to be a part of an academic community. In addition, all REU students will be invited together to the SILC Annual Retreat in October 2015 to present posters based on their summer work. This retreat includes SILC researchers from Northwestern University, University of Chicago and seven other institutions as well as Temple.

The scientific merit of the project lies in the advancement of a new science of learning, focusing on spatial learning and cognition. Spatial cognition is a diverse field with many exciting theoretical and translational research directions ripe to be drawn together synergistically. Spatial cognition is central to many vital human activities, including navigation and wayfinding, tool use and design, and scientific and mathematical thinking. Research on spatial cognition draws on many disciplines, including cognitive science, computer science, geography, geographic information science, neuroscience, linguistics, psychometrics, and robotics. Its findings and insights have relevance for education in science, technology, engineering and mathematics, as well as in a wide range of professional fields, including medicine and dentistry, urban planning and traffic modeling, and architecture and design. Thus, the REU students are exposed to a very rich research experience and mentored by faculty members and graduate students in these fields. Some of the REU students come from the undergraduate population at Temple University, one of the most diverse American research universities. At least half of the participants come from institutions with limited research opportunities. Given the complexity and size of the SILC enterprise, this pilot REU Site serves as a testing ground for a larger, full-fledged REU Site in the future.

This Science of Learning Collaborative Network of cognitive psychologists, education researchers, and geoscience educators from Temple University, Carleton College, and Northern Illinois University focuses on spatial learning. The team will develop new spatial learning principles by designing teaching tools that can be applied across classroom and field courses in the geosciences - a field of science that depends very heavily on spatial skills and spatial reasoning. The tools will be designed to allow students to self-correct conceptual errors in their understanding of scientific concepts, and will be made available through a project web site. Courses on using the tools will also be offered at meetings of geoscience professionals and teachers. The research will expand fundamental understanding of the science of learning by characterizing the different types of spatial reasoning that are required for the practice of a complex spatial science. The research will develop new supports for spatial learning challenges that have been barriers for student learning. The findings could ultimately improve retention and learning in geosciences and in many other Science, Technology, Engineering and Mathematics (STEM) domains that depend on spatial thinking.

Spatial thinking plays a critical role in STEM-related course achievement. Supporting the development of spatial thinking in a science curriculum requires an interdisciplinary effort that combines knowledge of the specific disciplinary science with education and psychology expertise. This collaborative network will develop two new fundamental and complementary spatial learning principles. One is spatial feedback, which is feedback in the form of a spatial error that allows the mind and brain to guide learning. Providing feedback about spatial information is essential to supporting learning about complex spatial concepts across the geosciences. The other is spatial accommodation, which is the constructing and reconstructing of mental models to accurately incorporate spatial information to improve inaccurate mental models from spatial feedback. The network will create a "trading zone" where theory and practice converge so that research on education and cognitive psychology can be influenced by disciplinary geoscience content, and vice versa. The expected results include new designs for teaching tools and new insights into the working of the human mind and brain.

The award is from the Science of Learning-Collaborative Networks (SL-CN) Program, with funding from the SBE Division of Behavioral and Cognitive Sciences (BCS), the SBE Office of Multidisciplinary Activities (SMA), and the EHR Core Research (ECR) Program.

A team of researchers from Temple University and Carleton College will convene a workshop focused on educating students to be skillful users and creators of modern scientific visualizations, such as data graphs, maps, charts and diagrams. Such visualizations are now abundant in science textbooks, newspaper and magazine articles, and in scientific journal articles. In many cases, the article or chapter simply cannot be understood by reading the text alone; the visualizations carry essential elements of the message and many students find them challenging to understand. The workshop will bring together a multi-disciplinary group of researchers, developers, and educators drawn from a variety of fields where using scientific visualizations is a key competency, including the full range of STEM disciplines. The workshop will focus on both social and cognitive processes and strategies that underlie learning with visualizations. Conveners and participants will work to synthesize what is known about fostering and assessing students' visualization proficiency, and will identify outstanding research questions. The main product of the workshop for the broader community will be a compilation of instructional and assessment strategies that are independent of discipline and show promise for helping to build students' proficiency with visualizations. The compilation will be made accessible through a widely used and freely-available website. The project is funded by the EHR Core Research (ECR) program, which supports work that advances the fundamental research literature on STEM learning.

Researchers from Temple University and Carleton College will convene a workshop focused on educating students to be skillful users and creators of modern scientific visualizations, such as data graphs, maps, charts and diagrams. The goal of the workshop is to construct a set of visualization competencies that builds across the educational trajectory, is grounded in human perceptual and cognitive systems, is not tied to specific disciplinary visualization practices, and produces graduates who can develop and interpret visualizations of types that have not been explicitly taught. Learners who acquire this kind of trans-disciplinary visualization competency will find it easier to move between and collaborate across disciplines. They will increase their capacity to communicate about science, and thus help to bridge between science and the rest of society. To move towards these goals, this workshop will bring together a multi-disciplinary group including faculty, doctoral students, curriculum developers, professional development providers, visualization creators, and education researchers, working in a variety of fields where using scientific visualizations is a key competency. The workshop will have two strands: one focused on fostering learners' visualization competencies and the second on assessing those competencies and evaluating the effectiveness of visualizations and their use in education based on those competencies. For each strand, the conveners have cast a broad net for ideas that have been shown to work in some context and are potentially suitable for expansion across multiple contexts. Drawing on the participants' experience and expertise, the group will produce a compilation of instructional strategies that are independent of discipline and show promise for helping to build students' proficiency with data-based and concept-based visualizations. The compilation will be fleshed out with examples from multiple fields. In parallel with this gathering and cross-fertilization of existing knowledge, the group will identify key research questions about visualization learning that have recurred across multiple visualization-using disciplines.

Sand and dust storms are a growing worldwide menace, soil instability and erosion threatens agriculture, and fine sediment compromises ecosystem health of rivers and oceans, all impacting large human populations on nearly every continent. The cumulative effects of these disturbed environments also threaten human well-being through damage of habitation and disruption of transportation. An interdisciplinary collaboration of geoscience, cognitive science, and robotics researchers aims to accelerate and deepen the collection of data about the fluid and materials properties associated with such unstable soils by endowing legged robots with the instrumentation and scientific agenda of agile, novice field assistants. These new robots are capable of general field mobility and are being programmed to think like assistant field geologists in order to develop research strategies in rugged natural environments where measurements are lacking. Overcoming the specific locomotion challenges presented by these environments and developing algorithms and software sufficient to interpret and act on human research needs will greatly advance the field of robotics. The resulting new information about wind, water and materials processes will have bearing on the management of infrastructure and agriculture, and also the response of landscapes to environmental changes.

The project focus is to use the geosciences field research setting to test a chain of hypotheses reaching from the formal representation of scientific knowledge to the properties of provably correct algorithms for human-machine pursuit of scientific data. The geoscientific goal is to produce the first comprehensive, time varying maps of soil strength with co-located soil moisture composition and size over the course of rainfall events in a natural landscape. The cognitive science goal is to develop a formalized representation of the cognitive processes underlying field data collection and interpretation that is simultaneously suitable to underlie robotic field assistance algorithms while at the same time advancing the study of human perceptual interpolation and reasoning. The robotics goal is to achieve a provably correct architecture for generating from formalized human task specification a chain of safe, stable online automated legged gait transitions on complex broken terrain that subserve the geoscientist data collection objectives. Two different families of legged robots with a variety of perceptual and geoscientific instrumentation suites will be deployed over natural hillsides under investigation by human geoscientists. Field performance of the resulting human-robot teams will be evaluated according to criteria assessing the degree of robotic mobility and autonomy, the quality and reliability of the resulting geoscientific measurements, and the impact of the collection process on the sampled environment. Advances in legged robot mechanics and intelligent control have brought the field to a threshold where the next major challenges for autonomous mobility can only be formulated and engaged with respect to suites of abstract but formal tasks relative to unstructured environments against which the appropriateness and success of autonomously generated, goal-directed motor behaviors can be precisely measured. Robots endowed with even the rudiments of understanding what measurements are needed where and when by scientists, in order to test their hypotheses, would deepen our insight into the structure of human cognition. They would also open the way toward collecting massive amounts of data at presently unachievably fine spatiotemporal scales, potentially transforming the theoretical and empirical foundations of geoscience.

The proposed workshop will bring together awardees of the NSF Science of Learning Program. It seeks to capitalize on NSF investments in the Science of Learning to build community among diverse interdisciplinary researchers studying learning, and to build capacity so that investigators can readily reconfigure and mobilize collaborations to capture new opportunities offered by NSF priority areas. As examples, how technology impacts the way we learn and work, as well as how we might harness Big Data to gain greater insight into real-world learning are areas of relevance to the NSF's 10 Big Ideas (https://www.nsf.gov/about/congress/reports/nsf_big_ideas.pdf).

This gathering will include researchers from several disciplines under one large umbrella, including neuroscience, cognitive, behavioral and social sciences, computer and information sciences, engineering and education. The proposed effort is needed as there is currently no professional society meeting to support this interdisciplinary Science of Learning community of investigators and their trainees. The gathering seeks to foster information exchange and resource sharing, and to build trust among community members for successful interdisciplinary research. The workshop format will include small group discussions on topics that include new insights and methodologies that offer opportunities to make transformative advances in fundamental knowledge about learning over the lifespan, and intersections in interdisciplinary research that offer opportunities for convergence research in support of NSF's Big Ideas. In this way, the workshop will leverage the group's collective interdisciplinary intelligence for "Big Idea" problem-solving and the unique opportunity to work on problems that are larger than any taken on by individual laboratories. A survey of the research and other needs of the Science of Learning community will be conducted. The diverse disciplinary representation of the participants and the requirement to communicate and discuss science across a broad audience will have broader impacts in building an active and vibrant community capable of addressing complex societal problems through collaborative efforts. In addition, and as part of the workshop, plans will be developed to promote better communication of research findings to educational practitioners, policy makers, and the public. This will help to bridge the gap in understanding between basic research findings and their relevance to social challenges.

The Future of Work at the Human-Technology Frontier (FW-HTF) is one of 10 new Big Ideas for Future Investment announced by NSF. The FW-HTF cross-directorate program aims to respond to the challenges and opportunities of the changing landscape of jobs and work by supporting convergent research. This award fulfills part of that aim.

This project will make a significant contribution toward the support of future workers in geology. Understanding how geologists reason, plan to collect new data, consider three-dimensional spatial relations, and evaluate uncertainty are critically important for supporting scientists working on applied problems, such as natural resource exploration. This project will enhance existing efforts in geology to collect data using robot drones. Drones allow access to important areas of the world too dangerous to access in person and not visible from satellite or plane. The project will use machine learning to incorporate expert knowledge into drone flights to support effective autonomous data collection. The data will yield improved geological understanding of an important fault system. Findings from the project will improve understanding of uncertainty in volumes and thus improve our understanding of earthquakes and the analyses of petroleum workers. Understanding how expert geologists reason will support new exploration and mapping strategies for human-robot teams working in natural environments. The collaborative efforts of the interdisciplinary team will advance the fields of cognitive science, geology, and machine learning. The integration of cognitive science, robotics, and geology will develop new approaches to field work with human-autonomous systems teams that are faster and more effective than any either human or autonomous system would be acting alone.

The project will characterize expert spatial reasoning about 3D relations and uncertainty as geologists collect data to develop a 3D understanding of a new field area, make predictions about future observations, and construct geological models. Errors in reasoning about 3D structures will be used to develop quantitative models of expert uncertainty. These models will be used to help explicitly visualize uncertainty for the experts and to construct cost functions for the robot navigation. The cost functions will include metrics that capture scientific value. The project will develop new approaches to drone exploration and mapping, including machine learning of features of interest to geologists. Drones will autonomously explore and map natural rock formations in canyon environments to support and speed up the data collection and interpretation efforts of field geologists. The project will study the structural geology of the Mecca Hills area of California, a well exposed portion of the San Andreas fault system. Robot drones will collect data about surface features to develop maps of subsurface structures. The cognitive science-infused robot design will employ successful expert strategies and focus on areas where experts are likely to make errors to prioritize exploration of those areas in navigation plans. The proposed strategies will enable 3D surface reconstruction of canyon surfaces. They will also enable better understanding of how to enhance planning and on-the-fly decision making of experts for collecting scientifically important data. The project's foundational work aims to develop an interdisciplinary understanding of how geologists build a scientific understanding of a region over time. It also aims to design autonomous exploration strategies for human-robot teams, and test new ways to support the sequential decisions about where to collect data to maximize scientific impact.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.