Steven L. Franconeri - US grants

Many visual tasks require that we divide attention across multiple locations at once. We need this ability for everyday tasks, from counting a set of chairs before a dinner party to navigating complex traffic during rush hour. We also need this ability at school or at work, when following the complex layout of a diagram or finding differences among values within graphs. In these kinds of tasks there is a surprising limit to our ability to split our attention and we can typically deal with only a few locations at once. The ubiquity of this limitation has led to its acceptance as a fundamental limit on visual processing, yet we have little understanding of why it happens. With the support of an NSF CAREER Award, Dr. Franconeri, Northwestern University, will test the possibility that these limits stem from a bottleneck within a cognitive 'map' of attended locations in the world.

Understanding the limits of divided attention could lead to important changes in the ways that we organize information in graphs and diagrams and may offer critical insight into our understanding of how children learn to count. It may also lead to better understanding of visual processing differences in autistic populations, who have greater difficulty dividing their attention. This research will be integrated with an education plan that includes (1) outreach to the general public through the design of a "Brain Week" series as well as other outreach talks in the Chicago area, (2) curriculum development, (3) advising of students and researchers from high school to postdoctoral levels, (4) outreach to underrepresented groups via recruitment to both Northwestern and the investigator's laboratory, including a summer fellowship for a local student from an under-represented background, and (5) collaborative outreach to related disciplines including education and information visualization.

Comparison is an essential part of data analysis and, therefore, of many visualization tasks. While the published literature provides a wealth of visualization tools for looking at individual objects (graphs, volumes, time series, gene sequences, molecular motions, etc.), there has to date been less consideration of support for comparison. The PIs argue that comparison tasks are best supported by tools explicitly designed for that purpose. The problem is that visual comparison becomes more challenging as the number of objects, their size, and the complexity of the objects and/or of the relationships among them increases. The difficulty is further compounded by our rapidly growing ability to collect and generate data. In prior work the PIs have developed some encouraging initial examples of comparison tools, but these are specialized successes that offer little guidance for future endeavors. Addressing a wider range of comparison problems at greater scale with our present limited understanding thus largely remains an art that requires considerable effort. The PIs' goal in this project is to move towards a science of visual comparison. By studying visual comparison as a general problem, they will establish a domain-independent foundation for the field that facilitates the design of future tools which allow the creation of more effective and scalable comparisons. To these ends the team will pursue three interconnected research threads. They will define theories that are grounded upon principles of visual cognition. They will explore case studies (derived from real problems suggested by domain collaborators) that challenge and extend these theories, provide examples for empirical study, and suggest or use general concepts. And they will identify common tasks, designs, and strategies that enable development of generalized techniques, guidelines, and software components. This approach uniquely combines empirical studies, design explorations, and software development to take the field of visual comparisons to a new level that is both rooted in theory yet viable in practice.

Broader Impacts: Because visual comparison plays a key role in diverse domains (including essentially all of the sciences, engineering, and medicine), the potential benefits from an improved science of visual comparison tools are far reaching. To ensure maximum applicability for project outcomes, the PIs are directly collaborating with physical, biological, social, educational, and medical scientists, as well as with engineers and scholars in the humanities. The project will generate visualization tools, software components, and resources for visualization development by others. Visual comparison will serve as a mechanism to expose students at all levels to issues in data understanding. This project will also provide training for visualization specialists, engage non-technical students in visualization, and explore the role of visualization in public outreach efforts.

A team of researchers from Northwestern University, the University of Illinois at Chicago, and the University of California - Santa Barbara will investigate spatial thinking in STEM fields. Students and scientists who are talented in STEM fields also tend to a high capacity for spatial imagination -- they score highly on tasks that ask them to imagine rotations of shapes, or predict how shapes will look when they are folded. But attempts to train these abilities have not translated to substantial improvement in STEM talents. This may be because current training focuses on rote practice, assuming that it is possible to improve the capacity of someone's spatial imagination. In contrast, this may be not possible -- even STEM experts may not have a substantially higher raw capacity for spatial imagination, compared to the average person. The research will test the exciting possibility that their available imagination 'machinery' is similar, but that experts have learned a set of strategies for using that same capacity far more efficiently. The studies will focus on the domain of chemistry, and will ask novices and experts to remember and transform objects that are both unfamiliar (abstract shapes) and familiar (molecules), in experiments designed to unpack the contributions of raw capacity versus a set of predicted strategies. If the studies can isolate the strategies that these STEM experts use to move beyond their capacity limits, then those strategies could be taught in chemistry classrooms. The same principles could extend to other domains as well, such as physics, geoscience, and algebra. This discovery would substantially enhance science and engineering education programs at all levels, strengthening the scientific and engineering research potential of our students. The project is funded by the EHR Core Research (ECR) program, which supports work that advances the fundamental research literature on STEM learning.

Success in STEM is correlated with spatial thinking ability, yet attempts to train spatial ability (e.g., with mental rotation or paper folding tasks) have led to little improvement in STEM outcomes. These spatial training programs may be ineffective because they are based on an impoverished model of the cognitive and visuospatial capacities processes underlying spatial thinking, both generally and in discipline-based education research. The present research will unpack spatial ability into three hypothesized mechanisms, to isolate where training might be best focused, using a set of controlled laboratory tasks that ask novices (undergraduates) and experts to encode and transform both unfamiliar/abstract and molecular stimuli. With chemistry as a case study, this project will unravel the relative contribution of three potential mechanisms for visuospatial representation and transformation: domain-specific chunking (using long-term memory representations of frequently-encountered chunks), domain-general compression skills (recognizing and leveraging redundancies such as repeated identities or planes of symmetry), and raw visuospatial capacity (the ability to store and transform any abstract set of points or shapes). A deeper understanding of the mechanisms involved in spatial thinking would lead directly to better pedagogy and curriculum design for teaching spatial thinking in kindergarten through undergraduate STEM classrooms.

Understanding quantitative data is a foundation of science, education, and the public communication of information about public policy and health. Our brains process and understand numbers far more efficiently when we can rely on data visualizations, allowing us to process patterns in data by leveraging the 40% of our brain that processes visual patterns in the real world. Decades of research in data visualization has produced evidence-backed guidelines for how to design the best data visualization for a given data analysis or communication task. But this process is limited by our incomplete understanding of the process by which we recognize patterns in visualized data. When people see a weather map color-coded by temperature, are they processing the hot and cold colors at the same perceptual moment, or just one? When they inspect a scatterplot, are people processing individual points, or the shape of the whole collection? This project will combine past research in the study of human vision, research in data visualization, and new research at the intersection of those two fields to create a model of how the visual system pulls patterns and statistics from visualized data. This model will lead to a more complete understanding of how to best harness the power of human vision to analyze a given dataset and to communicate a critical pattern clearly to an audience; this model will then be used to improve existing visualization tools.

Data visualization research has sought to find the best visualization for a given data analysis task. For example, scatterplots allow relatively precise judgment of correlations, while line graphs are a powerful way to inspect trends over time. But systematically testing the performance of many tasks across many visualizations has not revealed systematic patterns of performance that would allow us to predict why some matches lead to better performance, what design changes might alter that performance, or how novel visualizations might perform. One problem is that current work is limited to focusing on what viewers want to accomplish, without being able to capture how viewers actually perform these tasks. The goal of the proposed research is to refine and empirically evaluate a lower-level model of "perceptual tasks" that underlie higher level tasks (e.g. "What is the average value in the dataset?") based on established results in perceptual psychology. First, the team will conduct a qualitative study that documents how people break a high-level task down into perceptual tasks, followed by an empirical evaluation of those qualitative findings. Next, the team will measure the precision and operation of the proposed perceptual tasks -- Filter Image, Judge Shape, Compute Distributions and Compute Ratio -- along with other tasks identified in the first study; together, these will provide a set of empirically-backed design guidelines to improve visualization effectiveness. Finally, the team will validate the model by comparing its predictions to findings from previous literature, then integrate new guidelines as constraints into the Draco visualization recommender system, which should improve its ability to predict the performance of different visualization designs. The resulting guidelines, model, and integration into Draco promise in turn to improve visualization education and practice.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

People rely on visualizations to understand and communicate patterns in data, processes in diagrams, or routes within maps, in domains including journalism, education, business, and security. These visualizations are increasingly dynamic, using moving objects or animated patterns to show trends and interactions in the data. In many cases these dynamic displays can help people understand these relationships, but in some cases these dynamic elements can overwhelm people or lead them to incorrect conclusions. Across all of these domains, even expert designers have trouble predicting which displays will work. Through psychology-based experiments and interviews with expert visualization designers, this project will explore the power and limits of dynamic visualization. It will result in a set of guidelines that will enable designers from diverse backgrounds and levels of experience to create more effective displays that lead to better understanding, education, and decisions.

To understand how people process and interpret these dynamic displays, the investigators will catalog an abstracted set of intended uses for animation across data displays (e.g., track a value across an axis change in a graph) by interviewing designers of data displays and validating how well their designs meet their stated goals. In collaboration with these designers, the research team will conduct a series of empirical tests of the power and limits of the human visual system to process the intended patterns, with an initial set of experiments that will test the ability of dynamic visualizations to support viewers in seeing statistics, making comparisons, tracking objects, and drawing attention. The investigators will use these findings to generate a practitioner’s guide for designing effective displays for common goals. In ongoing consultations with our team of designers and advisors, the investigators will incorporate their feedback about (a) whether our abstracted displays, tasks, and measures remain relevant to their in-context case studies, and (b) whether our practitioner’s guide is consistent with their expectations and captures rules that should generalize across most case-study contexts.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.