Brian J. Scholl - US grants

Among the most crucial tasks of vision science is to determine the nature of the basic units over which visual processes operate. Much recent evidence has accumulated that processes of visual attention can in some cases operate over visual objects. The goal of the proposed research is to determine the nature of these visual objects, moving beyond the simple stimuli (static 2D dots, bars, and boxes) typically used in previous research. What is an 'object', in the context of 'object based visual attention'? Several experimental techniques (including change detection, multi-element tracking, spatial cueing, and eye- movement monitoring) will be employed with a broader and more naturalistic set of stimuli (focusing on perceptual groups, rendered 3D volumes, and dynamic 'object schemas'), in the context of various dynamic behaviors (including various degrees of spatiotemporally-continuous motion). This research will uncover the nature of the basic units of visual attentional processing in normal subjects, and will shed light on the nature of various object-based disorders, such as unilateral neglect and Balint syndrome.

In addition to perceiving the colors, shapes, and motions of objects in the world, we can also perceive higher-level properties such as causality, as when we see one object hit another object, causing it to move. Properties such as causality go beyond the objective kinematics of the displays, yet can be reliably perceived even in simple scenes consisting of moving 2D geometric shapes. Causal perception of this sort is ubiquitous in everyday life, from billiard balls to traffic collisions, but is not understood as well as the perception of properties such as color and shape. The goal of the proposed research is to explore the nature of causal event perception, focusing on the perception of causality in 'launching' displays. Computer- based experiments with normal adult observers are proposed to investigate three primary questions. First, to what degree is attention required to perceive such events? Second, to what degree do such events actually mediate the capture events? Third, how does the visual system use contextual information when detecting causal events? These questions are addressed using a variety of experimental tools, including variants of existing paradigms (visual search tasks with repeating dynamic events; sustained inattentional blindness' tasks) as well as novel paradigms developed to study the role of contextual information from other nearby events. This new research will help to determine the nature of such perceptual processing, which suggests new ways to think about the purpose of visual perception: just as the visual system works to recover the physical structure of the world by 'inferring' properties such as 3D shape, so too might it work to recover the causal structure of the world by 'inferring' properties such as causality.

One of the central lessons we have learned from the emerging discipline of cognitive science is that most seemingly effortless mental tasks, for instance recognizing faces and understanding speech, are in fact the result of incredibly complex feats of cognitive processing. Our minds not only make such accomplishments possible, but they do so in a seemingly effortless way that hides their complexity and difficulty from us. This research project will focus on the cognitive processing responsible for one such seemingly obvious aspect of our mental lives, our intuitive ability to perceive the world in terms of discrete objects (such as people, cars, and chairs). This ability seems somehow obvious and necessary to us; after all, the world is populated by such discrete objects. In fact, however, interpreting the world in terms of discrete objects is a Herculean task for the brain, since the incoming visual information consists of an undivided wash of colors, shapes, and motions. Our brains must grapple with such information, and through great efforts turn it into the orderly scenes of discrete objects that we perceive during almost every moment of our lives. This cognitive processing is critical to our lives, as it makes our visual experience coherent. This research project will explore how these "object cognition" processes work, answering a number of important and specific questions: What information does the brain use to infer the existence of a discrete object in a visual scene? What information is used to determine whether a briefly glimpsed object is in fact the same object that was viewed earlier? What are the limits on such abilities? The answers to such questions will help us understand how the brain accomplishes some of its most important but least appreciated tasks. This research will also address other critical questions: Where do these abilities come from? Are they hardwired into our visual systems from birth, or are they learned gradually? Are the cognitive processes which young infants use to make sense of the world in terms of discrete objects the same processes that guide our mature perception of objects in the world? Previous research on "object cognition" has studied both infants and adults, but these investigations have typically proceeded completely independently, in different sub-fields of cognitive science. This research will bridge this gap, by comparing object cognition in infants and adults, and by directly testing the hypothesis that researchers in these two areas have been exploring the same underlying brain mechanisms. This research thus has the potential to unite two previously independent areas of cognitive science, while pursuing the main goal of understanding the cognitive processing which provides us with the coherent perception of a world filled with discrete persisting objects, which we rely on but take for granted during almost every waking moment. In addition, this research on object cognition in normal adults and infants has implications for better understanding of, and perhaps improved treatment for, certain object-based visual disorders in impaired patient populations (such as "Balint Syndrome"). In addition, it may be useful in the development of artificial systems that implement similar processes for dividing visual scenes into discrete objects and tracking them over time and motion.

People recognize dramatic situations and attribute roles and intentions to perceived characters, even when presented with extremely simple cues. As any cartoon viewer can attest, two animated shapes are sufficient to describe a scene involving tender lovers, brutal bullies, tense confrontations and hair-raising escapes. These basic notions of agency and intentionality are foundational to our social perception of the world. They provide the first discriminations between agents and objects, delineate which elements of the world can move with goal-directed purpose, and provide the primitive structure for describing cause and effect. Extensive laboratory experiments have described many of the basic properties that produce these perceptions on controlled stimuli. However there have been only limited attempts to quantify these processes and no attempts to see if these same properties hold on real-world activity patterns.

This project models our human ability to perceive agency, intentionality, and goal-directed behavior in dynamic real-world environments. Using off-the-shelf real-time localization systems, the movements of people and objects are recorded as they engage in unstructured activity and staged group games. Drawing on both this empirical data and theories drawn from the psychophysical data, computational models are constructed that quantify, explain, and predict real-world social and goal-directed behavior. The benefits of this work include: (1) modeling tools for use within behavioral studies, (2) a real-world grounding for psychophysical studies, and (3) a computational model of social and intentional behavior that would enhance human-computer and human-robot interfaces.