Jonathan Flombaum, Ph.D. - US grants

[unreadable] DESCRIPTION (provided by applicant): We open our eyes and we instantly perceive a rich and stable world. This happens so effortlessly that we often take for granted the challenges that our visual systems face in piecing together what we see from the light that lands on our retinas. Visual attention and visual working memory are crucial resources without which this process would be impossible. So much information enters our eyes that the visual system would get nowhere if it tried to analyze it all. Attention and memory preclude this kind of information overflow by processing and storing only relevant information. But how do attention and memory know exactly which bits of the world are the relevant ones? Part of the answer is that they operate over important units in visual experience, and a central project in cognitive science is to determine what are the units that attention and memory select. This important question has received a significant amount of research, and current evidence suggests that, at least in some situations, attention and memory operate over bound object representations. They are object-based. However, most research has focused on the static nature of objects, typically contrasting bound objects with unbound features and locations in space. This neglects the intrinsically dynamic nature of objects in real-world experience. The proposed research seeks to expand our conception of what it means to be an object to include the important fact that objects maintain their identities over time despite motion, occlusion, and changes in appearance. Thus the proposed research explores the rules and mechanisms that support the construction and maintenance of persisting object representations. Moreover, the proposed experiments explore persisting object representations by focusing on their consequences for attention and memory. This reflects a new way of characterizing these resources. Rather than asking independent questions about the underlying nature of attention and memory, the proposed project looks at both of these crucial resources from a unifying perspective. Thus a central aim of this project is to demonstrate that representations of persisting objects comprise a piece of the underlying vocabulary of cognition, ultimately guiding how we attend to the world, how we remember it, and even how we reason about it. The results of these experiments will have implications for our understanding of disorders where basic resources such as attention and memory are impaired (e.g. ADHD, Alzheimer's disease). These results will also have implications for the effective presentation of information, especially in the design of interventions for impaired populations and in the design of interfaces for cognitively demanding situations (e.g. air-traffic control). [unreadable] [unreadable] [unreadable]

Visual information is initially received by the eyes; however, the experience of vision itself is the result of complex computations carried out deep within the brain. Though many aspects of human vision are well-understood, the mechanisms by which the visual system maps recent memory (what happened a moment ago) onto incoming visual signals (what is happening now) remain a critical question for cognitive science. As events unfold in real time, there needs to be a mechanism for continually linking the current visual information with what just happened. For example, tracking a moving object requires mapping very recent visual memory signals onto the visual signals currently entering from the retina and tying this information together. The same holds true when trying to detect changes in a scene, or when trying to remember the positions of groups of objects, or even when trying to link landmarks on a map to landmarks in real space. This research project aims to characterize the correspondence mechanisms that enable this mapping between the recent and the current. Understanding how the human visual system accomplishes correspondence mappings is critical for understanding complex visual activities and the development of specialized visual skills such those involved in driving, radar control, video analysis, satallite imagery analysis, and baggage screening. Understanding correspondence mechanisms could also inform the design of artificial vision systems.

The research achieves its goals by combining eye tracking methodology and probabilistic models derived from computer vision algorithms, along with behavioral tasks that engage motion tracking, spatial working memory, and visual working memory. Eye tracking is a crucial component of the project because, in humans, the quality of received visual signals depends heavily on a source's distance from an observer's fixation (eccentricity). The research project therefore begins with experiments that compare observers' performance as a function of their fixations as well as simulations by computational models that adopt those empirically obtained fixations. Subsequent experiments then investigate methods for facilitating and training fixation to improve observer performance. A major implication of the research is that fixation selection places tremendous constraints on the accuracy of probabilistic correspondence algorithms, and as a result, on the ability to effectively obtain, store, and retrieve visual information.