2002 — 2006 |
Zelinsky, Gregory J |
R01Activity Code Description: To support a discrete, specified, circumscribed project to be performed by the named investigator(s) in an area representing his or her specific interest and competencies. |
Eye Movements During Real-World Visual Search @ State University New York Stony Brook
[unreadable] DESCRIPTION (provided by applicant): Visual search has typically been studied using button-press dependent measures and fairly simple stimuli, methodological choices that have rendered current theories of search poorly equipped to predict the locations and durations of individual search movements to featurally complex real-world objects. In response to these limitations, the goals of the proposed project are to: (1) describe real-world visual search in terms of directly observable and spatio-temporally exact eye movement behavior, and (2) introduce a computational model capable of accommodating real-world oculomotor search. Work on this interdisciplinary project will be accomplished in three phases. Phase I will implement the computational model. Filter-based image processing techniques will be used to represent the real-world search stimuli, and visual routines acting on these representations will endow the model with simulated oculomotor behavior. The visual information available to each of these eye movements will be constrained by a simulated fovea that moves over the scene as the model's "eye" gradually converges on the search target. Phase 2 will apply this behavioral and computational approach to address basic questions regarding real-world oculomotor search (set size effects, target presence/absence, etc). Behavioral studies will determine how people direct their gaze as they search for simple and real-world targets. Computational studies will then input to the model the same search scenes viewed by the human observers and compare the simulated eye movement behavior to the sequence of saccades and fixations obtained from the behavioral studies. Phase 3 will build on the results of Phase 2 by using the gaze patterns predicted by the model to test several novel questions regarding search processes and representations (e.g., What is the role of complex backgrounds in a search task? Should search items be treated as objects or spatially extensive image patches?, What are the relationships between visual search and memory?). Finding spatio-temporal agreement between human and simulated gaze patterns in these studies will not only provide the literature with a validated computational model of oculomotor search, but also open to researchers a (real) world of stimuli to challenge our understanding of visual search behavior.
|
1 |
2009 — 2011 |
Zelinsky, Gregory J |
R01Activity Code Description: To support a discrete, specified, circumscribed project to be performed by the named investigator(s) in an area representing his or her specific interest and competencies. |
Eye Movements During Real-World Visual Search: a Behavioral &Computational Study @ State University New York Stony Brook
DESCRIPTION (provided by applicant): A joint behavioral/modeling approach is used to better understand the top-down constraints that guide overt visual attention in realistic contexts. In previous work we developed a biologically-plausible model of eye movements during search that used oriented and color-selective linear filters, population averaging over time, and an artificial retina to represent stimuli of arbitrary complexity. The simulated fixation-by-fixation behavior of this model compared well to human behavior, using stimuli ranging from Os and Qs to fully realistic scenes. However, this model was limited in that it had to be shown the target's exact appearance, and it could not exploit scene context to constrain attention to likely target locations. Consequently, it is largely unknown how people shift their attention as they look for scene constrained targets or targets that are defined categorically. These limitations are addressed in six studies. Studies 1-2 explore how people use scene context to narrow their search for a specific target in realistic scenes. A text precue provides information about the target's location in relation to a region of the scene ("in the field";Study 1) or a scene landmark ("next to the blue building";Study 2). Behavioral work quantifies the effects of these informational manipulations on search guidance;computational work implements the behavioral scene constraints and integrates them into the existing search model. Studies 3-6 address the relationship between search guidance and the level of detail in a target's description. Study 3 builds on previous work by designating targets either categorically (e.g., "find the teddy bear") or through use of a preview (e.g., a picture of a specific teddy bear), but increases the number of target categories to determine the boundary conditions on categorical search. Study 4 asks whether categorical targets are coded at the basic or subordinate levels, and Study 5 analyzes the distractors fixated during search to determine the features used to code these categorical targets. In Study 6 we use text labels to vary the degree of information in a target precue (e.g., a work boot target might be described as "footwear", a "boot", or a "tan work boot with red laces"). Study 7 describes the sorts of questions that can be asked once scene constraints and categorical target descriptions are integrated under a single theoretical framework, and Study 8 points to an entirely new research direction made possible by the modeling techniques that will be developed for this project. All of these studies are synergistic in that model predictions are used to guide behavioral studies, which in turn produce the data needed to refine the model and to make even more specific behavioral predictions. The project's long term objective is to obtain an understanding of how people allocate their overt visual attention in realistic contexts, specifically in terms of how partial information about an object's location in a scene or its appearance can be used to acquire targets in a search task. This understanding is expressed in the form of a computational model, one that can now use simple spatial relations and the visual features of learned target classes to acquire semantically-defined targets. PUBLIC HEALTH RELEVANCE: The attention system has been implicated in a host of neuropsychological disorders, and a visual search task is a key component in diagnoses of attention deficits. By increasing our understanding of the neuronal computations underlying overt search behavior, the proposed work is relevant to the public health in its potential to improve the validity of existing instruments for diagnosing attention disorders, and ultimately in better understanding these disorders so as to provide more effective treatments.
|
1 |
2012 — 2013 |
Zelinsky, Gregory J |
R01Activity Code Description: To support a discrete, specified, circumscribed project to be performed by the named investigator(s) in an area representing his or her specific interest and competencies. |
Eye Movements During Real-World Visual Search: a Behavioral & Computational Study @ State University New York Stony Brook
PROJECT SUMMARY / ABSTRACT A joint behavioral/modeling approach is used to better understand the top-down constraints that guide overt visual attention in realistic contexts. In previous work we developed a biologically-plausible model of eye movements during search that used oriented and color-selective linear filters, population averaging over time, and an artificial retina to represent stimuli of arbitrary complexity. The simulated fixation-by-fixation behavior of this model compared well to human behavior, using stimuli ranging from Os and Qs to fully realistic scenes. However, this model was limited in that it had to be shown the target's exact appearance, and it could not exploit scene context to constrain attention to likely target locations. Consequently, it is largely unknown how people shift their attention as they look for scene constrained targets or targets that are defined categorically. These limitations are addressed in six studies. Studies 1-2 explore how people use scene context to narrow their search for a specific target in realistic scenes. A text precue provides information about the target's location in relation to a region of the scene (in the field; Study 1) or a scene landmark (next to the blue building; Study 2). Behavioral work quantifies the effects of these informational manipulations on search guidance; computational work implements the behavioral scene constraints and integrates them into the existing search model. Studies 3-6 address the relationship between search guidance and the level of detail in a target's description. Study 3 builds on previous work by designating targets either categorically (e.g., find the teddy bear) or through use of a preview (e.g., a picture of a specific teddy bear), but increases the number of target categories to determine the boundary conditions on categorical search. Study 4 asks whether categorical targets are coded at the basic or subordinate levels, and Study 5 analyzes the distractors fixated during search to determine the features used to code these categorical targets. In Study 6 we use text labels to vary the degree of information in a target precue (e.g., a work boot target might be described as footwear, a boot, or a tan work boot with red laces). Study 7 describes the sorts of questions that can be asked once scene constraints and categorical target descriptions are integrated under a single theoretical framework, and Study 8 points to an entirely new research direction made possible by the modeling techniques that will be developed for this project. All of these studies are synergistic in that model predictions are used to guide behavioral studies, which in turn produce the data needed to refine the model and to make even more specific behavioral predictions. The project's long term objective is to obtain an understanding of how people allocate their overt visual attention in realistic contexts, specifically in terms of how partial information about an object's location in a scene or its appearance can be used to acquire targets in a search task. This understanding is expressed in the form of a computational model, one that can now use simple spatial relations and the visual features of learned target classes to acquire semantically-defined targets.
|
1 |
2019 — 2021 |
Mcpeek, Robert M [⬀] Zelinsky, Gregory J |
R01Activity Code Description: To support a discrete, specified, circumscribed project to be performed by the named investigator(s) in an area representing his or her specific interest and competencies. |
Saccade Target Selection in Naturalistic Visual Search @ State College of Optometry
Project Summary Understanding how we select targets for saccadic eye movements is a basic research question of far-reaching importance. These rapid eye movements allow the fovea to fixate objects of interest, and we typically make more than 170,000 saccades each day. Saccades are essential for efficient visual perception and action: from reading to cooking to driving, most common behaviors heavily engage the saccadic system, and a dysfunctional mechanism for selecting saccade targets would impair performance in all of these everyday activities. The superior colliculus (SC), a key midbrain structure responsible for controlling saccades, is comprised of two subdivisions: the superficial layers (SCs), which respond predominantly to visual stimuli, and the intermediate layers (SCi), which can show both visual and saccade-related responses. Despite its importance, we still know little about how the SC selects saccade targets in realistic conditions. This study will address this critical gap in our knowledge. A major obstacle to progress in this area has been the complexity of analyzing neural responses under naturalistic conditions, and tackling this problem requires a model that can provide testable predictions of neural responses in naturalistic conditions. We will use a state-of-the-art neural model, MASC (Model of Attention in the Superior Colliculus), which incorporates constraints based on SC anatomy and physiology, and does a superior job of predicting saccade endpoints and scanpaths in a variety of search and free-viewing tasks. In conjunction with neural recordings, we will elucidate how superficial- and intermediate-layer SC neurons differ in their integration of activity related to salience, relevance, inhibitory tagging, and movement selection during multi-saccade visual search. In addition, we will test the contributions of the frontal eye field, a cortical area providing input to the SC, to these search-related signals in the SC.
|
0.907 |