Jeremy M. Wolfe - US grants

A frequent consequence of strabismus and amblyopia is a loss of normal binocular functions. Under binocular viewing conditions, many individuals with these disorders are essentially monocular. Attempts to improve or restore binocular function in such individuals are hampered by chronic suppression of the weaker eye. Preliminary investigations (Nei R03-04297-01) have uncovered a way to disrupt suppression in normal and binocularly abnormal individuals. Suppression, in the form of binocular rivalry, is produced whenever different stimuli are presented to corresponding points in each eye. At any instant, a normal observer will see only one of the monocular stimuli at each location in the visual field. Observers with chronic suppression will see only the stimulus in the dominant eye. However, when briefly or intermittently presented, dichoptic stimuli appear to fuse. I have discovered that this 'abnormal fusion' is seen by many stereoblind subjects. The processes underlying this abnormal fusion are not understood and the potential uses of abnormal fusion in the treatment of impaired binocular vision have not been explored. The proposed research has four aims: A) to describe the process responsible for abnormal fusion in normal subjects, B) to describe the function of that process in normal vision, C) to examine changes in that process accompanying the binocular abnormalities that lead to stereoblindness and chronic suppression, and D) to determine if we can use knowledge about such a process to enhance the binocular capabilities of binocularly impaired individuals.

We are studying the integration of multiple, parallel sources of information in the human visual system. Early stages of visual processing consist of a set of semi-autonomous modules processing different basic features of the input in parallel across the visual field (e.g. color, orientation ,size, motion). Later stages allow for more sophisticated processing but only in a restricted region of the field at any moment in time (serial processing). In this proposal, the integration of information from multiple parallel modules is studied within the visual search paradigm. Visual search tasks involve the search for a target item among distractor items (e.g. find the red spot among spots of other colors). Of particular interest are conjunction searches where the target is defined by two or more simple features. In these searches, our data show that parallel modules are able to guide subsequent processing. For example, in a search for a red vertical item among green vertical and red horizontal items, a module processing color in parallel across the visual field can guide attention toward red items while an orientation module can guide attention toward vertical items. The combination of these two sources of guidance leads to an efficient search for a conjunction of color and orientation even though no single module processes this conjunction in parallel. This concept of parallel guidance of serial processing is at the heart of our "Guided Search" model (Wolfe, Cave, and Franzel, 1989; Cave and Wolfe, in press). Our aims for the next grant period are: 1) to study guidance using a new method that allows some tracking of the movements of attention during visual search, 2) To examine the role of size in visual search, 3) To investigate the division of complex scenes into the "items" used in visual search, and 4) To determine if a single parallel module can be queried about the presence of more than one attribute at one time (e.g. Can the color module be asked simultaneously about "red" and "green" items?).

[unreadable] DESCRIPTION (provided by applicant): This proposal addresses a fundamental tension in current work on vision, visual attention, & visual memory. On the one hand, we can recognize visual stimuli in a fraction of a second and, with slightly longer presentations, such stimuli can be remembered for days. On the other hand, phenomena like "change blindness" and "inattentional blindness" seem to show striking failures of visual representation and/or memory. Observers fail to report apparently obvious aspects of visual displays. In our own work on "postattentive vision" (the subject of the previous grant period), Ss searched repeatedly through static, unchanging displays. Hundreds of trials did not improve the efficiency of Repeated Search even though Ss had clearly memorized the display. The goal of the present proposal is to reconcile these disparate streams of research. How can an observer recognize a scene 24 hours after a one second presentation and yet be unaware if an object in that scene disintegrates before his eyes? There are three specific aims: [unreadable] Why do subjects choose vision over memory in Repeated Search? We will test three hypotheses about the cause of this apparent reliance on vision. A) Vision First: When available vision always takes priority, B) Inefficient Memory: Ss do, in fact, search memory but that search is as inefficient as visual search, or C) Pragmatic Choice: The costs, even of efficient memory search, make inefficient visual search worthwhile. [unreadable] When does memory contribute to visual search? In the "Contextual Cueing" paradigm of Chun and Jiang (1998), memory for repeated exposure to a search stimulus does aid subsequent search. We examine the relationship of Contextual Cueing to Repeated Search. We hypothesize that these are complementary, not contradictory findings. [unreadable] What is the gist of a picture? In Picture Memory tasks, Ss are said to remember the "gist" of an image. We propose that this gist includes the distribution of basic features like color and size and a rapid calculation of the structure or "spatial envelope" (Oliva & Torralba, 2001) of the scene. Gist also includes a limited number of attended objects. We will use several converging experimental paradigms to assess these components of the gist of a scene. Taken together, the results of this program of research will test hypotheses that tie our understanding of visual search to our understanding of short and longer term visual memory. [unreadable] [unreadable]

DESCRIPTION (Applicant's Abstract): The first steps in the perception of a novel scene can be described by a two-stage model with preattentive and attentive representations of visual stimuli. First, preattentive processes begin to analyze basic feature information at all locations. Prior research suggests that this involves the parallel processing of about a dozen basic features (color, size, etc.). Other tasks like object recognition of reading can be performed only with attention to the specific objects. The central question of this proposal is: What happens to the visual representation of an object after attention moves away? An observer looks at a fruit bowl. Preattentively, the stimulus is represented as a collection of colors, shapes, etc. With attention, the observer identifies bananas or grapes. Next, while viewing the same scene, the observer attends elsewhere. What is the nature of the post-attentive visual representation of the fruit? Do the effects of attention persist or does the visual representation revert to its preattentive state? Though post-attentive vision has received little study, it is central to an understanding of perception because, during sustained viewing of a scene, most visual experience must be post attentive. Post-attentive vision can be studied with several experimental methods: One is the repeated search paradigm in which a single search display is searched several times. For example, the letters "D, K, V & U" might appear on the screen. A probe letter would appear and the observer would respond "yes" if it were D, K, V or U or "no" if it were not. In repeated search, an observer searches a single display several times for different targets. Pilot experiments find little or no change in response time as a function of repeated search. This is a striking result because it suggests that repeated deployments of attention do not alter the visual representation in any way that can aid subsequent search. In a second, curve tracing paradigm, Ss must determine if two dots lie on the same of different curves. Reaction time in this task is dependent on the distance along the curve between the dots. As in repeated search, repeated curve tracing of the same curves does not improve performance. These and other results can be understood in the context of an "inattentional amnesia" hypothesis that holds that the visual representation has no memory. What is seen is seen in the present tense. Four lines of experiments are proposed to test the inattentional amnesia hypothesis and to expand the empirical basis for the study of post-attentive vision. These are studies of 1) basic properties of repeated search, 2) basic properties of repeated curve tracing, 3) the relationship of post-attentive vision and memory, and 4) the persistent effects of attention.

DESCRIPTION (provided by applicant): Consider looking at a bone scan or a CT to determine if a patient is responding to cancer treatment. Or, as a less life-and-death case, consider searching through a child's toy box for all the Lego pieces that can be used to build a roof on today's project. These are complex, extended visual searches for multiple instances of multiple target types, be they metastases of cancer or building blocks of a particular shape. How do humans perform such tasks and how can we help experts to perform more effectively the critical extended search tasks that our civilization has created in the clinic, the airport, and elsewhere? The past 25+ years have seen great progress in understanding the basic process of visual selective attention. That said, the bulk of the existing work involves brief episodes of search (seconds) for a single instance of a single class of target (e.g. find a red vertical line or this specific object). Typically, that target will be present on 50% or more of trials. In order to understand and improve upon socially important search tasks in radiology and elsewhere, we must investigate extended visual search tasks. The proposed work exploits three extended search paradigms, largely new to the study of human search. Foraging tasks are laboratory analogs of picking berries in a field or lung nodules in a chest CT. Observers try to select as many target items as possible in a fixed period of time. To model this behavior, the Guided Search (GS) model is augmented with ideas from the literature on animal foraging as well as decision theory and neuroeconomics. Hybrid Search tasks involve searching visual displays for any of several possible types of target (think of a shopping list or a checklist of signs of disease). These are combinations of visual search and memory search. It is possible to search for literally hundreds of distinct objects, held in memory. To model this behavior, ideas from the study of working memory and long-term memory (LTM) are brought into the GS framework. Finally, Hybrid Foraging tasks combine the demands of the foraging and hybrid tasks. Here, observers attempt to collect as many examples as possible of multiple types of targets. We use each of these methods in laboratory studies with non-expert observers and in experiments with expert radiologists. We seek to describe and model the fundamental processes of extended search and to improve the ability of radiologists and other experts to perform such searches. Of particular interest are searches for rare, low prevalence targets (e.g. cancer in a breast cancer screening population appears in about 0.3% of exams). False negative, miss errors are elevated at low prevalence in clinical settings. The proposed work suggests techniques for ameliorating this effect and examines the impact of prevalence in extended search tasks. In summary, this work develops an interdisciplinary extension of GS, applied to extended search tasks, with the ultimate goal of improving performance on the critical search tasks found in radiology and other socially important settings.

Project Summary Breast cancer screening is an important part of routine medical care for women around the world. The modern standard screening method is Full Field Digital Mammography (FFDM) in which two- dimensional, digital x-ray images are created of the entire thickness of the breast. Digital Breast Tomosynthesis (DBT) is an important advance that creates a 3D volume of images, consisting of a set of x-ray slices through the breast. DBT has been shown to improve accuracy - both the sensitivity and specificity - of breast cancer screening over FFDM alone, but clinical adoption has been relatively slow, in part because exams with DBT take significantly longer to interpret than do FFDM alone. The proposed studies will use medical image perception experiments to improve search strategies for DBT images, with the goal of increasing the speed and accuracy of breast cancer screening. There are three aims. Aim 1: To examine the pattern of eye movements and their relationship to errors when radiologists read DBT images in order to determine which search strategies are significantly faster and/or more accurate than others. Eye movements will be measured while three groups of radiologists read DBT: radiologists prior to DBT training, just after DBT training, and DBT experts. These data will reveal whether the strategies, taught by experts, are the same as the strategies used by experts in practice and whether, after training, trainees do what the experts recommend. Aim 2: Typically, a radiologist would use the FFDM image to guide subsequent search of DBT images. In Aim 2, this will be reversed. The hypothesis is that an initial preview of a DBT ?movie? will improve speed and/or accuracy of breast cancer screening by guiding the reader to important loci in the FFDM image and by aiding further examination of the DBT images. Aim 3: Compared to FFDM, DBT greatly increases the number of images that are available to be searched. It would be extremely time consuming and unnecessary to scrutinize every pixel of every slice. How many regions does a reader need to fixate upon in order to be reasonably sure that some sign of cancer, if present, will attract attention? To answer this question, we need to know the distance away from fixation at which different signs of cancer can be detected. A combination of studies with expert and non-expert observers will enable us to measure these distances. Combined with the eye tracking data from Aim 1, these data can be used to determine if error rates can be reduced by training radiologists to use specific search patterns. This program of research will yield low-cost interventions that can improve the use of DBT and, thus, enhance the life-saving effectiveness of breast cancer screening

Low prevalence searches form an important and problematic class of visual search tasks. These are tasks where the search target is rare. Many socially important tasks like airport security or cancer screening are low prevalence tasks. Previous work, much of it from our lab, has shown that low prevalence can have undesirable effects. Most notably, miss (false negative) errors are markedly elevated at low prevalence. This is a clear problem if the purpose of the search is to detect something rare but important like cancer or a terrorist threat. Our previous work has documented this pattern of increased miss errors in a number of expert domains including cytology (cervical cancer screening), airport baggage screening, and breast cancer screening. False alarm (false positive) error rates typically decline at low prevalence, moving in the opposite direction from miss errors. This indicates a shift in the observer?s decision criterion. At low prevalence, observers become more reluctant to call something a target. Several studies ? ours and others - have shown that this ?conservative? criterion shift is not adequate to explain the entire prevalence effect. Wolfe and VanWert (2010) developed a ?Dual- Threshold? model that better captures the important aspects of the prevalence effect data by proposing two effects of low prevalence: (1) the conservative shift in the criterion for deciding if an attended item is a target, and (2) a lowering of the ?quitting threshold.? The quitting threshold determines when observers end a search. Quitting too soon also increases the chance that the observer will miss a target. Prevalence effects have been studied in experimental isolation from other aspects of search. However, in tasks like breast cancer screening, other factors interact with prevalence. The four projects in the present proposal each investigate one of these interactions. Project 1 examines the relationship of prevalence to the ?vigilance decrements? that are seen as time elapses in a task. In search, observers must maintain an internal, mental representation of the search target (or targets). Project 2 is concerned with the impact of prevalence on these ?target templates?. Advances in artificial intelligence (notably deep learning) are producing tools to assist expert searchers. However, once deployed, these AI tools have been less effective than theory predicts. Project 3 tests the hypothesis that part of the problem is another side-effect of low prevalence and the project tests a potential intervention. Finally, clinicians, searching for one type of target (e.g. pneumonia) are supposed to report signs of other possible problems (e.g. lung cancer). Project 4 probes the role of prevalence in the failure to report such ?incidental findings?. Again, we test several interventions. This is ?use-inspired, basic research? whose results will provide guidance for experts performing socially important low prevalence tasks.