Matthew Steely Peterson - US grants

This proposal examines the role that memory plays in visual search. More specifically, the specific aims of our studies include determining under what circumstances memory guides attention and what mechanisms and environmental factors are used in memory-guided search. Our preliminary data suggests that memory guides search when search is forced to be serial and only one item can be examined at a time. In contrast, other research has suggested that when more than one item is examined in a single glance, search becomes memoryless. One way to reconcile these disparate findings is to assume that items examined in a single glance are examined in parallel, and parallel systems are inherently memoryless. Individual fixations or samples occur serially and are guided by memory. When visual span within a glance is reduced to one item, search becomes serial, and hence purely memory-guided. In many situations, search can be accomplished by a series of fixations in which more than one item can examined in each glance. In these situations, search is a mixture of serial and parallel processing, and hence becomes less memory-guided. Our second goal is to determine what mechanisms and environmental factors help guided search. This includes examining the contributions of inhibition of return, scan path planning, the use of landmarks, and the fate of previously examined items. To accomplish these goals, we plan to use to a combination of eye tracking, behavioral measures, and mathematical modeling. These studies are important because they give us a greater understanding of the mechanisms involved in visual search and perception and have the possibility of reconciling the serial/parallel dichotomy.

Preventing social media-based scams is a critical challenge for cybersecurity. There exist tools to protect individuals during online browsing, however, they are not tailored towards vulnerable subpopulations like individuals with developmental disabilities (e.g., Autism). Such individuals become targets without dedicated support to assist with threat identification in potential scam posts. This project aims to understand the distinctive comprehension and attention patterns displayed by individuals with Autism and Attention-Deficit/Hyperactivity Disorder (ADHD), to improve scam detection tools to assist these subpopulations. The project’s novelties include a multidisciplinary approach combining social computing, cognitive psychology, special education, and computational linguistics research to address existing biases in Artificial Intelligence methods of Natural Language Processing (NLP) used in scam detection tools, based on behavioral studies of browsing patterns displayed by vulnerable subpopulations. The project’s broader significance is in integrating insights of human behavior into cybersecurity tools, leading to better protection of vulnerable subpopulations and greater inclusiveness in cybersecurity.

This project pursues two goals. First, it develops an eye-tracking study to discover variations in attention patterns observable across populations with and without developmental disabilities when exposed to scams and legitimate social media posts. Second, it uses observed variations in attention patterns to highlight representation biases in the labeled datasets of NLP-based scam detection models. It further creates a novel set of linguistic attributes that can be used to train scam detection models tailored to aid vulnerable subpopulations. Project outcomes include a better understanding of social media scams for vulnerable subpopulations, the development of an inclusive NLP model for scam detection, and an open-source browser plugin prototype to aid individuals with developmental disabilities via tailored scam alerts. The project also creates a web portal (Inclusive-ScamBuster) hosting labeled scam datasets to highlight representational biases and open-source educational resources to support Special Education programs in teaching and training cybercrime prevention.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.