Ayse Pinar Saygin - US grants

Understanding the movements of others is critical for a wide range of functions such as detecting prey and predators, inferring the goals and intentions of others, and engendering emotional responses, such as feeling compassion and empathy toward others. The investigators will use a range of complementary methods to understand how the human brain detects and interprets the body language of others. For example, body movements can be characterized by very minimalist visual representations, consisting of a dozen points of light attached to the joints on the body seen in darkness (a method used in many motion capture systems). In addition, the investigators will use high-resolution videos to link the work to the field of social cognition by determining how processing is affected when the movements are produced by an artificial agent like a robot, rather than by a human.

Understanding the perceptual and neural basis for body movement processing is essential to an account of how humans negotiate objects and events in the world and can inform fields as wide-ranging as cognitive science, neuroscience, robotics, brain-computer interfaces, social cognition, technology design, visual arts, and computer vision. In addition, social artificial agents such as humanoid robots and virtual animated characters are becoming increasingly common in a range of domains such as education, defense, healthcare and entertainment. This research can help guide the design of these interactive technologies. The investigators also aim to further science and education through recruitment and retention of young people, especially minorities underrepresented in science. Since early engagement is critical for this goal, the focus will be on the development of hands-on research and computation skills for high school and undergraduate students.

The project proposes to create a distributed, multi-user social computing environment that will develop the capabilities of human Electroencephalography (EEG) to analyze users engagement with digitally based experiences. For this project, users will wear non-invasive, EEG headsets while navigating a shared virtual world. Beginning with a handful of EEG systems, the team will scale up over the course of the project to gather signals from dozens of users, providing a basis for larger scale studies. By comparing the EEG signals with each participants activities in virtual world, and with the brain activity and the activities of other users, a model of human brain activity will be developed for different types of behavior profiles and subjective states. This will allow significant improvement for the development of neural markers of human perceptual, cognitive and affective states, the parsing of EEG signals, the applicability of EEG interfaces to new types of experiences, all of which can enhance distance learning, collaborative distributed work, improved mobile computing interfaces and health care applications. The project will advance the capabilities for determining an individuals cognitive state by the creation of new computing methods utilizing comparative EEG analysis and data analysis of event states in a digital simulation. Bringing methods of large scale data analysis to articulate patterns across many users in the situated milieu of the online virtual world will create a new method to utilize EEG analysis to infer human subjective experience. The necessity of conducting this analysis in real-time, with data gathered from distributed, wireless EEG instruments will provide the impetus for utilizing accelerated hybrid multi-core techniques to bear on this domain.

The results from the project will be applicable for a variety of digital environments including computer aided learning and training, digitally mediated collaborative work environments, visualization of complex data sets, and digitally based entertainment experiences such as virtual worlds and computer games. The project will improve the functionality and outcomes of these digital media environments, better adjusting them to the cognitive states of the users. The PIs will train and employ a diverse body of participants to be involved in these activities. As part of an internship program at the Preuss School, a nationally recognized middle and high school on the campus of UCSD, UCSD faculty and staff mentor high school seniors to provide these students valuable experiences in a research laboratory. In addition, UMBC operates the very successful Meyerhoff program for minority students, primarily African-American, for which over 90% of a yearly entrance number of more than fifty go on to graduate school in science or engineering. Many of these students are computer science majors that have had the elective graduate course of Service Oriented Computing for Scientists and Engineers.