Paul Schrater - US grants

This project investigates two problems associated with the monitoring of human activities: The first problem is tracking of articulated motion as a whole without identifying individual limb motion. The goal is to address certain shortcomings in previous solutions to this problem, the main shortcoming being their over-constrained nature. The proposed solution, which is presented as a real-time human tracking
system, will be capable of working under many difficult circumstances. The second problem is recognition of
articulated motion. The goal here is to show that the recovery of three-dimensional properties of the object or even two-dimensional tracking of the object parts are not necessary steps that must precede action recognition. The proposed approach uses motion features only. Unlike other similar approaches, the motion
features will be used in such a way to represent complex and long actions as well as to distinguish different actions with many similarities. Each action is represented as a manifold in the lower dimension space and matching is done by comparing these manifolds. As part of a homeland security scenario, its is planned to use these methods to monitor outdoor human activities based on the ability to recognize, for example, that a human runs in the opposite direction that a crowd moves.

Our long-term goal is to understand how humans perform natural tasks given realistic visual input. Object perception is critical for the everyday tasks of recognition, planning, and motor actions. Through vision, we infer intrinsic properties of objects, including their shapes, sizes, materials, as well as their identities. We also infer their depths and movement relationships to each other and ourselves, as well as determine how to use this information. The remarkable fact is that the human visual system provides a high level of functionality despite complex and objectively ambiguous retinal input. Current machine vision systems do not come close to normal human visual competence. In contrast, our daily visual judgments are unambiguous, and our actions are reliable. How is this accomplished? Our conceptual approach to this question is motivated by our previous work on object perception as Bayesian statistical inference, and its implications for how human perception gathers and integrates information about scenes and objects to reduce uncertainty, resolve ambiguity and achieve action goals. Our experimental approach to this question grows out of our team's past accomplishments in using behavioral techniques such as interocular suppression, high-field functional magnetic resonance imaging and analysis, and Bayesian observer analysis of human behavioral performance. We combine our conceptual and experimental approaches to address a new set of questions. In three series of experiments, we aim to better understand: 1) the relationship between cortical activity and the perceptual organization of image features into unambiguous object properties and structures (Within-object interactions);2) how visual information about other objects and surfaces reduces uncertainty about the representation of an object's properties and depth relations (Between-object interactions);and 3) whether and how information and uncertainty may be processed differently depending on the viewer-object interactions demanded by task, as predicted by theory (Viewer-object interactions).

The research objective of this award is to reverse-engineer human spatial control skills, with the goal of bridging the gap between human and autonomous systems. Adaptive and versatile spatial control capabilities are essential to successfully deploying unmanned aerial vehicles and other applications of autonomous control. understanding how humans achieve these skills is also important for the design of man-machine systems, such as active safety systems for helicopters or even tele-surgery systems. The research approach combines psycho-behavioral experiments and the application of control-theoretic principles. The working hypothesis is that human spatial behavior is determined by combining a type of model predictive control (MPC) process and a spatial value function (SVF). The MPC process describes how a trajectory is generated online,based on the immediate sensed environment; the SVF describes how the global environment and task elements are encoded and used in the MPC process. This model will then be used to study how the pilot adapts his/her strategies in the presence of disturbances, environment ncertainties, or when presented with novel ituations. Deliverables include the details of the odeling framework, documentation of research results, and outcomes of interdisciplinary student education and research experience.

If successful, this research will lead to developing new algorithms to model and replicate human daptability in spatial control tasks, paving the way for novel technologies for autonomous vehicle ontrol with unprecedented performance and adaptability. The improved understanding of human uidance skills will provide a gateway to new operator interfaces, augmenting human unctionality, effectiveness, and safety. Furthermore, the modeling framework will provide
foundations for neurological studies, aimed at understanding the brain?s implementation of the patial control processes. Finally, the proposed interdisciplinary research activities will promote ew teaching and outreach activities, while attracting students from diverse backgrounds.

? DESCRIPTION (provided by applicant): Major breakthroughs in neuroscience have been achieved through the application of computational models to empirical research. Models are essential to connect theory to behavior and the increasingly rich and complex measures of nervous function at multiple spatial and temporal scales. That said, modeling is a highly complex activity requiring extensive training and multiple skills sets, which has created a critica shortfall in the cadre of researchers with the requisite skills to meet the modeling needs in computational neuroscience. The goal of the Summer School in Computational Sensory-Motor Neuroscience (CoSMo) is to provide cross-disciplinary training in mathematical modeling techniques relevant to understanding brain function, dysfunction and treatment. In a unique approach bridging experimental research, clinical pathology, cutting-edge technology and computer simulations, students will learn how to translate ideas and empirical findings into mathematical models. Students will gain a profound understanding of the brain's working principles and diseases using advanced modeling techniques in hands-on simulations of models during tutored sessions. This deep brain camp aims at propelling promising students into world-class researchers. Sensory and movement research form both a key paradigm in brain research and drive progress in many clinical areas related to disease and dysfunction. It is a mature area with a long history of achievements in developing, testing, and integrating experimental, neurobiological, neurotechnological and a rich array of computational modeling successes to understanding the brain. While many summer schools exist in related disciplines, CoSMo is the only summer school focusing on this exciting multidisciplinary area. It also has a unique pedagogical format that coherently spans hands-on model development, modeling methods, and integrating modeling with experiments, data analysis and clinical applications. CoSMo thus fills an important gap and teaches computational, experimental and clinical knowledge through combined empirical-theoretical teaching modules. Relevance: Participants also learn how to apply concepts to clinical pathologies using computational modeling, which results in practical and transferable skills. By developing this missing piece in the current training environment, we are accelerating progress of crucial basic and medical importance. We will train students and postdocs to use the power of computational and experimental frameworks to understand brain dysfunction. This exceptional theory-based translational component will put our students at the forefront of innovation in basic and applied brain science.