Dagmar Sternad, Ph.D. - US grants

Even apparently simple tasks like reaching for an object involve the simultaneous graded activation of a large number of joints and muscles. This project investigates the principles that underlie the generation of such multi-joint movements, and how perceptual information guides these actions. Whereas many of the current theoretical approaches are limited to either rhythmic or discrete movements, this project will develop an integrated framework that encompasses both movement types and therefore can be applied to complex movements. The investigation will proceed in three interrelated directions. First, theoretical work will develop a model for movement generation on the basis of a network of coupled dynamic systems. Second, experiments with human subjects will study the specific features of movement trajectories in tasks involving multi-joint arm movements. Third, the model will be used to synthesize movements of a human-like robot arm in order to compare these model-driven movements with the movements of the human participants. In this way the model's validity and robustness will be tested in a real physical environment. In addition to increasing our basic understanding of complex movement generation the results should be applicable to the control of artificial devices such as robotic systems or limb prostheses, and may also contribute to the diagnosis and treatment of movement disorders.

IBN-9813994 LAY ABSTRACT An International Conference "Progress in Motor Control-II: Structure-Function Relations in the Perceptual Control of Voluntary Movements" is planned for the last three days in August 1999. The Conference will take place in the Nittany Lion Inn, on the campus of the Pennsylvania State University, University Park, PA. This conference will represent the second step along the much-needed route towards integration of information in the area of sensorimotor control. The participation of the spectrum of the invited speakers will make it unique and attractive for potential participants. The Conference will cover the following topics: Neurophysiological Aspects of Sensorimotor Control of Movements; Perceptual Control of Movement; Biomechanics; Motor Disorders; Motor Development; Psychophysiology of Movements; Models and Theories in Sensorimotor Control. All topics will emphasize an integrative approach to the study of adaptive behavior. The focus of this particular meeting will be on the interactions between neurophysiological and biomechanical structures and their sensorimotor functions. The Conference will include 14 invited presentations, free podium presentations, and poster sessions. Invited speakers were selected based on their integrative approach to movement studies, their outstanding contribution to different areas of sensorimotor control, particularly to understanding the role of various neurophysiological structures and of biomechanical factors in the coordination of voluntary movements, and their international reputation. Conference abstracts will be distributed among the participants. Invited speakers will write chapters that will be published as a book by Human Kinetics Publishers.

This research will investigate the generation of perceptually controlled behavior in biological and artificial systems. Its focus is to understand intralimb coordination that consists of both discrete and rhythmic elements, such as in drawing or handwriting. The hypothesis underlying the work is that unconstrained multijoint movements can be understood in terms of two fundamental units of action, discrete movements and rhythmic movements. This "D-R" hypothesis is partially motivated by the fact that, from the perspective of dynamical systems theory, fixed-point and limit cycle dynamics are two primary stable regimes in a complex dynamic system. The research will involve the development of a dynamical model for multijoint movements, consisting of two separate pattern generators that produce rhythmic and discrete movement trajectories. A series of experimental studies will investigate this "D-R" hypothesis in three stages. First, the basic hypothesis that two regimes exist and that they interact will be tested in experiments examining controlled single-joint and two-joint movements that involve both rhythmic and discrete elements. Second, a subset of the same movement tasks will be examined, with additional recording of cerebral blood flow using functional magnetic resonance imaging. The "D-R" hypothesis expects that rhythmic and discrete movements will exhibit different brain activation patterns, and the research will test their interaction. Third, complex unconstrained arm movements will be studied in a three-dimensional drawing task. The behavioral experiments will conclude by testing the modeling propositions in the complex perceptual-motor skill of rhythmically bouncing a ball. Complementing the experiments, the model equations will be implemented on an anthropomorphic robot arm with seven degrees of freedom, in order to synthesize movements on the basis of the proposed organizational dynamics.

This research is fundamental to understanding how humans perform their everyday activities, the vast majority of which involve coordination of multijoint movements with perceptual information. In addition, from the standpoint of complex system theory, the investigation of the human body and its central nervous system, among the most complex of systems, is extremely useful to the goal of understanding the fundamental organizational properties of complex systems. Furthermore, obtaining a deeper understanding of what could be elementary units in the control of perceptuomotor tasks has the potential to advance knowledge for diagnosis and treatment of movement disorders, as well as to advance methods of training and rehabilitation. In addition, the work on the anthropomorphic robot is ideally suited for studying control principles that can be used for the development of new technologies concerning general purpose autonomous movement systems, limb prostheses, and, in the long run, techniques for functional stimulation in patients. The planned combination of fMRI experiments and behavioral experiments will also contribute to bridging psychological and neurobiological disciplines.

DESCRIPTION (provided by applicant): The acquisition of perceptual-motor skills and their adaptation to changing task demands is fundamental to everyday life. Loss of skill and adaptability is detrimental to functioning and is present in many neurological diseases of the sensorimotor system. Hence, further insights into these processes and their rehabilitation is of utmost significance. To elucidate the processes underlying acquisition, adaptation, and control of movements the proposed research tests the hypothesis that the central nervous system is exquisitely sensitive to its own variability and not only reduces unwanted intrinsic noise but has also developed strategies that accommodate and even utilize this noise. This hypothesis rests on two assumptions: first, the sensorimotor system has intrinsic neuromotor noise arising from complex hierarchical processes; second, behavioral tasks are typically redundant and afford many different ways to achieve equivalent task outcomes. Hence, the brain seeks solutions with stability that are robust with respect to its noise. Work of the previous funding cycle established that there are three conceptually distinct routes to achieve task stability: Tolerance: During practice humans explore and traverse the space of solutions in order to find those strategies that are tolerant to error and noise. Covariation: Task redundancy offers solutions where covariation among relevant variables achieves the same result in task performance while allowing variation in individual variables. Noise: When necessary, the amplitude of the random components can be reduced. This three-pronged TCN-distinction presents the quantitative framework to evaluate the hypothesis that in acquiring skilled behavior the central nervous system develops smart solutions that reduce, accommodate, and utilize the inevitable neuromotor noise. Twelve new experiments test this hypothesis and take findings as the platform to design novel intervention techniques. The research is organized into three aims: Experiments under Aim 1 focus on Tolerance and test whether the system seeks solutions that best accommodate for the individual's variability. Conversely, we also test whether manipulating the individual's variability can accelerate adaptation to tolerant solutions. Experiments under Aim 2 examine how Covariation of variables is achieved such that intrinsic noise has minimal effect on the task result. Augmented information is administered to investigate whether the acquisition of such trajectories can be facilitated. Experiments under Aim 3 examine whether intrinsic neuromotor Noise can be reduced by adding extrinsic noise and manipulating error information. The proposed research will be conducted on two tasks in parallel: Skittles, a target-oriented discrete throwing action predominantly under feedforward control, and Ball Bouncing, a continuous perceptually-guided skill of rhythmically hitting a ball. By performing equivalent experimental manipulations to both tasks, we test the generality of our hypothesis that the nervous system accommodates and utilizes intrinsic neuromotor noise in skilled behavior. Results from this quantitative TCN-approach will shed light on the control and acquisition of movement skills in ways that have not been addressed in any other extant research. Importantly, we also make the much-desired transition from new basic insights directly to intervention techniques. We propose three types of interventions that specifically aim to optimize task tolerance, maximize covariation, and reduce noise. We thereby establish the necessary bridge from theoretical concepts to practical techniques that will be applicable to a variety of neurological deficits. While the research proposed here is focused on healthy humans, complementary work is currently under way in close collaboration with Dr. Terence Sanger at Stanford University Medical Center that tests these concepts in children with dyskinetic cerebral palsy. The proposed work on healthy humans is an international collaboration with Dr. Hermann M¿ller at the University of Giessen, Germany, and Dr. Tjeerd Dijkstra at the University of Nijmegen, Netherlands, and will involve student exchanges.

All human behavior entails the continuous interaction between the actor and the environment. This interaction requires an exquisite degree of coordination, often under extremely complex conditions, and we are only just beginning to understand how such coordination is achieved. With funding from the National Science Foundation, Dr. Sternad will examine the performance of a perceptual-motor task in which the complex coordination of actor and environment is distilled in a paradigm suitable for research. The task is the rhythmic bouncing of a ball with a racket. In this task, the actor must continuously coordinate the movements of the racket with the movements of the ball, whose trajectories are in turn determined by the actor. This advantage of this task is that the resulting ball-racket system obeys well-understood physical principles. Moreover, by virtue of being a definable nonlinear dynamical system, it displays characteristic features such as stability and bifurcations. By manipulating the racket, the actor becomes part of this dynamical system and can exploit its properties. Some regimes of the system offer rhythmic solutions with "passive stability", where perturbations converge back onto the attractor without requiring corrective control. Other regimes require active control on the part of actor. Establishing, maintaining, and tuning the regimes of active control rely upon multiple sources of perceptual information. One of the primary aims of this research is to better understand these two general regimes of coordination. The research will also establish national and international collaborations in which both senior investigators and graduate students will participate.

This EArly-concept Grant for Exploratory Research (EAGER) collaborative research project is between an expert in robotics and control theory and an expert in experimental and computational motor neuroscience. It bridges cognitive science, experimental psychology and control engineering. The intellectual premise of the work is that a quantitative theory of human cognition may be built on top of limiting cases of human motor function. This premise lays the foundation for the development of a comprehensive quantitative theory of control-relevant cognition. The result will be an invaluable tool for the human-friendly design of complex motion control systems. Control strategies based on these fundamental objects would be more intuitively understandable by human operators, including prediction of impending failure. Extension of the results beyond motion control provide a new class of knowledge-processing systems capable of more natural interactions with humans.

The objective of this project is to articulate and test a quantitative, control-relevant theory of human cognition, to address a growing divide between cognitive science and control theory. The core hypothesis is that cognitive functions emerged from and are constrained by neural structures used for motor control. Complex motor actions are composed from a limited "library" of dynamic primitives, defined as attractors (e.g. fixed points, limit cycles, etc.). The project postulates that a similar composition of dynamic primitives underlies cognitive processes and that quantitative details may be obtained by re-purposing dynamic primitives found in motor behavior, especially in the manipulation of complex objects such as tools where the link between motor and cognitive function may be strongest. The project is based on a novel series of experiments: A data series is generated by various human participants physically manipulating a complex dynamic object. Alternative data sets are generated by computer simulation of movements to the same targets that minimize mean-squared applied force. Random fluctuations generated by low-pass filtered zero-mean Gaussian white noise and of magnitude comparable to the fluctuations in human performance are added to the simulated force and motion time-series. Without being told the origin, a second set of subjects are presented with the results as evolving abstract time-series and asked to predict their outcome. Subsequently, they are asked to generate a control input for the abstract system, based on their experience, to accomplish a specified task. According to the hypothesis, the subjects will more successfully predict the outcome of human-controlled systems than the synthetic systems, and will generate control inputs that more closely match the human-controlled system inputs.

? DESCRIPTION (provided by applicant): Manipulation of complex objects or tool use is a hallmark of many activities of daily living, but neural control of manual dexterity is still little understood. Even the seemingly simple task of transporting a cup of coffee without spilling creates complex interaction forces that humans need to predict, preempt, and compensate for. Prediction of such complex nonlinear dynamics based on an internal model appears daunting. Hence, this research tests the hypothesis that humans learn strategies that make the interactions predictable. The task of carrying a cup of coffee is modeled with a cart-and-pendulum system that is rendered in a virtual environment and subjects interact with the virtual cup via a robotic manipulandum. To gain insight into human control strategies, this proposal develops three new analysis avenues based on classical linear analysis, information theory, and nonlinear dynamics that operationalize predictability for quantitative theory-based assessment. Aim 1 applies classical frequency response analysis and tests the hypothesis that humans tune into resonance modes as they not only require lower forces, but also more predictable due to lower signal-dependent noise. Three experiments examine transient and steady-state performance with the linear and nonlinear task model. Aim 2 examines tasks with redundancy that offers a manifold of solutions. Predictability is operationalized by the mutual information between the applied force and object dynamics. Three experiments test whether subjects choose those strategies with the highest mutual information. Aim 3 applies contraction analysis, a theoretical framework that examines convergence, or stability, in the state space of the dynamical system. Two experiments examine whether subjects learn solutions that maximize contraction of their trajectories, especially when confronted with perturbations. As manual dexterity is compromised in many individuals with neurological disorders, the experimental paradigm and its analyses promise to become a useful platform to gain insights into neurological diseases, such as dystonia, multiple sclerosis, including aging.

There are many arenas where humans outperform machines. For example, when coordinated interaction with the physical environment is needed, humans (and animals) vastly out-perform modern robots. This occurs despite the biological systems having far slower 'hardware' and 'wetware' and much greater complexity than even the most modern robots. This research project seeks to understand the role of complexity in human sensory and motor performance. Human walking under challenging balance conditions will be studied and the use of canes to enhance stability will be included. The investigators emphasis on learning to balance in challenging environments should lead to new rehabilitation therapies (with or without robotic assistance) to aid recovery of balance and walking (e.g., after stroke). The researchers will create educational units suitable for online presentation to K-12 students and will devise exhibits based on their research for the Museum of Science in Boston.

The central hypothesis to be tested in this project is that complex movements involving physical interaction with objects are organized as a hierarchy formed of modules or primitives. Experiments will study how unimpaired humans learn to walk on narrow beams. Beams of different roundness will vary the challenge. Hand-held canes will alter the available support (like training wheels on a child's bicycle). Computer simulations combined with machine learning will study the benefits and drawbacks of organization as a hierarchy. New mathematical tools will be developed and tested to see if they enable insightful description of human performance in challenging conditions. The research involves a multinational collaboration among scientists from the U.S., Israel, and Germany, each with complementary expertise. The bridge between experimental and theoretical work and the diverse Principal Investigators will help to attract women into the traditionally male-dominated fields of computational neuroscience, robotics and control engineering.

Companion projects are being funded by the Federal Ministry of Education and Research, Germany (BMBF) and the US-Israel Binational Science Foundation (BSF).

Despite vastly slower "hardware" and "wetware," human dexterity vastly out-performs modern robots. This project studies apparently-simple tasks - managing the kinematic constraint on hand motion required to open a door; and dealing with the dynamic complexity of liquid sloshing in a cup of coffee - that profoundly challenge robots but humans perform with ease. The key idea is that humans manage skillful physical interaction with these objects by exploiting clever combinations of primitive dynamic actions that do not require continuous intervention. A novel theory to describe the effectiveness of this approach is developed and tested by experiments with human subjects. The theory is applied to transfer comparable skill to robots, despite manifestly different hardware. If successful, these robots will be more capable, more comprehensible, and more collaborative partners with humans.

The central experimental challenge is to determine the essential strategy underlying humans? remarkable competence in physical interaction tasks. Three hypotheses reflecting major themes in contemporary motor neuroscience are tested: Humans 1) develop models of object dynamics sufficient to pre-compute and execute required hand motions (similar to modern robot programming); 2) choose forces and motions to minimize muscular effort (similar to optimizing efficiency); or 3) exploit dynamic primitives to robustly achieve satisficing (good-enough) performance. The theoretical challenge is to formulate a coherent account combining the information-processing of brains (or computers) with the "energy-processing" of physical objects and their interactions. Classical equivalent circuit theory is re-purposed to define a neo-classical equivalent network theory, combining dynamic motion primitives with mechanical impedances (interactive dynamics). Mechanical impedances enjoy a key property, compositionality, that overcomes the curse of dimensionality.

PROJECT SUMMARY Autism spectrum disorder (ASD) is a developmental disorder associated with a heterogeneous phenotype that includes social, linguistic, cognitive, and behavioral symptoms. Anecdotal reports and selected research results suggest that individuals with ASD exhibit difficulties in motor coordination, especially when interacting with dynamic objects, like catching a ball. Yet, these challenges are not included in the diagnostic criteria. Motivated by a theory recently developed by members of this collaborative team, the overarching hypothesis guiding this project is that individuals with ASD have impairment in prediction when interacting with moving objects and events. Using motor skills that involve interaction with a ball as a first test bed for analysis, three sets of experiments aim to quantify predictive impairments. In addition, fast actions such as ball interception require feedforward control, which relies on internal prediction of limb movements. To test predictive impairments three sets of experiments manipulate time for prediction and coordinative challenges. Aim-1 examines naturalistic catching of a ball using 3D motion capture and electromyography to quantify predictive features of the complex task in ASD and neuro-typical children. Manual catching is compared to catching with a funnel that eliminates hand and finger coordination for the catch. Aim-2 examines ball interaction in a virtual set-up that affords controlled manipulation of the time window for prediction, while simplifying the coordination challenges for the hand movement. Confining ball and hand movements to one dimension, tasks will comprise kinematic and dynamic interception of a ball, the latter requiring prediction of the ball trajectory before and after contact. Computationally advanced metrics of hand movements, postural adjustments and eye kinematics relative to the ball will rigorously test the hypothesis of predictive impairment in autism. Aim-3 tests postural control and uni- and bimanual reaching to assess more elementary motor abilities, such as postural sway, reaction, movement time, and smoothness of hand movements. Potential impairments in these elementary movements will be entered as covariates in the statistical analyses. Both the theoretical framework and the rigorous experimental testing are innovative and promote a hypothesis-driven and unifying understanding of the heterogeneous profile of autism spectrum disorder. The investigator team combines Dr. Sternad's long- standing expertise in computational motor neuroscience at Northeastern University with Dr. Sinha's experience in computational neuroscience of visual cognition and autism, supported by Dr. Kjelgaard at Massachusetts General Hospital with long-standing expertise in autism. Given the safety risks to children with ASD acting in a world of dynamically evolving events, searching for the underpinnings of this pervasive impairment holds great significance. Better understanding of the disorder is relevant for making environments not only safer for autistic children and adults, but also for designing early biomarkers and interventions that address the underlying neuro-cognitive issue, i.e. prediction, and not merely the manifestations of the heterogeneous phenotype.

Human sensorimotor capabilities vastly out-perform those of modern robots. Prior research suggests that humans achieve skillful movement by exploiting combinations of "dynamic primitives", which are robust building blocks of coordination that simplify control. Interaction with complex objects - such as spreading a tablecloth - requires prediction which, in turn, requires mental representations of the objects and environment. This project explores the extent to which such mental models may also take the form of dynamic primitives. The project team will perform fundamental research exploring: how humans learn to manipulate a complex flexible object through the composition of dynamic primitives; the impact of explicit instruction on the acquisition of the mental models; and whether a primitive-based control structure to be implemented in a robot can achieve skillful manipulation of complex objects and fluid interactions between humans and robot. This project serves the national interest because the resulting understanding of human sensorimotor control and robot control methods may result in improved efficacy of robot-assisted physical rehabilitation after neuromotor injuries such as stroke, where safe physical cooperation with humans is fundamental and mutual learning is of paramount importance. The project will involve an educational component that provides engineering and research methods training to graduate and undergraduate students, as well as STEM outreach to individuals of all ages at the Museum of Science in Boston. Additional efforts will be made to attract and retain women into careers in science and engineering.

This research investigates a biomimetic approach to robot control that promise improved human-robot physical interaction during co-manipulation of flexible objects with complex continuum dynamics. Methods include skill acquisition experiments involving expert and novice human subjects to test how mental representations are formed and the extent to which they may be shaped by explicit instruction during training; a theoretical study of how motor learning may be facilitated using mental models based on dynamic primitives vs. lumped mechanical properties; and a study of human-robot co-manipulation of flexible objects wherein dynamic primitives provide the basis not only for robotic control, but also for communication and mutual learning between the human and robot partners. This research promises new insights into the manipulation of complex objects and tools, an area where humans still outperform machines.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

How can a basketball player reliably land his free throw in the basket, and yet still miss occasionally under nominally identical circumstances? While such skills are a paragon of motor expertise, even seemingly mundane actions also require surprising dexterity. When carrying a full cup of coffee, we exhibit motor skill that is far beyond what is typically studied in the laboratory. Specifically, when interacting with objects - the essence of any tool use -, successful actions require fine-grained control of interaction forces that have been beyond the purview of neuroscience to date. The proposed research examines the neural basis of motor expertise by bringing rich interactive tasks into the laboratory. The two PIs combine their long-standing experience in computational motor control and neurophysiology to study novel behavioral paradigms both in humans and non-human primates. Building on conceptual and computational overlap in their respective research, where skill is associated with low-dimensional structure in high-dimensional neural and behavioral redundant spaces, they will test the overall hypothesis that patterns of neural activity exhibit many of the characteristics of the behavior. Two aims will study two examples of motor skill: throwing an object and transporting an object with internal dynamics, both rendered in virtual environments. Parallel experiments in humans and primates will generate rich behavioral data that will be matched with intracortical recordings in the cerebral cortex of non-human primates. To date, non-human primate studies have necessitated that animals perform near-identical repetitions of simple behaviors to facilitate the analysis of neural activity. Now, modern multi-neuronal recording techniques make it possible to embrace more sophisticated real-world behaviors and address core principles of movement discovered in human motor control: high dimensionality, redundancy, and the ever-present variability. This research will develop a suite of computational tools that afford the analysis of behavioral and neural data with commensurate techniques and sophistication. This research will be transformative as it advances the motor challenges examined and brings insights from intracortical neurophysiology closer to understanding of human motor expertise. These scientific insights will channel into a large range of outreach activities to achieve broader impacts for the general public. RELEVANCE (See instructions): Patients with neurological disorders such as stroke face challenges in their daily activities, grasping a cup to bring to their mouths to drink; these actions are essentially interactive tool use. This research seeks insights into neural activation during such skilled actions and interactions to get closer to understand neural activity in tasks relevant in real life. Extending from PI Batista?s experience, neuroprosthetics and brain-machine interfaces are direct clinical application that may benefit from our findings and recovery

Each year in the United States, one in ten live births or roughly 380,000 babies are born prematurely, with a mortality of 27%. Nearly one third of those surviving suffer from lifelong neurological conditions including cerebral palsy, autism, and psychiatric disorders with substantial personal and societal costs. Despite advances in neonatal intensive care, little progress has been made in monitoring maturation of neurological function in preterm infants. This proposal addresses an urgent need for continuous monitoring of neurological function in the neonatal intensive care unit to advance our understanding of normal and abnormal neurological maturation and to develop timely clinical interventions to improve the health outcomes and reduce costs of prematurity.

The research program aims to show that bodily movements of infants are a vital marker for the functional integrity of the nervous system. In Objective 1 the team will quantify movement bouts in routinely obtained data sets over ~3 months, from birth (<29 weeks gestational age) until discharge from the neonatal intensive care unit (NICU). Pursuing the hypothesis that complexity of movement patterns increases with age as a sign of maturation, they will examine the evolving entropy of movement signals over several months. Objective 2 will quantify the strong physiological interactions of movement bouts with the respiratory and cardiac rhythms, and correlate movement to apneic events and cardiorespiratory instability. Objective 3 seeks to develop machine learning algorithms using features of movement and other physiological events for online prediction of apneic episodes. In return, the algorithms will reveal physiological features and mechanisms that enhance clinical insights. Preliminary data promise results that support our thesis that movement in newborn infants is an important vital sign. Taken together, this quantitative approach will advance understanding maturation of the neonatal nervous system and how movement can stabilize or disrupt cardiorespiratory control in neonates. The team has also identified multiple outreach activities to foster the integration of scientific research, technology and STEM education, including a workshop on Movement Science, Data, and Technology, in collaboration with a minority-serving institution to increase minority representation in the movement, data, and computational sciences.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.