Francisco J. Valero-Cuevas - US grants

0237258
Valero-Cuevas
This five-year CAREER Development project establishes the foundation for interdisciplinary education and research in neuromuscular biomechanics at Cornell University. The project develops two objectives: (a) uncover the neuromuscular biomechanics of dexterous manipulation; and (b) integrate engineering and neuroscience into an interdisciplinary educational network.

Dexterity is defined in the engineering sense of being able to perform stable dynamic manipulation. Research objectives focus on using engineering science to rigorously characterize dexterous manipulation, and to distinguish between the relative contributions of passive and active biological elements of the hand to stabilize manipulation. To achieve these goals, an integrative and interdisciplinary approach will combine nonlinear dynamics, robotics, biomechanics, and neurophysiology in a unique manner. Three specific investigations are proposed: (1) Analyze human dexterous manipulation experimentally using bifurcation theory. (2) Characterize brain and muscle activity during dexterous manipulation using functional MRI and electromyography. And, (3) use a computer biomechanical model of a multi-digit hand to predict the limits of dexterity with and without neural activity, and test these limits using a robotic manipulator.

Understanding the neuromuscular biomechanics of dexterous manipulation in humans will revolutionize understanding of biological motor function, aid in the diagnosis and treatment of hand impairment, and greatly expand the capabilities of robotic hands. The PI's previous work established a theoretical, computer modeling, and experimental foundation for the neuromuscular biomechanics of static force production of individual digits. This foundation will be expanded: (1) by using nonlinear dynamical analysis (to enable the use of reduced order models) to study the transitions from dynamical stability to instability in this complex system and (2) by integrating cerebral, muscular and biomechanical measurements in the context of a comprehensive computer model of multi-digit manipulation. The understanding gained from studying the human hand will be instrumental to improving dexterous manipulation in humans (clinical applications) and machines (robotic manipulators).

The educational objective is to develop an interdisciplinary educational network that integrates engineering and neuroscience, which will have a broad impact on bioengineering education. The educational methodology will promote discovery in neuromuscular biomechanics among a diverse population spanning from high school students to practicing researchers and clinicians. Education and research will be integrated by: 1) Creating an undergraduate and graduate educational program in neuromuscular biomechanics to improve and broaden the engineering curriculum, 2) Actively promoting opportunities for high school and undergraduate students from underrepresented groups to become involved in neuromuscular biomechanics research and 3) advocating the importance of engineering concepts and methods to researchers and clinicians in neuroscience, motor control, hand therapy, and hand surgery.

This project develops an integrated Bayesian framework for vision based control of Unmanned Aerial Vehicles (UAVs) through fundamental research in 1) model-based nonlinear state and parameter estimation, 2) intelligent adaptive control, and 3) image processing. We specifically address how real-time video data can be processed with ground-based sensors (and on-board avionics) to extract spatial and situational information (e.g., vehicle state and model parameters). Using only stationary video cameras, information from the sequence of images are integrated with an adaptive controller that transmits actuator commands directly to the UAV. Our research infrastructure consist of an X-Cell-60 R/C helicopter with custom avionics, video cameras on the ground, and a PC ground-station to perform all necessary processing

A key aspect is to go beyond traditional vision based motion estimation and tracking, utilizing new approaches to recursive Bayesian estimation allowing full coupling with the control system. Heuristically, this involves the propagation of probabilistic density estimates for the state (vehicle position, attitude, and velocities) and model parameters (mass, moments of inertia, aerodynamic forces, etc.). The vision components models the ``image likelihood'' and describes the probability of observing the image given the current state. The estimation combines the vision measurements with the dynamic vehicle model in a recursive filtering procedure using a Sigma-Point Filter (SPF) framework. SPF methods are a recent development in machine learning, and are shown to be far superior to standard EKF based estimation approaches.

The intellectual merit of the research contributes to both the individual component areas as well as the integrated whole. The integration of the different components in the proposed manner represents an interdisciplinary new approach, providing new research opportunities and applications in integrated sensing, information processing, and control. Beyond basic research, the broader impact to technology includes the obvious commercial and military applications that can be studied in this controlled environment (e.g. visually assisted vertical take-off and landing for ship board helicopters, or agile maneuvering through urban environments). The core technologies can also be extended to other information technology areas from image tracking and detection, to control of complex biological systems.

The Cornell University IGERT Program in Nonlinear Systems supports graduate education and research in the area of complex nonlinear systems. The research component of the program will be organized around interdisciplinary groups (IRTG) comprising faculty with expertise in theoretical, computational and empirical science, who will jointly mentor graduate student fellow projects. The research areas of the initial IRTG, including areas of applications, are (i) networks (social networks, gene networks, internet, electric power grid); (ii) gene regulation (cell signaling and gene expression networks); (iii) moving machines and organisms (manual dexterity and control of locomotion); and (iv) biological pattern formation (cardiac electrophysiology).

Nonlinear science has been a role model for interdisciplinary research. Principles arising from dynamical systems theory have revealed common features in seemingly unrelated phenomena across the breadth of science and engineering. The intellectual merit of this project lies in the extension of successful strategies employed in nonlinear dynamics to confront increasingly complex systems. A primary goal of the research is to understand how systems, especially those arising in the life sciences, can be more than the sum of their parts. For example, legged locomotion and manual dexterity will be studied through a combination of mechanical devices, observation of human and animal behavior and computer models. The broader impacts of this research will be in improving the performance of robots and the treatment of physical injuries. Another theme that will be explored is how network architecture influences dynamics of a system. The concept of small world networks, developed by the founder of this IGERT Program, Steve Strogatz and his students, has already influenced research on biological, social and communication networks. Applied to the internet, the results of this research facilitate efficient web searches. In general, the program will have broad impact in developing methods to predict the dynamics of complex systems, taking full account of underlying network structures and making extensive use of experimental data.

The primary mechanism of the IGERT program is the engagement of Ph.D. students in nonlinear systems research early in their studies. The program involves students in the conceptual phases of research, and it encourages faculty to develop long term collaborations, stimulated by their joint mentorship of students in the IRTG. The most direct impact of the program is in training a new generation of scientists with broad interests and expertise. In the words of a former IGERT fellow, "graduate students who go through the IGERT program learn to speak the language of two or more fields with considerable fluency, and all students are introduced to a common mathematical foundation so that even those who do not share the language of a specific field can interact meaningfully."

IGERT is an NSF-wide program intended to meet the challenges of educating U.S. Ph.D. scientists and engineers with the interdisciplinary background, deep knowledge in a chosen discipline, and the technical, professional, and personal skills needed for the career demands of the future. The program is intended to catalyze a cultural change in graduate education by establishing innovative new models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries. In this sixth year of the program, awards are being made to institutions for programs that collectively span the areas of science and engineering supported by NSF.

DESCRIPTION (provided by applicant): Dynamic grasp and manipulation with the fingertips ("precision pinch") depend on the mechanical ability to orchestrate fingertip motion and force. The muscle coordination patterns required to produce fingertip motion differ from those associated with exerting fingertip forces: fingertip motion depends on tendon excursions, whereas fingertip force relies on tendon tensions. Dexterous tasks like precision pinch require the coordination of muscles to fulfill two fundamental requirements: (i) the transition from fingertip motion to force in the same direction (e.g., to quickly pick-up a pencil); and (ii) combinations of fingertip motion and force in different directions (e.g., to roll the pencil). We will systematically explain how the musculature of the index finger is coordinated to fulfill these two fundamental requirements of precision pinch. This understanding will provide the biomechanical foundation for much needed studies on the degeneration of precision pinch in orthopedic and neurologic diseases and in aging. HYPOTHESIS I: The sequential transition from fingertip motion to force along the same direction involves interpolating between coordination patterns at the expense of motion and force accuracy, with the abruptness of the interpolation depending on the allowable margin of error. HYPOTHESIS II: The simultaneous production of submaximal fingertip motion and force in different directions can sufficiently constrain muscle interactions to eliminate muscle redundancy. To test these hypotheses, we propose an integrative approach combining behavioral studies, robotics analysis, biomechanical modeling, and cadaveric experiments. AIM 1: To describe the coordination patterns used by humans to produce a variety of index fingertip motion and force tasks. Muscle coordination patterns will be expressed as 7-dimensional vectors estimated from electromyograms (EMG). AIM 2: To explain how the pattern of muscle coordination satisfies the mechanical requirements of the task. A biomechanical model of the index finger will predict coordination patterns that reproduce the motions and forces observed in Aim 1 while evaluating the effects of variability in musculoskeletal parameters, and quantifying the degree of muscle redundancy. AIM 3: To validate the EMG and predicted coordination patterns by actuating the tendons of cadaveric fingers and comparing the mechanical output with the voluntary and model actions.

DESCRIPTION (provided by applicant): GOAL: To provide a much-needed simple, valid, and reliable measure of dynamic thumb forces that quantifies functional performance, impairment and recovery related to orthopedic and neurological diseases. The absence of an objective and sensitive outcome measure of dynamic thumb forces precludes informative clinical studies to optimize the choice, timing and options of treatment for each patient. APPROACH: We have designed a novel test to quantify the ability to produce dynamic thumb forces. The strength-dexterity (S-D) test is based on the principle of buckling of slender springs, and its current prototype consists of asking a person to attempt to compress each of 87 compression springs with different combinations of strength (stiffness) and dexterity (propensity to buckling) requirements. Our past clinical work has shown that the S-D Test is repeatable and is informative of the neuro-musculo-skeletal integrity of thumb with carpometacarpal osteoarthritis (CMC OA) better than measurements of pinch strength. We propose to refine the S-D Test into a clinically useful measure of dynamic thumb forces. AIM 1: Minimize burden to patient and clinician. We will shorten the S-D Test from 87 spring compressions to compressing at most 2 springs, 3 times each. Digital signal processing of fingertip kinematics and sensors embedded in this spring will fully characterize dynamic neuro-musculo-skeletal thumb function at each strength level via 3 continuous parameters. AIM 2: Validate the shortened S-D Test against established measures of hand function. We will deploy the S-D Test at the Hospital for Special Surgery to test the following hypotheses. PRIMARY HYPOTHESIS: The S-D Test correlates with available measures of hand function. SECONDARY HYPOTHESIS: The S-D Test correlates with patient satisfaction after treatment for CMC OA better than available measures of hand function. IMPACT: We seek to develop an assessment tool that is both simple and expeditious to administer, and can divulge a more comprehensive/reliable and quantifiable measure of this crucial thumb function. Developing the S-D Test into a quick, objective, and clinically valid measure of dynamic thumb forces will be instrumental to quantifying impairment and assessing effectiveness of treatments in today's clinical environment.

DESCRIPTION (provided by applicant): Anatomical descriptions of the hand have failed to explain the vulnerabilities and unsatisfactory outcomes to even slight damage to its network of tendinous interconnections. We propose that these classical anatomical descriptions do not capture the severe functional interdependence among its multiple elements. We will use an alternative structural inference approach base on bioinformatic testing of cadaver specimens to find the sensitivities and vulnerabilities of the tendinous apparatus. Hypothesis I: Classical anatomical descriptions of the tendinous apparatus do not capture the interdependencies among tendons and therefore fail to explain the vulnerability of finger function to injury. Hypothesis II: Anatomical descriptions inferred directly from finger force and motion data are better than classical descriptions at capturing the interdependencies among tendons and explain how injury results in deformity and pathologic finger motion. Aim 1: Quantify the fidelity of classical anatomical descriptions by comparing predicted vs. measured static fingertip forces and unloaded finger motions in a variety of postures. Aim 2: Mathematically infer alternative anatomical descriptions from cadaver data, and quantify their fidelity by comparing predicted vs. measured static fingertip forces and unloaded finger motions in a variety of postures. Aim 3: Validate the clinical usefulness of the best resulting anatomical descriptions by performing selective injuries in cadaver hands, and comparing predicted vs. measured deformity and pathologic finger motion. This work is made possible by our novel and validated data-driven bioinformatics approach to infer the functional structure of complex physical systems by autonomously interrogating them. We will infer anatomical descriptions of the fingers and tendinous apparatus by measuring fingertip forces and motion in response to a minimal number of automatically generated tendon tensions and excursions, respectively, delivered by a computer-controlled cadaver actuation system. Future Aims: Establishing the structure, function and vulnerabilities of the tendinous apparatus will have great clinical impact. This work will lead to understanding why and how injuries to the tendinous apparatus, or muscle imbalances, often result in deformity, pathologic finger motion and force deficits

This project aims to reverse-engineer the human brain's ability to control the hand.
The project begins by combining a robotic hand previously developed by the PI with a new type of sensitive skin, with a hundred biomimetic tactile sensors.

The main goal of this project is to understand how it is possible to achieve dextrous, approximately optimal control of a hand, performing familiar but challenging tasks in manipulating objects. New, more advanced learning-based control algorithms will be developed and tested on the four empirical testbeds of the project: (1) robotic manipulation by the biomimetic hand; (2) data from recording of human hands performing the same tasks; (3) computer simulations of physical hands; and (4) computer control of cadaver hands via their tendons. The project will use the same algorithms both as models of human motor control and to go beyond the present state of the art in robotic manipulation; this unified approach to biology and engineering is an essential part of the transformative goals of the COPN topic. Dextrous robotic hands have a wide variety of possible applications in industry, space and national security. Improved understanding of how humans can learn to perform better with their hands will also have broader benefits, particularly for the disabled. The team proposes a vigorous plan for education and outreach, capitalizing on the human interest aspects of the demonstrations they will be developing.

DESCRIPTION (provided by applicant): Our long-term goal is to reveal systematically how the musculotendon mechanics, spinal cord and brain interact to produce able and pathologic finger function. The prior grant revealed many necessary neuromechanical interactions for finger function and dysfunction. This compels and enables us to study spinal neurophysiology and neuromechanics as a step before tackling brain function. The immediate goals of our team of scientists and surgeons are to (i) test the extent to which the known somatosensory feedback and spinal interneuronal circuitry is sufficient, on its own, to account for critical features of fst isometric fingertip forces without requiring on-line supraspinal modulation; and (ii) understand how botulinum toxin (BTX) injections to reduce spasticity and dystonia in hemiplegic CP and iSCI interact with that circuitry. We will test theories of spinal reflexive and excitation-inhibiton mechanisms using synthetic analysis and physical implementation, which in our view is a strong test of our understanding of a system. That is, we will confront the very challenge the nervous system faces by controlling the tendons of cadaveric fingers with an autonomous neuromechatronic system of microprocessors and motors that implements the known motor and somatosensory spinal circuitry and muscle properties of healthy subjects and patients. Aim 1: Characterize H-reflex and performance of Single Joint and Whole Finger fast isometric tasks in control subjects, and pre-&post-BTX in patients. (Exploratory test on CP patients undergoing tendon transfers and musculotendon length changes will validate other physiological processes and model components in the later phases of the research.) Then, actuate tendons of cadaveric fingers to (i) find feasible tensions to replicate that performance and (ii) quantify robustness to errors in tendon tensions. Aim 2: Implement in real time the known connectivity and dynamics of spinal neurons, muscle proprioceptors and muscle fibers of a single afferented muscle. Validate against data in the literature. Single Muscle Hypothesis: Muscle function (e.g., tone, stretch reflex) emerges naturally from specific combinations of neuronal background activity and pathway gains. Test how physiologically tenable disruptions and BTX lead to, or mitigate, pathologic behavior (e.g., spasticity and clonus). Aim 3: Implement the hypothesized neural connectivity and dynamics across muscles to reproduce the H- reflex and performance of fast isometric tasks seen in control subjects, and pre-&post-BTX in patients. Replicating the behavior measured in Aim 1 by driving tendons of cadaveric index fingers will identify how clinically tenable disruptions lead to pathologic behavior, and the extent to which BTX (and preliminarily tendon transfers and musculotendon length changes) can mitigate those pathologies. a) Single Joint Hypothesis: Single-joint function (e.g., fast time-varying torques) emerges naturally from background activity and pathway gains across motoneuron pools of a pair of antagonist muscles. Test the emergence and BTX mitigation of single joint spasticity, clonus, instability, and deficits in single joint tasks. b) Whole Finger Hypothesis: The fast time-varying fingertip force tasks recorded in Aim 1 emerge naturally from physiologically tenable interactions across all finger muscles. Test the emergence and BTX mitigation of whole-finger spasticity, clonus, abnormal postures, and deficits in whole fingertip force tasks.

Reticulospinal pathways are likely key contributors to bilateral disability and recovery that remain understudied. This proposal breaks new conceptual ground by integrating across neuroanatomical, neurophysiological & behavioral domains in stroke. The reticulospinal tract receives input from both hemi-spheres and projects bilaterally, and undergoes neuroanatomical changes in chronic hemiparetic stroke. Neurophysiologically, alpha-band (10-20 Hz) neural drive is exaggerated or redistributed across muscles of both arms after stroke, and this frequency of neural drive is thought to be of reticulospinal, but not corticospinal, origin. We link this to behavior as alpha-band neural drive is especially exacerbated in muscles involved in the pathologic synergies that disrupt motor control. We will demonstrate muscle-muscle coherence is a valid neurophysiological assay to characterize mechanisms of bilateral motor impairment at the level of the reticulospinal tract. Interventions exploiting these mechanisms for recovery of the more-affected arm have a greater chance at being restorative and promoting brain repair than simply improving function through practice of compensatory strategies.

Engineering of robots primarily relies on prescribed algorithms for centralized control. This results in robots with limited versatility because every function must be preprogrammed. Animals, by contrast, rely on adaptable neuronal networks distributed throughout the body that convert and modulate brain signals into specific and well-coordinated muscle actions and corrections. It has now become possible to record signals from these large neuronal networks in primates in the part of the spinal cord controlling hand function. Therefore, our goal is to extract the functional features of these neuronal networks, and validate their function by controlling bio-inspired robotic hands, as well as human cadaveric hands. This validation will allow the first physical test of the biological mechanisms for grasp function, and will help understand hand disabilities and treatments in, for example, stroke, spinal cord injury and cerebral palsy. It will also launch a new generation of versatile robots that use the mechanisms of our nervous system.

The overall goal is to create a synthetic functional analogue of the cervical spine that controls multiple grasp modalities in bio-robotic hands. This is made possible by the advent of specialized massively parallel computer chips that allow the implementation of networks of hundreds of simulated neurons and their spiking dynamics (neuromorphic chips). Therefore, in this project, we will extract network architectures for the control of the hand from the nervous system of primates (Japan) and implement them as neuromorphic circuits to create a new class of versatile robotic hands (USA). Using specialized recording system, will record neural data from hundreds of spinal interneurons and alpha motoneurons in the cervical spinal cord of awake, behaving monkeys during manipulation—while also recording EMG and hand kinematics. This will be the most complete data set to date for cervical control of the hand (Aim 1). Then, we will create neuromorphic implementations of that neural circuitry using state of the art very large scale integration chips. Special attention will be paid to implementing physiologically valid versions of alpha-gamma motoneuron interactions, and realistic plasticity rules. We will also create a Domain Specific Language that allows the translation of general neuroanatomical circuits into neuromorphic code to make this technology accessible by the general neuroscience community (Aim 2). We will test, refine and validate the neuromorphic circuits by using the neuromorphic chips to control neuro-robotic hands using electric motors programmed to behave as muscles, and sensors to replicate the function of muscle spindles and Golgi tendon organs (Aim 3). We will also control cadaveric human hands to validate the neuromorphic controller for the anatomy of the human hand. This will pave the way to a better understanding of hand function and disability and serve as the proof of concept for a new class of neuromechanical robotic, prosthetic and brain-controlled hands.


A companion project is being funded by the National Institute of Information and Communications Technology, Japan (NICT). This project is jointly funded by the following NSF programs: Disability and Rehabilitation Engineering, Collaborative Research in Computational Neuroscience, Robust Intelligence, and Mind, Machine and Motor Nexus program.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.