Todd D. Murphey, Ph.D. - US grants

Proposal Number: 0820004/0819929

TITLE: Abstraction-Based Motion Programs for Complex, Interconnected Systems

PIs: Magnus Egerstedt and Todd Murphey

Systematic approaches to abstraction-based motion control of complex, physical systems are still largely missing from the control-theoretic foundations of embedded system design. Examples of high-degree of freedom physical systems include humanoids, multi-leg robots, minimally-invasive surgical robotics, and cooperative systems. This work aims at understanding how high-level motion program languages can be made to form a basis for an effective software system for such complex, interconnected mechanical systems. For this, novel tools and techniques are to be developed along the following directions: 1) construction of motion description languages based on optimal control techniques; 2) development of a novel, graph-based representation of mechanical systems that allows for a compact representation of mechanical systems for simulation and analysis; 3) automatic generation of dynamically feasible motion primitives from empirical data; 4) development of an experimental testbed based on autonomous marionette puppets that can execute the developed motion programs--this testbed will also serve as a unique learning environment for students in that it requires an understanding of highly nonlinear dynamic systems, networking architectures for synchronization, hybrid systems, and optimal control.

Hybrid mechanical systems arise in many applications, including hopping, walking, and climbing robots where contact with the ground changes; skid-steering vehicles where Coulomb friction introduces stick/slip behavior; and prosthetic devices that interact with objects. All of these applications are influenced by nonlinearity, nonsmooth transitions, and uncertainty, and these systems demand new tools in motion planning and control.

This work takes advantage of the specific structure of mechanical systems to bound the propagation of uncertainty and to develop feedback controllers that maximize robustness of execution. The work builds on state-of-the-art techniques in motion planning and estimation, including sample-based and optimization-based planning, leading to tools for uncertain hybrid mechanical systems that are analogous to control and estimation tools used for linear systems.

Example systems are used to drive algorithm development as well as to verify performance. These include 1) the Monkeybot, a robot that uses electromagnets and a single motor to locomote along a vertical wall;
2) a parkour robot that uses mechanical contact and jumping to climb narrow passages; and 3) a skid-steered vehicle that experiences discontinuous dynamics due to stick/slip friction effects. All phases of this work include participation of undergraduates and minority students. In addition to dissemination in conferences and journals, results are disseminated on a publicly viewable wiki.

The aim of this award is to develop computationally tractable methods for designing and controlling systems that experience impact. Impacts introduce discontinuities into the equations of motion, making traditional techniques in control theory difficult to employ. However, if a system can specify its configuration during impact a form of feedback control may be utilized. The work will use a combination of variational analysis for mechanical systems, shape optimization, and control theory to synthesize embedded controllers for a class of impacting systems with guarantees on stability, optimality, and computational complexity. The work will start with a simple example of a two-legged robot walking on terrain; later examples will include numerical models of high degree-of-freedom robots walking on uneven terrain and a biomechanical model of a human hand interacting with objects. Deliverables include mathematical proofs of algorithm properties, a software implementation of the feedback synthesis techniques in a variety of documented examples, articles describing the techniques, and training for students in control engineering and biomedical engineering.

Impacts are present in applications ranging from robotic locomotion and grasping to manufacturing and even human prosthetics. Computer control of these systems depends on incorporating impacts into algorithms that stabilize motion. If successful, the proposed techniques will make practical regulation of many impacting systems as straightforward as regulation of non-impacting systems. Moreover, the techniques will provide an improved understanding of the role that contact plays in human neuromuscular control. The project will leverage international collaboration through NSF Office of International Science and Engineering (OISE) Global Venture Fund (GVF) co-funding to develop long-term collaboration with researchers in Germany. Software developed as part of this work will be incorporated into an ongoing clinical study of hand rehabilitation at the Rehabilitation Institute of Chicago and will be freely available to the public. In addition to enabling important technologies, the work will be highlighted in a museum, and student will participate in outreach with an urban high school in Chicago.

Objective: How much a person should be allowed to interact with a controlled machine? If that machine is safety critical, and if the computer that oversees its operation is essential to its operation and safety, the answer may be that the person should not be allowed to interfere with its operation at all or very little. Moreover, whether the person is a novice or an expert matters.

Intellectual Merit: This research algorithmically resolves the tension between the need for safety and the need for performance, something a person may be much more adept at improving than a machine. Using a combination of techniques from numerical methods, systems theory, machine learning, human-machine interfaces, optimal control, and formal verification, this research will develop a computable notion of trust that allows the embedded system to assess the safety of the instruction a person is providing. The interface for interacting with a machine matters as well; designing motions for safety-critical systems using a keyboard may be unintuitive and lead to unsafe commands because of its limitations, while other interfaces may be more intuitive but threaten the stability of a system because the person does not understand the needs of the system. Hence, the person needs to develop trust with the machine over a period of time, and the last part of the research will include evaluating a person's performance by verifying the safety of the instructions the person provides. As the person becomes better at safe operation, she will be given more authority to control the machine while never putting the system in danger.

Broader Impacts: The activities will include outreach, development of public-domain software, experimental coursework including two massive online courses, and technology transfer to rehabilitation. Outreach will include exhibits at the Museum of Science and Industry and working with an inner-city high school. The algorithms to be developed will have immediate impact on projects with the Rehabilitation Institute of Chicago, including assistive devices, stroke assessment, and neuromuscular hand control. Providing a foundation for a science of trust has the potential to transform rehabilitation research.

1637764 - Murphey

Robotic assistive devices help people execute and learn physical tasks. Sometimes these tasks are relatively simple, sometimes they are in a particular context, and sometimes they are highly dynamic and very task specific. This work will create algorithms that enable the delivered assistance to take into account algorithmic descriptions of the underlying task. As an example, walking is a highly structured task that simultaneously requires efficiency and stability during motion and must take into account terrain characteristics. The work takes advantage of task knowledge to either modify a person's motion or exert forces that help the person complete the task. This capability is relevant to rehabilitation and physical therapy, where one may wish to only minimally help a person in order to improve therapy outcomes. This work will therefore impact the development of software that supports people engaged in robot-assisted physical therapy, including people recovering from various forms of injury. The key to this work is that knowledge of a task is combined with knowledge of a person's capabilities to synthesize software decisions that ensure safety while also maximizing a person's agency during motion. Broader impact of this work includes technology transfer to rehabilitation, outreach through the Museum of Science and Industry in Chicago, classroom innovation, and industry collaboration.

The proposed work will create software-enabled, task-specific support for assistive biomedical devices. Dynamic tasks require that a combination of the robot and the assisted person be both effective and safe, and the proposed research will create algorithms and software that ensure efficacy and safety while leaving the user free to both move and exert effort. The latter is important in contexts like physical therapy, where effort is important to therapeutic impact. The proposed work will leverage recent results in real-time nonlinear optimal control techniques for human-in-the-loop systems. Specifically, sequential action control (SAC) will be used to both filter and assist human subject dynamic behavior, using a method called the Maxwell's Demon Algorithm. The work will additionally develop formal methodologies for establishing stability and performance guarantees for the proposed algorithms. Lastly, the proposed work will develop compact representations of the controlled assistance algorithms appropriate for computationally minimal embedded systems. All the work will be developed in the Robot Operating System (ROS), making the developed tools widely available to both researchers and companies. The algorithms will be tested on haptic devices and an exoskeleton. The broader impacts for this work will include outreach, technology transfer to rehabilitation, the development of courses in dynamics and analysis, and industrial collaboration. The PI is currently working with the Museum of Science and Industry, and as part of the proposed work the PI and supported students will participate in a National Robotics Week exhibit in the main rotunda of the museum with an estimated viewership of over ten thousand on-site visitors. The PI is involved in significant classroom innovations, and the proposed work will include development of courses in analysis and dynamics. Lastly, the project will include a collaboration with Ekso Bionics, leveraging and impacting their unparalleled expertise in exoskeleton development.

This project will greatly extend a powerful new method for control of robots and vehicles, called sequential action control (SAC). One widely used approach to controlling complicated systems is to solve a real-time numerical optimization problem for the magnitude of the next few control pulses, where the all the pulses have a constant width. However even with powerful processors it can be difficult to compute these values fast enough. SAC addresses this challenge by instead computing the optimal width and relative start time of the next control pulse. For many problems of interest, this change in control strategy greatly simplifies computation, to the point that infeasible control problems become tractable. SAC allows analytical solutions to some problems, and speeds computations by up to eight orders of magnitude for others. SAC is naturally compatible with common features of modern control design, including hybrid systems that switch discretely between a collection of continuous dynamic behaviors; quantized systems where inputs, states, and outputs may take only a finite set of constant values; and systems with nonlinear dynamics. SAC can be shown to recover the globally optimal control signal in a number of analytically solvable cases. In other representative test cases, the computed SAC input provides performance that is numerically indistinguishable from the optimum. Optimal or near-optimal input signals are of no value if small disturbances cause the system to rapidly diverge from the desired behavior. Therefore practical controllers must also ensure that small disturbances to the controlled system cause only small deviations in the system response -- a property known as stability. This project seeks to rigorously derive SAC performance guarantees for a broad class of systems, as well as to show conditions under which SAC ensures stability. The Darwin humanoid robot will be used as a high-dimensional, nonlinear, hybrid testbed for this research. Control of the Darwin robot may be implemented in the open-source Robot Operating System (ROS), allowing a robust and verifiable SAC distribution for dissemination. The results of this project will enable greatly improved and verifiable control over systems such as rehabilitation robots, assistive devices, rotor vehicles, and driverless cars, using widely available and low-cost computing platforms such as mobile phones. Benefits to society from this project include enhanced safety and performance of these automated infrastructure systems. The project also includes classroom innovation, international collaboration, outreach activities through the Museum of Science and Industry in Chicago, and dissemination of open-source software.

The twofold purpose of this project is to develop sequential action control (SAC) into an actionable, near-universal method for synthesizing embedded real-time control as well as to provide foundational results on optimality, stability, and geometry. The method is computationally efficient and scales to high dimensional problems. Moreover, SAC extends naturally to Lie groups, common in applications such as robotics and automation. The project will address three fundamental questions. First, it will identify conditions under which SAC can be applied directly or iteratively to achieve optimal control. Second, it will derive conditions for stability. Third, it will adapt SAC to systems evolving on Lie groups, to achieve global performance for multibody mechanical systems. The broader impacts for this work include outreach, technology transfer to rehabilitation, the development of online courses in dynamics and analysis, and international collaboration. The PI is currently working with the Museum of Science and Industry, and as part of the project the PI, and graduate and undergraduates involved in the PI's laboratory, will participate in a National Robotics Week exhibit in the main rotunda of the museum with an estimated viewership of over ten thousand on-site visitors.

The understanding of designing structured surfaces for advanced functionality, such as friction reduction, antifouling, and hydrophobicity, has significantly progressed over the years; however, the critical technical barrier to the application of these structured surfaces is the scalability in manufacturing capability. The biggest challenge in surface patterning is the process scalability, which needs to reconcile the significant scale difference between the individual feature size down to the nano- or micro-level and the large surface-to-be-textured up to the meter level. The project will investigate one vision for future manufacturing -- distributed robotic manufacturing -- to achieve scalable patterning of micro-structured functional surfaces using a cohort of mini-robots. The project will not only push the knowledge boundaries in the scientific understanding of distributed physical intelligence, machine-material interactions, and swarm control, but may open up a new and interdisciplinary research field at the intersection of manufacturing, robotics, control, and cyberphysical systems. Additionally, the project includes outreach at the Museum of Science and Industry in Chicago, open-source software, and curricular innovations, including online classes and training modules. The research will build an educational and outreach platform to enable education and workforce development for STEM educators, next generation workforce, and technical engineers.<br/><br/>The research objectives are to explore and answer three fundamental scientific questions that will enable the vision for future manufacturing in distributed robotic manufacturing. (1) Distributed physical intelligence: a new design framework will be established to distribute the intelligence among mechanical structures, analog circuits, and digital logics, as well as to design unconventional communication channels through both active and passive manners. (2) Unconventional machine-material interaction: new theoretical underpinnings will be established to investigate the machine-material interaction and the possible removal, deformation, and addition of material in this new paradigm, where the tools are extremely flexible and with significantly constrained power. (3) Swarm control: new fundamental knowledge will be generated from novel task decomposition and distribution paradigms to synthesis techniques capable of minimizing control effort, communication, and computation. Instead of top-down control of every individual mini-robot, novel methods will be established through which local rule specifications lead to global distributed pattern completion.<br/><br/>This Future Manufacturing award is supported by the Division of Computer and Network Systems (CNS) of the Directorate for Computer and Information Science and Engineering (CISE), and by the Division of Civil, Mechanical and Manufacturing Innovation (CMMI) of the Directorate for Engineering (ENG).<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.