Devin Balkcom - US grants

This CAREER award will support research in algorithms to design motion strategies for robots and other mechanical systems. Applications include mobile-robot control, analysis of protein structure for drug design, and design of more reliable tools for automated manufacturing.

There are a number of fundamental open problems. Given a resource to conserve, such as time, energy, precision, or opportunities for sensing, what is the optimal trajectory between two configurations? How should motions be planned for highly-constrained mechanical systems, for which it is difficult to automatically generate feasible configurations? Can mechanical devices be designed so that all motions lead reliably to the goal, in spite of errors in sensing and control?

These challenges span several facets of the motion problem, but there is an underlying theme. Constraints due to the requirement for optimality, kinematics of the mechanism, or frictional contact mechanics impose a global structure on the space of possible configurations of the system. Often, the configuration space can be described as a collection of relatively homogeneous regions, separated by sharp boundaries at which the behavior of the system changes.

The basic goal is to extract as much information as possible about the structure of the configuration space using whatever analytical techniques are most appropriate, including Pontryagin's Maximum Principle, Morse Theory, and tools from mathematical programming. Once this structure has been determined, the next step is to find ways to represent the space and reason about it automatically.

The research program will be complemented by the development of a strong curriculum in robotics and geometric-reasoning algorithms. The award will support a small team of graduate student researchers, the development of a core group of classes to prepare undergraduate and graduate students for robotics research, and a summer robotics camp for K-12 students.

We propose to build a Digital Imaging Laboratory that will support research, teaching and cross-disciplinary collaboration at the boundary of Art, Engineering, Law and Science. Four primariy research projects will be supported by this Laboratory: (1) Distinguishing between real and computer generated images and video; (2) Building compact and intuitive representations of human motion; (3) Art authentication; and (4) Lighting art. The Laboratory will also be made available to students enrolled in the newly created Digital Arts minor. And, faculty and students across the Dartmouth campus will be able to take advantage of the Digital Imaging Laboratory. Progress reports for this project will be regularly updated at www.cs.dartmouth.edu/farid.

This proposal presents a plan to attack the problem of manipulating flexible materials like string and wire using minimal sensing and actuation. Flexible materials are ubiquitous in manufacturung and surgery, and human examples demonstrate that complex manipulation of flexible materials is possible, from knot-tying to folding laundry or even origami. Robots are currently far less capable in these domains, leaving many boring, repetitive factory tasks inaccessible to robotic automation. Current simulation models of flexible objects are too computationally complex for standard approaches to manipulation planning to be effective. Furthermore, although the simulated dynamic behavior of a piece of string using existing models might be visually believable, the behavior is unlikely to match any particular piece of string in the real world and, therefore, is of limited use.

Knot tying has many applications ranging from manufacturing to surgery to agriculture and service robots. Manipulation of deformable objects is fundamental to many manufacturing and household applications. The proposed work has application to factory operations and seems to apply to potential future needs in manipulation of linear, flexible 1-D structures such as carbon nanotubes and proteins.

Broader Impacts: The development of reliable, inexpensive techniques for manipulation of flexible materials goes beyond the obvious potential impact on factory automation. Minimalism is particularly appropriate for problems involving manipulation at new scales. Fixtures for tying knots might be built at the microscale, allowing parallel manipulation of many very tiny threads or rods. Although we emphasize that the focus of the current work is on theoretical foundations and manipulation at the macro-scale, one can imagine using cleverly-designed geometries to manipulate carbon nano-tubes or proteins - things not possible with standard serial arm designs.

Motion planning is fundamental problem that need to be solved for a robot to move from one location to another. This problem arises in many domains such as self-driving cars and automated assembly. Motion planning algorithms typically sample configurations of the robot to build a map of the space of robot configurations. As more computational power becomes available, samples can be placed more quickly, building a better map, but at tremendous cost in memory.

This project explores the problem of finding low-memory approximate maps that allow rapid and accurate motion planning. Methods include mathematical analysis of algorithms, and experiments applying algorithms in simulation. Expected results include new algorithms that allow generation of configuration space maps that require orders of magnitude less space than existing representations. Expected results also include algorithms for motion planning that make use of these maps, and formal guarantees about quality of motion plans and computational costs of generating maps and plans. These results are expected to advance the understanding of fundamental theoretical characteristics of motion planning. These results will also have direct practical impact in application areas, including automated manufacturing and self-driving vehicles, by allowing vastly greater computational power to be leveraged to generate maps that can be stored efficiently and transmitted quickly across a network. Results will be disseminated on new motion planning web pages devoted to the project, in international robotics journals, and at international robotics conferences. Anticipated broader impacts of the work include training of the next generation of scientists and engineers at both the undergraduate and graduate level.

The project will develop technology and study methods for teaching motion tasks, with the teaching of sign language as a first application. Simultaneous placement or quick movement of parts of the body is hard to observe, explain, and execute. While tactile-sensing and augmented-reality systems have been developed to enable machine-human communication of physical processes, the focus has largely been on execution, rather than on teaching and learning. The initial focus of the project will be on teaching sign language, but principles and techniques discovered will be generalized to research the learning of increasingly complex physical motions, ranging from simple posing tasks to high-speed fine manipulation tasks. The proposed work is transformative in that it will directly address the scientific question of how to use technology to understand correct or incorrect human motion and provide constructive guidance, leading to a better understanding of human motion learning. The project will disseminate findings and resources through traditional scientific publications. In addition, models, algorithms, and designs for rapidly-prototyped tools for manipulation will be made available on the online. Results will also be communicated broadly through collaborations with local high schools and museums, and through participation in events such as the USA Science and Engineering Festival.

The task of teaching motion motivates the research of three fundamental challenges. First, closed-loop control is a core feature of cyber-physical systems. With a human participant in the system, how can the loop be closed around slow and low-bandwidth human attention? Actuation that guides the human must be easily communicated and sufficient to stabilize the human-suit system. Second, due to limitations in how much information may be communicated, complex human motions must be broken down, and components taught in isolation. How can these component motions be discovered, taught, and re-integrated? Third, algorithms and systems must be developed to measure the accuracy and retention of the learner during the teaching process, guiding repetition and selection of practice material. The project will design and build a lightweight sensing and guidance system that allows interactive communication about motion between human and computer. This technology will allow the investigators to address fundamental research questions in cyber-learning about how to better teach and learn human motion tasks. Research questions include how to measure and evaluate human motion with respect to the task, how to select sensory input to use as guidance, and how to selectively apply or remove training aids, until the learner can complete the motion task with no assistance.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

The objective of the project is to develop the theory, techniques, and mechanical designs needed for robots to rapidly build large, rigid structures from blocks that geometrically interlock, without the need for fasteners, cement, or glue. The work is motivated by the desire to quickly assemble structurally strong buildings, furniture, and devices, in such a way that the structure may later be disassembled and the parts re-used. Fabricating in parts presents many advantages: for instance, individual components may be fabricated efficiently, packed for storage and transport, repaired or replaced as needed, and design changes can be made readily in response to changing customer needs. Design of smooth surface finishes allows applications such as modular furniture and packaging of delicate parts for shipping, and embedding mechanical or electrical components will allow rapid construction of robots and other devices.

The problem to be studied is how structures may be designed to rigidly interlock, using cycles of geometric constraints imposed by blocks to reinforce, support, and immobilize other blocks. Goals for the project include: 1) the design of joint geometries that allow a connection process that is robust to assembly and fabrication errors; 2) mechanical block designs that allow incremental construction of interlocked structures; and 3) layout algorithms that tailor assembly sequences to allow construction of desired shapes. Methods and approaches include mechanical design, development of layout algorithms, and analysis of assembly processes to ensure structural stability during construction. Experimental work includes robot assembly to validate the approach for complex structures involving hundreds of blocks.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Underwater surveys provide critical information to federal, state, and local agencies charged with protecting unique underwater archeological and cultural heritage sites. The research enabled by the instruments to be acquired with this award is focused on an exploration of techniques and principles that guide the development of algorithms and computational systems for control and design of water robots to study: Underwater robotic construction or scaffolding to protect delicate ecologies; multirobot communication and coordination for monitoring; and underwater exploration and mapping. This project fosters collaboration among researchers in diverse disciplines (engineering, environmental studies, biological science, and anthropology). The instrumentation acquired will provide opportunities to students at all levels with hands-on experience in research, programming, and experiential learning, and public outreach communicate the challenges and insights derived from field experiments. The findings from experimental monitoring will most likely augment public awareness of the environmental challenges for the area.

The instrumentation will enable investigators to study principles, algorithms, and systems needed to enable autonomous locomotion and manipulation in water, including gathering data about physical changes and pollutants, aiding in search-and-rescue, constructing underwater structures, and documenting underwater and cultural heritage. New challenges such as global positioning, high speed communication between robots or with a remote computer, and human access for repair pose new challenges. The investigators aim to establish a versatile computational framework to enable the precise simulation, design, optimization and planning of various physical processes related to the construction activities immersed in a fluid environment. Using more complex models, they expect to create a real-time decision framework that includes constraints and optimization criteria and can adapt to changing contexts reasoning.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

The proposed work will develop a system for teaching physical motions such as sign language, dance, and human-robot collaboration automatically. Such motion skills support education and broad accessibility, can enhance well-being through creative expression and physical fitness, and enable new forms of work. Teaching motion often makes use of the intensive efforts of a human coach: the coach demonstrates a motion, evaluates the learner’s performance, and provides feedback and a tailored plan for practice to improve fluency. The proposed work aims to better understand how to develop systems that devise and carry out these components of the coaching process on their own. Such systems would make learning motion-based skills more accessible for those who do not have uninterrupted access to a dedicated human coach. The broader impact of the project also includes the training of the next generation of scientists at the intersection of psychology, cognitive science, education, and computer science.<br/><br/>The goal of the work is to expand the capabilities of automatic teaching agents into the domain of physical motion, and to incorporate signals reflecting cognitive and emotional states of the learner into the system. Using pre-recorded motion examples from experts, and sensed actions and poses of the learner, this system will identify qualitative and quantitative differences between the teacher and learner. Low-level errors will be tracked to build an evolving cognitive model that measures the learner’s level of comfort with the process, as well as mastery of skills and combinations of skills. The project will develop computational tools that use this model to determine which feedback may be most effective to improve the learner’s performance. The project will also develop and evaluate hardware and software platforms that provide this feedback, with cues that may be presented in audio, visual, or tactile forms. To make the system as accessible as possible, the project will evaluate low-cost and ubiquitous approaches to sensing, including web cameras for sensing, and expert demonstrations parsed from internet videos.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.