Blake Hannaford - US grants

This is a PYI application from a new Assistant Professor of Electrical Engineering. Dr. Hannaford received his Ph.D. from UC-Berkeley in 1985, and was employed by the JPL at Cal Tech until 1989. He has numerous publications in the area of human motor control. His plan is to continue his research to better understand how humans control their limbs, but he will be increasingly involved in robotics research as well. His plan is to use the understanding of the natural motor system to design robotic manipulators.

9451387 Hannaford Engineering Education is undergoing a period of rapid change. American industry needs engineers who can design products which are competitive in quality and performance in an increasingly aggressive global market place. There is a sense that dividing Mechanical, Electrical, Industrial, and Computer engineers into separate compartments may produce only locally optimal designs. In education, the traditional emphasis on single-student analysis within an "Engineering Discipline" is giving way to interdisciplinary, design-team oriented education. With these global trends in mind, we propose a new laboratory-based course, "Design of Consumer Electronics," to address the need for interdisciplinary, design-team oriented education within the College of Engineering at the University of Washington. We request funding to develop this course, to disseminate the course content and format to other institutions, and to encourage and compile an expanding collection of curricular materials from among a diverse group of faculty at UW and other institutions.

This CRCD project supports the development at the University of Washington of an exportable package for the capstone design course emphasizing computing appliences, creates two new undergraduate courses on embedded systems programming, creates a new graduate course on the elements of computing appliances, and brings the results of research in embedded system design and design automation as well as internetworking protocols to the undergraduate curriculum. The exportable package for the capstone course consists of course modules and web-based materials that can be augmented and updated by the community and used to rapidly set up new project courses, labs, and individual senior projects. The two courses on embedded systems programming include material on device drivers, real-time programming and operating systems, component-based programming models and also material in the area of wireless networking. Overall, the project is planned to modernize the computer engineering curriculum by focusing on a class of devices, services, and applications that are expected soon to be center-pieces of the consumer electronics and computing industries.

0104542
Hannaford

This one-year award will support a dissertation enhancement project of Greg Lee, a student in the Biorobotics Lab of the Electrical Engineering Department at the University of Washington. He will work with Prof. Paolo Dario at the ARTS/MITEC Laboratory at the Scuola Superiore Santa Anna, in Pisa, Italy. The objective of Mr. Lee's research is to incorporate MEMS techniques into his research on low power haptics. Mr. Lee will benefit from studying biorobotic MEMS technology being developed at the ARTS/MITEC laboratory, which is unique in its integration of MEMS with biorobotics. interface. While at ARTS/MITEC, Mr. Lee will offer seminars in real-time embedded Windows CE networked control systems.

0104452
Hannaford

This one-year award will support a dissertation enhancement project of Jesse Dosher, a student in the Biorobotics Lab of the Electrical Engineering Department at the University of Washington. He will work with Prof. Massimo Bergamasco at the PERCRO lab at Scuola Superiore Santa Anna in Pisa, Italy. The objective of Mr. Dosher's research is to improve the design of the Biorobotics Lab's new control method for haptic devices. While at PERCRO, one of the world's leading laboratories in haptic device mechanization, Mr. Dosher will learn state of the art materials, mechanization, and fabrication techniques for haptic device design. While at PERCRO, Mr. Dosher will offer seminars on the Biorobotics Lab's new control method for haptic devices.

Integrating humans with their naturally developed control algorithm and robots with their extended capability of applying forces and torques into one system offers multiple opportunities for creating a new generation of assistance technology for both healthy and disabled people suffering from neuromuscular diseases and neuro-degenerative disorders. The exoskeleton is a wearable robotic arm. It is worn by the human as an orthotic device and acts as a human-amplifier allowing the operator natural control of the device as an extension of his/her body while sharing an external load. One of the primary innovative ideas of this research is to set the human machine interface at the muscular level of the human physiological hierarchy using an expression of body's own control command (surface electromyography - sEMG) signals as one of the primary command signals of the exoskeleton for improving the synergy between the operator and the exoskeleton. The goals of this research are to design, build, and experimentally study the integration of a powered exoskeleton controlled by myosignals (sEMG) for the human arm with healthy people. It is anticipated that the scientific activity involved in this research will integrate and fuse multidisciplinary knowledge by promoting dialogues and collaborative work between students and faculty members from different disciplines with a long-term goal of improving the quality of life of the physically disabled community.

Robotics and Human Augmentation Program

ABSTRACT


Proposal #: 303750
Title: Multi-Finger High Fidelity Haptic Exploration
PI: Hannaford, Blake
U of Washington


Haptic interfaces apply ideas from robotic manipulation science and technology to human interfaces and human augmentation. Current haptic interfaces have proven their feasibility for a variety of applications. Better haptic interfaces could dramatically improve medical training through high fidelity simulators for invasive medical procedures. Such procedures must be taught today on human patients.

However, the richness and fidelity of most haptic interfaces is still limited. It is very difficult to engineer a haptic interface with many degrees of freedom (DOF) and also the high bandwidth and low friction necessary for rendering subtle but important details. Before we expend this effort, we should know what if any perceptual benefit is provided by multiple fingered/multi DOF devices. This project will study how human performance in haptic perception and exploration varies as increasing number of fingers are supported through increasing degrees of freedom. An existing high fidelity haptic device for the index finger will be duplicated to support haptic exploration with up to four fingers and two degrees of freedom per finger. This will allow independent variation of number of fingers and number of DOF. A preliminary evaluation of the device will measure performance in detecting small haptic features, and demonstrate haptic exploration of novel surfaces. We will learn whether and how haptic perception performance and strategies vary with the number of fingers and number of fingers per DOF.

Imagine a virtual environment so realistic that within it one could practice a violin technique such as vibrato, or how to filet a freshly caught halibut, or a surgical procedure using patient specific data. Such applications, and countless others, would require a high quality haptic device for the hands through which realistic forces could be applied to convincingly simulate touch. Today's haptic interfaces - information technology that supports human touching and manipulation of virtual or remote objects - are still in their infancy compared to human haptic capabilities. The typical haptic device is connected to a single finger, or supports interaction through a pen-like stylus. In an effort to dramatically advance the state of the art, the PI has developed an exciting prototype multi-finger device of high resolution and bandwidth. In this project he will determine the limits of the prototype and refine the technology so that it is useful for evaluation in applications. To these ends, the PI will explore the requirements on high-fidelity haptic technology imposed by human perception. He will measure user thresholds {i.e., the smallest forces which they can perceive with multiple fingers), and will study multi-finger and bi-manual behavior and performance in exploring and manipulating objects and materials. These data will enable the PI to reconfigure his multi-finger haptic device and create software modifications as necessary, and also to develop a new haptic rendering algorithm and study how it interacts with human perception. Project outcomes will include engineering requirements for successful yet cost-effective haptic devices, along with some of the knowledge necessary to achieve future high quality multi-finger haptic interface applications

Broader Impacts: The technology to be developed in this project will ultimately give people who are visually impaired access to the many images and 3D datasets and models which are accessible only in electronic form. Additionally, haptics is an important enabling technology for medical simulations; better medical simulators will lead to fewer medical errors and more efficient training of medical personnel. Haptics is expected to have a significant impact on the appreciation and the conservation of works of art. Demonstrations of applications in these and other fields will be byproducts of this research.

CPS Small: Control of Surgical Robots: Network Layer to Tissue Contact

Research Objectives: This proposed CPS project aims to enable intelligent telesurgery in which a surgeon, or a distributed team of surgeons, can work on tiny regions in the body with minimal access. The University of Washington will expand an existing open surgical robot testbed, and create a robust infrastructure for cyber-physical systems with which to extend traditional real-time control and teleoperation concepts by adding three new interfaces to the system: networking, intelligent robotics, and novel non-linear controllers.

Intellectual Merit: This project aims to break new ground beyond teleoperation by adding advanced robotic functions. Equally robust and flexible networking, high-level interfaces, and novel controllers will be added to the existing sytsem. The resulting system will be an open architecture and a substrate upon which many cyber-physical system ideas and algorithms will be tested under realistic conditions. The platforms proven physical robustness will permit rigorous evaluation of results and the open interfaces will encourage collaboration and sharing of results.

Broader Impacts: We expect the results to enable new research in multiple ways. First, the collaborators such as Johns Hopkins, U.C. Santa Cruz, and several foreign institutions will be able to remotely connect to new high level interfaces provided by this project. Second, for the first time a robust and completely open surgical telerobot will be available for research so that CPS researchers do not need to be limited to isolated toy problems but instead be able to prototype advanced surgical robotics techniques and evaluate them in realistic contexts including animal procedures.

While areas of the US such as California and the East coast have major
medical centers near every population center, there are many areas,
even in developed countries, with a widely dispersed patient
population with significant obstacles to advanced medical care.
Victims of large scale natural disasters and battlefield casualties
also lack state of the art care. Telesurgery could potentially meet
these needs. There is a growing interest in computational research on
intelligent telesurgery. Today's telesurgery systems do not integrate
with medical imaging or have intelligent functions to assist the
surgeon. This project is building seven teleoperated surgical robot
and surgeon-interface systems powerful and precise enough to support
research on advanced robotic surgery techniques at seven major
research universities. University of Washington and University of
California Santa Cruz are building the seven robots and will
distribute them to Harvard, Johns Hopkins, University of Nebraska,
U.C. Berkeley, and UCLA. When completed, the seven research lab
robots will be networked together for sharing of software,
interoperability, and on-line telesurgery experiments.

Impact

The results of this project will enable a network of researchers to
prototype advanced surgical robotics techniques, and evaluate them in
realistic contexts including appropriate animal procedures. Besides
new and less invasive cures, such a technology can provide easier
access to care in remote parts of the US, and the developing world.
The seven cooperating labs, and external partners will be able to
interconnect their systems, share modules of software, and replicate
each others experiments.

DESCRIPTION (provided by applicant): Lesions of the anterior and middle skull base are diverse in their etiology and cause significant morbidity due to close proximity to the carotid arteries, brain, orbits, optic nerves and chiasm. For most lesions, a surgical biopsy, excision, or debulking is an essential aspect of treatment. Technological advances over the past few decades now permit transnasal endoscopic surgical access in many instances, which has dramatically reduced the morbidity of gaining surgical access compared to an open craniotomy approach. However, transnasal access is not sufficient for all lesions, which require an open craniotomy if lateral structures are involved. The geometric constraints at the pyriform aperture limit the angle between instruments to 15 degrees when manipulating a target at a depth of 9 cm on the skull base. To improve the ability to manipulate a target, one option is to widen the angle between instruments by using additional surgical portals. Many portals are described to access the skull base, but little work has been done on combining different portals in order to optimize an approach for a certain target location; portals include transnasal, transoral, transorbital, supraorbital, transmaxillary, transcervical, and transventricular. A multidisciplinay team of engineers and surgeons from subspecialties including otolaryngology, neurosurgery, urology and robotic surgery, and orbital surgery has been assembled. Within the team are professors, attending surgeons, engineering graduate students, and resident surgeons in training. We aim to develop an improved 3D computer model to identify and test optimal approaches for endoscopic access and excision of skull base lesions. Using individualized imaging information, it will offer preoperative surgical rehearsal to improve surgical outcomes and minimize adverse effects. Initially, clinically relevant skull base targets around the pituitar will be defined as locations where tumor invasion often occurs. Virtual endoscopy will be performed to access the specified skull base targets through a variety of endoscopic approaches, including existing standard endoscopic approaches and novel multiportal approaches. The multiportal approach provides wider angles between instruments, which will accommodate current surgical robotic platforms. The model will be validated in cadaver specimens and robotic feasibility will be assessed on two robotic surgery systems. We will use this model to test the hypothesis that multiportal approaches significantly broaden the angle of access to the target and minimize the probability of instrument collisions on set protocols of dissection. It is anticipated that the computer model will demonstrate the shortest, most direct, and least traumatic pathways to skull base targets. Normative data on the approach combinations will be generated and ultimately, these results will serve as a platform for future robotic integration and computer simulation into skull base surgery.

Telemanipulation systems consist of a human interacting with a mechanical device on the master side to operate a robot at the remote side. They provide natural opportunities for research in intelligent human/robot collaboration, but existing commercial systems, used in areas such as telesurgery, are not intelligent and therefore only replicate the actions of the human operator. These systems are also proprietary, expensive, and not available for modification by researchers. The goal of this NRI project is to provide an open-source software infrastructure that is designed to work with a broad range of hardware and simulated devices to enable a larger community to pursue research and education in intelligent telemanipulation at a lower cost.

The increasing pace of robotics research can be attributed, at least in part, to the increasing availability of software infrastructure, such as Robot Operating System (ROS), and open hardware platforms. This NRI project focuses on providing a software infrastructure for research in intelligent telemanipulation, leveraging infrastructure developed for the Raven II robot and the da Vinci Research Kit (dVRK) and continuing to extend it to other systems, including simulated robots. The three main tasks are to: (1) engage the community to guide development, (2) develop and implement a common API for the diverse hardware platforms, and (3) provide a set of high-level, platform-independent software modules. The goal is to support research towards semi-autonomous telerobotic systems that can more effectively combine the knowledge, reasoning, and decision-making capabilities of a human with the sensing and manipulation capabilities of a robot.

This Future of Work at the Human Technology Frontier (FW-HTF) Planning Grant project will contribute new understanding of how geographical and organizational separation between telerobotic surgical team members affects the use of telerobotic technology to reduce disparities between rural and urban healthcare access and outcomes. According to the Center for Disease Control, people who live in rural areas are more likely than urban residents to die prematurely from all five of the leading causes of death. With over 46 million Americans living in rural areas, the scope of this problem is considerable. Census data shows that the elderly -- who are generally most in need of specialty healthcare -- disproportionately live in rural communities. At the same time, healthcare workforce studies have demonstrated a critical lack of specialists in rural healthcare settings. This disparity leads to a lack of timely access to the life-saving specialty surgeries that are afforded to urban residents. Telerobotic surgery has the potential to effectively bring the benefits of skilled specialty operators to rural areas, however the barriers to the implementation and the testing of novel telerobotic solutions are not well understood. Recent studies suggest that the impact on the functioning of the operating room team due to the displacement of the operator during robotic surgery may outweigh any purely technological factor. This exploration into rural telerobotic operation will ultimately benefit the US healthcare system as a whole and could have a profoundly positive impact on patients living in rural US communities. The multidisciplinary approach employed will ensure that the voices of all stakeholders will be considered.

This planning project brings together a team of experts in sociology, robotics, economics, cybersecurity, computer science, and healthcare to carry out fundamental research into telerobotic surgery using a multidisciplinary framework. The overarching goals of the project are to build a powerful new multidisciplinary research team to explore rural telerobotic operations, identify and prioritize the questions that need to be addressed, refine hypotheses, and inform future experimental design. To accomplish these goals, we will use three essential research tools: a stakeholder workshop; technical ideation and basic prototype development; and mock telerobotic surgical procedures. The multidisciplinary research group will focus its initial efforts on understanding the impact of geographic separation of the physician-operator from the operating room team as it relates to communication, hierarchy, and trust.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.