Peter Kazanzides - US grants

The objective of this proposal is to investigate the feasibility and potential for implementation of information design technology as a means to improve management and dissemination of scientific information amassed through research conducted within and among NSF Engineering Research Centers, thereby increasing collaboration activity and results. Preliminary exploration is focused on standardized scientific poster design, offering potential for streamlining of information and improved sharing capabilities. The PI plans in using "information design" technology to organize complex and large amounts of scientific information. The intellectual merit of this proposal is in the conceptualization, design, and implementation of systematic methods for the creation, classification, and sharing of scientific information resulting from ERC activities over a 10-year life span. The broader impact of this research is manifested through industrial-strength software development technology, which is not normally taught to graduate students in software engineering classes at HBCUs. This research will make it possible for HBCU students to learn the basics of good software developmentpractices and information design. This research will provide input for revision of the curriculum in the "Advanced Software Engineering" course where concepts of industrial-strength software development will be taught. Also, this project will partner with the Baltimore City School system to provide a practicum for senior high school students. These students will partner with college students on projects to understand software engineering and information design technology.

Proposal #: CNS 07-22943
PI(s): Okamura, Allison M.
Cowan, Noah J.; Hager, Gregory D.; Kazanzides, Peter.; Taylor, Russell H/
Institution: Johns Hopkins University
Baltimore, MD 21218-8268
Title: MRI/Dev.: Infrastructure for Integrated Sensing, Modeling, and Manipulation with Robotic and Human-Machine Systems

Project Proposed:

This interdisciplinary project, developing infrastructure for sensorimotor integration that fosters new studies and enhances existing work in the area of manipulation for robotics and human-machine systems, aims to offer a publicly available systems framework, including software, mechatronics, and integrated hardware. Enforcing a general development approach that can be easily extended to other robotic and human-machine applications, two complementary robotics platforms will built addressing two different application domains: a
. Bimanual dexterous manipulation system with integrated environment sensing and the capability for modeling rigid objects commonly found in human environments, and
. Teleoperated surgical robotic system with integrated sensors that can acquire patient-specific deformable tissue models.
Moreover, via the project's website, dissemination is planned for:
. Open-source software for real-time system control and sensor/model/manipulation/display integration;
. Design of a complementary mechatronic firewire controller board that includes A/D, D/A, encoders, amplifiers, and low-level control capabilities via FPGS, and
. Detailed descriptions of hardware integration, including WAM arms, Barrett hands, a tactile sensing suite from Pressure Profile Systems, surgical robots, cameras, ultrasound, OCT, a vision-based tracking system, visual and haptic displays, and more.

Broader Impact: Integrated robotic systems that fuse multimodal sensory information to enhance models and manipulate the environment positively impact human lives, particularly in health care, safety, and human assistance. The research also impacts related disciplines, including neuroscience, rehabilitation, and surgery. Researchers will have access to open software and designs. The platforms clearly impact education, people at many career stages, from high school students to senior faculty. The system, and detailed directions on how to produce it, will be made available. The design framework will be used in undergraduate classes in conjunction with existing educational hardware; experimental platforms will be used for course projects. Visiting students and faculty (including WISE girls) will make positive use of the integrated testbeds. There is an ongoing collaboration with Morgan State U. Involving REU and RET participants, outreach is well planned. The dissemination plan includes an active web site; the software will be made available via open-source repository. Furthermore, the website documents all hardware and respective vendors.

This project attempts to combine human strengths in reasoning with machine capabilities in information fusion, task planning, and simulation to manage uncertainty and achieve successful human-robot partnerships to perform complex tasks in uncertain environments that were previously considered impractical or infeasible. The approach consists of three objectives: the use of sensing and control to reduce model registration uncertainty; the definition, simulation and implementation of virtual fixtures to allow humans to intuitively constrain the task; and the development of bi-directional task planning and execution with uncertainty to allow humans and robots to request help from one another.

This research has a number of broader impacts affecting both the academic community and society at large. The work is expected to have significant appeal to those in the manufacturing and medical robotics sectors, as testbeds will impact these areas. The PIs will mentor hands-on research by undergraduate, graduate, and post-doc students and guide them in the dissemination of their research to the scientific community. The team will also provide engineering experiences for middle school girls in the Baltimore area through weekend programs on the Johns Hopkins University campus, including the "Ready, Set, Design!" program. Finally, the development will be made freely available as all software will be fully integrated with open-source ROS.

DESCRIPTION (provided by applicant): Acoustic MedSystems proposes to develop an image-guided robot for MRI, CT and fluoroscopy guidance. The robot will be used for brain, prostate, liver, kidney, breast, and spine intervention. A general purpose robotic manipulator will be designed with development and implementation of a specific configuration for interventional medical biopsy and treatment use. The proposed program represents an innovation in clinical robotic interventions in several ways. It includes a cable-controlled drive and gear assembly linkage, so that actuators can be displaced from the robot motors and controller as much as several meters. Fiber-optic position sensors eliminate all electronics for sensing from the robot. These design innovations will make it possible to build small, purpose-specific robots. Although our proposed new robotic design and implementation is very well-suited to medical interventions, the concept is useful for a broad range of applications extending well beyond this initial application area. Multiple applications of the proposed single drive, multiple DOF robots can benefit from applying this concept and using possible standardized components that may be implemented with it. AMS is engaged in multiple medical research projects that include medical robots, and hence has become aware of many complications of using robots in clinical settings. We have a developed a conceptual approach for a robotic drive system that addresses these clinical issues, while also offering other design benefits and possible cost and size reductions for medical and non-medical robotics applications. PUBLIC HEALTH RELEVANCE: Minimally-invasive procedures are pervasive in medicine and further, needles and catheters are among the least invasive vehicles for accessing the interior of the human body. They can be used for diagnoses (e.g. biopsy), as well as interventions (e.g. injection of liquid therapeutic agents, insertion of surgical tools, radioactive seed implantation, thermal therapy, etc.). Accuracy in targeting the desired location is essential in nearly all procedures to ensure therapeutic or diagnostic efficacy and safety. It has been well-established that image-guided robotic devices are useful for aligning tools accurately with preoperatively planned insertion trajectories. A single robotic insertion has been shown to exhibit approximately half the error of a manual insertion by an experienced surgeon under Ultrasound guidance. However, initial robotic alignment of the device toward its target can never completely eliminate tip placement error, because there is no means of compensation for registration error (which can never be completely eliminated) or perturbations that occur during insertion, including tissue deformation, patient motion, breathing, deflection of a needle at membrane boundaries, etc. Furthermore, there are some locations that are inaccessible to straight-line trajectories (e.g. the pubic arch can obstruct a portion of the prostate in some patients during brachytherapy). These factors have motivated the recent development of steerable needles and surgical tools, and many mechanisms for steering have been proposed. To date, research has focused on tip placement accuracy assessment and model validation, which are necessary first steps toward interventional and diagnostic goals. However, inherent technical difficulties in existing systems have limited the efficacy and application of procedures, namely, 1) lack of adequate image guidance and 2) inadequate control of placement. Furthermore a third and very important issue is that in many cases, site access is limited, making the need for remote device control and location of drive mechanisms (motors, etc.) of paramount importance. The goal of this project is to address these three limitations by creating a system that combines a proprietary general purpose multi-degree-of-freedom precisely controllable robotic mechanism with e3D spatial tracking and image guidance. The proposed integrated robotic system will enable highly accurate tool placement and provide greater control at the target site. Compared to existing percutaneous techniques, this system could improve diagnostic accuracy, treatment efficacy, limit the risks of complications, and enable treatment in those patients who otherwise would have been precluded from the procedure.

Telemanipulation systems consist of a human interacting with a mechanical device on the master side to operate a robot at the remote side. They provide natural opportunities for research in intelligent human/robot collaboration, but existing commercial systems, used in areas such as telesurgery, are not intelligent and therefore only replicate the actions of the human operator. These systems are also proprietary, expensive, and not available for modification by researchers. The goal of this NRI project is to provide an open-source software infrastructure that is designed to work with a broad range of hardware and simulated devices to enable a larger community to pursue research and education in intelligent telemanipulation at a lower cost.

The increasing pace of robotics research can be attributed, at least in part, to the increasing availability of software infrastructure, such as Robot Operating System (ROS), and open hardware platforms. This NRI project focuses on providing a software infrastructure for research in intelligent telemanipulation, leveraging infrastructure developed for the Raven II robot and the da Vinci Research Kit (dVRK) and continuing to extend it to other systems, including simulated robots. The three main tasks are to: (1) engage the community to guide development, (2) develop and implement a common API for the diverse hardware platforms, and (3) provide a set of high-level, platform-independent software modules. The goal is to support research towards semi-autonomous telerobotic systems that can more effectively combine the knowledge, reasoning, and decision-making capabilities of a human with the sensing and manipulation capabilities of a robot.

This award will support the organization of the annual Principal Investigators (PI) meeting for the National Robotics Initiative (NRI), which was launched in 2011. The PI meeting will bring together the community of active NRI participants to provide cross-project coordination of intellectual challenges and best practices in education, technology transfer and general outreach. This activity will also establish a repository illustrating the research ideas explored and milestones achieved by the NRI projects.

The meeting is planned for two days in October 2018 in the vicinity of Washington DC. The format will include presentations by the attending PIs of projects in their final year, a poster session with all other projects, keynote speeches, and panel discussions.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.