Camillo Jose Taylor - US grants

IIS-9875867
Taylor, Camillo
University of Pennsylvania
$86,316 - 12 mo.

CAREER: Object-Level 3D Reconstruction Techniques for Mobile Robots


This is the first year funding of a four-year continuing award. The research component of this CAREER program will focus on the problem of constructing an object level, 3D model of a scene using the image data obtained from one or more video cameras mounted on a mobile robot platform. The ability to construct and maintain accurate, object level models of the environment registered to the image data is a fundamental capability that could be exploited in a variety of ways in a mobile robot application. Toward this end, research will be carried out on three related problems: (1) extending the reconstruction
techniques to make use of the imagery acquired with an omnidirectional camera system (2) developing effective model tracking algorithms that can be used to update the model of the scene as the robot moves (3) developing algorithms that can automatically determine how a given scene could be modeled in terms of parameterized primitives.
The education component of this award includes new course curriculum development, undergraduate and graduate research projects, mentoring of minority students and outreach in the K-12 system. Outreach efforts to minority high-school students will include the development of new course curriculum modules which take advantage of students' natural curiosity about robotic systems to reinforce learning of fundamental techniques in mathematics, physics and computer science.

This project seeks to develop a novel class of computer interfaces centered on a vision-based interaction paradigm, and human augmentation using a range of panoramic sensors and intelligent controllers to provide assistive technology to disabled users. The goal of such interfaces is to enable people with physical disabilities such as impaired limbs, paralysis, or tremors to overcome difficulties associated with accessing computers and products with embedded computers such as wheelchairs, household and office electronic equipment, and robotic aids with traditional input devices. The goal is to create the framework, architecture, scientific algorithms, and augmentative hardware and software to facilitate (a) interaction; (b) control and tasking; and (c) programming of computers and computer-controlled smart devices. There are two main sets of research problems that need to be solved: (a) the development of novel, flexible, portable, adaptable interfaces that allow users with physical disabilities to interact with computers and computer controlled devices by touching and feeling; and (b) human augmentation via a combination of inexpensive sensors and controllers, along with a set of algorithms and software for computer mediated control. This research will result in the next generation of interfaces for users to interact with computers and robot assistants, and more generally, devices with embedded controllers. Although the immediate goal is to develop the basic framework, methods, and algorithms using the smart wheelchair as a test product, the basic ideas will be applicable to a wider range of products.

EIA-0130858
Camillo Taylor
University of Pennsylvania

The idea of deploying teams of small, inexpensive robotic agents to accomplish various sensing, manipulation and communication tasks is one that has gained increasing currency over the last few years. This project will involve three interrelated research thrusts that investigate various aspects of this paradigm. The first thrust deals with the problems associated with coordinating the motion of teams of robots. Some of the questions that are addressed by this effort include the problem of controlling the motion of robots moving in formation and coordinating the action of robots engaged in cooperative manipulation of an object. The second thrust focuses on the issues associated with combining the information obtained from distributed robots to form a coherent model of the environment. The third area of research concerns the problems associated with designing and analyzing networking strategies that are appropriate for use with distributed teams of robotic agents. Since the platforms are mobile, many of the traditional networking strategies, which were designed with fixed infrastructure in mind, are not applicable. As part of this proposal we intend to investigate questions concerning the appropriateness of various wireless networking technologies such as IEEE 802.11b and Bluetooth. This proposal requests funding to purchase the equipment required to develop a fleet of networked robots that would serve as a shared testbed for our research efforts.

Abstract
0205336


This research pursues the development of a network of robots with cameras and
their optimal, dynamic positioning for monitoring a cluttered scene, under different conditions of visibility. The robots must discover their neighbors, localize
themselves with respect to their neighbors and integrate information with that
available from the other members of the team on the fly. Further this information
must allow a remotely located human operator to immerse herself in the
environment. The project addresses the development of enabling algorithms and
technology tailored to such applications as fire-fighting.

Engineering - Mechanical (56)

The project is modeled on successful existing implementations of undergraduate robotics efforts at MIT and Swarthmore, but with a specific emphasis on the freshman and sophomore experience. The investigators are purchasing equipment to develop and implement the Laboratory for Undergraduate Robotics Education (LURE), permitting them to develop new, technologically advanced laboratory space for undergraduate education. The laboratory allows them to change to a mode of teaching that provides analysis, design, and manufacturing skills in a robotics setting. The equipment requested also permits them to inject engineering content with a hands-on laboratory component into the curriculum at an early stage (freshman year). This provides some perspective and motivation to beginning students, who currently receive the impression that engineering consists only of theoretical physics and mathematics. In the evaluation study, they are investigating how the differences in background preparation and training of incoming students affect development for high-tech courses related to robotics. They are developing and disseminating robotic-related curricular materials for use both in interdisciplinary college-level education, as well as K-12 outreach programs.

This project, developing an experimental testbed to work on different aspects of control and sensing for mobile networks of cameras and microphones, envisions a system of cameras, MACNet, moving in three dimensions enabling a Mobile Ad Hoc Camera Network. The development of this testbed provides the experimental infrastructure for the following interdisciplinary projects:
Monitoring, Evaluation, and Assessment for Geriatric Health Care,
Assistive Technology for People with Disabilities,
Digital Anthropology, and
Visual Servoing for Coordinating Aerial and Ground Robots.
MACNet cameras will be used to track patients and salient dynamic features for the 1st project. MACNet will simulate intelligent homes and museums with active cameras providing feedback to smart wheelchairs and providing information about target areas in the 2nd Project. MACNet will allow datasets of video to be acquired and analyzed for three-dimensional reconstruction and archiving in the 3rdProject; and for the last project, MACNet will provide dynamic platforms with cameras which will simulate aerial vehicles to track, localize, and coordinate with existing ground based autonomous vehicles with applications to surveillance and monitoring for homeland security. Finally, MACNet will be used as a testbed for research on distributed active vision, a theme that will bring together all researchers.

Broader Impact: The results will be directly applicable to a large class of problems in which communication, control, and sensing are coupled, with applications in smart homes, communities for assistive living, and surveillance and monitoring of homeland security. This cross-fertilization will contribute to train students with broader perspectives and potential new approaches to problem solving. The lab will be used by students and the institutions will leverage existing outreach programs in which both faculty and students participate, such as Robotics for Girls and PRIME.

The University of Pennsylvania has joined the multi-university Industry/University Cooperative Research Center for Safety, Security and Rescue Research located at the University of South Florida and the University of Minnesota. The I/UCRC will bring together industry, academe, and public sector users together to provide integrative robotics and artificial intelligence solutions in robotics for activities conducted by the police, FBI, FEMA, firefighting, transportation safety, and emergency response to mass casualty-related activities.

The need for safety, security, and rescue technologies has accelerated in the aftermath of 9/11 and a new research community is forming, as witnessed by the first IEEE Workshop on Safety, Security and Rescue Robotics. The Center is built upon the knowledge and expertise of multi-disciplinary researchers in computer science, engineering, industrial organization, psychology, public health, and marine sciences at member institutions.

Intellectual Merit
This project mixes results and methods from cognitive psychology, computational vision and learning, neuromechanical systems biology, and robotics to develop a computer assisted environment for studying animal sensorimotor strategies, discovering how they undergird animal cognitive capabilities, and using those insights to inspire new algorithms for robot navigation, localization and situational awareness. We observe live, intact, highly mobile terrestrial invertebrate predators such as ghost crabs, desert scorpions and tiger beetles in carefully constructed habitats that challenge their ability to negotiate terrain and navigate space. We automate the collection, annotation and mathematical model extraction of their behavior from massive, parallel real-time recordings of visual, muscle, neural, and biomechanical recordings. We mine these data sets to develop intuitive hypotheses as well as formal mathematical representations of the basis on which these animals organize their own sensorimotor data streams to compile novel behaviors from previously consolidated constituents in a process of autonomous mental development. We add numerous existing sensor suites to highly agile existing robot bodies and instantiate algorithmically the hypothesized animal models to develop supporting or refuting evidence that challenges and refines them.
Broader Impacts
Scientifically, the new computational tools and ideas we identify in the interrelations we set up promise a bridge between whole areas of disciplines that have long been divided by spatiotemporal scale and the concomitant gap in analytical tradition, terminology and methods. For example, the study of these complex competencies in simpler species offers a new glimpse at the building blocks of cognition in species more closely related to humans. From the perspective of technological invention, algorithms pioneered in this research could lend an animal-like quality to a machine?s proximal tenacity in engaging its environment and even its overall situational awareness within unstructured worlds. For example, the team is inspired to imagine what it might be like to have a search and rescue robot with the (taskable) capabilities of a ghost crab. From the perspective of training and education, the automated database collection and management tools developed in this project bring to a mass audience the conceptual and computational building blocks that have heretofore been the exclusive province of a small group of experts. For example, a universally accessible (?cloud-based?) tool for unifying the design, parsing, display, and cross comparison of robots and animals searchable at will from the most intimate to the broadest scale of design and operation would have a profound impact on the ability of teachers at many different levels to motivate the fascination and unity of both synthetic and biological science.

This Integrative Graduate Education and Research Training (IGERT) award to the University of Pennsylvania supports the development of a new training paradigm for perception scientists and engineers, and is designed to provide them with a unique grasp of the computational and psychophysical underpinnings of the phenomena of perception. It will create a new role model of a well-rounded perceptual scientist with a firm grasp of both computational and experimental analytic skills. The existence of such a cadre of U.S. researchers will contribute to the country's global competitiveness in the growing machine perception and robotics industry.

Research and training activities are organized around five thematic areas related to complex scene perception: (1) Spatial perception and navigation; (2) Perception of material and terrain properties; (3) Neural responses to natural signals, saliency and attention; (4) Object Recognition in context and visual memory; and (5) Agile Perception. Interdisciplinary research will enable new insights into the astounding performance of human and animal perception as well as the design of new algorithms that will make robots perceive and act in complex scenes.

IGERT trainees will commit in advance of acceptance to a five-year graduate training program, comprising the following components: (1) Core disciplinary training; (2) one-year cross-disciplinary training in a chosen second discipline; (3) participation in two foundational and one integrational IGERT courses; (4) attendance of an interdisciplinary IGERT seminar; (5) co-advising throughout the 5 graduate years by an interdisciplinary faculty team ; and (6) completion of the Ph.D. dissertation.

IGERT is an NSF-wide program intended to meet the challenges of educating U.S. Ph.D. scientists and engineers with the interdisciplinary background, deep knowledge in a chosen discipline, and the technical, professional, and personal skills needed for the career demands of the future. The program is intended to catalyze a cultural change in graduate education by establishing innovative new models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries.

The proposed I/UCRC for Robots and Sensors for the Human Well-being (RoSeHuB) will focus on complementing a broad variety of off-the-self sensors with intelligent processing software that enables them to extract useful information about the operating environments in medicine and agriculture. RoSeHuB research will make heavy use of commercial cameras that can work in different parts of the electro-magnetic spectrum (i.e., visible, IR, Thermal, etc.), laser or radar sensors, etc. Sensors or sensor systems may exhibit different degrees of mobility. They may be embedded in robots or flying drones or they may be fixed with limited degrees of motion (PTZ cameras). In the areas of algorithms and learning methods the focus and the challenge is on creating methodologies that can balance real-time operation and computational power while providing high level semantic information either for planning, interaction or situational awareness for human operators. With respect to robots, efforts will focus on building systems with advanced mobility, manipulation, human-machine interaction, and coordination skills.

Robots and sensors can lead to more effective precision agriculture techniques that provide more food than current levels while they save water and prevent soil erosion. Similarly, robots play a critical and growing role in modern medicine, from training the next generation of doctors, dentists, and nurses, to comforting and protecting elderly patients in the early stages of dementia. The proposed Center will attract large companies to the pertinent domains and energize innovative startup companies, both through research and through the production of highly trained graduate students with advanced coursework in sensory-based robotic systems and hands-on exposure to multi-disciplinary, integrative systems. The Center will also fund a projects to pursue new pertinent high-risk initiatives, ensuring that technology continues to meet emerging societal needs. RoSeHuB faculty will aggressively recruit women and minority graduate and undergraduate students and host an annual summer camp for middle-schoolers from underrepresented groups.