Amin Vahdat - US grants

EIA-9986024
Carla Ellis
Duke University

CISE Research Instrumentation: System Support for Mobile and Embedded Workloads

This proposal seeks funding to deploy a wireless infrastructure in the Computer Science Department at Duke University.

The requested infrastructure will be used as a testbed for our research. The individual research results will be combined to provide a coherent system for the deployment of mobile and embedded applications. A goal of this research is to evaluate the success of the individual system components by demonstrating the viability of the disaster recovery application, in effect approximating next generation environments using currently available technology. In addition to these research results, the requested infrastructure will aid the department's continuing efforts into education and outreach. The equipment will serve as the basis of a project-oriented course to develop applications and system support for the infrastructure and will also serve as the basis for summer internship projects in Duke's continuing outreach efforts to underrepresented groups.

This project explores employing distributed computational resources to combat the unreliability and highly variable performance inherent to wide-area systems. The goal of this work is to balance wide-area performance, security, and resource utilization by efficiently replicating and migrating computation using inexact information distributed across multiple administrative domains. To this end, this research addresses three areas important to scaling the global network infrastructure: i) dynamic placement and migration of computation for efficient utilization of global resources, ii) resource allocation among principals simultaneously utilizing resources in multiple administrative domains, and iii) a security infrastructure that enables the fine-grained transfer of rights for wide-area computation. The impact of this work will be increase fairness, scalability, and fault tolerance for distributed systems leveraging this infrastructure.

The goal of this project is to develop an integrated
hardware/software infrastructure to support power management for
battery-powered mobile and wireless applications. These future
environments will support applications with demanding requirements
such as disaster recovery. Energy conservation, especially for
mobile and embedded devices, promises to have significant economic,
environmental, and societal impacts.

The activities focus on three key directions: i) the development of
power measurement tools, workloads, and experimental methods to
evaluate energy consumption, ii) the energy-aware APIs to allow
application-directed power management, and iii) the development of
system support for high-level solutions.

These research projects all rely on experimental techniques for
evaluating ideas. Making empirical measurements and observations on
device and workload characteristics pinpoints the problem areas of
greatest potential. Initially formulating simulation models narrows
the solution space and allows consideration of new architectures.
Finally, constructing working prototypes allows observation of all
activity associated with real operating environments and offers
deeper insights into their behavior. The popularity and
accessibility of the palmtop and handheld platforms gives this
research significant potential for immediate technology transfer.

0088078
Astrachan, Owen L.
Duke University

CRCD: Modules and Courses for Ubiquitous and Mobile Computing

This project educates students in the techniques and technologies required to deploy next generation wireless information systems. Courses developed for this project focus on ubiquitous and wireless computing. And, these same technologies are used in classrooms to deliver the curriculum as a part of the project that supports "active learning." The areas of research chosen for migration into the curriculum include mobile code (placement and migration of code to adapt to rapidly changing clients, networks, and service characteristics), transcoding (transforming multimedia web content to save bandwidth and thus energy consumption at the destination device), active name architectures (where resource names are decoupled from specific hosts when resolving services), and energy aware operating systems (where the goal is to make basic interactions of hardware and software as energy efficient as possible for local computation). The project migrates research topics into advanced undergraduate courses and graduate courses, developing modules, assignments, software and curricular support that engage and educate students in the technologies of mobile and wireless information systems. The materials developed are integrated into five computer science courses. The courses apply active lectures, a form of active learning, to deliver content.

This project explores two complementary techniques for addressing
fundamental limitations in replicating network services. The first aspect
of this proposal seeks to automatically replicate service programs and
state information to allow transparent caching or replication of dynamic
services. The goal of the research is to allow transparent caching or
replication of dynamic services, a key step toward automatically converting
unscalable service implementations into scalable ones. The second thrust of
this work is to allow network services to dynamically trade replica
consistency for increased system availability and performance. The TACT
(Tunable Availability and Consistency Tradeoffs) toolkit allows Internet
services to flexibly and dynamically choose their own
availability/consistency tradeoffs. We use three consistency metrics,
Numerical Error, Order Error and Staleness to capture application-specific
consistency requirements of Internet services. Applications use these
metrics in addition to application-specific parameters to assign a numeric
value to system consistency, e.g., the percentage of user requests that
must eventually be rolled back because of underlying replica
inconsistency. Finally, TACT allows consistency to be specified on a
per-user, client, and replica basis, enabling differentiated quality of
service.

Clustering technologies enable incremental scaling of Internet server sites at modest cost. It is increasingly common in cluster-based service architectures to distribute incoming request traffic among servers using redirecting intermediaries integrated into the network switching fabric or interposed between the client and servers. However, Internet services and their delivery architectures continue to evolve rapidly. This creates new challenges and opportunities for redirecting intermediaries, and motivates basic research in both the mechanisms for request redirection and the request routing policies for specific service environments.

This work will undertake a coordinated research program to expand the potential of redirecting intermediaries as an enabling technology for scalable Internet services. The work focuses primarily on integrating service-aware redirection and request routing as network-level functions in a high-speed switching architecture. The methodology combines simulation, construction of software prototypes, and evaluation of prototypes using synthetic and real workloads.

The expected outcomes of the work are:
An improved understanding of the role of request routing as an enabler for large-scale Internet services,
Simulation results evaluating these policies in large systems,
Software prototypes that demonstrate the value of these solutions in practice for Web-based services and network storage services, and
Opportunities to train students as participants in this research at both the graduate and undergraduate levels.

In summary, the research work has the following basic objectives:

1. Define protocol features essential for redirection at the level of the transport protocol. The switch routes incoming requests on each transport connection to any active server at the discretion of a service-specific routing policy; referred to as Anypoint communication.

2. Implement an Anypoint-capable transport protocol that supports features commonly required by service protocols: reliable communication, ordering and duplicate suppression, and congestion control.

3. Define interfaces and capabilities for service-specific policy modules in Anypoint intermediaries. This defines an architecture for decomposing service protocol implementations into a client, a server, and service module to extend the intermediary.

4. Evaluate the intermediary architecture defined by Anypoint by constructing software prototypes of virtualized service implementations. The initial targets are HTTP 1.1 application services and NFS.

Managing the energy consumption of computers requires cooperation
between energy aware applications and operating systems. This
research seeks to fully explore the energy management space
surrounding the interaction of applications and the operating system.
Applications should adjust their energy consumption when appropriate,
but must be provided accurate information on their individual energy
consumption. The operating system must implement the mechanisms and
policies to determine energy consumption and allocate it fairly as a
global system resource.

This research will first re-examine operating system structure with an
emphasis on managing energy as a first class resource. Energy
management cuts across all traditional system resources, with the CPU,
disk, network, and memory all exhibiting unique energy consumption
characteristics. Next, the work will explore policies for allocating
energy to competing tasks. The goal is to maintain fairness while
observing user-specified priorities and soft real-time deadlines.

The end product of this research will be a comprehensive framework for
globally managing energy in a diverse set of scenarios, ranging from a
single mobile computer, to wireless sensor networks that may have
aggregate goals across a large number of sensors, to hosting centers
that wish to provide maximum performance with minimum energy
consumption.

The power of the national information infrastructure has expanded
enormously over the last decade. Commercial Internet service and
government-sponsored advanced networks for research and education
support a range of national priorities. Unfortunately, rapid
development and deployment of robust, adaptive network software is
hampered by a fundamental obstacle: prototype systems are difficult to
evaluate due to scale and complexity of their host environment---the
Internet.

This proposal seeks to remove this obstacle by constructing a software
environment for evaluating prototype network software systems under
realistic, controlled, repeatable conditions through scalable Internet
emulation. The proposed system, ModelNet, emulates a wide-area
network on a high-speed cluster, enabling researchers to deploy
unmodified software prototypes in a configurable Internet-like
environment and subject them to faults and varying network conditions.

This research will first investigate techniques for scaling network
emulation to thousands of unmodified applications with aggregate
communication bandwidth of over 10 Gb/s. Second, it will enable the
community to leverage large-scale network emulation as a primary
technique for rapidly developing and evaluating next-generation
Internet services and applications. Finally, the research will seek
fundamental improvements in network service robustness and performance
by providing a controlled environment for subjecting network services
to a range of realistic deployment scenarios.

National Science Foundation
Distributed Systems Research
CISE/CNS

ABSTRACT
PROPOSAL NO.: 0411307
PRINCIPAL INVESTIGATOR: Vahdat, Amin M
INSTITUTION: University of California, San Diego
PROPOSAL TITLE: Framework for Designing, Evaluating, and Deploying Global-scale Adaptive Networked Systems

Today, designing and building networked and distributed systems remains fraught with difficulty. Yet, addressing these challenges is becoming increasingly important as an ever larger fraction of the world's infrastructure comes to rely upon networked systems. Challenges faced by networked systems include failures, lack of fate-sharing among sys-tem components, highly variable communication patterns, race conditions, difficulty in reproducing bugs, asynchrony, security concerns, etc. While the advent of higher-level programming languages such as Java has raised the level of abstraction and eased this burden, most programmers still face the task of reinventing the appropriate techniques for dealing with asynchronous, failure-prone network environments known by a handful of elite programmers.

This proposal is exploring the design and implementation of a high-level language and runtime environment for building and operating robust, high-performance, and highly-available global-scale distributed systems. Target applications include Internet-scale routing protocols, planetary-scale network testbeds, and computational Grids. This work aims to holistically support the entire distributed system life-cycle from design to de-ployment and operation. The overall goals of this work are to effect a qualitative shift in: i) the time required to build, debug, and deploy robust distributed systems, ii) the ease and accuracy of translating high-level policy to low-level system specifications and con-figurations, and iii) the ability to compare the utility of competing systems architectures in a fair and consistent manner.

Dr. Brett D. Fleisch
Program Director, CISE/CNS
June 29, 2004
.

The proposed research program will develop and catalyze the core component of a next-generation Internet architecture that greatly increases the functional capabilities, robustness, flexibility, and heterogeneity of the Internet in the face of modern application requirements.

Our approach has two inter-related thrusts. The first is to address the following question: What is the right architecture for the next generation of global networking infrastructure? Because proposing a clean-slate design, or treating this question as a thought experiment, has little chance of practical impact, the second thrust is to build the research infrastructure that allows us to discover, evaluate and deploy this architecture.

Specifically, overlay networks have recently emerged as a promising technique for introducing disruptive technology into the Internet. This focus on overlay functionality has left unanswered the single most critical question: What lies underneath? What is the appropriate minimal, universally shared environment to underlay the overlays?

The core of an underlay that supports an increasing multiplicity of overlay opportunities lies in three key elements. First, there must be some means of information discovery and dissemination through with overlays learn about the underlying Internet; a so-called Information Plane. Second, elevating overlays to first-class objects places special emphasis on the coordinated assignment of complex collections of network resources (e.g., bandwidth, storage, computational cycles, shared information) to competing overlays in an economically coherent, computationally practical fashion. This requires an Economic Framework for resource allocation. Third, there is a complementary question of how to define the basic unit of resource, and the challenges in the implementation of decisions made in resource allocation by means of Virtualization. Combined, these elements form the critical core of an operating environment for overlays; the necessary universal substructure for an overlay-enabled world. These three elements play a pivotal role in this research program: they are both a key objective (output), and at the same time, essential for building a scalable wide-area testbed that allows researchers to evaluate new ideas under real-world conditions.

This proposal is developing a set of technologies to create federated content distribution utilities that support the simultaneous delivery of a wide variety of content to overlapping sets of clients with statistical quality of service assurances. The federated content distribution infrastructure will: i) simultaneously meet the performance demands of high-bandwidth and low-latency content delivery and the resource allocation constraints of constituent service providers; ii) weather a variety of attacks both from the outside and from self-interested or malicious nodes directly participating in the protocol; iii) incorporate basic algorithms for distributing content under a wide variety of dynamic network conditions. Because the infrastructure is shared by a variety of applications and hosted by a number of mutually distrustful administrative domains, the system must provide mechanisms that both adjudicate among competing applications and allow each administrative domain to maintain its own resource allocation policies.

If successful this research will effect a qualitative shift in: i) the way in which data is distributed across the Internet; ii) basic algorithms for determining optimal data distribution strategies across arbitrary data meshes; iii) the levels of reliability
and performance that can be achieved in critical nation-wide or global-scale event notification systems (e.g., the air-traffic control system); and iv) the ability of resource-poor providers to harness federated distribution utilities to publish urgent content such as Internet worm or virus signatures on a global scale.

Large-scale computational resources are increasingly being delivered through distributed clusters of commodity workstations, which, when taken as a whole, provide the raw horsepower of traditional supercomputers at significantly reduced cost. Unfortunately, economic reality still dictates that large clusters must be shared across multiple, distinct applications, each with their own resource needs. This project focuses on designing and implementing an efficient management framework that enables the creation, allocation, and management of virtual clusters.

A logical abstraction layered on top of a set of physical machines, virtual clusters harness virtual machine technology to more efficiently share computational resources between competing
application demands while ensuring fault isolation. Critically, this proposal leverages the power of virtual machine monitors to provide novel functionality for an emerging class of applications. In
particular, by exposing the dynamic levels of parallelism, dilating logical time, and supporting apparently infinitely large clusters, this work supports the distinctive needs of grid computing, network modeling, and Internet epidemiology.

An output of this work will be a fully operational environment for managing cluster-based computational resources integrated with publicly available virtual machine technology. In addition to dynamically adjusting resources in response to changes in demand and application load, virtual clusters can instantly create clones of existing virtual machines, a functionality critical to the deployment of large-scale high-fidelity honeypots. Finally, the ability to slow down logical time within a virtual cluster---thereby speeding up the relative speed of network communication---enables the emulation of network links orders of magnitude faster than those typically available on commodity clusters.

Research in network security to date focuses largely on defenses---mechanisms that impede the activities of an adversary. Practical security, however, requires a balance between defense and deterrence. While defenses may block current attacks, without a meaningful risk of being caught adversaries are free to continue their attacks with impunity. Deterrence is usually predicated on effective means of attribution---tying an individual to an action. In the physical world attribution is achieved through forensic evidence, but constructing such evidence is uniquely challenging on the Internet.

This project is developing a novel architectural primitive---private attribution, based on group signatures--that allows any network element to verify that a packet was sent by a member of a given group. Importantly, however, actually attributing the packet to a particular group member requires the participation of a set of trusted authorities, thereby ensuring the privacy of individual senders. In addition, this work explores content-based inverse firewalls that can inspect the content of traffic leaving a secured network, ensuring that sensitive information is kept within an enterprise. Approved data can then be labeled by the inspecting firewall, providing an audit trail should concerns arise.

Broader Impacts: This research is developing a key architectural component to improve the level of security and assurance available to network services. In addition, the PIs are initiating a dialogue among both researchers and network operators about critical policy aspects of network security. In particular, information about the sources of both normal and attack traffic that must be safeguarded according to some policy.

One of the primary limiting factors to improving the security, robustness, and performance of the global computation and communication infrastructure is the difficulty of subjecting novel protocols, applications, and services to realistic network conditions, at scale. Recent advancements in emulation technology enable running unmodified applications on stock hardware and operating systems. These applications can be accurately subject to the wide-area communication characteristics of topologies consisting of tens of thousands of routers and 10 Gigabits per second of aggregate bandwidth. Given these advancements, the principal challenge in carrying out realistic large-scale experiments lies in what to emulate rather than how to emulate it. The community currently lacks the ability to: i) generate topologies that reflect important characteristics of the Internet, ii) annotate the topologies with link capacities, latencies, and loss rates, and iii) subject a given topology to realistic levels of background traffic.

The goal of this proposal is to develop a unified infrastructure for creating realistic network topologies and to accurately annotate them with appropriate distributions for bandwidths, latencies, and round trip times. Further, it will be possible to subject individual topology links to realistic packet arrival processes.

Broader Impact: Taken together, the components of this proposal have the potential to effect a qualitative change in the community's ability to conduct controlled experiments that accurately reflect a variety of both current and future network scenarios. Such ability will then dramatically accelerate the

Designing and implementing robust and high-performance distributed
systems remains a challenging, tedious, and error-prone task.
Building correct systems is difficult because of the asynchronous,
heterogeneous, failure-prone and adversarial environments
these systems are subject to.
Tracking the source of performance problems often
reduces to searching for a needle
in a haystack: among millions of individual message
transmissions, algorithmic decisions, and a large number of
participating nodes, which network link, computer, or low-level
algorithm results in performance degradations?

This research aims to create a programming environment for building
distributed systems, that includes
(1) programmings language that make the
structure of the system explicit in a manner that
allows the compilation of readable high-level descriptions into
high-performance implementations, and
(2) tools that can exploit the explicit structure to
perform automatic system level analyses that will
help developers locate, understand and fix behavioral anomalies in deployed systems.
By increasing our understanding of which aspects of the development process
are automatable, this research will show the high-level aspects
where human creativity needs to be focused, without paying the price of
performance penalties.
This research can lead to the availability of novel
architectures and services as an increasing
developer population is empowered to design and build
these systems

Real-world security policies invariably involve questions of ``who'' and ``what''--who are the principals, what data are they seeking to access, and so forth. By contrast, the present-day Internet architecture concerns itself primarily with issues of ``how'' and ``where''-- what are the protocols by which a data item is delivered and to which topological endpoints. This inherent dissonance of purpose makes Internet security a bolt-on affair---with abstract access control policies pushed off to be implemented by particular applications or mapped onto the poor approximations provided by network-level abstractions (e.g., network firewalls). Moreover, these imperfect mechanisms are themselves attacked with impunity since today's Internet architecture provides a functional anonymity that insulates attackers from any meaningful liability.

This project is developing two key architectural capabilities--host attribution (which physical machine sent a packet) and data provenance (what is the ``origin'' of the data contained within a packet)--to enable the direct expression of a wide-range of security policies. Moreover, these properties are being implemented in a fashion that mandates their use (in a strong sense) by the network, but manages to preserve end-user privacy. The PIs are focusing on two key applications in this work: forensic trace-back and attribution for the purpose of attack deterrence, and defensive data-exfiltration to place precise controls over what kinds of data may move across a network.

Broader Impacts: This research is developing key architectural components to improve the level of security and assurance available to network services. In addition, the PIs are initiating a dialogue among both researchers and network operators about critical policy aspects of network security. In particular, information about the sources of both normal and attack traffic that must be safeguarded according to some policy.

Networking research has long relied on simulation as the primary vehicle for demonstrating the effectiveness of proposed protocols and mechanisms. Typically, one simulates network hardware and software in software using, for example, the widely used ns-2 simulator. Experimentation proceeds by simulating the use of the network by a given population of users using applications such as ftp or web browsers. Synthetic workload generators are used to inject data into the network according to a model of how the applications or users behave.

In order to perform realistic network simulations, one needs a traffic generator that is capable of generating realistic synthetic traffic in a closed-loop fashion that ?looks like? traffic found on an actual network. Unfortunately, the networking community suffers from a lack of validated tools and models suitable for synthetic traffic generation. As a result, all too often, networking technology is evaluated using ad hoc workloads with an unknown relationship to traffic seen on real links and hence begs the question of how believable the results of the evaluation are.

This project is a collaborative effort to develop a synthetic traffic generation resource for the experimental networking research community. The resource consists of (1) synthetic traffic generators for the ns-2, ns-3, and GTNets software simulators, and Linux and BSD-based testbeds, (2) a repository of datasets to be used by the traffic generators to generate traffic that is statistically equivalent to traffic found on a variety of network links including campus networks, wide-area backbone networks, corporate intranets, wireless networks, etc, and (3) a set of traffic analysis tools to enable researchers to generate empirical models of traffic on network links of interest and to use these models to drive the synthetic traffic generation process.

Proposal #: CNS 08-21155
PI(s): DeFanti, Thomas A.
Krueger, Ingolf H.; Papadopoulos, Philip M.; Smarr, Larry L.; Vahdat, Amin M.
Institution: University of California ? San Diego
La Jolla, CA 92093-0934
Title: MRI/Dev.: Development of Instrumentation for Project Green Light
Project Proposed:
This project, developing an instrument called GreenLight, measures, monitors, and optimizes the energy consumption of large-scale scientific applications from many different areas. The work enables inter-disciplinary researchers to understand how to make ?green? (i.e., energy efficient) decision for IT computation and storage. Consequently, an experienced team might be able to make deep and quantitative explorations in advanced architecture, including alternative circuit fabrics such as Field Programmable Gate Arrays (FPGAs), direct-graph execution machines, graphics processors, solid-state disks, and photonic networking. The enabled computing and systems research will yield new quantitative data to support engineering judgments on comparative ?computational work per watt? across full-scale applications running at-scale computing platforms, thus helping to re-define fundamentals of systems engineering for a transformative concept, that of green CyberInfrastructure (CI). Keeping in mind that the IT industry consumes as much energy (same carbon footprint) as the airline industry, this project enables five communities of application scientists, drawn from metagenomics, ocean observing, microscopy, bioinformatics, and the digital media, to understand how to measure and then minimize energy consumption, to make use of novel energy/cooling sources, and employ middleware that automates optimal choice of compute/power strategies. The research issues addressed include studying the dynamic migration of applications to virtual machines for power consumption reduction, studying the migrations of virtual machines to physical machines to achieve network locality, developing new power/thermal management policies (closed loop, using feedback from sensors), classifying scientific algorithms in the context of co-processing hardware such as GPUs and FPGAs, and developing algorithms for resource sharing/scheduling in heterogeneous platforms. The full-scale virtualized device, the GreenLight Instrument, will be developed to measure, monitor, and make publicly available (via service oriented architecture methodology), real-time sensor outputs, empowering researchers anywhere to study the energy cost of at-scale scientific computing. Hence, this work empowers domain application researchers to continue to exploit exponential improvements in silicon technology, and to compete globally. Although the IT industry has begun to develop strategies for ?greening? traditional data centers, the physical reality of modern campus CI currently involves a complex network of ad hoc and suboptimal energy environments in departmental facilities. The number of these facilities increases extremely fast creating campus-wide crisis of space, power, and cooling due to the value of computational and data intensive approaches to research. This project addresses these important issues offering the possibility to improve.

Broader Impacts: The project enables researchers to carry-out quantitative explorations into energy efficient CyberInfrastructure (CI) and to train the next generation of energy-aware scientists. It enlists graduate students from five disciplinary projects, involves minority serving institutions, and is likely to have direct impact on commercial components of the nation?s CI.

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).
Increasingly, computation and storage are moving into a planetary cloud accessible across ever-widening Internet pipes. Unfortunately, asynchrony, failures, and heterogeneity make it difficult to harness available computing power and storage, with inherent tradeoffs in performance, data consistency, and availability. The goal of this research is to raise the level of abstraction for building planetary-scale services, in particular to: i) make fault tolerant and high-performance storage a baseline abstraction for a variety of services, and ii) simplify the process of building extensible and distributed data structures that match the requirements of a range of services.
Taken together, this project has the potential to effect a qualitative shift in our ability to deploy next-generation high-performance and highly available network services. First, application developers will be able to leverage fault tolerant, locality-aware data storage as a given for their distributed applications. Second, the research will deliver primitives to ease the problem of deploying new distributed data structures. We will provide the abstractions to manage distribution, replications, and faults in these environments.
The broader impacts of this project will include leveraging our infrastructure to conduct studies of large-scale service architectures, first in advanced graduate courses and later in undergraduate courses, and a public release of the replication and extensible data structure software underlying our work for research and educational purposes.

Proposal #: CNS 09-23523
PI(s): Papen, George Fainman, Y.; Vahdat, Amin M.
Institution: University of California - San Diego
La Jolla, CA 92093-0934
Title: MRI/Dev.: Development of a Scalable Energy Efficient Datacenter (SEED)

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).

Project Proposed:
This project, building a Scalable Energy Efficient Datacenter (SEED), develops an integrated solution that encompasses physical layer hardware, protocols, and topologies that can provide the expected size and performance scaling for future data centers while minimizing the cost and energy per switched bit. The work creates the knowledge base required for the development of next generation scalable, energy efficient datacenters. Unique features of this instrument include novel statistical multiplexing modules to reduce connection complexity, a circuit switched optical interconnection fabric, and the ability to accommodate novel protocols, components and subsystems in a realistic system environment. With a design based entirely on commodity components and a non-blocking and scalable switch, the baseline configuration of the SEED instrument will connect more than 250 servers, each operating at 10 Gb/s. The fully configured instrument is a hybrid electrical packet/optically circuit-switched network designed to efficiently route large data flows into a circuit-switched optical core utilizing an optical switch from a previously funded MRI, Quartzite. The instrument supports several newly established multidisciplinary projects including the ERC Center for Integrated Access Networks (CIAN), the MRI GreenLight project, the Center of Interdisciplinary Science for Art, Architecture, and Archaeology (CISA3), and projects at the San Diego Supercomputer center. Specifically, SEED is expected to create the technology base for an order of magnitude improvement in both the cost and energy per switched bit. This will be accomplished by the development of new protocols and topologies, measuring and optimizing application dependent traffic patterns, providing critical system-driven specifications of a technology roadmap for the development of novel photonic technologies, and acting as a platform for training the next generation network engineers that are equally versed in both optical and electrical networks. The following four issues are associated with the SEED instrument.
- Design of flow scheduling techniques for fat trees that fit both electrical and hybrid systems,
- Algorithms for fault tolerance (components in large scale communication switches fail),
- Optimal Wavelength Division Multiplexing (WDM) design (uses multiple lasers and transmits several wavelengths of light (lambdas) simultaneously over a single optical fiber)
- Technology road map based on findings on performance metrics pertaining to building, testing, and operating the initial optical aggregation, transmission, and switching hardware to inform the Center for Integrated Access Networks (CIAN) ERC.

Broader Impacts: The engine of the 21st century economy, the creation of wealth through information processing, utilizes data centers as its cornerstones. Hence, technologies that can enable larger and more energy efficient information processing will affect many, if not every, aspect of modern life. Access to efficient remote processing should dramatically reduce the amount of physical transport and avoid the expense and human costs of unnecessary commuting, minimize environmental impact from infrastructure and pollution, substantially reduce our dependence on energy imports, improve educational opportunities, enhance the distribution of medical services, and increase overall national security. Thus, the infrastructure to carry these services constitutes a precious national resource, perhaps as precious as the air, rail, and road transportation. Indeed, it should enable this country to better compete globally.

Modern data centers host operations as varied as flight planning, drug discovery, and Internet search running on thousands of machines. These services are often limited by the speed of the underlying network to coordinate parallel data access. In current data centers, network I/O remains a primary bottleneck and a significant fraction of capital expenditure ($10B/year). Compounding the problem are operational issues caused by interference between services, down times due to failures, and violations of performance requirements.
This project will develop a hardware/software architecture with the following capabilities: i) non-blocking bandwidth to hundreds of thousands of hosts; ii) ``slicing'' across services with minimum bandwidth guarantees; iii) detecting fine-grained performance violations; iv) tolerating a range of failure scenarios; v) supporting end host virtualization and migration. Our goal is to enable modular deployment and management of networking infrastructure to keep pace with the burgeoning computation and storage explosion in data centers. This work will result in a prototype fully functional virtualizable data center network fabric to support higher-level services.
Broader impacts include: i) outreach to under-represented minorities through the UCSD COSMOS program; ii) a public release of the data center communication workloads, protocols, and algorithms we develop; iii) working with our industrial partners and advisory board to address key performance and reliability issues in a critical portion of the national computation infrastructure. A significant outcome will be students trained in data center networking and cloud computing.

This research addresses a fundamental computer systems challenge presented by the rise of datacenter computing as the dominant platform for ``cloud computing.'' Today, datacenters host mission-critical services for IT, health, and financial institutions using complex systems consisting of large ensembles of machines spread across multiple physical networks and geographic regions. These sophisticated services must meet stringent design and performance requirements such as horizontal scalability, fault tolerance, self adaptation, and security while leveraging low-cost commodity hardware. A key challenge is to quantify the impact of hardware changes, software designs, and energy management policies. Critically, such evaluations require the real code, workloads, component interactions, heterogeneous hardware, and high-load conditions to accurately predict performance.

This proposal investigates techniques and architectures for building a high-fidelity datacenter emulation platform to transform datacenter network and service design from a black art into a rigorous, accurate, and repeatable (scientific) process. Such a facility allows researchers and architects to develop the principles and best practices for next-generation datacenter design. By accurately emulating realistic datacenter scales (10,000+ machines, 100+switches, multiple cooperating services) with a modest cluster of a few hundred machines, the proposed work aims to place datacenter experimentation within the reach of students, academics, and businesses without the financial reach of large IT firms.

Intellectual Merit: Data-intensive Scalable Computing (DISC) is increasingly important to peta-scale problems, from search engines and social networks to biological and scientific applications. Already datacenters built to support large-scale DISC computing operate at staggering scale, housing up to hundreds of thousands of compute nodes, exabytes of storage, and petabytes of memory. Current DISC systems have addressed these data sizes through scalability, however the resulting per-node performance has lagged behind per-server capacity by more than an order of magnitude. For example, in current systems as much as 94% of available disk I/O and 33% of CPU remain idle. This results in unsustainable cost and energy requirements. Meeting future data processing challenges will only be possible if DISC systems can be deployed in a sustainable, efficient manner.

This project focuses on two specific, unaddressed challenges to building and deploying sustainable DISC systems:

-a lack of per-node efficiency and cross-resource balance as the system scales, and
-highly-efficient storage fault tolerance tailored to DISC workloads.

This project's approach is to automatically and dynamically ensure cross-resource balance between compute, memory, network, and underlying storage components statically during system design, as well as dynamically during runtime. The goal is to support general DISC processing in a balanced manner despite changing application behavior and heterogeneous computing and storage configurations. This work will result in a fully functional prototype DISC system supporting the Map/Reduce programming model to support general-purpose application programs.

Broader impacts include:
-training diverse students, such as undergraduates and underrepresented groups - to understand DISC services as an interesting part of the overall curriculum and as a resource for interdisciplinary collaboration.
-a public release of the proposed balanced runtime system, including support for higher-level programming models;
-working with industrial partners as part of UCSD's Center for Networked Systems to address sustainability and efficiency issues in this critical portion of industrial and governmental data processing.