Cloud Computing Research Test Bed Formed

July 30, 2008

A global, multi-data center, open source test bed for the advancement of cloud computing research and education has been created by HP, Intel, and Yahoo!. The goal of the initiative is to promote open collaboration among industry, academia and governments by removing the financial and logistical barriers to research in data-intensive, Internet-scale computing.

The HP, Intel and Yahoo! Cloud Computing Test Bed will provide a globally distributed, Internet-scale testing environment designed to encourage research on the software, data center management and hardware issues associated with cloud computing at a larger scale than ever before. The initiative will also support research of cloud applications and services.

HP, Intel and Yahoo! have partnered with the Infocomm Development Authority of Singapore (IDA), the University of Illinois at Urbana-Champaign, and the Karlsruhe Institute of Technology (KIT) in Germany to form the research initiative. The partnership with Illinois also includes the National Science Foundation.

The test bed will initially consist of six “centers of excellence” at IDA facilities, the University of Illinois at Urbana-Champaign, the Steinbuch Centre for Computing of the Karlsruhe Institute of Technology, HP Labs, Intel Research and Yahoo!. Each location will host a cloud computing infrastructure, largely based on HP hardware and Intel processors, and will have 1,000 to 4,000 processor cores capable of supporting the data-intensive research associated with cloud computing. The test bed locations are expected to be fully operational and made accessible to researchers worldwide through a selection process later this year.

The test bed will leverage Yahoo!’s technical leadership in open source projects by running Apache Hadoop – an open source, distributed computing project of the Apache Software Foundation – and other open source, distributed computing software such as Pig, the parallel programming language developed by Yahoo! Research.

“The HP, Intel and Yahoo! Cloud Computing Test Bed furthers our commitment to the global, collaborative research community that is advancing the new sciences of the Internet,” said Prabhakar Raghavan, head of Yahoo! Research. “With this test bed, not only can researchers test applications at Internet scale, they will also have access to the underlying computing systems to advance understanding of how systems software and hardware function in a cloud environment.”

Researchers at HP Labs, the central research arm of HP, will use the test bed to conduct advanced research in the areas of intelligent infrastructure and dynamic cloud services. HP Labs recently sharpened its focus to help HP and its customers capitalize on the industry’s shift toward cloud computing, a driving force behind HP’s vision of Everything as a Service. With Everything as a Service, devices and services will interact seamlessly through the cloud, and businesses and individuals will use services that anticipate their needs based on location, preferences, calendar and communities.

“To realize the full potential of cloud computing, the technology industry must think about the cloud as a platform for creating new services and experiences. This requires an entirely new approach to the way we design, deploy and manage cloud infrastructure and services,” said Prith Banerjee, senior vice president of Research at HP and director of HP Labs. “The HP, Intel and Yahoo! Cloud Computing Test Bed lets us tap the brightest minds in the industry, academia and government to drive innovation in this area.”

Intel is a leading provider of platform technologies, including processors, chipsets, networking and SSD (solid state drives), for cloud computing data centers. Current platform features such as Data Center Management Interface (DCMI), Node Manager (NM) and virtualization have been designed to improve the manageability and energy efficiency of data centers. This open, collaborative research effort will give researchers full access to the system’s hardware for further innovation of existing and future platform features.

“We are pleased to engage with the academic research community – open collaboration with the academia is in our DNA at Intel Research,” said Andrew A. Chien, vice president and director of Intel Research. “Creating large-scale test beds is important because they lower barriers to innovation and provide the opportunity to experiment and learn at scale. Intel’s support of Tashi, an open source cluster management system for cloud computing, and this HP, Intel, Yahoo! Cloud Computing Test Bed are a natural extension of our ongoing, mutually beneficial partnerships with the research community, such as the Universal Parallel Computing Research Centers.”

IDA will facilitate research in the test bed by providing its users with the computing resources required to develop cloud computing software and applications. IDA will also leverage the test bed and its industry partnerships to train local students and professionals on the technologies and programs associated with cloud computing.

“With the ready and available Internet-scale resources in Singapore to support cloud computer research and development work, we can collaborate with like-minded partners to advance the field,” said Khoong Hock Yun, assistant chief executive of the Infrastructure Development Group at the Infocomm Development Authority of Singapore. “Cloud computing is considered by many to be the next paradigm shift in computer technology, and this may be the next ‘platform’ for innovative ecosystems. Partnerships like this will allow Singapore to leverage this new paradigm for greater economic and social growth.”

Partnership builds on initiatives in cloud computing

The Cloud Computing Test Bed is the next step in expanding each company’s ongoing initiatives in cloud computing. In November 2007, Yahoo! announced the deployment of a supercomputing-class data center, called M45, for cloud computing research; Carnegie Mellon University was the first institution to take advantage of this supercomputer. Yahoo! also announced this year an agreement with Computational Research Laboratories (CRL) to jointly support cloud computing research and make one of the 10 fastest supercomputers in the world available to academic institutions in India. Earlier this year, Yahoo! hosted the first-ever Hadoop Summit and Data-Intensive Computing Symposium. Co-sponsored with the Computing Community Consortium (CCC), these meetings brought together leading experts from industry, academia and government to discuss the future directions of Hadoop and data-intensive computing.

In 2008, HP announced the formation of its Scalable Computing & Infrastructure Organization (SCI), which includes a dedicated set of resources that provide expertise and spearhead development efforts to build scalable solutions designed for high-performance and cloud computing customers. The company introduced scalable computing offerings including the Intel Xeon-based HP ProLiant BL2x220c G5, the world’s first server blade to combine two independent servers in a single blade, and the HP StorageWorks 9100 Extreme Data Storage System (ExDS9100), a highly scalable storage system designed to simplify the management of multiple petabytes.HP also introduced the HP POD (Performance-Optimized Datacenter), an open architecture, compact, shipped-to-order alternative for deploying IT resources.