Cloud Computing Initiative to Study Internet-Scale Systems

5/12/2009

Written by

Building on a long history of innovation in large-scale systems and parallel computing, the Illinois department of computer science today announced a joint research endeavor to investigate Internet-scale cloud computing systems.

In collaboration with Hewlett-Packard, Intel, and Yahoo!, Illinois will develop an experimental testbed for data-intensive applications using distributed "cloud" computational resources. The global partnership, which also includes the National Science Foundation (NSF), the Infocomm Development Authority of Singapore (IDA), and the Karlsruhe Institute of Technology in Germany, will result in the establishment of a globally distributed, Internet-scale testing environment for advanced research in cloud computing infrastructure and services.

"We live in a world populated with enormous amounts of data from a wide variety of sources such as satellite telemetry, medical and agricultural sensors, and billions of web pages," said computer science professor Michael Heath, interim head of department, and one of the three Illinois researchers leading the project. "There is a compelling human need to represent, analyze, query, manage, understand, and respond to such data for knowledge extraction and decision making."

Illinois computer science professor Roy Campbell will lead the $6 million center
Illinois computer science professor Roy Campbell will lead the $6 million center

Responding to these needs, the Cloud Computing Testbed (CCT) at Illinois will enable researchers to explore systems-level research issues such as automatic resource allocation, scheduling, monitoring, and management tasks that arise in processing and responding to large amounts of data. The proposed research will cover a breadth of research areas including networking, operating systems, virtual machines, distributed systems, data-mining, Web search, network measurement, and multimedia.

"This effort will differ markedly from existing experimental clusters," said Heath. "With previous efforts focused on networking or user-level applications, the gaping need to process and respond to large amounts of data has been inadequately addressed. Our effort will go deep into the system software stack to explore new and better ways to provide system-level support for data-intensive computing."

Illinois researchers will bring their characteristic multi-disciplinary approach to the project. The effort will involve multiple computer science faculty members spanning several disciplines and will be targeted at critical cloud computing systems needs such as: analyzing network logs, querying application logs, semantic web systems, search navigation maps, crawling online social networks, tele-immersive applications, and system management and monitoring. Computer science professors Roy Campbell and Indranil Gupta, well known for their systems and networking research for distributed computing, will serve as co-leads on the project.

Illinois computer science professor Indranil Gupta
Illinois computer science professor Indranil Gupta

"To realize the full potential of cloud computing, the technology industry must think about the cloud as a platform for creating new services and experiences. This shift will require an entirely new approach to the way we design, deploy and manage cloud infrastructure and services," said Prith Banerjee, senior vice president of Research at HP and director of HP Labs. "The HP, Intel and Yahoo! Cloud Research Testbed gives us the ability to tap the brightest minds in the industry, in academia and government to drive innovation in this area."

The CCT effort will build on recent Illinois efforts to advance parallel computing across the entire computing spectrum, from the Blue Waters project, which is building the world's first sustained petascale computing system, to the Universal Parallel Computing Research Center, which is bringing parallel computing concepts and performance to mainstream systems and applications.

"Creating large-scale testbeds is important because they lower barriers to innovation and provide the opportunity to experiment and learn at scale," said Andrew A. Chien, Vice President and Director of Intel Research. Intel's support of Tashi, an open source cluster management system for cloud computing, and this HP, Intel, Yahoo! Cloud Computing Testbed are a natural extension of our ongoing, mutually beneficial partnerships with the research community such as the Universal Parallel Computing Research Centers."

The CCT will be located in the Thomas M. Siebel Center for Computer Science on the Illinois campus. It will include a 1,024-core HP system with 200 TB of disk space. The testbed will run Apache Hadoop, an open source distributed computing project of the Apache Software Foundation, and other open source, distributed computing software such as the Pig parallel programming language developed at Yahoo! Research. The globally distributed testbed locations, including the CCT at Illinois, will be fully operational and access will be made available to researchers in the U.S. and worldwide through a proposal selection process later in 2008.

Prabhakar Raghavan, Head of Yahoo! Research. "With this testbed, not only can researchers test applications at Internet-scale, they will also have access to the underlying computing systems to advance our understanding of how systems software and hardware function in a cloud environment."

"Illinois is thrilled to be a part of this groundbreaking effort," said Heath. "It will accelerate research for data-intensive, Internet-scale computing and drive innovation for future systems. It will enable a better understanding of Internet characteristics, especially emerging online social networks. It will also enable development of open-source tools for network analysis, fast querying, and transfer of distributed logs, search engines yielding navigation maps, semantic Web tools, and configuration for multimedia environments."


Share this story

This story was published May 12, 2009.