Grid Systems Deployment & Management Using Rocks

Federico D. Sacerdoti, Sandeep Chandra and Karan BhatiaSan Diego Supercomputer Center{fds,chandras,karan}@sdsc.edu

Abstract

Wide-area grid deployments are becoming a standard forshared cyberinfrastructure within scientific domaincommunities. These systems enable resource sharing,data management and publication, collaboration, andshared development of community resources. This paperdescribes the systems management solution developedfor one such grid deployment, the GEON Grid(GEOsciences Network), a domain-specific grid ofclusters for geological research. GEON provides astandardized base software stack across all sites toensure interoperability while providing structures thatallow local customization. This situation gives rise to aset of requirements that are difficult to satisfy withexisting tools. Cluster management software is availablethat allows administrators to specify and install acommon software stack on all nodes of a single clusterand enable centralized control and diagnostics of itscomponents with minimal effort. While griddeployments have similar management requirements tocomputational clusters, they have faced a lack ofavailable tools to address their needs. In this paper wedescribe extensions to the Rocks cluster distribution tosatisfy several key goals of the GEON Grid, and showhow these wide-area cluster integration extensionssatisfy the most important of these goals.

1. Introduction

Computational clusters have become the dominantcomputational platform for a wide range of scientificdisciplines. Due to this pressure, cluster managementsoftware has risen to the challenge: cluster tools existthat specify a common configuration base, install asoftware stack on all nodes, enable centralized control ofcomponents, and provide diagnostics for failurereporting, all with minimal effort. While clustermanagement toolkits have been successfully applied tolarge-scale clusters operating in tightly coupled LANenvironments [1, 2], current grid deployments, popularfor building shared cyberinfrasture [3], have similarmanagement requirements yet have faced a lack ofavailable tools. These grid systems seek to offer acommon operating environment for the scientific

domain community and typically involve a diverse set ofresources operating in a geographically dispersedenvironment. Examples of such grid deploymentsinclude GEON [4], BIRN [5] and GriPhyn [6] althoughmany others exist [7-9].

We present grid design from the perspective of GEON,although its similarity to other grid efforts makes ourresults applicable to other projects. Figure 1 shows theGEON grid architecture that is composed of a set ofphysically distributed clusters located within theadministrative domain of each of sixteen participatingsites. These clusters have at least one point-of-presence(pop) node and may have zero or more additionalcompute or data nodes that may consist of differenthardware architectures. The operation of this virtualorganization [10] requires the machines to run acommon software stack that enables interoperabilitywith other GEON resources. We restrict our attention tocomputational hardware resources.

In addition to being geographically distributed, theclusters at each partner site build upon the commonsoftware base to provide site-specific applications andservices. The challenge, therefore, is to manage thedistributed set of hardware resources, physicallydistributed at partner institutions and connected over thecommodity Internet, in a way that minimizes system

Figure 1: The GEON grid. Grid resources aregeographically distributed, and connected via boththe commodity Internet and specialized wide-areanetworks. While individual sites may have uniquecomponents and abilities, GEON must insureinteroperability throughout the grid.

administration costs while achieving interoperability andlocal site autonomy. Although local cluster performanceis important, the main objective of the GEON systemscomponent is to efficiently manage the grid in the faceof system upgrades, security patches, and new softwarecomponents. A central tenant of the design is to achievethis level of management with minimum administration.

We use the Rocks cluster distribution as a starting pointfor the GEON effort. Although lacking the functionalityto manage a grid of machines, Rocks has beensuccessfully used to manage large clusters consisting ofmore than 500 nodes. With our direction, the Rockstoolkit has been modified to support wide-areadeployment and management of systems in a fashionappropriate for GEON. In this paper we present ourrequirements for a grid management toolkit and showhow the new wide-area cluster initialization capability inRocks satisfies a majority of these requirements. Wehope the solutions explored for the GEON project can bedirectly applied to other grid deployments in addition toour own.

Section 2 describes the overall Rocks architecture andsummarizes its capabilities along with other popularcluster management software. Section 3 discusses thespecific requirements for wide-area grid deploymentsand management using GEON as an example. Section 4describes the architecture and implementation changesmade to Rocks to support wide-area grid management.Section 5 discusses some initial performance

measurements and other feedback from using thissystem for real systems deployment and management inGEON and identifies open issues and future directions.Section 6 summarizes the paper.

2. Rocks Cluster Management

High-performance clusters have become the computingtool of choice for a wide range of scientific disciplines.Yet straightforward software installation, management,and monitoring for large-scale clusters have beenconsistent and nagging problems for non-cluster experts.The free Rocks cluster distribution takes a freshperspective on cluster installation and management todramatically simplify version tracking, clustermanagement, and integration.

The toolkit centers around a Linux distribution based onthe Red Hat Enterprise line, and includes work frommany popular cluster and grid specific projects.Additionally, Rocks allows end-users to add their ownsoftware via a mechanism called Rolls [11]. Rolls are acollection of packages and configuration details thatmodularly plug into the base Rocks distribution. In thispaper, for example, we demonstrate injecting softwareinto a domain-specific Grid via a GEON roll. Strongadherence to widely used tools allows Rocks to movewith the rapid pace of Linux development. The latestrelease of Rocks, version 3.2.0, supports severalcommodity architectures including x86, IA64 andx86_64 Opteron.

2.1. Architecture

Figure 3 shows a traditional architecture used for high-performance computing clusters. This design waspioneered by the Network of Workstations [12], andpopularized by the Beowulf project [13]. In this methodthe cluster is composed of standard high-volume servers,an Ethernet network and an optional off-the-shelfperformance interconnect (e.g., Gigabit Ethernet orMyrinet). The Rocks cluster architecture favors high-volume components that lend themselves to reliablesystems by making failed hardware easy andinexpensive to replace.

Rocks Frontend nodes are installed with the basedistribution and any desired rolls. Frontend nodes serveas login and compile hosts for users. Compute nodestypically comprise the rest of the cluster and function asexecution nodes. Compute nodes and other clusterappliances receive their software footprint from thefrontend. Installation is a strong suit of Rocks; a singlefrontend on modern hardware can install over 100

cluster-db-structure

dhcp-server

sge

rocks-dist

condor

condor-server

condor-client

emacs fortra

server

i386ia64

er-db-data

central x11-thin

i386x86_64

syslog-server

x11

i386

Figure 2: Rocks and Rolls. A node in the graphspecifies a unit of packages and configuration; agraph traversal defines software for a clusterappliance. Yellow nodes are Rocks base, whilecolored nodes belong to various rolls.

compute nodes in parallel, a process taking only severalminutes. Typically cluster nodes install automatically,using PXE to obtain a boot kernel from the frontend

One of the key ingredients of Rocks is the modular Rollmechanism to produce Linux distributions customizedfor a particular domain. When we build a clusterfrontend with the GEON roll, all compute nodes willinstall a set of Geo specific software. This mechanismallows us to easily inject domain-specific software intothe Rocks integration system, enabling a Geo-specificgrid of clusters. Specialized CDs can be generated fromthis custom distribution, which behave identically tothose from RedHat. More importantly to this paper, thecustom distribution may be transmitted to other clusterfrontends over a standard wide-area network such as theInternet.

By leveraging the standard RedHat Anacondainstallation technology, Rocks abstracts many hardwaredifferences and automatically detects the correctconfiguration and hardware modules to load for eachnode (e.g., disk subsystem type: SCSI, IDE, integratedRAID adapter; Ethernet interfaces, etc). Although itsfocus is flexible and rapid system configuration (and re-configuration), the steady-state behavior of Rocks has alook and feel much like any other commodity clustercontaining de-facto cluster standard services.

2.2. Related Work

We chose the Rocks cluster distribution to power theGEON scientific grid based on its fitness to ourrequirements. However several attractive clustering

solutions exist, both in open source and commercialform.

2.2.1. Cluster Distribution Methods

SystemImager [14] performs much of the same tasks asRocks. It supports each hardware platform by storing aunique image of the desired directory structure on amaster node in the cluster. There is no mechanism forsharing portions between images, however, makingfunctional or physical heterogeneity difficult to manage.

The LCFG project [1] has an installation philosophysimilar to Rocks. It provides a configuration languageand a central repository of configuration specifications.The specifications, analogous to Rocks kickstart nodes,can be combined to install and configure individual Unixmachines over a network. Changes to the centralspecifications automatically trigger correspondingchanges in the configuration of managed nodes. TheLCFG system is used in diverse configurations whereeven the Unix flavor is heterogeneous.

Scyld Beowulf [15] is a commercial, single-system-image cluster operating system. In contrast toSystemImager, LCFG, and Rocks, processes on a scyldcluster see a single process space for all running tasks.While this feature simplifies cluster operations, it relieson a heavily modified Linux kernel and GNU C library.Because of the deep changes required by the system, theScyld company sits in the critical path of many bug andsecurity fixes. These fundamental changes require Scyldto take on many (but not all) duties of the distributionprovider.

Other clustering projects such as Warewulf [16],LinuxBIOS [17], and OpenMosix [18] offer interestingcapabilities, but do not provide a complete solution.They require a Linux distribution to be installed andconfigured a priori by experienced systemadministrators, a critical shortfall for our choice. Theactual ingredients in these projects cover a small part ofa full distribution: OpenMosix provides a kernel module(albeit with some elegant capabilities), and LinuxBIOSspecifies a small assembly/C initialization pre-kernel.Warewulf is the most ambitious of the three and canconfigure a shared environment in the cluster via animage-like file system on the master node.

3. Grid Requirements

The GEON grid imposes a regular structure on itsconstituents. At minimum, each partner site runs aGEON pop node that acts as a point-of-presence in thegrid, and corresponds to a Rocks frontend cluster

Figure 3: Rocks Cluster Hardware architecture.The Frontend node acts as a firewall andgateway between the private internal networksand the public Internet.

Ethernet Network

Application Message Passing

Network

Front-end Node(s)

eth1 Public Ethernet

eth0

eth0 eth0 eth0 eth0

NodeNodeNodeNode

appliance type. The pop node optionally coordinatesadditional local machines at the site for providing data orcomputation capability.

A GEON pop node must maintain a core set of services.In order to ensure interoperability between nodes atdifferent sites, SDSC provides a comprehensivestandardized software stack definition that includes thecore operating system, currently Linux, along withhigher-level development environments such as Weband Grid Service libraries, and end user environments(portals). The software stack definition includes thenecessary security information to link nodes to theGEON grid security infrastructure.

3.1. Consistency

The first major challenge for systems management of theGEON grid is maintaining a consistent software stack onall machines, currently 40 in total. These hosts arephysically distributed among various administrativedomains at different sites and connected through thecommodity Internet1. Uniformity of the software stackis required to ensure interoperability between servicesrunning on different sites in the grid. The GEONgriduses the NMI grid software release [19] as the base of itsgrid software stack. In addition, we provide higher-levelsoftware such as the portal environments, various webservices libraries, an OGSI grid service container andlibraries, and GEON-specific services. In order to dealwith the complexity of so many interoperatingcomponents, we have chosen to integrate and test thesystem as a whole unit, which when verified is pushedout to all nodes in the GEON grid.

3.2. Controlled Customization

The second major challenge of the project is to managethe constituent machines to allow controlledcustomization of the machines at each of the partnersites. There are three reasons for local customization:First, each site may have specific system requirementsfor its configuration. For example, software and securitypatches, firewall integrity, and other management toolsspecific to the site must be accommodated. Each partnersite may also configure the GEON machines in such away to leverage additional resources it may have. SDSC,for example, will provide gateways to large computeclusters (TeraGrid [20] and Datastar) and data archives(HPSS).

1 Few sites are connected using the higher bandwidthInternet2 backbone

Second, the partner sites may deploy additional grid orweb services applications on the GEON machinesbeyond those built into the software stack. Theseservices must be persistent across reinstallations andsystem updates. Finally, partner sites may deploydatasets into the GEON network by hosting them onGEON machines. These datasets must also be preservedacross software stack updates.

Unconstrained customization, however, leads tosoftware incompatibilities that require significantadministration overhead. Therefore the needs ofcustomization must be balanced with the need forefficient system management.

3.3. Requirements

The following are specific requirements we havedetermined for the GEON deployment, which webelieve are similar to other grid systems:

1. Centralized software stack definition. GEONcentral node will define the base software stackused to instantiate all other nodes.

2. The ability to push the software stack to GEONpop nodes over the Internet with little or noadministration.

3. Enable local site autonomy by definingacceptable node types for compute, data, andsite-specific customized nodes.

4. Ability to add site-specific constraints. Allowcustomized software with durable site-specificenvironment and settings. These softwarecomponents should survive the process ofupgrading or reinstalling the base distribution.

5. Update software and security patches. UseINCA framework [21] to monitor softwareversions of important middleware components.The ability to identify and incorporate changesto the base software stack.

6. Incremental and hierarchical base distributionupdates. Updates to the pop frontend nodesmust be automatically pushed to the computeand data nodes at the site.

7. Remotely update critical patches and thesoftware stack. At the same time sites withvarying administrative requirements andpolicies should be able to add additional rulesto the basic update mechanism.

8. Nodes or clusters that join the grid shouldintegrate easily and be immediately consistentwith the base grid software stack.

While keeping in mind that the sites own their localresources and have full control of them, the GEONsystem must provide a robust, basic level of systemsmanagement that can be extended.

4. Wide Area Kickstart

To address the primary requirement of the GEON grid, alow management cost for grid-wide software installationand updates, we extended the Rocks distribution toperform full cluster installations over wide areanetworks. While compute nodes in Rocks alwaysemployed the LAN to install software from the frontendmachine, the frontend itself had to be integrated with CDor DVD media. This strategy, while appropriate forcluster instantiations of basic Rocks software, isinsufficiently flexible for the dynamic needs of theGEON grid. Specifically, we considered affecting grid-wide software with mailed disk media unacceptable.

4.1. Central

The wide area cluster integration involves a Centralserver that holds the software stack. Frontend pop nodesof the grid obtain a full Rocks cluster distribution fromthe central. This distribution is suitable to install localcompute, data, and customized nodes. Since the softwarestack defines the whole of the base system, and becausesecurity patches and feature enhancements are commonduring updates, any part of the stack may be changed.The challenge is to retrieve all components necessary forintegration from central over the network, including theinstallation environment itself. We require a small staticbootstrap environment for the first pop initialization,which contains only network drivers and hardware-probing features and fits onto a business card CD. Thisbootstrap environment is stored on the node to facilitatethe upgrade process.

Figure 4 shows the wide-area kickstart architecture. Acentral server holds the base Rocks distribution,including the Linux kernel and installation environment,along with standard rolls such as the HPC (highperformance computing), and the domain-specificGEON roll. We initialize GEON frontend pops over thewide-area Internet from this central.

The software disbursement methodology is optimizedfor unreliable networks, and possesses greater awarenessand tolerance for failure than the local-area Rocks install

process for between frontend and compute nodes. Thefrontend pop locally integrates the Rocks base and thevarious Rolls selected from the central server, and thecombined software stack is rebound into a new Linuxdistribution. The pop then initializes cluster nodes withits combined distribution. If the pop is current it is easyto see any new compute, data, or specialized nodesjoining the grid immediately posses a consistent versionof the base grid stack.

4.2. WAN vs LAN

In a traditional Rocks cluster, compute nodes also installover a network: the LAN between themselves and thefrontend. The frontend uses PXE or DHCP requestsfrom compute nodes to perform a registration step,information from which it uses to generate customkickstart files for each node in the cluster.

In the wide area, we do not have the luxury of PXE orDHCP. The only communication method over the WANis HTTP. In essence a frontend is grown over severalsteps. The strategy gives a nascent frontend enoughinformation and software to generate its own fullyfunctional kickstart file, which it then uses to installitself.

The frontend initially boots from a small environment(<10MB on a CD), and specifies the name of a central itwishes to be cloned from. The central then sends a fullinstaller over HTTP, which enables the frontend torequest a kickstart file. The kickstart file returned isminimal, since central knows little about the new host.

In this way a frontend WAN install differs from a localcompute node. The kickstart file includes no IPinformation, nor any definition of the software stack.However, the frontend can ask the user for necessaryinformation by way of a remote input mechanism basedon ssh. The architect contacts an installing frontend andanswers naming, networking, and software stackquestions. Rolls are chosen during this phase, and thenascent frontend can pull them from various sources ifneeded (figure 5).

At this point the frontend transfers software from thecentral and roll servers using http. During this phase thekickstart generators [2] are received, along withelements of the kickstart graph contributed by Rocksbase and any chosen rolls (figure 2). Upon completion,the frontend uses the kickstart generators, graph, andpreviously obtained user input to generate a full kickstartfile for itself. The inward-looking kickstart generationforms the crux of this process.

The frontend uses the new kickstart file to restart its owninstallation, which now does not require additional input.Once the installation completes, the frontend is ready tointegrate local compute nodes.

4.3. Controlled Customization

With the ability to bind the base and third party rolls intoa distribution for local consumption comes thepossibility of obtaining components from many sources.Figure 5 shows integration from several central servers.This feature allows a type of multiple inheritance, wherea frontend’s identity is composed from several sources.This inheritance model allows GEON’s goal ofcontrolled local customization. A site can maintain acentral server with locally developed rolls for its use.However, rolls are highly structured combinations ofsoftware and configuration. Their framework allowsquality control and verification of their constituentpackages, to a level not available to traditional software-testing techniques [11].

With rolls, GEON supports controlled site customizationsuitable for complete software components, and allowthese components to be easily incorporated into the basesoftware stack.

However by the requirements, sites must be capable ofaffecting local configuration changes as well. The Rocksmethod of distributing updates reinstalls the rootpartition of the system. While this provides securityassurances and software verification advantages, itrequires all service configuration be relegated to otherpartitions. The GEON system places all localconfiguration and user environments on non-rootpartitions, which are durable across updates. No facility

exists in the Rocks system to distribute incrementalpackage updates, a shortcoming which is left to futurework.

4.4. Multiple Architectures

Figure 5 illustrates the multiple-architecture aspects ofthe Rocks system as well, which extend to the wide-areakickstart facility. Contained within the bootstrapenvironment is a local architecture probe, which alertsthe central or cluster frontend what architecture it is.This classification allows heterogeneous clusters toensure each node runs the most native software stackavailable. It also requires frontend nodes to posses adistribution for each architecture they expect toencounter in their local cluster. For this reason, theGEON central provides the base distribution and rollsfor several hardware types, notably x86 (Pentium,Athlon), ia64 (Itanium), and x86_64 (Opteron). TheGEON central forms the repository of the base GridDNA, with local central servers providing additionalsoftware resources.

The Rocks wide-area cluster integration facility enablesGEON to push its software stack to pop nodes over theInternet in an efficient manner with low administrationoverhead. Site-specific additions and customizations ofsoftware components are standardized with the rollstructure, and are implemented with site-local centralservers. Finally, customizations are maintained acrosssoftware stack updates by the GEON system structure.The proceeding section provides an evaluation of theprocess and function of this wide-area integrationcapability.

5. Experience

To verify the effectiveness of the Rocks wide areakickstart technique we performed experiments in variousrelevant settings. In this section we show the wide-areacluster integration facility of Rocks satisfies our majormanagement goals of a common software stack that iscentrally disbursed to sites, a low-administration methodof updates and patches, and the ability for local sitecustomization. We present three experiments, eachwhich model an important use case of our grid.

The objective of these trials is to measure the timeneeded to retrieve the base and Roll distributions overthe public Internet, and to observe any failures andrecovery during the process. In addition we wish tocharacterize the amount of administrative labor involvedin the configuration before the installation processbegins, and how much configuration is necessary for a

Figure 4: Wide Area Cluster Integration. Centralserver provides Rocks base and the GEON Roll tofrontend cluster nodes in the Geongrid. Frontendsthen update the software on their respectivecompute nodes. Frontends obtain their softwareover wide area networks such as the Internet, whilecompute nodes install over a local LAN.

Central ServerBase, HPC, GEON

Cluster 1

Frontend

ComputeNode

ComputeNode

Cluster 2

Frontend

ComputeNode

Internet

Cluster N

Frontend

local site to add their customized software Roll at thetime of installation. Finally, we verify the consistency ofthe configuration between the base distribution andsoftware stack installation on the remote site. The timetaken to setup the node once it retrieved the distributionis dependent on the hardware configuration of the node(e.g. processing speed, etc) and is not of greatsignificance to the experiment.

5.1. Experimental Setup

The purpose of this experiment is to verify operationalcorrectness and ascertain the typical managementoverhead necessary for a cluster install. Once thefrontend node of a cluster has been initialized, standardand proven Rocks techniques can be used to install localcompute nodes. Therefore we bias our report towards theintegration of the cluster frontend nodes, which serve aspop nodes of the GEON grid.

All experiments involve a central server node and afrontend pop machine. The central node is a dualPentium 4 server running Rocks 3.2.0 beta software withLinux kernel 2.4.21. Since all frontend nodes canfunction as central servers, we setup the central as aRocks frontend with modified access control. Softwaredisbursement occurs using standard HTTP on port 80,access to which has been opened to our frontend popnode using Apache access control lists. The centralserver is connected to the network via copper-basedGigabit Ethernet capable of 1000mbps.

For the first experiment we kickstart a cluster frontendnode from the central server located on the same local

area network (LAN). A Gigabit Ethernet link connectsthe two machines.

In the second experiment we stress the system slightlyby kickstarting a cluster frontend over the wide areanetwork (WAN) in the same organizational domain. Wedefine wide area in this case as a two network endpointsconnected with one more IP routers between them. Thisscenario corresponds to obtaining a roll from a local sitecentral server for the purposes of customization.

Finally, we initialize a frontend pop over the WAN froma central server in a different organizational domain.This experiment simulates the process of distributing thecommon Geon software stack to pop nodes on the grid.We expect to see the effects of increased packet loss andTCP congestion control when operating over this longerhaul network.

The LAN testing was performed at SDSC betweennodes on the same level 2 (switching) network segment.The WAN experiment within the same organizationaldomain was performed at SDSC using the central serverfrom the first experiment. The frontend has a 100 mbpsnetwork link and therefore limits the bandwidth of thistest. The third experiment we conduct using the centralserver at SDSC and a frontend node at Virginia Tech,roughly 4000 kilometers away. The maximumbandwidth between the sites is ~100mbps over thestandard Internet link. The Rocks distribution is close to660 MB, the HPC roll is 338 MB and the GEON roll isclose to 98 MB. A series of 10 runs were conducted ineach of these experiments and the results in Table 1indicate the average time in each case.

5.2. Discussion

Our observations show that we were successfully able tosatisfy our most important requirements. When creating

Central ServerBase, HPC, GEON

ClusterFrontend (x86_64)

ComputeNode

ComputeNode

ComputeNode

. . .

Internet

x86 x86_64

Central Server 2Java, Custom

Internet

Figure 5: Multiple Central servers. A frontend maybe composed of rolls from several sources, allowingcourse-grained local site customization suitable forlarge software components. Clusters can haveheterogeneous hardware; base distribution and rollsare available for x86, ia64, and x86_64. Nodesautomatically negotiate for the most native softwareavailable.

Kickstart Type Rocks BaseRolls

(HPC+GEON)

Local Area Network

(Single level 2 segment)65s 8s

Wide Area Network

(Same Organization)390s (6.5min) 25s

Wide Area Network

(Different Organization)2700s (45 min) 360s (6 min)

Table 1: Time (in seconds) to retrieve the Rocks base andRoll distributions over different types of networkenvironments. Values represent average of several trials.

a node within the local area network, the distributionwas retrieved in the shortest time as a result of the highbandwidth, and the node was consistent in software andconfiguration with the central server. The amount ofpersonnel involvement was minimal as we would expectfrom the Rocks toolkit. Our experience with the widearea kickstart were the same, all the metrics were similarexcept the time taken to retrieve the distribution over thepublic Internet, a result dominated by the performance ofthe network over the geographic distance involved. Theremote site was also able to successfully add their sitespecific Roll and it was consistent with the overallsoftware installation.

Our initial assessment is that the Rocks wide areakickstart extension successfully satisfies many of theGEON requirements and we believe similar principlescan be applied to a grid consisting of many nodes.Limitations of the system are detailed in future work.

5.3. Future Work

Not all the requirements of GEON were met by theRocks cluster distributions current release. We require afrontend node to be able to initiate an upgradeautomatically. Another specification sought is the fine-grained incremental upgrade of critical RPMS. Thisfeature is also absent from the Rocks system. Inaddition, nodes in wide area grid systems like GEONhave the concept of user space and OS distributionspace. Our ideal solution would posses the functionalityto update or reinstall the OS distribution completelywithout affecting the user environment in any way. Weexpect to address these requirements in future iterationsof the Geon grid design.

5.4. Security

A flaw in the current Wide-Area kickstart design is thelack of strong security. In order to confidently install afrontend over the network, the central must encrypt andsign the kickstart files, and both parties mustauthenticate each other before any software changeshands.

In the next version of Rocks, we plan to tighten securityby using x509 certificates to prove the identity of bothclient and central, and protect sensitive kickstart filesusing the standard HTTPS (SSLv3/TLS) protocol.

6. Conclusion

This paper discusses the requirements, architecture andapplicability of Rocks wide area grid systems

deployment and management. We discussed some of therequirements of GEON grid deployment andmanagement and how Rocks wide area functionalityaddresses those requirements. We also discussed someinitial results and talk about possible extensions to theRocks toolkit that would benefit GEON and similar gridsystems.

We believe in GEON and other grid architectures thekey is to manage an extremely heterogeneous hardware,computing, storage, and networking environment, ratherthan an “ultra-scale” environment. The complexity ofmanagement (e.g., determining if all nodes have aconsistent set of software and configuration,incrementally updating patches, etc) of such gridsystems is challenging. The new Rocks wide areakickstart facility establishes a rational softwaredistribution and management plan, enabling the GEONnetwork to easily expand by adding computationalhardware, services, middleware components, and otherresources with minimal effort.

6.1. Acknowledgements

This work is supported in part by the National ScienceFoundation under Grant No. 225673. We would like tothank Greg Bruno, Mason Katz, and Phil Papadopoulosfor their hard work and help with the initial GEONinfrastructure.

References

1. P. Anderson and A. Scobie. LCFG: The nextgeneration. In UKUUG Winter conference, 2002.

2. P. Papadopoulos, M. Katz, and G. Bruno. NPACIRocks: Tools and Techniques for Easily DeployingManageable Linux Clusters. In IEEE Cluster, 2001.

3. D.E. Atkins, K.K. Droegemeier, et al.,Revolutionizing Science and Engineering ThroughCyberinfrastructure: Report of the NationalScience Foundation Blue-Ribbon Advisory Panelon Cyberinfrastructure, National ScienceFoundation, 2003.

4. CyberInfrastructure for the Geosciences,http://www.geongrid.org.

5. BIRN: Biomedical Informatics Research Network,http://www.nbirn.net.

6. Griphyn: Grid Physics Network,http://www.griphyn.org/.

7. Building the National Virtual Collaboratory forEarthquake Engineering, http://www.neesgrid.org.

8. An open platform for developing, deploying, andaccessing planetary-scale services,http://www.planet-lab.org.

9. Southern California Earthquake Center,http://www.scec.org.

10. I. Foster, C. Kesselman, and S. Tuecke. TheAnatomy of the Grid: Enabling Scalable VirtualOrganizations. International Journal of HighPerformance Computing Applications, 15(3): p.200-222, 2001.

11. G. Bruno, M. Katz, F. Sacerdoti, and P.Papadopoulos. Rolls: Modifying a Standard SystemInstaller to Support User-Customizable ClusterFrontend Appliances. In IEEE Cluster, 2004.

12. T. Anderson, D. Culler, and D. Patterson. A Casefor NOW (Networks of Workstations). IEEE Micro,15(1): p. 54-64, 1995.

13. T. Sterling, D. Savarese, et al. BEOWULF: Aparallel workstation for scientific computation. In24th International Conference on ParallelProcessing, Oconomowoc, WI, 1995.

14. B. Finley. VA SystemImager. In 4th Annual LinuxShowcase and Conference, Atlanta, GA, 2000.

15. Scyld Beowulf Clustering for High PerformanceComputing, Scyld Software,

16. The Warewulf Cluster Project, http://warewulf-cluster.org/.

17. R. Minnich, J. Hendricks, and D. Webster. TheLinux BIOS. In 4th Annual Linux Showcase andConference, Atlanta, GA, 2000.

18. The openMosix Project,http://openmosix.sourceforge.net/.

19. NSF Middleware Initiative, http://www.nsf-middleware.org/.

20. Distributed Terascale Facility to Commence with$53 Million NSF Award,http://www.nsf.gov/od/lpa/news/press/01/pr0167.htm.

21. S. Smallen, C. Olschanowsky, K. Ericson, P.Beckman, and J. Schopf. The Inca Test Harnessand Reporting Framework. In Supercomputing,2004.