— experiences with OpenNebula 3.X

HPC Cloud at SURFsara

Ander Astudillo <ander.astudillo@surfsara.nl>

Lykle Voort <lykle.voort@surfsara.nl>

OpenNebula Tech Day Ede (The Netherlands)

26th March 2014

#

e-Infrastructure landscape

3

Cartesius (National Supercomputer)

Bull bullx, 13984 cores, 41.75 TB memory, 2.4 PB disk, 271 TFlop/s

LISA (National compute cluster)

Dell cluster, 6528 cores,16 TB of memory, 46 TFlop/s

Grid

5000 cores, 4 PB disk, 6 PB tape11 sites

Visualization

CollaboratoriumRemote visualizationRender cluster

Network

SURFnet7GLIFNetherLight

Innovative infrastructures

HadoopGPU clusterBeeHubHPC Cloud

Agenda

1.- SURFsara’s HPC Cloud service

2.- User experience

3.- SURFsara’s HPC Cloud implementation

1

What do we (SURFsara) want to offer?

5

Services for scientists…scientists ⇏ systems gurus

… complex users’ problems• Data: big, dirty, non-structured…• Computation: complex (e.g.: modeling, simulation)

• Libraries nightmare• 3rd party, incompatibility, maintenance…

… flexibility

… trial and error

… test

… cooperate… share

… show

… scratch … privacy

1.- Our service

Where does the HPC Cloud fit?

61.- Our service

What does our HPC Cloud offer?

7

Your own world

In a jar

1.- Our service

IaaS:

Network filter

Virtualization

What does a user see?

81.- Our service

2User experience

Our decisions

102.- User experience

HPC• Big nodes• Fast interconnect• MPI-alike computation• Own flexibility

Cloud light• Smaller nodes• CPU overcommit• (Interactive) services

• Projects• Documentation

• System load

• Network filters• Dynamic DNS• VirDir

• Image upload

• Wizard

tweaks

Accounting

112.- User experience

Per project

User accounts

CPU time

Images storage

VirDir

Accounting (& figures)

122.- User experience

Per project

User accounts

CPU time

Images storage

VirDir

180+ projects active: 110+

370+ users active: 240+

Since 2009 until March 2014: 300+ requests 32.000+ VM

130+ TB in use

850+ years

3.700+ images

User applications

132.- User experience

• Profit from cloud elasticity• Provide own service to own users• Software that requires licenses• Workflows• Training• Intensive compute

Fields:• Biology• Genetics• Informatics• Chemistry• Ecology• Linguistics• Robotics• Business• Social sciences• Engineering• Humanities

User applications

142.- User experience

• High-memory nodes• Database as a Service

Experimental additions:

SURFsara’s HPC Cloud

implementation

3

Hardware

163.- Implementation

30 HPC nodes19× 32-core Intel Xeon-E711× 32-core Intel Xeon-E5256 GB RAM1 TByte local diskEthernet 2×10GE

Each new HPC node delivers 12K SPECints under ideal circumstances

10 light nodes8 cores AMD Opteron 6212 64 GB RAM6 TByte local diskEthernet 1×10GE

Network144 ports 10GE, 1-hop

Storage800TB shared

(Arista DCS-7504)

800TBSharedStorage

2×10Gbps

4ServiceNodes

Virtualised Services

30 largecomputenodes

....

..10 light

computenodes

Network overview

3.- Implementation

4×10Gbps

Software

3.- Implementation

Groups’ network isolation

3.- Implementation

The good

3.- Implementation

• Live migration • HUGE disks possible• Security• Ease of use

<<EOF

Ander Astudillo <ander.astudillo@surfsara.nl>

Lykle Voort <lykle.voort@surfsara.nl>

Credits

Images: Wikipedia, Science ParkFront slide photo: dvanzuijlekom in Flicker

Slides: SURFsara colleagues