Maximizing HPC Compute Resources with Minimal Cost

Michael ThompsonLead Systems Software

EngineerWayne State University

Problems

HelloWorld

Simple jobs consuming high-end server resources

Usage Spikes cause long queue wait times

Data Transfer and Availability

Options

In-HouseCheaper per CPU

core per hour

Pay for all cores all the time

HPCWorkload

Cloud$$$ per CPU core per hour

Only pay for what you use

Options – Per Core

In-House

Cloud

Example: Assumptions - Infrastructure

6.35 cents/kWh

In-House & Cloud labor roughly equal

Available Air Conditioning

Available rack space

Example: Assumptions - Hardware Specs

2.6ghz processors or greater

Less than 2gb per core required

5 Year hardware useful lifespan

35% off In-House Purchases

How to get the best HPC bang for the Buck

In-House Cloud

Analytics

How to get the best HPC bang for the Buck

Analyze Usage

In-House Cloud

Analytics

Usage

Analysis Average Cores used Historical %

Used (bracket)

1,000 1%1,500 3%2,000 7%2,500 22%3,000 40%3,500 20%4,000 6%4,500 0%5,000 0%5,500 0%6,000 0%6,500 1%

EXAMPLE

How to get the best HPC bang for the Buck

Analyze Usage

Determine Facility CostsIn-House Cloud

Analytics

Facility

Cost Facility Cost

$ per kWh 6.35 cents

Air Conditioning

~42.6% of Power Usage

EXAMPLE

How to get the best HPC bang for the Buck

Analyze Usage

Determine Facility Costs

Research Option Costs

In-House Cloud

Analytics

Cost per Core

EXAMPLEAnnual Cost per Core

(Includes A/C cost)

Description Active Idle

$481.80 FREE

Cloud $262.80 FREE

$67.25 $49.39

$59.94 $55.69

$10.51 $7.53

NEW5YR R430

C6320

C6105

NEW5YR

USED3YR

How to get the best HPC bang for the Buck

Analyze Usage

Determine Facility Costs

Research Cloud Options

Determine what to buy

In-House Cloud

Analytics

0 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000 5,500 6,000 6,500$0.00

$200,000.00

$400,000.00

$600,000.00

$800,000.00

$1,000,000.00

$1,200,000.00

$1,400,000.00

$1,600,000.00

Cloud + Used C6105 Cost vs Purchased Cores Annual Cloud Annual In-House Used Annual In-House Idle

0 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000 5,500 6,000 6,500$0.00

$10,000.00

$20,000.00

$30,000.00

$40,000.00

$50,000.00

$60,000.00

$70,000.00

$80,000.00

$90,000.00

$100,000.00

Cloud + Used C6105 Cost vs Purchased Cores Annual Cloud Annual In-House Used Annual In-House Idle

Sweet Spot$8,004 Idle+

$30,748 Used+$12,045 Cloud=$50,797 TOTAL

How to get the best HPC bang for the Buck

Analyze Usage

Determine Facility Costs

Research Cloud Options

Determine what to buy

Implement

In-House Cloud

Analytics

Results

• 25,842,000 CPU Core Hours• $50,797 Total Cost• Minimal queue wait time• Happy customers

Compute Optimized - Current Generation Per CPU Core per Hour GB/core

c4.large 2 8 3.8 EBS Only $0.11 0.0550 4.00c4.xlarge 4 16 7.5 EBS Only $0.22 0.0550 4.00

c4.2xlarge 8 31 15 EBS Only $0.44 0.0551 3.88c4.4xlarge 16 62 30 EBS Only $0.88 0.0551 3.88c4.8xlarge 36 132 60 EBS Only $1.76 0.0490 3.67

c3.large 2 7 3.8 2 x 16 SSD $0.11 0.0525 3.50c3.xlarge 4 14 7.5 2 x 40 SSD $0.21 0.0525 3.50

c3.2xlarge 8 28 15 2 x 80 SSD $0.42 0.0525 3.50c3.4xlarge 16 55 30 2 x 160 SSD $0.84 0.0525 3.44c3.8xlarge 32 108 60 2 x 320 SSD $1.68 0.0525 3.38

2yr old C6105 System Cost Cores Watts3 $875.00 64 480

Per Core Cost Hardware Power @ 6.35 cents/kwh A/C Total$ over 3 years $13.67188 $37.54755 $5.33823 $31.52596

$ per year (1/3) $4.56 $12.52 $1.78 $10.51$ per day $0.01249 $0.01143 $0.00488 $0.02879

/hr $0.00052 $0.00048 $0.00020 $0.00120

2yr old C6105 System Cost Cores Watts

3 $875.00 64 240

Per Core Cost Hardware Power @ 6.35 cents/kwh A/C Total

$ over 3 years $13.67188 $18.77378 $2.66912 $22.59892

$ per year (1/3) $4.56 $6.26 $0.89 $7.53

$ per day $0.01249 $0.00572 $0.00244 $0.02064

/hr $0.00052 $0.00024 $0.00010 $0.00086