Cycle Computing Record-breaking Petascale HPC Run
-
Upload
insidehpc -
Category
Technology
-
view
711 -
download
1
description
Transcript of Cycle Computing Record-breaking Petascale HPC Run
Record-breaking Petascale CycleCloud HPC Production Run 156,000-core Cluster (1.21PetaFLOPS) Accelerates Schrödinger Materials Science and Green Energy
November 2013 Cycle Computing
Cycle Computing believes utility high performance computing
accelerates invention
Records broken, Science done On November 3rd, ran a “MegaRun” cluster that had: • 156,314 cores and 1.21 PetaFLOPS of theoretical peak compute power • Ran 2.3 Million hours, totaling 264 years of computing, in 18 hours • Executed world-wide, across all 8 public AWS Regions (5 continents) • Compared to $68Million to purchase – done on CycleCloud with Spot Instances for just $33K THE SCIENCE • Finding Organic Photovoltaic Compounds that are more efficient, easier to manufacture to help remove
the US’s reliance on fossil fuels. • Designing, synthesizing, and experimenting with a new material can take 1 year of a scientists time
requiring hundreds of thousands of dollars in equipment, chemicals, etc. or With Schrödinger Materials Science’s tools, on Cycle and AWS Spot Instances, it cost $0.16 per molecule
• The run analyzed 205,000 compounds in total • This is the exact kind of science being outlined in the Materials Genome initiative from the White House
Challenge of Materials Science Traditional Materials Design • Design, Synthesis, Analysis are challenging
for an arbitrary material
• Low hit rate for viable materials
• Total Molecule Cost: • Time: A year for a grad student
• $100,000s in equipment, chemicals, etc.
With Schrödinger Computational Chemistry & Cycle
• Schrödinger Materials Science tools simulate accurate properties in hours
• Simulation guides the researcher’s intuition
• Focus physical analysis on promising materials
• Total cost: • Time to enumerate molecules: Minutes/
hours
• $0.16 per molecule in infrastructure using AWS Spot Instances
Designing Solar Materials The Challenge is efficiency • Need to efficiently turn photons from the sun to Electricity The number of possible materials is limitless • Need to separate the right compounds from the useless ones • If the 20th century was the century of silicon, the 21st will be all
organic How do we find the right material,
without spending the entire 21st century looking for it?
The Challenge for the Scientist Dr. Mark Thompson
Professor of Chemistry, USC “Solar energy has the potential to replace
some of our dependence on fossil fuels, but only if the solar panels can be made very inexpensively and have reasonable to high efficiencies. Organic solar cells have this potential.”
Challenge: run a virtual screen of 205,000 molecules in continuing analysis of possible materials for organic solar cells
The right needle in the right hay stack Before: Trade-off between compute time vs. sampling
Now: Better analysis, more materials è Better results
Coarse screen, Small
samples
Higher Quality
Analysis, More
materials
More Materials
More Materials
Solution: Utility HPC On-demand compute power is transformative for users, but hard to make production � Big Opportunity to help Manufacturing, Life Science, Energy, Financial
companies:
� Rise of BigData, compute, Monte Carlo problems that power modern business and science
� Applications, like Schrödinger Materials Science tools, offer a compelling alternative
to physically testing products
� Amazon Web Services makes infrastructure easily accessible
� AWS Spot instances decrease the cost of compute
� Science & engineering face faster time-to-market, increased agility requirements
� Capital efficiency (OpEx replacing CapEx) are organizational goals
Why isn’t everyone doing this? Because it is really complicated, and really hard to orchestrate
technical applications, securely, at scale We’re the first and only ones doing this including the well-
publicized: 2000, 4000, 10000, 30000, and 50000 core clusters in 2010-2013
Clients including: Johnson & Johnson, Schrödinger, Pfizer,
Novartis, Genentech, HGST, Pacific Life Insurance, Hartford Insurance Group …
Cycle Computing Makes Utility HPC a Reality Easily orchestrates complex workloads and data access to local and Cloud HPC � Scales from 100-1,000,000 cores � Handles errors, reliability � Schedules data movement � Secures, encrypts and audits � Provides reporting and chargeback � Automates spot bidding � Supports Enterprise operations
Challenge: 205,000 compounds
totaling 2,312,959 core-hours, or 264 core-years
Solution: “MegaRun” Cluster
Tool Description Schrödinger Materials Science tools
Set of automated workflows that enable organic semiconductor materials to be simulated accurately
CycleCloud HPC clusters at small to massive scale: application deployment, job/data aware routing, error-‐handling
Jupiter Cycle’s massively scalable, resilient cloud scheduler Chef Automated configuration at scale Multi-‐Region AWS Spot Instances Massive server resource capacity across all public regions of AWS
New record: MegaRun is the largest dedicated Cloud HPC Cluster to date on Public Cloud
16,788 Spot Instances, 156,314 cores!
205,000 molecules 264 years of computing
156,314 cores = 1.21 PetaFLOPS (Rpeak)
Equivalent to Top500 Jun2013 #29
205,000 molecules 264 years of computing
Done in 18 hours Access to $68M system
for $33k
205,000 molecules 264 years of computing
8-Region Deployment
US-West-1 US-East
EU US-West-2
Brazil Singapore
Tokyo
Australia
Jupiter Scheduler � Make large cloud regions work together � Spans many regions/datacenters to resiliently route
work with minimal scheduling overhead � Batch/MPI Schedulers get 10k cores doing 100k jobs � Jupiter seeks to get Millions of cores doing 10Ms tasks
� Currently 100k’s cores doing 1M tasks on large runs
� Can survive machine, availability zone, and region failure while still executing the full workload
Resilient Workload Scheduling
MegaRun – Facts and Figures Metric � Count �Compute Hours of Work� 2,312,959 hours �Compute Days of Work� 96,373 days �Compute Years of Work� 264 years �Molecule Count � 205,000 materials �Run Time � < 18 hours �Max Scale (cores) � 156,314 cores across 8 regions �Max Scale (instances) � 16,788 instances �
Accelerated Time to Result Cluster Scale � Cost � Run-time �
156,000 core CycleCloud � $33,000� ~ 18 hours�
300-core Internal cluster �(stopping all other work) � $132,000� ~ 10.5 months�
CycleCloud–156,000 cores
CycleCloud – 16,788 instances
8 Public Regions across AWS
Ramping up to full capacity
Solution: 205,000 compounds, 264 core years,
156k core Utility HPC cluster in 18 hours
for $0.16/molecule using
Schrödinger Materials Science tools, Cycle & AWS Spot Instances