NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE...
Transcript of NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE...
![Page 1: NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE COMPUTING DEPARTMENT RICHARD GERBER Department Head ADVANCED TECHNOLOGIES NICHOLAS](https://reader034.fdocuments.net/reader034/viewer/2022042205/5ea70f518cb91a1ed9388983/html5/thumbnails/1.jpg)
PrabhatData & Analytics Services Group
January 25, 2019
NERSC Data Ecosystem Overview
![Page 2: NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE COMPUTING DEPARTMENT RICHARD GERBER Department Head ADVANCED TECHNOLOGIES NICHOLAS](https://reader034.fdocuments.net/reader034/viewer/2022042205/5ea70f518cb91a1ed9388983/html5/thumbnails/2.jpg)
Afternoon Schedule1:40 pm Data Transfer Shreyas Cholia
2:10 pm File Systems + Burst Buffer Wahid Bhimji
2:30 pm I/O Best Practices Quincey Koziol
2:50 pm Break
3:10 pm Python and Jupyter Rollin Thomas
3:30 pm Machine Learning Mustafa Mustafa
3:50 pm Shifter Shane Canon
4:10 pm End
![Page 3: NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE COMPUTING DEPARTMENT RICHARD GERBER Department Head ADVANCED TECHNOLOGIES NICHOLAS](https://reader034.fdocuments.net/reader034/viewer/2022042205/5ea70f518cb91a1ed9388983/html5/thumbnails/3.jpg)
![Page 4: NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE COMPUTING DEPARTMENT RICHARD GERBER Department Head ADVANCED TECHNOLOGIES NICHOLAS](https://reader034.fdocuments.net/reader034/viewer/2022042205/5ea70f518cb91a1ed9388983/html5/thumbnails/4.jpg)
Phase I: 2388 x 32-core Intel Xeon “Haswell” 128 GB DDR4Phase II: 9688 x 68-core Intel Xeon Phi “KNL” 96 GB DDR4 + 16 GB MCDRAM
Gerty Cori: Biochemist and first American woman to win a Nobel Prize in science
Cori Brings HPC & Data Together
![Page 5: NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE COMPUTING DEPARTMENT RICHARD GERBER Department Head ADVANCED TECHNOLOGIES NICHOLAS](https://reader034.fdocuments.net/reader034/viewer/2022042205/5ea70f518cb91a1ed9388983/html5/thumbnails/5.jpg)
Production Data StackCapabilities Technologies
Data Transfer + Access
Workflows
Data Management
Data Analytics
Data Visualization
TaskFarmer
![Page 6: NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE COMPUTING DEPARTMENT RICHARD GERBER Department Head ADVANCED TECHNOLOGIES NICHOLAS](https://reader034.fdocuments.net/reader034/viewer/2022042205/5ea70f518cb91a1ed9388983/html5/thumbnails/6.jpg)
Cori’s Data-Friendly Features
Cray DataWarp:Burst Buffer forI/O acceleration
12 32-core Haswell500 GB Login Nodes
768 GB “bigmem”Haswell Nodes
Pipeline/WorkflowManagement Nodes
JupyterNotebook
Node
Serial QueueShared-node Queue
Transfer Queue
Real-Time Queues for Co-Scheduling
w/Experiments
Interactive Queue:64 Nodes x 4 Hours
ContainerizedEnvironments
External Network Access to/fromCompute Nodes
Streaming Data to Compute Nodes
![Page 7: NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE COMPUTING DEPARTMENT RICHARD GERBER Department Head ADVANCED TECHNOLOGIES NICHOLAS](https://reader034.fdocuments.net/reader034/viewer/2022042205/5ea70f518cb91a1ed9388983/html5/thumbnails/7.jpg)
Asks
● Please engage with NERSC staff members○ Provide feedback, critique○ Tell us about interesting science problems!
![Page 8: NERSC Data Ecosystem Overview Data & Analytics Services Group · 1/25/2019 · HIGH PERFORMANCE COMPUTING DEPARTMENT RICHARD GERBER Department Head ADVANCED TECHNOLOGIES NICHOLAS](https://reader034.fdocuments.net/reader034/viewer/2022042205/5ea70f518cb91a1ed9388983/html5/thumbnails/8.jpg)
Questions? Comments?