Programming in Large Classrooms Open-source Tools For GPU
Transcript of Programming in Large Classrooms Open-source Tools For GPU
![Page 1: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/1.jpg)
Open-source Tools For GPU Programming in Large Classrooms
Abdul Dakkak, Carl Pearson, Cheng Li
rai-project.com
![Page 2: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/2.jpg)
WebGPU
![Page 3: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/3.jpg)
![Page 4: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/4.jpg)
Originally Designed for MOOC
➔ Around 100k students registered for Coursera's Heterogeneous Parallel Programming course
➔ Targeted weekly labs➔ Labs auto-graded based with dataset
![Page 5: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/5.jpg)
Intro to CUDA
Summer SchoolAround 100 students from all over the world
Advanced CUDAAround 100 students for UIUC and collaborating institutions
Around 200 students from UIUC
Coursera HPPAround 20,000 students worldwide
Students Per Offering
![Page 6: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/6.jpg)
![Page 7: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/7.jpg)
Problem
![Page 8: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/8.jpg)
Restrictions with WebGPU
➔ Cannot modify programming environment◆ Build scripts / libraries / dataset / …◆ Cannot use profilers and debuggers
➔ User restricted within a sandboxed environment
![Page 9: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/9.jpg)
Intro and Advanced CUDA Project
➔ Develop a CUDA version of a CNN➔ Given unoptimized sequential code➔ Significant part of the total grade➔ Around 4-6 weeks to complete➔ Users should be "root"➔ github.com/webgpu/ece408project➔ github.com/webgpu/ece508-convlayer
![Page 10: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/10.jpg)
Pipeline
![Page 11: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/11.jpg)
Jupyter Notebook Interface to RAI
➔ Make it easy to develop interactive labs
➔ Built on top of Jupyter➔ Implements a
client/server that speaks the IPython protocol
![Page 12: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/12.jpg)
Command line InterfaceOutput
Subm
ission Spec
User P
rogram https://asciinema.org/a/6k5e96itnqu6ekbji60c3kgy4
![Page 13: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/13.jpg)
Demo
![Page 14: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/14.jpg)
Architecture
![Page 15: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/15.jpg)
Current Deployment Setup
![Page 16: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/16.jpg)
Docker Layer
Wrote our own docker volume plugin
![Page 17: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/17.jpg)
Not Just Project Submission
▷ A set of reusable components serving as a runtime
▷ Submission specific code is contained and small (<2KLoc)○ Client logic is ~400 lines of code○ Server logic is ~800 lines of code
![Page 18: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/18.jpg)
Service Available BackendsAuthentication Secret, Auth0
Queue NSQ, SQS, Redis, Kafka, NATS
Database RethinkDB, MongoDB, MySQL, Postgres, SQLite, ...
Registry Etcd, Consul, BoltDB, Zookeeper
Config Yaml, Toml, JSON, Environment
PubSub EC, Redis, GCP, NATS, SNS
Tracing XRay, Zipkin, StackDriver
Logger StackDriver, JournalD, Syslog, Kinesis
Store S3, Minio
Container Docker
Serializer BSON, JSON
![Page 19: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/19.jpg)
IMPACT
![Page 20: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/20.jpg)
Usage / Pedigree from Last Semester
➔ Around 170 students had to use the system for submission
➔ Students were using Linux, OSX, Windows, and WLS
➔ Students uploaded and generated around 100GB of data
Used 25 Workers
![Page 21: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/21.jpg)
Currently
➔ Running on the 2 IBM Minsky machines➔ Used by around 100 people in the 508 class (UIUC and Minnesota)
◆ For the last lab◆ For open-ended projects
➔ Students developed their own containers solving anything from Matrix factorization (for recommender systems) to Molecular simulations
![Page 22: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/22.jpg)
CarML
![Page 23: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/23.jpg)
CarML - Deploy ML Artifacts w/RAI
➔ Make it easy to deploy ML artifacts➔ Makes it possible for people to test tools / ML
models without investing time in installing software dependencies and getting HW resources
![Page 24: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/24.jpg)
Resources
![Page 25: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/25.jpg)
GPU TEACHING KIT FOR ACCELERATED COMPUTING
Co-developed by UIUC and NVIDIA for educators
Comprehensive teaching materials
3rd Ed. PMPP E-book by Hwu/KirkLecture slides and notesLecture videosHands-on labs/solutionsLarger coding projects/solutionsQuiz/exam questions/solution
GPU compute resources
NVIDIA online free Qwiklab credits AWS credits
developer.nvidia.com/teaching-kits
Breaking the Barriers to GPU Education in Academia
![Page 26: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/26.jpg)
CUDA Programming
Model
CUDA Memory
Data Management
CUDA Parallelism
Model
Dynamic Parallelism
CUDA Libraries
Unified Memory
Parallel Computation
Patterns
Histogram
Stencil
Reduction
Scan
Sparse Matrix
Merge Sort
Graph Search
Case Studies
Advanced MRI Reconstruction
Electrostatic Potential
Calculations
Deep Learning
Related Programming
Models
MPI
CUDA Python using Numba
OpenCL
OpenACC
OpenGL
developer.nvidia.com/teaching-kits
![Page 27: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/27.jpg)
Questions, Criticisms, and Concerns?
![Page 28: Programming in Large Classrooms Open-source Tools For GPU](https://reader031.fdocuments.net/reader031/viewer/2022012412/616c488216f46a197b2fafe6/html5/thumbnails/28.jpg)
Thank youAbdul Dakkak, Carl Pearson, Cheng Li