UW-Eau Claire Teaching and Research at Using HTCondor...
Transcript of UW-Eau Claire Teaching and Research at Using HTCondor...
![Page 1: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/1.jpg)
Using HTCondor for Teaching and Research at
UW-Eau ClairePeter Bui
![Page 2: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/2.jpg)
University of Wisconsin - Eau Claire
Liberal arts primarily undergraduate institution
1
UW System Center of Excellence for Faculty and Undergraduate Student Research Collaboration
![Page 3: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/3.jpg)
HTCondor Infrastructure
DPL Cluster● 36 Cores
EB Wilson Cluster● 96 Cores, 2 GPUs
LittleFe● 12 Cores, 6 GPUs
2
Key HTCondor Features● Dynamic Slots● Condor Connection Broker● UID_DOMAIN● Flocking
![Page 4: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/4.jpg)
Future HPC Infrastructure
Blugold Commitment SuperComputer
● $100,000 Hardware○ 100-200 CPUs○ 2-4 GPUs
● $20,000 Software○ Specialized compilers○ Domain specific applications
3
Computational Science Working Group
● Interdisciplinary collaboration
● Consolidate management and administration
● Promote HPC research and teaching
General Purpose HPC cluster and a supportive computational science community.
![Page 5: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/5.jpg)
HTCondor in Research
Image Transcoding
4
![Page 6: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/6.jpg)
HTCondor in Research (Continued)
Distributed Photo Processing Pipeline (DP3)
5
![Page 7: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/7.jpg)
HTCondor in Research (Continued)
Distributed System For Automated Blender Rendering (DSABR)
6
![Page 8: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/8.jpg)
HTCondor in Teaching
● CS 252 Computer SystemsMPI Scheduler
● CS 352 Computer Organization & DesignDistributed Computing
● CS 485 Software EngineeringContinuous Integration
● CS 491 Cloud ComputingWorkQueue MapReduce 7
![Page 9: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/9.jpg)
Future HTCondor Projects
● HTML5 Cluster Visualization.
● RESTful Cluster Web Service.
● Cloud provisioning system.
● Curriculum modules for distributed computing.
8
![Page 10: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/10.jpg)
Concluding Observations
● HTCondor enables connecting multiple distributed systems.
● HTCondor is a low-level component in distributed system stack.
● HTCondor is powerful, but also complex.
● HTCondor needs an easier API for third party extension.
9
![Page 11: UW-Eau Claire Teaching and Research at Using HTCondor …cs.uwec.edu/~buipj/static/pdf/htcondor-condor2013.pdf · Teaching and Research at UW-Eau Claire Peter Bui. ... Condor Connection](https://reader033.fdocuments.net/reader033/viewer/2022050917/5a9864bb7f8b9a9c5b8d648b/html5/thumbnails/11.jpg)
Questions?
EmailPeter Bui ([email protected])
Websitehttp://cs.uwec.edu/~buipj
AcknowledgmentsOffice of Research and Sponsored Programs
10