Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.
-
Upload
justine-dager -
Category
Documents
-
view
220 -
download
1
Transcript of Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.
![Page 1: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/1.jpg)
![Page 2: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/2.jpg)
Considerations and Strategies for Deploying Virtual Clusters
Giovanni MarchettiSr. Technical EvangelistMicrosoft Corp.WSV202
![Page 3: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/3.jpg)
Session Objectives
Brief intro to HPC ClustersWhy virtualize HPC clusters?Where do they make sense?How do we deploy and manage them?What configuration issues arise?How do we schedule jobs on them?Are they so exotic?
![Page 4: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/4.jpg)
What Is High Performance Computing?
A branch of computer science that studies systems designs and programming techniques to extract the best computational performance out of microprocessors. It is usually associated with parallel computing either on specialized machines or on clusters of commodity machines.
![Page 5: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/5.jpg)
Industries Using HPC
Education & ResearchGeological ServicesManufacturingFinancial ServicesMoviesMedicineArchaeology???!!!
![Page 6: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/6.jpg)
Real-world Application Example
Courtesy ofUniversity of Bologna, Italy
CAT scan of a 3000 year-old catData and computation – intensiveLimited comms. among nodesTypical parametric sweep
![Page 7: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/7.jpg)
MPINetwork
PrivateNetwork
PublicNetwork
Corporate IT Infrastructure
Compute NodeHead Node
AD
DNS
DHCP
Clients Monitoring SystemsManagement
Compute Cluster
Possible Cluster Topologies
Admin / User Cons
WDS
Job Scheduler
MPI
Management
NAT
Node Manager
MPI
Management
Compute Node
Node Manager
MPI
Management
![Page 8: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/8.jpg)
The Opportunity
Low usage on physical servers (e.g. at night)Make computers compute for a living
Virtualized infrastructureTreat computing as just another functionIntegrate with “real” HPC clusters
Maintain isolation for security and ease of management reasons
Cycle scavenging does not
![Page 9: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/9.jpg)
The Applications
Suitable for parametric sweeps (e.g., Montecarlo simulations) or web services
Many independent processes, possibly multi-threadedLarge set of inputsNot interactive
NOT suitable for MPI applicationsVirtualization comms. overhead (reduced with R2)Unpredictable latenciesLimited topologies
![Page 10: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/10.jpg)
The Options
We have 3 alternatives:Install the HPC Pack on all the x64 serversDual bootUse Hyper-V and virtual machines with HPC 2008
Coming in R2 / v3 Boot from networkBoot from vhd
![Page 11: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/11.jpg)
HPC Pack on all servers
ProsLittle overhead when not in useEasy to deploy and manageAlways ready
Cons:No isolation at all
Whoever can launch a job, can run anything on your computer / x64 workstation
Only works on the domain your computer is inHead node may not necessarily be in that domain
No control on resource consumption
![Page 12: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/12.jpg)
Dual Boot
Pros:“Clean” solution, as long as you have a 2nd partition100% of resources available in designated timeNo perf. Overhead
Cons:Licenses: 2 per machine!!!100% of resources available ONLY in designated timeRequires careful management of OS transitions
Diskpart is powerful and dangerous.Limited isolation: must manage access to partitions carefully
![Page 13: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/13.jpg)
Hyper-V
ProsComplete isolationControl resource allocation Rapid provisioning / de-provisioningDomains and VLANs for VM can differ from hostVirtualization rights included or free hyper-v server
Cons:Performance and management overheadMust have virtualization management solutionMust work on your own scheduler integration
![Page 14: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/14.jpg)
Hyper-V: Quick Tips
Works only on x64 servers with DEP and VT (or AMD-V) enabled
Most x64 workstations can run it;Install “desktop experience”
Once installed, everything is a virtual machine, including the management partitionPatch early, patch often and be sure you know what to patch
![Page 15: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/15.jpg)
Hyper-V Management
Requires Virtual Machine ManagerVMM will install agent, hyper-v role on targeted servers in trusted domainsCan manage servers in different domains
Deploy these patches on hyper-v hosts:KB956589 – perf. counters for synthetic storageKB956774 – BITS client handling volume IDs
![Page 16: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/16.jpg)
Virtual Machine Manager Server
Connector
Self Service Web Portal
Administrator’s Console
Management Interfaces
SAN Storage
Operator’s Console
Web Console
Windows PowerShell
Operations Manager Server
Virtual Server Host
VM
VM
VMM Library Server
VM Template
ISO ScriptVHD
VMware VI3
Virtual Center Server
ESX Host
VM VM VM VM
VM VM VM VM
VM VM VM VM
Windows® PowerShell
VMM 2008 Architecture
![Page 17: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/17.jpg)
VM Deployment
Build “prototype node” VMCreate hardware and o/s profile based on itExtract a template from it with SCVMM
Will run sysprep and make the node unavailableSelect target servers to run hyper-v
Note: RDMA adapters (e.g. Infiniband) will not be virtualized; they will be used as any other IPoIB interface
Deploy node template on target servers (GUI or powershell script)
![Page 18: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/18.jpg)
VM Placement
CPU, Network,and Disk Load
Physical Disk and Memory Requirements
Configuration
VM
Result of Disk Capacity and Memory Check
Disk Capacity and Memory Check
Capacity PlanningTechnology
Normalized Host + VM Load
Existing Load Configuration
Host Performance and Configuration
RatingFunction
![Page 19: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/19.jpg)
VM Placement
Memory is the main constraintVM can be moved and RAM allocation altered, but not dynamically
HPC VMs will most often either run at 100% CPU or 0% within the VM
Does not necessarily map to real cpu utilizationHPC job scheduler does not understand “50% of CPU”
![Page 20: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/20.jpg)
VM Placement Considerations
Specify number of CPUs in VM and assume 100%Total Number of active logical CPUs ≤ number of physical CPUs on hostLogical will be mapped to physical by hyper-v scheduler
![Page 21: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/21.jpg)
Hyper-V Architecture Provided by:
OS
MS / XenSource / Novell
ISV/IHV/OEM
Hyper-V
Windows hypervisor
“Designed for Windows” Server Hardware
Non hypervisor aware OS
Windows Server 2003, 2008
Applications
WindowsKernel VSC
WindowsKernel
Windows Server 2008
VSP
VMBus Emulation
Parent Partition
Kernel Mode
User Mode
Xen-enabledLinux Kernel Linux
VSCs
VMBus
Hypercall Adapter
Applications
Child Partitions
VMBus
Virtualization Stack
VMService
WMI Provider VM WorkerProcess
Applications
![Page 22: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/22.jpg)
VM Placement Considerations
SCVMM hardware profiles ≠ hardware constraintsIndication of expected hw performance
Hyper-V always reports up to 1 quad-core cpu of underlying kind
Co-located tasks in VM may not actually beIdentical hardware profiles on same kind of host simplify life for the scheduler and the admin
Not always practicalPlace for performance, not density (in R2)
![Page 23: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/23.jpg)
On the HPC side
Create host groupsRule of thumb: 1 per VM template deployed
Use job templates to enforce constraints for jobs on VM host group (e.g. max runtime)Schedule jobs at appropriate time
At <time> job submit <job parameters>Or specify host group, then activate the nodes in question
At <time> node resume <node list>
![Page 24: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/24.jpg)
Rapid Provisioning / De-provisioning
Create several templatesDifferent kind of nodes for different kind of jobs> 1 template deployed to same physical host, 1 active
Test scalability by deploying N, 2N, etc… VMs. on same hostsNo need to delete VMs
Take nodes offline and shut them downWhen required, turn them back on
![Page 25: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/25.jpg)
Provisioning on demand
1 more layer of complexityUse SCOM to track n. of tasks in the queueCause VM provisioning or activation when high threshold is reached (run psh script)De-provision or shut down when low threshold is reached
SCOM / SCVMM / Pro tipsMay cause VMs to be movedMay be irrelevant to particular HPC workload
VMs always run 100% or 0 %
![Page 26: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/26.jpg)
Dynamically Grow / Shrink
Dynamically grow or shrink pool of resources for jobs in HPC Server 2008Ties in with dynamic provisioning / de-provisioning
Provision serversGrow pool available to running jobOffline servers -> shrink poolDe-provision or pause VMs
Job keeps running
![Page 27: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/27.jpg)
SCOM rules for on-demand VMsdemo
![Page 28: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/28.jpg)
Availability advantages
High availability of head nodes is quite complexSCVMM takes care of HA for VMs on clusters
Head node can be a VM tooIn this case, you’re protected against node failure only
What if time on a machine has run out, but you still need to compute?
Shrink pool, move VM to available node, bring backWith R2, live migration means you need not interrupt the computation either!
![Page 29: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/29.jpg)
Moving Computing Nodesdemo
![Page 30: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/30.jpg)
Call to Action
Just try it!www.microsoft.com/hpcwww.microsoft.com/hyper-vwww.microsoft.com/scvmm
![Page 31: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/31.jpg)
questions & answers
![Page 32: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/32.jpg)
www.microsoft.com/teched
Sessions On-Demand & Community
http://microsoft.com/technet
Resources for IT Professionals
http://microsoft.com/msdn
Resources for Developers
www.microsoft.com/learningMicrosoft Certification and Training Resources
www.microsoft.com/learning
Microsoft Certification & Training Resources
Resources
![Page 33: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/33.jpg)
Related ContentVIR208 Virtual Machine Manager 2008: Technical Overview and R2 PreviewVIR209 How to Build an Efficient Application Infrastructure through VirtualizationVIR302 Hyper-V & Virtual Machine Manager 2008: Tips, Tricks, and Best PracticesVIR303 Hyper-V & System Center: Practical Orchestration & Process Automation
VIR03-INT Virtual Computing Clusters
VIR04-HOL Introduction to Hyper-V VIR05-HOL Introduction to Microsoft System Center Virtual Machine Manager 2008
![Page 34: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/34.jpg)
Windows Server ResourcesMake sure you pick up your copy of Windows Server 2008 R2 RC from the Materials Distribution Counter
Learn More about Windows Server 2008 R2: www.microsoft.com/WindowsServer2008R2
Technical Learning Center (Orange Section): Highlighting Windows Server 2008 and R2 technologies•Over 15 booths and experts from Microsoft and our partners
![Page 35: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/35.jpg)
Complete an evaluation on CommNet and enter to win!
![Page 36: Giovanni Marchetti Sr. Technical Evangelist Microsoft Corp. WSV202.](https://reader037.fdocuments.net/reader037/viewer/2022102814/551a7ef5550346761a8b4f84/html5/thumbnails/36.jpg)
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.