Moving Virtualization Forward:How emerging data center architectures are enabling virtualization adoption

Randy Allen

Corporate VP, Server and Workstation Division

Issues That Are Raising Data Center Costs

Underutilized resources Power, cooling, and space constraints Managing and securing the computing infrastructureDisruptions that decrease user productivity

Solution: Transform Data Center Infrastructure

• Significant unused capacity

• Inflexible

• Many legacy technologies

• Higher rates of utilization

• Reconfigure dynamically

• Unified resources “managed as one”

Internet Virtual Resource Pool

Shared Infrastructure

How Do We Define “Virtualization?”

Virtualization:The Use of Abstractions of Physical Entities in Place of Direct References to those Entities

If Done with Finesse, These Substitutions Are Invisible to Users and Developers and the Software They Use or Develop

Source: Picasso, 1912 (Table in a Café)

Virtualization Has Been Around For a Very Long Time

We See Virtual Images In Mirrors Everywhere

Often Extremely RealisticSometimes Highly Distorted• Rear View Mirrors• Fun House Mirrors

Interior Designers Use Mirrors to Deceive Our Senses and Make Rooms Seem LargerSoftware Designers Use Virtualization to Deceive Programs and Make Machines Seem Larger

Source: Carroll/Tenniel (Illustrator), 1871 : Through the Looking-Glass

Virtually All Aspects of System HardwareCan Be Virtualized

Virtualization can mask annoying hardware details from software

Virtualized devices can be highly fault tolerant

Virtualization can provide added flexibility to configuring and managing hardware resources

Virtual Virtual Memory Memory

Physical

MemoryPhysical Disks, LANs, and SANs

Virtual Virtual StorageStorage

Virtual Virtual MachineMachine

Physical Server

One Physical Machine Can Typically Host 10 to 30 VMs

Multi-Core CPUs Will Further Improve This Ratio

Reduces “Server Sprawl”Better Use of Data Center Real Estate

Improves Electrical Power Utilization

10-20 Watts Per Idle VM(vs. 100-200W Per Idle PM)

Simplifies Deployment of New Applications, ServersFacilitates Rapid Recovery From Hardware Failures

Source: Picasso, 1968 (Homme a la Pipe)

Behind Every Virtual Machine Stands(a Fraction of) A Physical Machine

The Secret Sauce That MakesVirtualization Possible

It’s All Done With Mirrors!

With a Little Help from AMD Silicon and Advanced Software from Several AMD Ecosystem Partners

Source: M. C. Escher, 1935 (Hand with Reflecting Sphere)

Hypervisor

Virtual Virtual MachineMachine

Guest OS

Application

Virtual Virtual MachineMachine

Guest OS

Application

Technology that partitions a computer into several independent machines that can support different OS’s and applications concurrently

Hypervisor is the underlying software which runs directly on the hardware and manages multiple operating systems

Virtual machine is self-contained operating environment that runs on top of the hypervisor and behaves as if it is a separate computer

Virtual Virtual MachineMachine

Guest OS

Application

RealMachine

What Is A Virtual Machine?

Virtualization converts physical boundaries into logical constructs

You can still have your special function servers –they’re just sharing physical resources with each other

Virtualized servers can be moved easily between physical machines, thus dynamically reconfiguring your data center

Virtual Virtual MachineMachine

Virtual Virtual MachineMachine

Virtual Virtual MachineMachine

Move Virtual Machines based on workload or maintenance schedules

Breaks “one application per server” model

Virtual Virtual MachineMachine

ApplicationApplication

Virtual Virtual MachineMachine

Virtual Virtual MachineMachine

Virtual Virtual MachineMachine

From Physical Servers to Virtual Machines

Virtualization In the x86 World

Ring 0

Ring 1

Ring 2

Ring 3

Operating SystemDevice Drivers

Only privileged code can define memory-mapping

parameters or set up interrupt handlers

• In the x86 environment, the OS is “selfish” and expects direct control of the CPU

• x86 architecture supports different classes of privileged operations (often called “rings”)

• Privileged software runs in Ring 0 and can exercise control over software running at higher rings

• Traditional x86 processors can't simultaneously run both host and guest OS’s at Ring 0

ApplicationApplication

Application

AMD Is Driving x86-based Virtualization

AMD and VMware are working together to develop a robust virtualization ecosystem

that provides users with innovative virtualization solutions

VirtualizationExtensions

(Today)

I/O Virtualization

(2008*)

Nested Paging(2007*)

Direct Connect

Architecture

AMD64 TechnologyRoad Map

Increase performance with multi-core

capabilities and fast memory access

Simplify virtualization software and helps reduce CPU overhead

Virtualize devices to improve

performance and security

Reduce overhead of switching between virtual machines

*Planned features for future AMD Opteron™processor-based systems

Software-Enabled Virtualization

Ring 0Ring

1Ring

2Ring 3

HypervisorHost OS

ModifiedGuest OS

Application

Para-virtualization Hypervisor handles Host/Guest OS interactions Guest OS is recompiled to run at Ring 1 Host OS handles devices

Ring 0

Ring 1Ring

2Ring 3

Hypervisor

Unmodified Guest OS

Application

Full Virtualization Hypervisor is also Host OS Intercepts and emulates instructions and virtualizes devicesGuest OS runs unmodified

Hardware-Enabled Virtualization

Ring 0Ring

1Ring 2Ring

3

Hypervisor

Guest Mode

ApplicationCreates a Guest Mode that allows Guest OS to run unmodified with Hypervisor

Hypervisor controls access to devices

Tagged TLB reduces the number of memory accesses needed to switch between host/guests

Instructions ease creation and management of virtual machines

Performance

64-bit and Dual-Core Performance-

per-watt

Assists datacenters in

controlling power consumption and

heat output

Virtualization

Grid Computing

Enhances performance while

offering the flexibility to support both 32- and 64-bit

applications

Enables separate, secure operating

environments

High performance in smaller form factor

Perfect balance of performance, power, and cost

AMD Opteron™ Processor The Platform for Virtualization

Native quad-core upgrade for 2007Introducing “Barcelona”…

Advanced Process Technology

65nm Silicon-on Insulator Process

Fast transistors with low power leakage to reduce power and heat.

Direct Connect Architecture

• Integrated memory controller designed for reduced memory latency and increased performance

• Memory directly connected

• Provides fast CPU-to-CPU communication

• CPUs directly connected

• Glueless SMP up to 8 sockets

Native Quad-Core Processor

To increase performance-per-watt efficiencies using the same Thermal Design Power.

Platform Compatibility

Socket and thermalcompatible with “Socket F”.

A Closer Look at the Barcelona Processor

Expandable shared L3 cache

IPC-enhanced CPU cores

More deliveredDRAM Bandwidth

ComprehensiveUpgrades

for SSE128

Virtualization Performance

Advanced Power

Management

I/O HubI/O HubUSBUSB

PCIPCI

PCIe™Bridge

PCIe™Bridge

PCIe™Bridge

PCIe™Bridge

I/O HubI/O Hub

8 GB/S

8 GB/S 8 GB/S

8 GB/S

PCI-E Bridge

PCI-E BridgePCI-E Bridge

PCI-E BridgePCIe™Bridge

PCIe™Bridge

USBUSB

PCIPCII/O HubI/O Hub

XMBXMBXMBXMB XMBXMB XMBXMB

Direct Connect Architecture – Virtualization Platform

Memory Controller

Hub

Memory Controller

Hub

MCPMCP MCPMCPMCPMCP MCPMCP

Legacy x86 Architecture• 20-year old front-side bus architecture• CPUs, Memory, I/O all share a bus• Major bottleneck to performance• Faster CPUs or more cores ≠ performance

CORE CORE CORE CORE CORE CORE CORE CORE

AMD64 Technology with Direct Connect Architecture• Industry-standard AMD64 technology• Eliminate bottlenecks of traditional front-side bus architecture• HyperTransport™ technology interconnect for high bandwidth

and low latency

AMD Virtualization™ TodayEnabling More Efficient Workload Consolidation

• Instructions added to help reduce complexity of virtualization software

• Reducing overhead by selectively intercepting information destined for guest

• Enables Guest OS’s to run unmodified for easier implementation and support

• Increased isolation to improve security of virtual machines

• Offers architectural enhancements to improve efficiency of switching between hypervisor and the guest OS’s

Hypervisor Host OS

Virtual Virtual MachineMachine

Guest OS

Application

Virtual Virtual MachineMachine

Guest OS

Application

Virtualization Instructions

Virtualization is Memory Intensive

Memory Controller HubMemory Controller Hub

CPUCPU CPUCPU CPUCPU CPUCPU

VMVM VMVM VMVM VMVM VMVM VMVM VMVM VMVM VMVM VMVM VMVM VMVM

CPUCPU

VMVM VMVM VMVM

CPUCPU

VMVM VMVM VMVM

CPUCPU

VMVM VMVM VMVM

CPUCPU

VMVM VMVM VMVM

Memory Controller

Memory Controller

Memory Controller

Memory Controller

Memory Controller

Memory Controller

Memory Controller

Memory Controller

Shared memory can create bottlenecks Dedicated memory for scalability

• Direct Connect Architecture helps improve application performance within a virtual machine

• Tagged TLB means more efficient switching between virtual machines

• Hardware-based memory management and security (Integrated memory controller with DEV) can improve overall virtualization performance and efficiency

• Shared front-side bus can decrease application performance within a virtual machine

• Untagged TLB means less efficient switching between virtual machines

• Software-based memory management and security (via external Memory Controller Hub) can reduce overall virtualization performance and efficiency

Virtualization Performance Today – 4P Servers

740W883W 902W 763W 796W

96% CPU Util.

96% CPU Util.

82% CPU Util.

Advantages of AMD Opteron™ processor-based

system for x86 Virtualization

At Same CPU Utilization:6 additional VMs hosted2.6x better web serving2.1x better file & print serving12% less power draw

At 20 VMs:Over 2x better web servingOver 50% better file & print serving15% less power draw14% more CPU headroom

Full 4P study available at: http://www.veritest.com/NR/rdonlyres/F1F21288-8D6D-4769-94F3-252FCB02D51F/272/AMD_Virtualization_Project_Final_Report.pdf

Virtualization Performance – 2P Servers

Advantages of AMD Opteron™ processor-based system

for x86 Virtualization

At similar CPU utilization rates:7-12% more SQL Requests per second13-51% more Web requests per second2-19% better file & print serving

At 12 VMs:10% Advantage in Light Workload 26% Advantage in Heavy WorkloadApproximately a 26% performance/watt advantage

Full study available at: http://www.veritest.com/

Workload Scores (Sums of Individual VMs)(DBH and WB in rqsts/sec, NB in Mb/sec)

0.00

500.00

1000.00

1500.00

2000.00

2500.00

3000.00

Light Workload Heavy Workload100.00

105.00

110.00

115.00

120.00

125.00

130.00

DBHammer Intel

DBHammer AMD

WebBench Intel

WebBench AMD

NetBench Intel

NetBench AMD

110% Geomean

126% Geomean

Nested Paging Enabling More Efficient Switching

• Provides the guest OS with the illusion that it is managing memory

• Page tables are actually kept up by the hypervisor in software

• Requires more software intervention from the hypervisor

• Each guest physically has their own world to manage

• Requires less intervention • Memory look ups done in

hardware which can be faster than software management

Hypervisor

Virtual Machine

Guest OS

Application

Virtual Machine

Guest OS

Application

AppPage Table

Guest OSPage Table

AppPage Table

Guest OSPage Table

Shadow Page Tables

AppPage Table

Guest OSPage Table

AppPage Table

Guest OSPage Table

Hypervisor

Virtual Machine

Guest OS

Application

Virtual Machine

Guest OS

Application

Nested Page Tables

Guest OS Page Table

Host Page Table

Guest OS Page Table

Guest OS Page Table

Guest OS Page Table

AMD’s “Barcelona” Processor Improves Virtualization Performance

Nested Paging (NP)Guest and Host page tables both exist in memory• The processor walks both guest and host page

tablesNested walk can have up to 24 memory accesses!• Hardware caching accelerates the walk“Wire to wire” translations are cached in TLBsEliminates hypervisor cycles spent managing shadow pages• As much as 75% of hypervisor time

Barcelona also reduces world-switch time by 25%World-switch time : round-trip to the hypervisor and back

Hypervisor Host OS

Virtual Machine

Guest OS

Application

Virtual Machine

Guest OS

Application

AMD-V

Nested Page Tables

Guest OS Page Table

Host Page Table

Guest OS Page Table

Guest OS Page Table

Guest OS Page Table

113.22

198.22

387.67

117.00

205.56

389.61

0.00 50.00 100.00 150.00 200.00 250.00 300.00 350.00 400.00 450.00

AMD-V & NP

BT

AMD-V (no NP)

Actual VM time in seconds

Cygwin Compile with AMD Nested PagingVMware products will take advantage of hardware-based Guest Page Translation assist (such as AMD Nested Paging) in 2007 and beyond as hardware becomes available.

2VM Avg1VM

Cygwin Compile

Platform: Experimental AMD Processor with Nested Paging running experimental build of VMware Workstation.

Among Best Case Improvement for Nested Paging, which mainly

helps memory-management intensive workloads; not

representative of all workloads.

Nested Paging reduces compile time by

43%

The First Wave: Using Virtualization to Lower Data Center Costs

Consolidate workloads to reduce hardware and space requirements

Run legacy software on reliable, power-efficient hardware

Reduce hardware requirements for development and test

AMD: Consolidating Hardware

In Austin, approximately 135 servers consolidated to 7 active ESX 3.0 servers plus 2 swing serversIn Sunnyvale, approximately 45 servers consolidated to 2 active ESX 3.0 servers plus 1 swing serverOverall consolidation ratio between the two sites is 15:1 including swing serversAnticipate 79% reduction in power consumption (Austin Data Center)

One of the largest direct providers of technology solutions for business, government and educationDevelops most applications in-house

Can now easily provision development and test environments without having to build physical serversReduced server deployment time from a day to a half hour$25,000 a month savings from deploying virtual machinesCPU utilization increased from 5 – 15% to an average of 50%

Virtualization Environment• Virtualization Software: VMware ESX Server

• Servers: 2P servers and 4P HP ProLiant DL 585s running AMD Opteron™ processors

• Guest OS’s: Windows® and Linux®

• Applications: Windows domain controllers, IIS, FTP Endeca, ISA Server 2004, File & Print, SQL Server, Exchange, Blackberry Enterprise Server, BizTalk Server, Cognos, PeopleSoft

http://www.vmware.com/pdf/cdw.pdf

CDW: Wowing the Software Developer

The Data Center of the FutureIn the 1970’s and 1980’s, Large Systems Prevailed

The Larger, the Better(Grosch’s Law)

Over the Last Decade, Scale-Out Deployments Became Feasible

The Smaller, the Better(Moore’s Law)

Today, the Pendulum Will Start to Swing Back Toward Larger Systems

Reduce Server Sprawl and Power RequirementsSimplify System ManagementImprove Resource Utilization

Virtualization Plays a Key Role in Enabling this Shift

A Few Big Servers

Many Small Servers

Closing Considerations

Virtualization Facilitates Resource Sharing

But You’ve Got to Have Resources to Share

Some workloads are not the best candidates for virtualization

May Be of Little Value in Compute-Intensive HPTC environments

Virtual Machines Require Real Software Licenses Paid for with Real Money

Source: Picasso, 1947 Ulysses and the Sirens)

Summary

Virtual Technology Can Help IT Managers:Simplify Data Center OperationsImprove Resource UtilizationReduce Server SprawlImprove the Power Efficiency of their Data Centers

Evolving Hardware and Software Offerings Will Enhance Virtualization and Security in Coming Years

AMD Is Playing a Key Role in Driving this EvolutionIf You Have Yet to Explore this Technology, Watch Out!

Your Competitors Are Moving in this DirectionYou Are at Risk of Falling Behind

Trademark Attribution

AMD, the AMD Arrow logo, AMD Athlon, AMD Opteron, AMD Turion, AMD Sempron. AMD Geode, and combinations thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdictions. Other names used in this presentation are for identification purposes only and may be trademarks of their respective owners.

©2006 Advanced Micro Devices, Inc. All rights reserved.

Presentation Download

Please remember to complete yoursession evaluation form

and return it to the room monitorsas you exit the session

The presentation for this session can be downloaded at http://www.vmware.com/vmtn/vmworld/sessions/

Enter the following to download (case-sensitive):

Username: cbv_repPassword: cbvfor9v9r

Some or all of the features in this document may be representative of feature areas under development. Feature commitments must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery.