Virtual Hierarchies to Support Server Consolidation
Mike MartyMark Hill
University of Wisconsin-Madison
ISCA 2007
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
Core
L2 Cache
L1
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
data
data
Optimize Performance
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
Isolate Performance
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
Dynamic Partitioning
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
64-core CMP
Motivation: Server Consolidation
www server
database server #1
database server #2
middleware server #1
middleware server #1
data
Inter-VM Sharing
VMWare’s Content-based Page Sharing Up to 60% reduced memory
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Executive Summary
Motivation: Server Consolidation• Many-core CMPs increase opportunities
Goals of Memory System:• Performance• Performance Isolation between VMs• Dynamic Partitioning (VM Reassignment)• Support Inter-VM Sharing• Hypervisor/OS Simplicity
Proposed Solution: Virtual Hierarchy• Overlay 2-level hierarchy on physically-flat CMP• Harmonize with VM Assignment
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Outline
Motivation• Server consolidation• Memory system goals• Non-hierarchical approaches
Virtual Hierarchies
Evaluation
Conclusion
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
TAG-DIRECTORY
duplicate tagdirectory
A Read A
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
duplicate tagdirectory
TAG-DIRECTORY
A
getM A
1fwd data3
duplicate tagdirectory
2
Read AA
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
STATIC-BANK-DIRECTORY
getM A
1
2
fwd
data3
A
A Read A
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
STATIC-BANK-DIRECTORY
getM A
1
2fwd
data3
A
A Read A
with hypervisor-managed cache
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Goals
Optimize Performance
Isolate Performance
Allow Dynamic Partitioning
Support Inter-VM Sharing
Hypervisor/OS Simplicity
Yes
Yes
?
Yes
No
No
No
Yes
Yes
Yes
STATIC-BANK-DIRECTORY
TAG-DIRECTORY
STATIC-BANK-DIRECTORY w/ hypervisor-managed cache
No
No
Yes
Yes
Yes
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Outline
Motivation
Virtual Hierarchies
Evaluation
Conclusion
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Virtual Hierarchies
Key Idea: Overlay 2-level Coherence Hierarchy on CMP
- First level harmonizes with VM/Workload
- Second level allows inter-VM sharing, migration, reconfig
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
VH: First-Level Protocol
Intra-VM Directory Protocol w/ interleaved directories
Questions: • How to name directories?• How to name sharers?
Dynamic home tile selected by VM Config Table• Hardware VM Config Table at each tile• Set by hypervisor during scheduling
Full bit-vector to track any possible sharer• Intra-VM broadcast also possible
INVgetM
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
VH: First-Level Protocol
Example:
Hypervisor/OS can freely change VM Config Table • No cache flushes• No atomic updates• No explicit movement of directory state
Address……000101
Home Tile: p14offset
6
VM Config Table
p12p13p14
012
63 p12
p12p13
p14
345
p13
p12
p14
Core
L2 Cache
L1
per-Tile
Dynamic
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Virtual Hierarchies
Two Solutions for Global Coherence: VHA and VHB
memory controller(s)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Protocol VHA
Directory as Second-level Protocol• Any tile can act as first-level directory• How to track and name first-level directories?
Full bit-vector of sharers to name any tile• State stored in DRAM• Possibly cache on-chip
+ Maximum flexibility
- DRAM State
- Complexity
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
VHA Example
dir
ecto
ry/m
emo
ry c
on
tro
ller
getM A
1
2
data6
A
AFwd
data
4
3
getM A
5
Fwd
data
A
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Protocol VHB
Broadcast as Second-level Protocol
Attach token count for each block [token coherence]• T tokens for each block. One token to read, all to write• Allows 1-bit at memory per block• Eliminates system-wide ACK
+ Minimal DRAM State
+ Enables easier optimizations
- Global coherence requires more activity
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
VHB Example
mem
ory
co
ntr
oll
ergetM A
1
2
3global getM A
getM A
Data+tokens
4
A
A
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Memory System Goals
Optimize Performance
Isolate Performance
Allow Dynamic Partitioning
Support Inter-VM Sharing
Hypervisor/OS Simplicity
VHA and VHB
Yes
Yes
Yes
Yes
Yes
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Outline
Motivation
Virtual Hierarchies
Evaluation
Conclusion
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Evaluation: Methods
Wisconsin GEMS• Full-system, execution-driven simulation• Based on Virtutech Simics• http://www.cs.wisc.edu/gems
64-core tiled CMP• In-order SPARC cores• 512 KB, 16-way L2 cache per tile• 2D mesh interconnect, 16-byte links, 5-cycle link latency• Four on-chip memory controllers, 275-cycle DRAM latency
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Evaluation: Methods
Workloads: OLTP, SpecJBB, Apache, Zeus• Separate instance of Solaris for each VM
Approximating Virtualization• Multiple Simics checkpoints interleaved onto CMP• Assume workloads map to adjacent cores• Bottomline: No hypervisor simulated
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Evaluation: Protocols
TAG-DIRECTORY:• 3-cycle central tag directory (1024 ways!)
STATIC-BANK-DIRECTORY
• Home tiles interleaved by frame address
VHA
• All data allocates in L2 bank of dynamic home tile
VHB
• Unshared data always allocates in local L2
All Protocols: one L2 copy of block on CMP
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
0
0.2
0.4
0.6
0.8
1
1.2
1.4
No
rma
lize
d R
un
tim
e
Result: Runtime
OLTP Apache
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
Zeus SpecJBB
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
Eight VMs x Eight Cores Each = 64 Cores(e.g. eight instances of Apache)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Result: Memory Stall Cycles
0
0.2
0.4
0.6
0.8
1
1.2
1.4
No
rma
lize
d M
em
ory
Sta
ll C
yc
les
Off-chip Local L2 Remote L1 Remote L2
OLTP Apache
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
Zeus SpecJBB
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
TAG-DIR
STATIC-B
ANK-DIR
VH AVH B
Eight VMs x Eight Cores Each = 64 cores(e.g. eight instances of Apache)
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Executive Summary
Server Consolidation an Emerging Workload
Goals of Memory System:• Performance• Performance Isolation between VMs• Dynamic Partitioning (VM Reassignment)• Support Inter-VM Sharing• Hypervisor/OS Simplicity
Proposed Solution: Virtual Hierarchy• Overlay 2-level hierarchy on physically-flat CMP• Harmonize with Workload Assignment
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Backup Slides
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Omitting 2nd-Level Coherence: Protocol VH0
Impacts:• Dynamic Partitioning• Inter-VM Sharing (VMWare’s Content-based Page Sharing)• Hypervisor/OS complexity
Example: Steps for VM Migration from Tiles {M} to {N}
1. Stop all threads on {M}
2. Flush {M} caches
3. Update {N} VM Config Tables
4. Start threads on {N}
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Omitting 2nd-Level Coherence: Protocol VH0
Example: Inter-VM Content-based Page Sharing• Up to 60% reduced memory demand
Is read-only sharing possible with VH0?
VMWare’s Implementation:• Global hash table to store hashes of pages• Guest pages scanned by VMM, hashes computed• Full comparison of pages on hash match
Potential VH0 Implementation:• How does hypervisor scan guest pages? Are they modified
in cache? • Even read-only pages must initially be written at some point
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Physical Hierarchy / Clusters
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
Shared L2 Shared L2
PL1 $
Shared L2 Shared L2
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation
Physical Hierarchy / Clusters
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
Shared L2 Shared L2
PL1 $
Shared L2 Shared L2
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
PL1 $
www server
database server #1
middleware server #1
Top Related