Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of...
Transcript of Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of...
![Page 1: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/1.jpg)
Dynamic Selection ofApplication-Specific GarbageCollectors
Sunil V. SomanChandra Krintz
University of California, Santa BarbaraDavid F. Bacon
IBM T.J. Watson Research Center
![Page 2: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/2.jpg)
ISMM '04 2
Background
VMs/managed runtimes Next generation internet computing Automatic memory management for memory
safety High performance, multi-application servers
GC performance impacts overallperformance So GC must perform well
![Page 3: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/3.jpg)
ISMM '04 3
GC Research Focus on general-purpose GC
One GC for all apps
No single GC provides bestperformance in all cases Across applications Even for the same application given
different memory constraints
![Page 4: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/4.jpg)
ISMM '04 4
Motivation
1 2 3 4 5 6 7 8 9 10 11 12Heap Size Relative to Min
110
160
210
260
310
360
410
10^6
/Thr
ough
put
SPECjbb2000 SSMSGMS
GSSCMS
switch point
1 2 3 4 5 6 7 8 9 10 11 12
Heap Size Relative to Min
15
20
25
30
35
40
Exec
utio
n Ti
me
(sec
)
_209_db SSMSGMSGSSCMS
![Page 5: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/5.jpg)
ISMM '04 5
Motivation
Other researchers have reported similarresults [Attanasio ’01, Fitzgerald ’00] Spread between best & worst at least 15% Generational not always better
Employ a VM instance built with theoptimal GC
Goal of our work: employ multipleGCs in the same system and switchbetween them
![Page 6: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/6.jpg)
ISMM '04 6
Framework
Implemented in the JikesRVM Uses the Java Memory management Toolkit
(JMTk) Extended to enable multiple GC support
Can switch between diverse collectors Easily extensible
GC System = Allocator + Collector
![Page 7: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/7.jpg)
ISMM '04 7
Framework Implementation
StopTheWorld
Plan
SS_Plan MS_Plan GenerationalCopyMS_Plan
GenMS_Plan GenCopy_Plan
StopTheWorld
VM_Processor
Plan{SS_Plan,MS_Plan,...}
(Selected at build time)
VM_Processor
(Plan currentPlan;)
Most code is shared across GCs 44MB vs 42MB (average across GCs)
![Page 8: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/8.jpg)
ISMM '04 8
Framework Implementation Example
MS -> GSS MS -> GMS
Nursery
Mark-Sweep
High Semispace
Low Semispace
Large Object Space
GC Data Structures
Immortal
HIG
HER
LOW
ER
Nursery
Mark-Sweep
High Semispace
Low Semispace
Large Object Space
GC Data Structures
Immortal
HIG
HER
LOW
ER
Nursery
Mark-Sweep
High Semispace
Low Semispace
Large Object Space
GC Data Structures
Immortal
HIG
HER
LOW
ER
Unmap
Nursery
Mark-Sweep
High Semispace
Low Semispace
Large Object Space
GC Data Structures
Immortal
HIG
HER
LOW
ER
AllocNursery
Mark-Sweep
High Semispace
Low Semispace
Large Object Space
GC Data Structures
Immortal
HIG
HER
LOW
ER
![Page 9: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/9.jpg)
ISMM '04 9
Framework Implementation Example
MS -> GSS MS -> GMS
Nursery
Mark-Sweep
High Semispace
Low Semispace
Large Object Space
GC Data Structures
Immortal
HIG
HER
LOW
ER
Nursery
Mark-Sweep
High Semispace
Low Semispace
Large Object Space
GC Data Structures
Immortal
HIG
HER
LOW
ER
No GCDo not unmap
Alloc
![Page 10: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/10.jpg)
ISMM '04 10
Framework Implementation
Nursery
Mark-Sweep
High Semispace
Low Semispace
Large Object Space
GC Data Structures
Immortal
HIG
HER
LOW
ER
Mem mapped on demand Unused memory unmapped
Need MS & copying states Use object header Bit stealing
Worst-case cost = 1 GC No need to perform GC in
many cases
![Page 11: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/11.jpg)
ISMM '04 11
Making Use of GC Switching
Dynamic switching guided by Cross-input offline behavior Annotation
Methodology 2.4 GHz/1G single proc x86 (hyperthreading) JikesRVM v2.2.0, linux kernel 2.4.18
SpecJVM98, SpecJBB, Jolden, Javagrande Pseudo-adaptive system
![Page 12: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/12.jpg)
Annotation-guided GC (AnnotGC)
Annotation Hints about optimization opportunities Widely used to guide compiler optimization
Different benchmarks & inputs examined offline Cross-input results combined Select “optimal” GC and annotate program with it Best GC for a range of heap sizes
JVM at program load time Switch to annotated GC Ignore annotation if GC-Switching not avail.
![Page 13: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/13.jpg)
ISMM '04 13
AnnotGC Implementation
Observations Only 0 or 1 switch points per benchmark Switch point relative to min heap does not
vary much across inputs
Annotate min heap size & switch point 4 byte annotation in Java class file Encoded in a compact form 40 MB min assumed if not specified
![Page 14: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/14.jpg)
ISMM '04 14
AnnotGC Results
1 2 3 4 5 6 7 8 9 10 11 12Heap Size Relative to Min
15
20
25
30
35
40
45
Exec
utio
n Ti
me
(sec
)
_209_db SSMSGMSGSSCMS
1 2 3 4 5 6 7 8 9 10 11 12Heap Size Relative to Min
110
160
210
260
310
360
410
10^6
/Thr
ough
put
SPECjbb2000 SSMSGMSGSSCMS
![Page 15: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/15.jpg)
ISMM '04 15
AnnotGC Results
Degradation over best: 4% Improvement over worst: 26% Improvement over GenMS: 7%
1 2 3 4 5 6 7 8 9 10 11 12Heap Size Relative to Min
15
20
25
30
35
40
45
Exec
utio
n Ti
me
(sec
)
_209_db SSMSGMSGSSCMS
GC Annot
1 2 3 4 5 6 7 8 9 10 11 12Heap Size Relative to Min
110
160
210
260
310
360
410
10^6
/Thr
ough
put
SPECjbb2000 SSMSGMSGCCMS
GC Annot
![Page 16: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/16.jpg)
AnnotGC Results
24.13%3.36%Average (without MS)
26.22%4.38%Average
31.78% (1063 ms)9.20% (144 ms)Voronoi
48.42% (1001 ms)4.07% (30 ms)MST
27.95% (82.6*106/tput)2.22% (3.17*106/tput)SPECjbb
17.71% (15500 ms)3.97% (2511 ms)JavaGrande
32.70% (2787 ms)3.22% (147 ms)jack
42.29% (5170 ms)4.51% (270 ms)mtrt
3.21% (209ms)3.54% (214 ms)mpegaudio
24.12% (2944 ms)5.64% (392 ms)javac
12.47% (3028 ms)2.88% (532 ms)db
56.17% (5767 ms)2.82% (85 ms)jess
3.53% (279 ms)6.28% (443 ms)compress
Improvement overWorst
Degradation over BestBenchmarks
Average Difference Between Best & Worst GC Systems
![Page 17: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/17.jpg)
ISMM '04 17
Extending GC Switching Switch automatically
No offline profiling Employ execution characteristics
Performance hit: lost optimization opportunity Inlining of allocation sites Write barriers
Solution: Aggressive specialization guarded by Method invalidation On-stack replacement
![Page 18: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/18.jpg)
ISMM '04 18
Investigating Auto Switching Exploit general performance characteristics
Best performing GCs across heap sizes & programs
Deciding when to switch Default GC: MS Heap size & heap residency threshold If residency > 60%,
switch to GSS if heap size > 90MB else use GMS
Different collectors for startup & steady-state
![Page 19: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/19.jpg)
ISMM '04 19
Preliminary Implementation Method invalidation - for future invocations On-stack replacement - for currently executing
methods Deferred compilation & guarded inlining [Fink’04] Unconditional OsrPoints
Our extension OsrInfoPoints for state info
Without inserting checks in application code
![Page 20: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/20.jpg)
ISMM '04 20
AutoSwitch Results Improvement over worst: 17% (Annot: 26%) Degradation over best: 15% (Annot: 4%) Overhead
OSR - negligible Recompilation - depends on where switch occurs
Lost optimization opportunities all variables live at every GC point OsrInfoPoints are “pinned” down DCE, load/store elim, code motion, reg allocation
![Page 21: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/21.jpg)
ISMM '04 21
Related Work Application-specific GC studies
Profile-direction GC selection [Fitzgerald’00] Comparison of GCs [Attanasio’01] Comparing gen mark-sweep & copying [Zorn’90]
Switching/swapping Coupling compaction with copying [Sansom’92] Hot-swapping mark-sweep/mark-compact
[Printezis’01] BEA Weblogic JRockit [BEA Workshop’03]
![Page 22: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/22.jpg)
ISMM '04 22
Conclusion Choice of best GC is app-dependent Novel framework
Switch between diverse collectors Enables annotation driven & automatic
switching
Significantly reduce impact of choosingthe "wrong" collector AnnotGC: 26%, degradation: 4%, GMS: 7% Enabled by aggressive specialization
![Page 23: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/23.jpg)
ISMM '04 23
Future Work Improving AutoSwitch
Improved OSR Cuts AutoSwitch overhead by half
Paper available upon request
Better heuristics for automatic switching Decide when to switch & which GC to switch to
High-performance application-specific VMs Guided by available resources Application characteristics Self-modifying
![Page 24: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/24.jpg)
App-specific Garbage CollectorsFront Loaders Rear Loaders
Recyclers Side Loaders
All products are trademarks of Heil Environmental Industries, Ltd. 2002-2003
![Page 25: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/25.jpg)
ISMM '04 25
Extra slides
![Page 26: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/26.jpg)
ISMM '04 26
Preliminary Implementation Aggressive specialization
Inline allocation sites Insert write barriers only when necessary
Method Invalidation Invalidated method replaced by stub
On-stack Replacement (OSR) Potentially at every GC point
![Page 27: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/27.jpg)
ISMM '04 27
On-stack replacement Insert OsrInfoPoints at each GC point
Keep track of state Call sites, allocation sites, prologs, backedges,
explicit yieldpoints, exceptions All local/stack variables kept alive
bar
foo
osr_helper {
...
}
old return address
trigger OSR
b. OSR triggered lazily by external event
Lazy OSR
![Page 28: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/28.jpg)
ISMM '04 28
GC Performance Evaluation (2) Comparison of parallel garbage collectors [5]
Mark-sweep, copying, Gen MS, Gen copy, hybrid(copying + MS)
Results Hybrid has lowest heap residency Gen copy handles fragmentation better Minor collections for Gen copy faster Mark-sweep has fewer GCs
No one collector best across applications
![Page 29: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/29.jpg)
ISMM '04 29
AnnotGC Results
Degradation over best: 4% Improvement over worst: 26% Improvement over GenMS: 7%
1 2 3 4 5 6 7 8 9 10 11 12Heap Size Relative to Min
110
160
210
260
310
360
410
10^6
/Thr
ough
put
SPECjbb2000 SSMSGMS
GSSCMS
GC Annot
1 2 3 4 5 6 7 8 9 10 11 12Heap Size Relative to Min
15
20
25
30
35
40
45
Exec
utio
n Ti
me
(sec
)
_209_db SSMSGMS
GSSCMS
GC Annot
![Page 30: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/30.jpg)
ISMM '04 30
GC Performance Evaluation Comparing generational mark-sweep & copying [66] Mark-sweep
Slower free-list allocator Requires less heap space: 20% less on average
Copying collection Fastest allocator Copying overhead Copy reserve space required
Generational copying not clearly superior
![Page 31: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/31.jpg)
ISMM '04 31
GC Performance Evaluation (2) Comparison of parallel garbage collectors [5]
Mark-sweep, copying, Gen MS, Gen copy, hybrid(copying + MS)
Results Hybrid has lowest heap residency Gen copy handles fragmentation better Minor collections for Gen copy faster Mark-sweep has fewer GCs
No one collector best across applications
![Page 32: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/32.jpg)
ISMM '04 32
GC Performance Evaluation (3) Fitzgerald et al [32] compared GCs across 20
benchmarks Null collector, non-gen copying, Gen copy with various
WB implementations
Every collector best at least once Spread between best & worst at least 15% Generational not always better
Non-generational may outperform by 15-20%
Recommend “profile-directed” selection The “best” GC chosen from different pre-compiled binaries
based on profiles
It is difficult or impossible to replace GCs at runtime
![Page 33: Dynamic Selection of Application-Specific Garbage Collectors...Dynamic Selection of Application-Specific Garbage Collectors Sunil V. Soman Chandra Krintz University of California,](https://reader035.fdocuments.net/reader035/viewer/2022070218/6127fedc83298f2a7f713a36/html5/thumbnails/33.jpg)
ISMM '04 33