Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user...

22
Go GC Settings Bryan Boreham (@bboreham)

Transcript of Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user...

Page 1: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

Go GC Settings

Bryan Boreham (@bboreham)

Page 2: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

At Weaveworks, I work on Open Source DevOps tools doing system visualisation, monitoring and CI/CD

I also contribute to Container Network Interface, Kubernetes, Prometheus

Program optimisation is my video-game.

Hi, I’m Bryan Boreham

2

@bboreham

Page 3: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

Why care about Garbage?

3

https://www.redbubble.com/people/clgtart/

(this is the point where Bryan moans about never having enough RAM)

Page 4: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

CPU memory architecture

4

CPU

RAM RAM

Level 2Cache

Level 1Cache

(Not to scale)

Logic

Page 5: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

Caches in use

5

CPU

RAM RAM

Level 2Cache

Level 1Cache

Logic

Page 6: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

After GC has run

6

CPU

RAM RAM

Level 2Cache

Level 1Cache

Logic

Page 7: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

Stack vs Heap

7http://www.clipartpanda.com/clipart_images/stack-files-max-39545601 Photo: JohnNyberg, rgbstock.com

Page 8: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

GC Options

8

-Xmsn-Xmxn-XX:-UseConcMarkSweepGC-XX:-UseParallelGC-XX:-UseParallelOldGC-XX:-UseSerialGC-XX:+UseG1GC-XX:+UseGCOverheadLimit-XX:MaxGCPauseMillis=n-XX:InitiatingHeapOccupancyPercent=n-XX:MaxTenuringThreshold=n-XX:+ScavengeBeforeFullGC

-XX:ParallelGCThreads=n-XX:ConcGCThreads=n-XX:G1ReservePercent=n-XX:G1HeapRegionSize=n-XX:MaxHeapFreeRatio=n-XX:MaxNewSize=size-XX:MaxPermSize=n-XX:MinHeapFreeRatio=n-XX:NewSize=n-XX:NewRatio=n-XX:SurvivorRatio=n-XX:TargetSurvivorRatio=n-XX:+HandlePromotionFailure

Page 9: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

GOGC

Go GC Options

9

Page 10: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

Server: github.com/stefanprodan/k8s-podinfo

- microservice skeleton

Client: github.com/rakyll/hey

- HTTP load generatorhey -z 2m -c 10 -q 20 ".../metrics"

Test set-up

10

Page 11: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

Situation: large stable data set

11

Page 12: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

(at the cost of CPU)

Lower GOGC to save RAM

12

Page 13: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

Situation: high GC rate, small heap

13

Page 14: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

Raise GOGC to save CPU

14

Page 15: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

GOGC=1000

15

Page 16: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

What’s happening here? (no load, GOGC=100)

16

Page 17: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

/usr/local/go/src/runtime/proc.go line 4310(!):

// If we go this long without a garbage collection,

// one is forced to run.

var forcegcperiod int64 = 2 * 60 * 1e9

There is another setting!

17

Page 18: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

GOGC=off

What if we turn off GC completely?

18

Page 19: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

$ go clean -cache

$ time go build

real 0m7.331s

user 0m12.152s

sys 0m0.660s

Building podinfo

19

$ go clean -cache

$ export GOGC=off

$ time go build

real 0m5.412s

user 0m8.460s

sys 0m0.704s

Page 20: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

SetMaxHeap

GOGCMIN

The future?

20

https://twitter.com/ashleymcnamara

Page 21: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

THANK YOU!

21https://twitter.com/ashleymcnamara

@bboreham

Page 22: Go GC Settings - FOSDEM · 2019. 4. 15. · $ go clean -cache $ time go build real 0m7.331s user 0m12.152s sys 0m0.660s Building podinfo 19 $ go clean -cache $ export GOGC=off $ time

More information

22

Good explanation of GOGC and other settings

https://dave.cheney.net/tag/gogc

Really detailed history https://blog.golang.org/ismmkeynote