ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test...
Transcript of ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test...
![Page 1: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/1.jpg)
© 2017 ISC
ISC's Performance Lab
DNS-OARC Workshop 26 - Madrid
1
![Page 2: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/2.jpg)
© 2017 ISC
Goals of the Project
▪ Provide a framework for automated performance testing of BIND– Test experimental code– Look at long-term trends to avoid
regressions
2
![Page 3: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/3.jpg)
© 2017 ISC
System Features #1
▪Multiple configurations– git branch or tag– ./configure settings– named.conf options– named command line options– etc
3
![Page 4: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/4.jpg)
© 2017 ISC
System Features #2
▪Multiple authoritative zone configurations– root zone– 1M delegations (small TLD) – 1M small zones (web hoster)– 1 zone with 1M A+AAAA
▪ dnsperf query sets for the above▪ (Some) Recursive support4
![Page 5: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/5.jpg)
© 2017 ISC
System Features #3
▪ HTML5 Web UI– real time updates over WebSockets
▪ Round-robin scheduler–with ad-hoc priority bump
5
![Page 6: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/6.jpg)
© 2017 ISC
System Features #4
▪ Graphs▪ CSV Output▪ A/B comparison with Student's t-test▪ Extensible for other servers▪ NSD 4, Knot 2 already supported▪ Dummy 'echo' server support, too
6
![Page 7: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/7.jpg)
© 2017 ISC
Technology
▪MongoDB ▪ NodeJS (ES6)–Promises– Express
▪ HTML5–Bootstrap–Angular JS (ES5)–WebSockets
7
![Page 8: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/8.jpg)
© 2017 ISC
QUICK DEMO…
![Page 9: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/9.jpg)
© 2017 ISC
Logical Architecture
DNS Server
Web Client
dnsperfREST API
Server
Test Agent
MongoDB
9
![Page 10: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/10.jpg)
© 2017 ISC
Physical Architecture
Server under Test
Traffic Generator Backend
10 GE
Gig-E Management LAN
Apache Reverse Proxy
10
![Page 11: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/11.jpg)
© 2017 ISC
Test Methodology
▪ Build and start server▪ Run dnsperf 30 times▪ 30 seconds each time▪ first run ignored for statistic (allows
for cache and buffer warmup)
11
![Page 12: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/12.jpg)
© 2017 ISC
Challenge - Test Variability
▪ The results from 30x dnsperf tests against a single run of BIND are generally consistent
▪ Testing a new run of BIND often shifts the mean significantly, by several standard deviations
12
![Page 13: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/13.jpg)
© 2017 ISC
Challenge - Test Variability
13
Master Branch Test Variance Detail
610
615
620
625
630
635
640
kQPS
![Page 14: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/14.jpg)
© 2017 ISC
(Some) Mitigation
▪ Tune for stability, not peak performance:–Disable Hyperthreading– Lock CPU clock rate•Disable Intel SpeedStep•Disable Turbo mode
– Lock Client and Server CPU core affinity– Lock NIC RX/TX queue CPU core affinity– Set NIC queue flow hashing to use
deterministic ports14
![Page 15: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/15.jpg)
© 2017 ISC
Challenge - Test Variability #2
15
Dual Core Variance Detail
660
680
700
720
740
760
780
800
kQPS
●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●●●
●
●●●
●●
●
●●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●●
●●
●
●
●●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●●
●
●
●
●●●
●
●●●●●
echo server two threads locked to CPU#0 and #1
![Page 16: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/16.jpg)
© 2017 ISC
Challenge - Test Variability #2
16
Dual Core Variance Detail
660
680
700
720
740
760
780
800
kQPS
●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●●●
●
●●●
●●
●
●●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●●
●●
●
●
●●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●●
●
●
●
●●●
●
●●●●●
echo server two threads locked to CPU#0 and #1
Huh?!
![Page 17: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/17.jpg)
© 2017 ISC
Coming Soon
▪ In 9.11.2 and back-ported to 9.10– optimised name compression– optimised owner case preservation
▪ In 9.12– "minimal-responses" on by default– "glue cache" replaces "acache"• improves delegation performance
17
![Page 18: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/18.jpg)
© 2017 ISC
2012 2013 2014 2015 2016 2017 2018
0
200
400
600
800
BIND Performance − 1M Delegations
12−core Intel Xeon E5−2680 v3 @ 2.50GHz
Release Date
kQPS
9.9 9.10 9.11 dev
●●
● ● ● ●
● ● ●
● ●
●
● ●●
●
●
● ●●
●
18
![Page 19: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/19.jpg)
© 2017 ISC
2012 2013 2014 2015 2016 2017 2018
0
200
400
600
800
BIND Performance − 1M Delegations
12−core Intel Xeon E5−2680 v3 @ 2.50GHz
Release Date
kQPS
9.9 9.10 9.11 dev
●●
● ● ● ●
● ● ●
● ●
●
● ●●
●
●
● ●●
●
19
Change to default "-U" setting
![Page 20: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/20.jpg)
© 2017 ISC20
2012 2013 2014 2015 2016 2017 2018
0
100
200
300
400
BIND Performance − Root Zone
12−core Intel Xeon E5−2680 v3 @ 2.50GHz
Release Date
kQPS
● ● ●●
● ●
● ● ●
● ●
● ● ● ●
● ●● ●
9.9 9.10 9.11
![Page 21: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/21.jpg)
© 2017 ISC
2012 2013 2014 2015 2016 2017 2018
0
100
200
300
400
BIND Performance − Root Zone
12−core Intel Xeon E5−2680 v3 @ 2.50GHz
Release Date
kQPS
9.9 9.10 9.11 dev
● ● ●●
● ●
● ● ●
● ●
● ● ● ●
● ●● ●●
●
21
6x faster!
![Page 22: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/22.jpg)
© 2017 ISC
How Do I Get It?
https://github.com/isc-projects/perflab▪ This is not supported software▪ It's not turn-key - installation is likely
to require lots of fiddling▪ Pull requests welcomed…
22
![Page 23: ISC's Performance Lab · Provide a framework for automated performance testing of BIND –Test experimental code –Look at long-term trends to avoid ... MongoDB NodeJS (ES6) –Promises](https://reader033.fdocuments.net/reader033/viewer/2022042310/5ed785667bbb9f68866aa9c8/html5/thumbnails/23.jpg)
© 2017 ISC
QUESTIONS?