Dustin Whittle - Performance Testing Crash Course - code.talks 2015
109
Performance Tes-ng Crash Course Dus$n Whi*le
-
Upload
aboutyougmbh -
Category
Internet
-
view
585 -
download
0
Transcript of Dustin Whittle - Performance Testing Crash Course - code.talks 2015
PerformanceTes-ngCrashCourseDus$nWhi*le
Dus-nWhi5le
• dus-nwhi5le.com
• @dus-nwhi5le
• SanFrancisco,California,USA
• Technologist,Traveler,Pilot,Skier,Diver,Sailor,Golfer
Whydoesperformancema5er?
Performancedirectlyimpactsthebo5omline
Howfastisfastenough?
§ Performanceiskeytoagreatuserexperience
- Under100msisperceivedasreac$nginstantaneously
- A100msto300msdelayispercep$ble
- 1secondisaboutthelimitfortheuser'sflowofthoughttostayuninterrupted
- Usersexpectasitetoloadin2seconds
- AMer3seconds,40%willabandonyoursite.
- 10secondsisaboutthelimitforkeepingtheuser'sa*en$on
§Modernapplica-onsspendmore-meinthebrowserthanontheserver-side
Login
Flight Status
Search Flight
Purchase
Mobile
Big data
SOA
NOSQL
Cloud
Agile
Web
Application complexity is exploding
Up-meiscri-calforenterprisesandconsumers
Treatperformanceasafeature!
Toolsofthetradeforperformancetes-ng
Understandyourbaselineperformance
Sta-c
vs
HelloWorld
vs
Applica-ons
Apache Bench
ab -c 1 -t 10 -k http://acmedemoapp.com/
Benchmarking acmedemoapp.com (be patient) Finished 187 requests
Server Software: Apache-Coyote/1.1 Server Hostname: acmedemoapp.com Server Port: 80
Document Path: / Document Length: 5217 bytes
Concurrency Level: 1 Time taken for tests: 10.039 seconds Complete requests: 187 Failed requests: 0 Keep-Alive requests: 187 Total transferred: 1021768 bytes HTML transferred: 975579 bytes Requests per second: 18.63 [#/sec] (mean) Time per request: 53.687 [ms] (mean)
Benchmarking acmedemoapp.com (be patient) Finished 659 requests
Server Software: Apache-Coyote/1.1 Server Hostname: acmedemoapp.com Server Port: 80
Document Path: / Document Length: 5217 bytes
Concurrency Level: 10 Time taken for tests: 10.015 seconds Complete requests: 659 Failed requests: 0 Keep-Alive requests: 659 Total transferred: 3600776 bytes HTML transferred: 3438003 bytes Requests per second: 65.80 [#/sec] (mean) Time per request: 151.970 [ms] (mean) Time per request: 15.197 [ms] (mean, across all
Siege
** SIEGE 3.0.6** Preparing 10 concurrent users for battle.The server is now under siege...Lifting the server siege... done.
Transactions: 623 hitsAvailability: 100.00 %Elapsed time: 9.57 secsData transferred: 3.10 MBResponse time: 0.15 secsTransaction rate: 65.10 trans/secThroughput: 0.32 MB/secConcurrency: 9.91Successful transactions: 623Failed transactions: 0Longest transaction: 0.30Shortest transaction: 0.10
Crawl the entire app to discover all urls
sproxy -o ./urls.txt
SPROXY v1.02 listening on port 9001...appending HTTP requests to: ./urls.txt
...default connection timeout: 120 seconds
wget -r -l 0 -t 1 --spider -w 1 -e "http_proxy = http://127.0.0.1:9001"
"http://acmedemoapp.com/"
sort -u -o urls.txt urls.txt
http://acmedemoapp.com/http://acmedemoapp.com/abouthttp://acmedemoapp.com/cart
http://acmedemoapp.com/currency/change/EURhttp://acmedemoapp.com/currency/change/GBPhttp://acmedemoapp.com/currency/change/USD
http://acmedemoapp.com/loginhttp://acmedemoapp.com/register/
http://acmedemoapp.com/t/brand/bookmaniahttp://acmedemoapp.com/t/category/bookshttp://acmedemoapp.com/t/category/mugs
http://acmedemoapp.com/t/category/stickershttp://acmedemoapp.com/terms-of-service
Benchmark traffic across all unique urls with siege
siege -v -c 50 -i -t 3M -f urls.txt -d 10
** SIEGE 3.0.6** Preparing 10 concurrent users for battle.The server is now under siege...Lifting the server siege... done.
Transactions: 623 hitsAvailability: 100.00 %Elapsed time: 9.57 secsData transferred: 3.10 MBResponse time: 0.15 secsTransaction rate: 65.10 trans/secThroughput: 0.32 MB/secConcurrency: 9.91Successful transactions: 623Failed transactions: 0Longest transaction: 0.30Shortest transaction: 0.10
Multi-Mechanize is an open source framework for performance and load
testing
pip install multi-mechanize
multimech-newproject demo
import requests
class Transaction(object): def run(self): r = requests.get(‘http://acmedemoapp.com/) r.raw.read()
import mechanize import time
class Transaction(object): def run(self): br = mechanize.Browser() br.set_handle_robots(False)
start_timer = time.time() resp = br.open(‘http://acmedemoapp.com/) resp.read() latency = time.time() - start_timer self.custom_timers['homepage'] = latency
start_timer = time.time() resp = br.open(‘http://acmedemoapp.com/cart') resp.read() latency = time.time() - start_timer self.custom_timers['cart'] = latency assert (resp.code == 200)
multimech-run demo
What about when you need more than one machine?
Who lives in the cloud?
Bees with Machine Guns
A utility for arming (creating) many bees (micro EC2 instances)
to attack (load test) targets (web applications)
pip install beeswithmachineguns
# ~/.boto
[Credentials]
aws_access_key_id=xxxaws_secret_access_key=xxx
[Boto]
ec2_region_name = us-west-2ec2_region_endpoint = ec2.us-west-2.amazonaws.com
bees up -s 2 -g default -z us-west-2b -i ami-bc05898c -k appdynamics-
dustinwhittle-aws-us-west-2 -l ec2-user
Connecting to the hive.Attempting to call up 2 bees.Waiting for bees to load their machine guns.......Bee i-3828400c is ready for the attack.Bee i-3928400d is ready for the attack.The swarm has assembled 2 bees.
bees report
Read 2 bees from the roster. Bee i-3828400c: running @ 54.212.22.176 Bee i-3928400d: running @ 50.112.6.191
Read 2 bees from the roster.Connecting to the hive.Assembling bees.Each of 2 bees will fire 500 rounds, 25 at a time.Stinging URL so it will be cached for the attack.Organizing the swarm.…Offensive complete. Complete requests: 1000 Requests per second: 306.540000 [#/sec] (mean) Time per request: 163.112000 [ms] (mean) 50% response time: 151.000000 [ms] (mean) 90% response time: 192.000000 [ms] (mean)Mission Assessment: Target crushed bee offensive.The swarm is awaiting new orders.
Read 2 bees from the roster.Connecting to the hive.Assembling bees.Each of 2 bees will fire 50000 rounds, 500 at a time.Stinging URL so it will be cached for the attack.Organizing the swarm.…Offensive complete. Complete requests: 100000 Requests per second: 502.420000 [#/sec] (mean) Time per request: 360.114000 [ms] (mean) 50% response time: 451.000000 [ms] (mean) 90% response time: 402.000000 [ms] (mean)Mission Assessment: Target crushed bee offensive.The swarm is awaiting new orders.
Read 2 bees from the roster.Connecting to the hive.Assembling bees.Each of 2 bees will fire 50000 rounds, 500 at a time.Stinging URL so it will be cached for the attack.Organizing the swarm.Bee 0 is joining the swarm.Bee 1 is joining the swarm.Bee 0 is firing his machine gun. Bang bang!Bee 0 lost sight of the target (connection timed out).Bee 1 lost sight of the target (connection timed out).Offensive complete. Target timed out without fully responding to 2 bees. No bees completed the mission. Apparently your bees are peace-loving hippies.The swarm is awaiting new orders.
bees down
Read 2 bees from the roster.Connecting to the hive.Calling off the swarm.Stood down 2 bees.
locust.io
https://github.com/locustio/locust
pip install locustio
There are many tools…
•Gatling.io•Wrk•Vegeta•Tsung•…
What about the client side?
In modern web applications more latency comes from the client-side
than the server-side.
npm install psi
WBench
gem install wbench
Automateclient-sideperformancetes-ngwith
Grunt/Gulp
UseBower(fordependencies),Grunt/Gulp(forautoma-on),andYeoman(forbootstrapping)
Howmanypeopleunderstandexactlyhowfasttheirsiterunsinproduc-on?
Trackperformanceindevelopmentandproduc-on
Instrumenteverything=code,databases,caches,queues,thirdpartyservices,and
infrastructure.
Statsd + Graphite+ Grafana
Track performance of end users
webpagetest.org
SiteSpeed.io
Loadtes-ngservicesfromthecloud
Testforfailures
• NetFlixSimianArmy+ChaosMonkey
• Whathappensifyouloseacachinglayer?
• Whathappensifdependenciesslowdown?
PerformanceMa*ers• Treatperformanceasafeature
• Usingthe14kbRuleforinstantloading
• Markupmanagement
• Elimina$ngexcessAJAXcalls
• Workingwithandaroundapplica$oncache
• Developingaresponsivedesign+imagestrategy
• Implemen$ngagoodtouch-firststrategy
• Codemanagementforgoodproduc$onanddevelopmentexperiences
• Usingtaskrunnerstobuildanddeployproduc$oncode
BestPrac$ces• Treatperformanceasafeature
• Capacityplanandloadtesttheserver-side
• Op-mizeandperformancetesttheclient-side
• Understandyourstar-ngpoint
• Instrumenteverything
• Monitorperformanceindevelopmentandproduc-on
• Measurethedifferenceofeverychange
• Automateperformancetes-nginyourbuildanddeploymentprocess
• Understandhowfailuresimpactperformance
Theprotocolsareevolving• Thelimita$onsofHTTP/1.Xforcedustodevelopvariousapplica$on
workarounds(sharding,concatena$on,spri$ng,inlining,etc.)toop$mizeperformance.However,intheprocesswe’vealsointroducednumerousregressions:poorcaching,unnecessarydownloads,delayedexecu$on,andmore.
• HTTP/2eliminatestheneedforthesehacksandallowsustobothsimplifyourapplica$onsanddeliverimprovedperformance.
• Youshouldunshard,unconcat,andunspriteyourassets
• Youshouldswitchfrominliningtoserverpush
• ReadIlyaGrigorikawesomebookonbrowserperformance-h*p://hpbn.co/h*p2
Integrateautomatedperformancetes-ngintocon-nuousintegra-onforserver-side
andclient-side
Understandtheperformanceimplica-onsofeverydeploymentandpackageupgrade
Monitorenduserexperiencefromendtoendinproduc-on
Ques-ons?
FindtheseslidesonSpeakerDeck
h5ps://speakerdeck.com/dus-nwhi5le
