Stack Exchange Infrastructure - LISA 14
-
Upload
gabeech -
Category
Technology
-
view
296 -
download
4
description
Transcript of Stack Exchange Infrastructure - LISA 14
![Page 1: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/1.jpg)
The Stack Exchange
InfrastructureVroom Vroom
![Page 2: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/2.jpg)
inet.perf.profile
• SRE Generalist @ Stack Exchange
• @GABeech
• http://brokenhaze.com
• http://stackexchange.com
![Page 3: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/3.jpg)
A brief Overview
• 560 Million Page Views a Month
• 34TB of Data transfered a Month
• 1665 rps (2250 peak) Across web Farm
• WISC(HER)
![Page 4: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/4.jpg)
Our First Priority is
PerformanceNobody likes a slow site, least of all us.
When your site is slow people leave.
Make your site fast, and the people will stay
Good write up on moz.com:
http://moz.com/blog/site-speed-are-you-fast-
does-it-matter
![Page 5: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/5.jpg)
The Performance
toolkit
• Mini Profiler
• OpServer
(https://github.com/opserver/Opserver)
• Client Timings
(http://teststackoverflow.com/)
![Page 6: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/6.jpg)
Mini Profiler
![Page 7: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/7.jpg)
OpServer
![Page 8: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/8.jpg)
OpServer HAproxy
![Page 9: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/9.jpg)
OpServer Redis
![Page 10: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/10.jpg)
OpServer SQL
![Page 11: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/11.jpg)
Client Timings
![Page 12: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/12.jpg)
You can’t be fast if
you are not up
• Highly Redundant network
• Datacenter, ISP, Edge, Core, Server,
Port
![Page 13: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/13.jpg)
![Page 14: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/14.jpg)
Load Balencers
• HAProxy
• 2 Servers (Hot/Standby)
• Multiple Tiers (HAProxy Processes)
![Page 15: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/15.jpg)
![Page 16: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/16.jpg)
SSL Termination
• Terminated at LB
• Feature added to HAProxy 1.5
• See:
http://brokenhaze.com/blog/2014/03/25/
how-stack-exchange-gets-the-most-out-
of-haproxy/
![Page 17: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/17.jpg)
Web Servers
• IIS
• 9 Production (2 Test/Dev)
• Dell R610’s
• 32GB Memory
• 2xE5-5640
![Page 18: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/18.jpg)
Data Tier
• MS SQL Server
• 4 Servers
• 2 Always-On Clusters
• Each Cluster 1 RW, 1 RO
![Page 19: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/19.jpg)
Caching Tier
• Redis
• 2 Servers
• Hot / Standby configuration
![Page 20: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/20.jpg)
Tag Engine
• Our Special index of SO
• Tagging is hard
• Written by Marc Gravell
• http://blog.marcgravell.com/2014/04/technica
l-debt-case-study-tags.html
![Page 21: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/21.jpg)
Elastic Search
• 203GB Index
• 3 Machines
• 42M searches/day
![Page 22: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/22.jpg)
Deployment
• Git
• TeamCity
• Custom Powershell Scripts
![Page 23: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/23.jpg)
So what does this get
you
• 52 ms homepage render time
• 33 ms questions page render time
![Page 24: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/24.jpg)
Always See our
Performance
• http://stackexchange.com/performance
![Page 25: Stack Exchange Infrastructure - LISA 14](https://reader033.fdocuments.net/reader033/viewer/2022060200/559906841a28abf30f8b4707/html5/thumbnails/25.jpg)
Thank YOU!
Contact:
@GABeech
Office Hours:
Wednesday, November 12th
(today…)
2:00pm - 3:30pmLISA Lab