Boring dot com - the virtues of boring technology
30
The virtues of boring technology Kristian Köhntopp, Scalability Team Veteran Boring
-
Upload
kristian-koehntopp -
Category
Internet
-
view
2.917 -
download
0
Transcript of Boring dot com - the virtues of boring technology
The virtues of boring technology
Kristian Köhntopp, Scalability Team Veteran
Boring
What people think Booking looks like…Photo: ©2012 Kristian Köhntopp
What Booking actually looks like…Photo: ©2015 Kristian Köhntopp
Interesting times…
Interesting times…
Interesting times…Photo: ©2012 Kristian Köhntopp
Boring Setup…nginx
uwsgi MySQL
Replication
Write
Read
Apache
mod_perl
Not using sharding…Web Server GroupApache (mod_perl)
bp Database av Database
Web Server GroupApache (mod_perl)
bp Database
Doing it over an over again…Web Server GroupApache (mod_perl)
Many Databases
Fragile!
Many web servers…
LB
Redirect
Many bp servers
Load balanceddirect connect
Doing something about it…
Y U SO BORING?CC BY-SA 3.0 Kartin st, https://en.wikipedia.org/wiki/Pallas%27s_cat#/media/File:Manul2.jpg
Growth kills…
Problem
appears Solution
must be
readySolution
in use Solution
falling apart
3 years, Factor 10
© 2006-2015 booking.com
Keep it simple, that’s complicated enough!"AK-47 type II Part DM-ST-89-01131". Public Domain - https://commons.wikimedia.org/wiki/File:AK-47_type_II_Part_DM-ST-89-01131.jpg
What’s wrong with this model?
Wait…Wait…
Result
Set
httpsSQL
HTML
All the things we didn’t do…● Not doing many network hops.
● Not doing distributed.
● Not doing asynchronous.
● Not doing microservices.
● Not doing schemaless.
● Not doing sharding.
● Not doing virtualization.
● Not doing hyperconvergence.
● Not doing multiple datacenters.
● Not doing multirepo.
● Not doing test driven development.
● Not doing change management.All Lies!
Not doing many network hops.● Reliable, predictable performance.
● Low response time.
● Low response time variance.
● Works in a regular data-center network.
Not doing distributed…● Strictly speaking a bug.
●We do have per-schema masters,
● we do multi-schema updates.
● This happes in places where it does not matter:
● Low concurrency,
● non-critical information.
● Also: “fsck”
avMaster
userMaster
Not doing asynchronous…● Perl can’t (with MySQL) and mostly doesn’t (elsewhere).
● MySQL couldn’t.
● Non-critical Javascript on page for animations and spellchecking.
Not doing microservices…● Codebase has structure and contracts.
● This does not express itself in the form of service boundaries.
● Saves a lot serialization/copying/instantiation.
● Less overhead.
● Less Jitter.
● Less locking issues.
Not doing schemaless…● There is no such thing as schemaless.
● The sooner you admit that, the less painful it is going to be.
Not doing sharding…● Except where we are doing it.
● Unfortunately, in an increasing number of places.
● Even where we aren’t doing it, we are seeing application-side hash-joins.
● At least they are visible.
Not doing virtualization…● Again, predictable, reliable performance.
● Also, lots of problems with high density networking and computing.
● Semi-manual is painful.
● Fully automated is unreliable, and doesn’t scale.
=2005, 2 Racks 2015, 2U
50 Cores, 50 GBit/s,
512 GB SSD = moar IOPS!
Not doing hyperconvergence…● Again, predictable, reliable performance.
● Does painful things to your network.
● Has high commit-latency.
●Good for archive storage.
● Bleeding edge technology. 18U Rack (resize vertically) 18U Rack (resize vertically) 18U Rack (resize vertically)
Not doing multiple datacenters…● Well, there is CDN…
●Many, many, many images.
● The load for the active content actually isn’t that much.
Not doing multirepo…● »The booking.com git-insanity.«
● Actually, the technical term is
● »Monorepo with trunk-based development and separation of rollout and
activation.«
● Man, am I thankful for that, if only after the fact.
Not doing TDD…● “Making it safe to test in production.”
● “Separation of rollout and activation.”
● “A/B-testing” aka “Requirements validation.”
● “Conversion tracking” aka “fast metrics.”
● Fast, lag-free monitoring.
Not doing change-management…● “The Booking-Rules.”
● “Roll-Call.”
● “Separation of rollout and activation.”
● Duplicate data keeping.
● Fast, lag-free monitoring.
All the things we didn’t do…● Not doing many network hops.
● Not doing distributed.
● Not doing asynchronous.
● Not doing microservices.
● Not doing schemaless.
● Not doing sharding.
● Not doing virtualization.
● Not doing hyperconvergence.
● Not doing multiple datacenters.
● Not doing multirepo.
● Not doing test driven development.
● Not doing change management.
http://booking.com/jobsPhoto: ©2012 Kristian Köhntopp
Want a boring job?
