SecurityManagement Networking IntrospectionPerformance Utilization

“the lies we tell our code”
 @misterbisson

Powering modern applications Your favorite code

Container-native infrastructure

Your favorite platforms

SecurityManagement Networking IntrospectionPerformance Utilization

Public Cloud Triton Elastic Container Service. We run our customer’s mission critical applications on container native infrastructure

Private Cloud Triton Elastic Container Infrastructure is an on-premise, container run-time environment used by some of the world’s most recognizable brands

SecurityManagement Networking IntrospectionPerformance Utilization

Public Cloud Triton Elastic Container Service. We run our customer’s mission critical applications on container native infrastructure

Private DataCenter Triton Elastic Container Infrastructure is an on-premise, container run-time environment used by some of the world’s most recognizable brands

it’s open source! fork me, pull me: https://github.com/joyent/sdc

https://github.com/joyent/sdc

Node.js enterprise support

Best Practices

Performance
 Analysis

Core File
 Analysis

Debugging 
 Support

Critical 
 Incident
 Support

⚠

As the corporate steward of Node.js and one of the largest-scale production users, Joyent is uniquely equipped to deliver the highest level of enterprise support for this dynamic runtime.

The best place to run Docker

Portability 
 From laptop to any public or private cloud

Great for DevOps 
 Tools for management, deployment & scale

Productivity 
 Faster code, test 
 and deploy

The best place to run containers. 
 Making Ops simple and scalable.

SecurityManagement Networking IntrospectionPerformance Utilization

breath for a moment

lying to our code is a practical

choice

without moralconsequence

without allconsequence

…but not

most importantly

most importantly never

lie to yourself

The earliest common lie

Virtual memory

from http://www.webopedia.com/TERM/V/virtual_memory.html

http://www.webopedia.com/TERM/V/virtual_memory.html

Virtual memory
 according to Poul-Henning Kamp Take Squid for instance, a 1975 program if I ever saw one: You tell it how much RAM it can use and how much disk it can use. It will then spend inordinate amounts of time keeping track of what HTTP objects are in RAM and which are on disk and it will move them forth and back depending on traffic patterns. Squid’s elaborate memory management…gets into fights with the kernel’s elaborate memory management, and like any civil war, that never gets anything done. from http://web.archive.org/web/20080323141758/http://varnish.projects.linpro.no/wiki/ArchitectNotes

http://web.archive.org/web/20080323141758/http://varnish.projects.linpro.no/wiki/ArchitectNotes

Virtual memory
 according to Poul-Henning Kamp Varnish knows it is not running on the bare metal but under an operating system that provides a virtual-memory-based abstract machine. For example, Varnish does not ignore the fact that memory is virtual; it actively exploits it. A 300-GB backing store, memory mapped on a machine with no more than 16 GB of RAM, is quite typical. The user paid for 64 bits of address space, and I am not afraid to use it. from http://queue.acm.org/detail.cfm?id=1814327

http://queue.acm.org/detail.cfm?id=1814327

vm.swappiness = 0

The harmless lie

Hyperthreading

from http://www.intel.com/cd/channel/reseller/asmo-na/eng/products/36016.htm

http://www.intel.com/cd/channel/reseller/asmo-na/eng/products/36016.htm

Hyperthreading One physical core appears as two processors to the operating system, which can use each core to schedule two processes at once. It takes advantage of superscalar architecture in which multiple instructions operate on separate data in parallel. Hyper-threading can be properly utilized only with an OS specifically optimized for it.

from http://en.wikipedia.org/wiki/Hyper-threading

http://en.wikipedia.org/wiki/Hyper-threading

Faster, but not double the performance

Hyperthreading

from https://capacitas.wordpress.com/2013/03/07/hyper-threading-on-vs-off-case-study/

https://capacitas.wordpress.com/2013/03/07/hyper-threading-on-vs-off-case-study/

The lie that built the cloud

Hardware virtual machines

from http://virtualizationtutor.com/what-is-hosted-virtualization-and-dedicated-virtualization/

http://virtualizationtutor.com/what-is-hosted-virtualization-and-dedicated-virtualization/

HVM: call translation Say a virtual machine guest OS makes the call to flush the TLB (translation look-aside buffer) which is a physical component of a physical CPU. If the guest OS was allowed to clear the entire TLB on a physical processor, that would have negative performance effects for all the other VMs that were also sharing that same physical TLB. [Instead, the hypervisor must translate that call] so that only the section of the TLB that is relevant to that virtual machine is flushed.

from http://serverfault.com/a/455554

http://serverfault.com/a/455554

The lie that made VMware huge

HVM: type 1 vs. type 2

from https://microkerneldude.wordpress.com/2009/03/23/virtualization-some-get-it-some-dont/

https://microkerneldude.wordpress.com/2009/03/23/virtualization-some-get-it-some-dont/

Lies upon lies

Paravirtualization

from http://www.cubrid.org/blog/dev-platform/x86-server-virtualization-technology/

http://www.cubrid.org/blog/dev-platform/x86-server-virtualization-technology/

HVM vs. clocksource… EC2 User: the kernel time will jump from 0 to 
 thousands of seconds.

Kernel dev: for some reason it looks like the vcpu time info misses…without implementation details of the host code it is hard to say anything more.

AWS: Ubuntu…uses the underlying hardware as a timesource, rather than sources native to the instance, leading to timestamps that are out of sync with the local instance time.

from https://forums.aws.amazon.com/thread.jspa?messageID=560443

https://forums.aws.amazon.com/thread.jspa?messageID=560443

HVM vs. CPU oversubscription An operating system requires synchronous progress on all its CPUs, and it might malfunction when it detects this requirement is not being met. For example, a watchdog timer might expect a response from its sibling vCPU within the specified time and would crash otherwise. When running these operating systems as a guest, ESXi must therefore maintain synchronous progress on the virtual CPUs. from http://www.vmware.com/files/pdf/techpaper/VMware-vSphere-CPU-Sched-Perf.pdf

http://www.vmware.com/files/pdf/techpaper/VMware-vSphere-CPU-Sched-Perf.pdf

HVMs vs. network I/O Reality: interrupts are challenging in HVM with oversubscribed CPU. Consider these AWS network tuning recommendations: • Turn off tcp_slow_start_after_idle • Increased netdev_max_backlog from 1000 to 5000 • Maximize window size (rwnd, swnd, and cwnd) from http://www.slideshare.net/AmazonWebServices/your-linux-ami-optimization-and-performance-cpn302-aws- reinvent-2013

http://www.slideshare.net/AmazonWebServices/your-linux-ami-optimization-and-performance-cpn302-aws-reinvent-2013

HVMs vs. memory oversubscription [P]age sharing, ballooning, and compression are opportunistic techniques. They do not guarantee memory reclamation from VMs. For example, a VM may not have sharable content, the balloon driver may not be installed, or its memory pages may not yield good compression. Reclamation by swapping is a guaranteed method for reclaiming memory from VMs. from https://labs.vmware.com/vmtj/memory-overcommitment-in-the-esx-server

https://labs.vmware.com/vmtj/memory-overcommitment-in-the-esx-server

HVM vs. performance Most successful AWS cluster deployments use more EC2 instances than they would the same number of physical nodes to compensate for the performance variability caused by shared, virtualized resources. Plan to have more EC2 instance based nodes than physical server nodes when estimating cluster size with respect to node count. from http://docs.basho.com/riak/latest/ops/tuning/aws/

http://docs.basho.com/riak/latest/ops/tuning/aws/

HVM vs. security

from http://venom.crowdstrike.com

http://venom.crowdstrike.com

Because lying about software is easier than lying about hardware

OS-based virtualization

from http://www.slideshare.net/ydn/july-2014-hug-managing-hadoop-cluster-with-apache-ambari

http://www.slideshare.net/ydn/july-2014-hug-managing-hadoop-cluster-with-apache-ambari

OS-based virtualization Simple idea • The kernel is there to manage the relationship with hardware

and isolate processes from each other • We’ve depended on secure memory protection, process

isolation, privilege management in unix for a long time •