NATS + Docker meetup talk Oct - 2016
Transcript of NATS + Docker meetup talk Oct - 2016
1 . 1
Simple and ScalableMicroservices
NATS and the Docker tooling
+
Waldemar Quevedo / @wallyqs
Docker SF Meetup, Oct 2016
1 . 1
2 . 1
ABOUTWaldemar Quevedo / So�ware Developer at in SF
Development of the Apcera PlatformPast: PaaS DevOps at Rakuten in TokyoNATS client maintainer (Ruby, Python)
@wallyqsApcera
3 . 1
ABOUT THIS TALKWhat is NATSFeatures from NATSNATS + Docker �
4 . 1
What is NATS?
4 . 2
4 . 3
NATSHigh Performance Messaging SystemCreated by First written in in 2010
Originally built for Cloud FoundryRewritten in in 2012
Better performanceOpen Source, MIT License
Used by Apcera, Ericsson, HTC, GE, Baidu
Derek CollisonRuby
Go
https://github.com/nats-io
4 . 4
NATSDesign constrained to keep it as operationally simple andreliable as possible while still being both performant andscalable.
4 . 5
Acts as an always available dial-tone
4 . 6
It's Fast
4 . 7
single byte message
Around 10M messages/second
4 . 7
4 . 8
and Simple
4 . 9
DESIGNConcise feature set (pure PubSub!)No built-in persistence of messagesNo exactly-once-delivery promises eitherThose concerns are simplified away from NATS
→ NATS Streaming
4 . 10
DESIGNTCP/IP basedPlain text protocolfire and forgetat most once
4 . 11
Very simple protocol
telnet demo.nats.io 4222
sub hello 1+OKpub hello 5world+OKMSG hello 1 5worldpingPONG
nats.io/documentation/nats-protocol
4 . 12
Demo
4 . 13
Clients
4 . 14
GOnc, err := nats.Connect()// ...nc.Subscribe("hello", func(m *nats.Msg){ fmt.Printf("[Received] %s", m.Data)})nc.Publish("hello", []byte("world"))
4 . 15
RUBYrequire 'nats/client'
NATS.start do |nc| nc.subscribe("hello") do |msg| puts "[Received] #{msg}" end
nc.publish("hello", "world")end
4 . 16
PYTHON (ASYNCIO)yield from nc.connect()
@asyncio.coroutinedef handler(msg): print("[Received] {data}".format( data=msg.data.decode()))
# Coroutine based subscriberyield from nc.subscribe("foo", cb=handler)yield from nc.publish("foo", "bar")
4 . 17
NODE.JSvar nats = require('nats').connect();
// Simple Publishernats.publish('foo', 'Hello World!');
// Simple Subscribernats.subscribe('foo', function(msg) { console.log('[Received] ' + msg);});
4 . 18
CnatsConnection_Publish(nc,"foo",data,5);natsConnection_Subscribe(&sub,nc,"foo",onMsg, NULL);
voidonMsg(natsConnection *nc, natsSubscription *sub, natsMsg *msg, void *closure){ printf("[Received] %.*s\n", natsMsg_GetData(msg)); // ...}
4 . 19
C#using (ISyncSubscription s = c.SubscribeSync("foo")) { for (int i = 0; i < 10; i++) { Msg m = s.NextMessage(); System.Console.WriteLine("[Received] " + m); } }
4 . 20
JAVA// Simple Publishernc.publish("foo", "Hello World".getBytes());
// Simple Async Subscribernc.subscribe("foo", m -> { System.out.println("[Received] %s\n", new String(m.getData()));});
4 . 21
Many more available!
C C# Java
Python NGINX Go
Node.js Elixir Ruby
PHP Erlang Rust
Haskell Scala Perl
( italics → community contributed)
4 . 21
4 . 22
Great community!
4 . 23
4 . 24
Connectors
4 . 24
4 . 25
Community contributed dashboards
github.com/cmfatih/natsboard
4 . 25
4 . 26
Docker Hub activity from NATS users
4 . 26
4 . 27
Docker Hub activity from NATS users
4 . 27
4 . 28
Docker Store (Beta)
4 . 28
5 . 1
NATS Features
5 . 2
FEATURESPure PubSub based Request/ReplySubject routing with wildcards
AuthorizationDistribution queue groups for balancingCluster mode for HA
Auto discovery of topology/varz style monitoringSecure TLS connections with certificates
5 . 3
REQUEST/REPLYIt is pure PubSub based using and bysignaling in the subject:
NATS unique identifierslimited interest
SUB _INBOX.y1JxglDi76shQQIhPbTDme 2UNSUB 2 1PUB help _INBOX.y1JxglDi76shQQIhPbTDme 6please
tells the server to unsubscribe from subscription with sid=2a�er getting 1 message before sending a request on thehelp subject.
5 . 4
REQUEST/REPLYIf another subscriber is connected and interested in thehelp subject, it will then receive a message with that inbox.
SUB help 90
# Message received from serverMSG help 90 _INBOX.y1JxglDi76shQQIhPbTDme 6please
# Use the inbox to reply backPUB _INBOX.y1JxglDi76shQQIhPbTDme 11I can help!
5 . 5
REQUEST/REPLYFinally, original requestor will be receiving the message:
SUB _INBOX.y1JxglDi76shQQIhPbTDme 2
UNSUB 2 1
PUB help _INBOX.y1JxglDi76shQQIhPbTDme 6
please
MSG _INBOX.y1JxglDi76shQQIhPbTDme 2 11
I can help!
5 . 6
REQUEST/REPLYNATS clients libraries have helpers for generating the uniqueinboxes which act as ephemeral subscriptions:
nats.NewInbox()// _INBOX.y1JxglDi76shQQIhPbTDme
Used internally when making a Request:
nc, _ := nats.Connect(nats.DefaultURL)t := 250*time.Millisecond// Request sets to AutoUnsubscribe after 1 responsemsg, err := nc.Request("help", []byte("please"), t)if err == nil { fmt.Println(string(msg.Data)) // => I can help!}
REQUEST/REPLY → LOWEST LATENCYResult of publishing a request to all nodes with limitedinterest means we are getting the fastest reply back:
5 . 7
REQUEST/REPLY → LOWEST LATENCYResult of publishing a request to all nodes with limitedinterest means we are getting the fastest reply back:
5 . 85 . 9
SUBJECTS ROUTINGWildcards: *
SUB foo.*.bar 90 PUB foo.hello.bar 2hiMSG foo.hello.bar 90 2hi
e.g. subscribe to all requests being made on the demo site:
telnet demo.nats.io 4222INFO {"auth_required":false,"version":"0.9.4",...}
SUB _INBOX.* 99MSG _INBOX.y1JxglDi76shQQIhPbTDme 99 11I can help!
5 . 10
SUBJECTS ROUTINGFull wildcard: >
SUB hello.> 90PUB hello.world.again 2hiMSG hello.world.again 90 2hi
e.g. subscribe to all subjects and see whole traffic goingthrough the server:
telnet demo.nats.io 4222INFO {"auth_required":false,"version":"0.9.4",...}
sub > 1+OK
5 . 11
DISTRIBUTION QUEUESBalance work among nodes randomly
5 . 12
DISTRIBUTION QUEUESBalance work among nodes randomly
5 . 13
DISTRIBUTION QUEUESBalance work among nodes randomly
5 . 14
DISTRIBUTION QUEUESService A workers subscribe to service.A and createworkers distribution queue group for balancing the work.
nc, _ := nats.Connect(nats.DefaultURL)// SUB service.A workers 1\r\nnc.QueueSubscribe("service.A", "workers", func(m *nats.Msg) { nc.Publish(m.Reply, []byte("hi!")) })
5 . 15
DISTRIBUTION QUEUESNote: NATS does not assume the audience!
5 . 16
DISTRIBUTION QUEUESAll interested subscribers receive the message!
5 . 17
CLUSTERINGAvoid SPOF on NATS by assembling a full mesh cluster
5 . 18
CLUSTERINGClients reconnect logic is triggered
5 . 19
CLUSTERINGConnecting to a NATS cluster of 2 nodes explicitly
srvs := "nats://10.240.0.1:4222,nats://10.240.0.2:4223"nc, _ := nats.Connect(srvs)
NOTE: NATS servers have a forwarding limit of one hop.
Each server will only forward a message that it hasreceived from a client to all connected servers thatexpressed interest in the message's published subject.A message received from a route will only be distributedto local clients.
5 . 20
CLUSTERINGReconnect and disconnect handlers can be useful to traceconnection failures.
nc, err := nats.Connect(uri, nats.DisconnectHandler(func(nc *nats.Conn) { fmt.Printf("Got disconnected!\n") }), nats.ReconnectHandler(func(nc *nats.Conn) { fmt.Printf("Got reconnected to %v!\n", nc.ConnectedUrl()) }), nats.ClosedHandler(func(nc *nats.Conn) { fmt.Printf("Connection closed. Reason: %q\n", nc.LastError()) }),)
5 . 21
CLUSTER AUTO DISCOVERYSince release, topology can be discovered
dynamically by clients!v0.9.2
5 . 22
We can start with a single node…
5 . 22
5 . 23
Then have new nodes join the cluster…
5 . 23
5 . 24
As new nodes join, server announces INFO to clients.
5 . 24
5 . 25
Clients auto reconfigure to be aware of new nodes.
5 . 25
5 . 26
Clients auto reconfigure to be aware of new nodes.
5 . 26
5 . 27
Now fully connected!
5 . 27
5 . 28
On failure, clients reconnect to an available node.
5 . 28
5 . 29
MONITORING style monitoring endpoint available for inspecting the
internal state of the server./varz
Other available endpoints:
- Info of clients connected to this server/connz
- Subscriptions metrics/subsz
- Cluster routes/routez
5 . 30
MONITORING: EXAMPLESGathering connections metrics from demo:
curl demo.nats.io:8222/varz | grep connections
Result:
"max_connections": 65536, "connections": 25, "total_connections": 12429,
5 . 31
MONITORING: EXAMPLESGathering metrics regarding languages used whenconnecting to demo
curl demo.nats.io:8222/connz?subsz=1 | grep lang | sort | uniq -c
Result:
10 "lang": "go", 7 "lang": "node", 8 "lang": "python2",
5 . 32
MONITORING polls from these endpoints providing a terminal UInats-top
5 . 33
TLSSupported for client and route connections, and formonitoring.
https_port: 6443
tls { cert_file: "./configs/certs/server-cert.pem" key_file: "./configs/certs/server-key.pem" ca_file: "./configs/certs/ca.pem" # Require client certificates verify: true}
5 . 34
CLUSTER TLSSecuring route connections with TLS
cluster { listen: 0.0.0.0:6222 tls { # Route cert cert_file: "./configs/certs/srva-cert.pem" # Private key key_file: "./configs/certs/srva-key.pem" # Optional certificate authority verifying routes # Required when we have self-signed CA, etc. ca_file: "./configs/certs/ca.pem" }}
5 . 35
SUBJECTS AUTHORIZATIONPubSub on certain subjects can be disallowed in the server'sconfiguration:
authorization { admin = { publish = ">", subscribe = ">" } requestor = { publish = ["req.foo", "req.bar"] subscribe = "_INBOX.*" }
users = [ {user: alice, password: foo, permissions: $admin} {user: bob, password: bar, permissions: $requestor} ]}
5 . 36
SUBJECTS AUTHORIZATIONClients are not allowed to publish on _SYS (reserved):
PUB _SYS.foo 2hi-ERR 'Permissions Violation for Publish to "_SYS.foo"'
6 . 1
NATS and the Docker tooling
6 . 2
THE NATS DOCKER IMAGESmall binary → Lightweight Docker image
No deployment dependencies
6 . 3
2-STEP BUILD PROCESS
6 . 4
FIRST STEP: COMPILE
github.com/nats-io/gnatsd/Dockerfile
FROM golang:1.6.3
MAINTAINER Derek Collison <[email protected]>
COPY . /go/src/github.com/nats-io/gnatsdWORKDIR /go/src/github.com/nats-io/gnatsd
RUN CGO_ENABLED=0 go install ...(elided)
EXPOSE 4222 8222ENTRYPOINT ["gnatsd"]CMD ["--help"]
6 . 5
SECOND STEP: FROM SCRATCH
github.com/nats-io/nats-docker/Dockerfile
FROM scratch
COPY gnatsd /gnatsdCOPY gnatsd.conf /gnatsd.conf
EXPOSE 4222 6222 8222
ENTRYPOINT ["/gnatsd", "-c", "/gnatsd.conf"]CMD []
6 . 6
THE NATS DOCKER IMAGEBy default it is exposing these ports:
# Clients, Cluster and Monitoring portsEXPOSE 4222 6222 8222
- Clients will be connecting against this port4222
- Port used for the cluster routes6222
- HTTP Monitoring endpoint8222
6 . 7
USING NATS + DOCKER
6 . 8
All examples can be found at
github.com/wallyqs/nats-docker-examples
6 . 8
EXAMPLE: SINGLE NODE
API Server which receives HTTP requests and communicatesinternally through NATS to a pool of workers.
6 . 96 . 10
EXAMPLE: CLUSTERED SETUP
API Server which receives HTTP requests and communicatesinternally through NATS to a pool of workers.
6 . 11
LOCAL NATS CLUSTER VIA DOCKER COMPOSE
Docker Compose tooling comes in handy here for doingdevelopment as it helps us in assembling NATS clusterslocally.version: "2"
networks: nats-network: {}services:
nats-server-A: networks: ["nats-network"] image: "nats:0.9.4" entrypoint: "/gnatsd --routes nats://nats-server-B:6222,nats://nats-server-C:6222 --cluster nats://0.0.0.0:6222"
nats-server-B: networks: ["nats-network"] image: "nats:0.9.4" entrypoint: "/gnatsd --routes nats://nats-server-A:6222,nats://nats-server-C:6222 --cluster nats://0.0.0.0:6222"
nats-server-C: networks: ["nats-network"] image: "nats:0.9.4" entrypoint: "/gnatsd --routes nats://nats-server-A:6222,nats://nats-server-B:6222 --cluster nats://0.0.0.0:6222"
6 . 12
NATS + Docker Compose Demo
6 . 12
6 . 13
DOCKER SWARM BASED NATS CLUSTER
Auto discovery becomes helpful in this context, we canassemble one with the following commands.docker network create --driver overlay nats-cluster-example
docker service create --network nats-cluster-example --name nats-A \ nats:0.9.4 -cluster nats://0.0.0.0:6222
docker service create --network nats-cluster-example --name nats-B \ nats:0.9.4 -routes nats://nats-A:6222 \ -cluster nats://0.0.0.0:6222
docker service create --network nats-cluster-example --name nats-C \ nats:0.9.4 -routes nats://nats-A:6222,nats://nats-B:6222 \ -cluster nats://0.0.0.0:6222
6 . 14
DOCKER SWARM BASED NATS CLUSTER
Even easier: just use same configuration in all nodes.
for node in A B C; do docker service create --network nats-cluster-example \ --name nats-$node nats:0.9.4 \ -routes nats://nats-A:6222 \ -cluster nats://0.0.0.0:6222done
Initial server becomes the seed server, ignoring route to self.
6 . 15
NATS + Docker Swarm mode Demo
6 . 15
7 . 1
Summary
7 . 2
NATS is a simple, fast and reliable solution for the internalcommunication of a distributed system.
Docker flexible tooling is a good complement for buildingNATS based applications.
7 . 2
7 . 3
BONUS: NATS STREAMING
NATS Streaming recently became an official image too!
It is a layer on top of NATS totalling in ~10MB.
Further info: github.com/nats-io/nats-streaming-server
8 . 1
THANKS! / github.com/nats-io @nats_io
Play with the demo site!
telnet demo.nats.io 4222