0MQ And Mongrel2 Overview
description
Transcript of 0MQ And Mongrel2 Overview
![Page 1: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/1.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 1/21
0MQ and Mongrel2The future is far too simple.
![Page 2: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/2.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 2/21
ZeroMQ
Developed by iMatrixBuilt AMQP for a bank
Didn't like what they madeBlogged about itTried to fix it
Did a pretty good job
![Page 3: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/3.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 3/21
ZeroMQ is Not
AMQPAMQP "Light"
A job/task queueJust lazy socketsMagic bullet that fixes every
problem you've ever had withdata distribution.
![Page 4: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/4.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 4/21
ZeroMQ is
A message passing libraryWith bindings for everything
Atomic messagesNo message fragmentationYou can ask for it if you
wantYour chunks remain atomicPeer is notified
Asyncronous message delivery
![Page 5: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/5.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 5/21
ZeroMQ is
Simple
Similar API to BSD sockets,similar terminologyAround 20 total API calls
zmq_init, zmq_socket, zmq_connect, zmq_bind, zmq_setsockopt,
zmq_getsockopt, zmq_close, zmq_term, zmq_poll, zmq_recv, zmq_send,zmq_device, zmq_msg_init, zmq_msg_init_data, zmq_msg_init_size,zmq_msg_close, zmq_msg_size
Easy to keep entirely in yourhead
![Page 6: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/6.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 6/21
ZeroMQ is
N:NSockets are more than pairs
Can connect and bind a socketto many addresses over manytransports at the same time
Behavior governed by sockettype
![Page 7: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/7.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 7/21
ZeroMQ is
Transport AgnosticCan use multiple transports
transperantly andsimultaniouslytcp:// udp:// ipc:// inproc://
IPC are unix pipesinproc are internal memorybuffers
![Page 8: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/8.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 8/21
ZeroMQ really is
A set of message distributionpatterns.
Different socket types promisedifferent delivery semantics
A set of chosen socketsenforce the promised behavior
![Page 9: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/9.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 9/21
ZeroMQ Sockets
Request/ReplyZMQ_REQ/ZMQ_REP
Alternate zmq_send()/zmq_recv()
Requests are load balancedReplies are matched to requesterAnything else is an error
ZMQ_ROUTER/ZMQ_DEALER
Advanced REQ/REPUnrestricted send/recvMore requirements on messageformat
![Page 10: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/10.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 10/21
ZeroMQ Sockets
Pub/Sub
ZMQ_PUB/ZMQ_SUB
ZMQ_SUBSCRIBE/ZMQ_UNSUBSCRIBE socketoptionsOne side can only recv(), the other can onlysend()Messages are filtered by subscriptionDelivered to all connected peersIf no one is listening, messages are dropped.If you don't subscribe, you don't get messages.
![Page 11: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/11.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 11/21
ZeroMQ Sockets
Work PipelineZMQ_PUSH/ZMQ_PULLZMQ_PUSH produces work, and can only send()ZMQ_PULL consumes work, can only recv()Outgoing messages are fair-queued to allconnected peers.No connected peers causes send() to block,unless you ask nicey not to.
No messages are dropped.
![Page 12: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/12.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 12/21
ZeroMQ Sockets
Exclusive PairZMQ_PAIR
Exclusive pair for inter-threadcommunicationOnly "works" on inproc://
transportIf you think you need thissocket, you're probably wrong.
![Page 13: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/13.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 13/21
That's cute, but what
does it have to dowith web servers?
![Page 14: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/14.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 14/21
Mongrel2
"Application, language, andnetwork architecture agnostic
web server"
Correct, FSM-driven HTTP
parserLong-pollingZeroMQ based
![Page 15: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/15.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 15/21
Mongrel2
Long pollingSuperpoll
Hipster compliant pollingmechanismUses load to determine
polling mechanismChoice based on statisticalperfomance analysis
![Page 16: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/16.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 16/21
Mongrel2
ZeroMQ HandlersRequests sent to a knownZMQ_PUSH addressResponses receieved at a
known ZMQ_SUB address
Wrapper protocol for HTTPrequests based onnetstrings+JSON, tnetstrings
![Page 17: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/17.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 17/21
Mongrel2
Wait, what?
Request and Reply are decoupledA client is identified by an integerMongrel2 doesn't have to keep a pile of state around while a request is processedA reply mentions client ids that shouldreceive itAnything that can send a ZMQ messageto the server can generate an HTTP
response
![Page 18: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/18.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 18/21
Mongrel2
N:M RepliesEach reply mentions the UUID of the handlerEach reply mentions every client id that shouldreceive itEach reply can come from anywhereSo each reply can target multiple clients with asingle message just by adding more clients to theresponse list
Usefull for propogation of information to manyinterested peersHTTP transported PUB/SUB
![Page 19: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/19.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 19/21
Mongrel2 Wire
Protocol[T]Netstrings for everythingRequest
uuid id path size:headers,size:body,Response
uuid size:id id id, bodyEverything is ASCII
UUID identifies a handlerSizes are in bytesIDs are client id integers
Request ID identifies a browser that sent a requestResponse IDs target the reply to the given
browsers
![Page 20: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/20.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 20/21
Mongrel2
That really is it.Everything else is a detail or fluff
Disconnect messages state when a peer youreplied to went away
Large uploads are stored on disk and sent as a0mq message with a path to the file
Handler messages at start and finishCan be terminated in-flight by a handler
Configuration through sqlite3, zeromq configuration
in progressCluster response routing can be handled with 0mqPUB/SUBOpen SourceI'm your local contributor with repo keys
![Page 21: 0MQ And Mongrel2 Overview](https://reader030.fdocuments.net/reader030/viewer/2022020218/5571fe4949795991699b0c8c/html5/thumbnails/21.jpg)
5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com
http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 21/21
Questions?RTFM
ZeroMQ
http://zguide.zeromq.org/page:allhttp://api.zeromq.org/
Mongrel2
http://mongrel2.org/
Me
http://twitter.com/sshirokovslava hackin ibs ns.c m