Learn RabbitMQ with Python in 90mins
-
Upload
larry-cai -
Category
Technology
-
view
599 -
download
0
Transcript of Learn RabbitMQ with Python in 90mins
Larry cai <[email protected]>
Agenda Introduction Exercise 1: Hello World Exercise 2: Work queues - Direct Exercise 3: Publish/Subscribe - Fanout Exercise 4: Routing - Direct Exercise 5: Topics Reference
2 04/15/23
Mostly are based https://www.rabbitmq.com/getstarted.html
https://www.rabbitmq.com/
Environment Preparation (docker) Boot2docker Installer (127M)
Contains latest docker already, fast Container persistence via disk automount on /var/lib/docker $ docker -v User/Passwd: docker/tcuser (192.168.59.103)
Download python/rabbitmq:2 docker images $ docker pull python:2 $ docker pull rabbitmq:3-management
(Windows/Mac) Clone the code from github to your user directory (~/git) https://github.com/larrycai/codingwithme-rabbitmq
Notepad++ & MobaXterm are recommended
3 04/15/23
http://boot2docker.io/
Introduction RabbitMQ is a lightweight, reliable, scalable
and portable message broker. (Apache ActiveMQ/Apollo, ZeroMQ)
It supports AMQP (Advanced Message Queuing Protocol)
Written in Erlang/OTP Messaging: Pattern (asynchronous)
Producer: create message Consumer: Handle a message Queue: Stack the message
4 04/15/23
Exercise 1: Hello World Run first app inside docker environment
$ cd /c/Users/<id>/git/codingwithme-rabbitmq
$ docker run -d -e RABBITMQ_NODENAME=rabbit --name rabbit -p 15672:15672 rabbitmq:3-management
$ docker run -v $PWD:/code -w /code --name worker --link=rabbit:rabbit -it python:2 bash
# pip install pika
# cd exer1 && ./send.py
Browse: http://192.168.59.103:15672 See overview and Check Exchange/QueueGet message
Run in another shell inside docker$ docker exec –it worker bash
# cd exer1 && ./receive.py
Check queue again in GUI Update the “Hello world” to “Hello Larry”
5 04/15/23
Docker HostDocker Host
RabbitMQ Server
(rabbit)
15672
Python:2(worker)Python:2(worker)
RabbitMQ - Concept Publisher/Consumer is app (running in python
container in exercise 1) Publish & consume the messages
In RabbitMQ, An exchange receives messages from producers and pushes them to queues.
6 04/15/23
Rabbit Libraries - Python Pika module Pika is RabbitMQ server python module Procedure for Publish
1. Establish connection
2. Define Queue (“Hello”)3. Publish (routing_key is
queue name) Consume
1,2 same for publish 3: Call back
7 04/15/23
Queue in depth Round-robin dispatching
Scale to distribute to different consumers Message acknowledgment
An ack(nowledgement) is sent back from the consumer to tell RabbitMQ that a particular message had been received
Message durability & Fair dispatch (self-learning)
8 04/15/23
Exercise 2: Work queues How to scale the twitter message image
processing twitter.py – publish twitter message image.py – process the image if twitter has Env# cd ../exer2#./twitter.py “I see the nice picture 1.png”#./image.py
Did we receive the message in queue ? Why ? Run more image.py to see how it is distributed
(another shell, docker exec -it .. ) Kill some image.py
9 04/15/23
Exchange types Exchange type is the rule on how
to deliver message to the queues: direct, fanout, topic and headers
Direct: if the routing key matches, then the message is delivered to the corresponding queue.
Fanout: multicast the received message to the bound queues. Publish/Subscribe Patten Broadcast
10 04/15/23
Exercise 3: Fanout - Publish/Subscribe How to broadcast the message for different
consumers twitter.py – publish twitter message image.py – manipulate the image if twitter has
Env# cd ../exer3#./twitter.py “I see the nice picture 1.png”#./image.py
Did we receive first twitter & Check in the browser for the queue
Implement extra tag.py check whether twitter has tag Define queue `tag` and bind to `twitter` exchange Search for “#xxx#” keywords and print it out ./twitter.py “#codingwithme# is good, send picture 1.png”
11 04/15/23
Binding A binding is a relationship between an
exchange and a queue. One to one mapping
Multiple mapping with Routing_key
12 04/15/23
Exercise 4: Direct - Routing How to handle different log system Env# cd ../exer4# ./emit_log_direct.py # ./receive_logs_direct.py
Exercise: Check GUI for queue name and bindings More receiver
# ./receive_logs_direct.py warning error > logs_from_rabbit.log# ./receive_logs_direct.py info warning error
Define fixed queue name
13 04/15/23
Topics User pattern as routing key (“a.b.c.d”) Queues can use wildcard characters in binding
* matches a single word, # zero or more words. “rabbit.*” matches for rabbit.info/rabbit.error
Topic exchange is powerful and can behave like other exchanges. When a queue is bound with "#" (hash) binding key - it
will receive all the messages, regardless of the routing key - like in fanout exchange.
When special characters "*" (star) and "#" (hash) aren't used in bindings, the topic exchange will behave just like a direct one.
14 04/15/23
Exercise 5: Topic handle different log system more Env# cd ../exer5# ./emit_log_direct.py # ./receive_logs_direct.py “#”
Exercise Check the queue name in GUI Create for rabbit.*/apache.*/*.info Publish message for rabbit.info/error & apache.info/error
15 04/15/23
Summary RabbitMQ is the message broker support
AMQP Handle async messages
AMQP Model Different Exchange types Messages Queues Bindings Rule for Binding ..
16 04/15/23
Reference http://www.rabbitmq.com/ Exercises are based on
http://www.rabbitmq.com/getstarted.html Book:
RabbitMQ in Action (2012):http://www.manning.com/videla/
Code: https://github.com/larrycai/codingwithme-rabbitmq
17 04/15/23
ChangeLog 2015/05/11: first release
18 04/15/23