RabbitMQ之tutorial-2

RabbitMQ之tutorial-2

Tutorial Two

https://www.rabbitmq.com/tutorials/tutorial-two-python.html

Screen Shot 2020-12-29 at 11.50.12 AM

The main idea behind Work Queues (aka: Task Queues) is to avoid doing a resource-intensive task immediately and having to wait for it to complete

Screen Shot 2020-07-21 at 10.46.28 AM

new_task.py

import pika
import sys
connection = pika.BlockingConnection(
    pika.ConnectionParameters(host='localhost')
)

channel = connection.channel()

channel.queue_declare(queue='task_queue', durable=True)

msg = ' '.join(sys.argv[1:]) or 'hello world'
channel.basic_publish(
    exchange='',
    routing_key='task_queue',
    body = msg,
    properties=pika.BasicProperties(delivery_mode=2) # make msg persistent
)

print(" [x] Sent %r" % msg)
connection.close()

worker.py

import pika
import time

connection = pika.BlockingConnection(
    pika.ConnectionParameters(host='localhost'))
channel = connection.channel()

channel.queue_declare(queue='task_queue', durable=True)
print(' [*] Waiting for messages. To exit press CTRL+C')


def callback(ch, method, properties, body):
    print(" [x] Received %r" % body)
    time.sleep(body.count(b'.'))
    print(" [x] Done")
    ch.basic_ack(delivery_tag=method.delivery_tag)


channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue='task_queue', on_message_callback=callback)

channel.start_consuming()

Round-robin dispatching

By default, RabbitMQ will send each message to the next consumer, in sequence. On average every consumer will get the same number of messages.

Terminal1 python worker.py

Terminal2 python worker.py

Terminal3 python new_task.py First message. python new_task.py Second message.. python new_task.py Third message... python new_task.py Fourth message.... python new_task.py Fifth message.....

Message acknowledgment

we don't want to lose any tasks. If a worker dies, we'd like the task to be delivered to another worker.

There aren't any message timeouts; RabbitMQ will redeliver the message when the consumer dies. It's fine even if processing a message takes a very, very long time.

Acknowledgement must be sent on the same channel that received the delivery.

Screen Shot 2020-07-21 at 10.53.13 AM

Message durability

We have learned how to make sure that even if the consumer dies, the task isn't lost. But our tasks will still be lost if RabbitMQ server stops.

When RabbitMQ quits or crashes it will forget the queues and messages unless you tell it not to. Two things are required to make sure that messages aren't lost: we need to mark both the queue and messages as durable.

Fair Dispatch

It just blindly dispatches every n-th message to the n-th consumer.

use the Channel#basic_qos channel method with the prefetch_count=1 setting,tell RabbitMQ not to give more than one message to a worker at a time. Or, in other words, don't dispatch a new message to a worker until it has processed and acknowledged the previous one. Instead, it will dispatch it to the next worker that is not still busy.

Screen Shot 2020-07-21 at 11.22.55 AM

If all workers are busy, queue can fill up.

• add more worker
• message TTL