工作队列 rabbitmq

Work Queues

工作队列 rabbitmq_第1张图片

In the first tutorial we wrote programs to send and receive messages from a named queue. In this one we'll create a Work Queue that will be used to distribute time-consuming tasks among multiple workers.

在第一个教程中,我们编写了从命名队列发送和接收消息的程序。在这个示例中,我们将创建一个工作队列,用于在多个worker之间分配耗时的任务。

The main idea behind Work Queues (aka: Task Queues) is to avoid doing a resource-intensive task immediately and having to wait for it to complete. Instead we schedule the task to be done later. We encapsulate a task as a message and send it to a queue. A worker process running in the background will pop the tasks and eventually execute the job. When you run many workers the tasks will be shared between them.

工作队列(又名:任务队列)背后的主要思想是避免立即执行一个资源密集型任务并不得不等待它完成。相反,我们把任务安排在稍后完成。我们将任务封装为消息并将其发送到队列。在后台运行的工作进程将弹出任务并最终执行任务。当您运行许多worker时,任务将在它们之间共享。

This concept is especially useful in web applications where it's impossible to handle a complex task during a short HTTP request window.

这个概念在web应用程序中特别有用,在web应用程序中不可能在短HTTP请求窗口期间处理复杂的任务。

Preparation准备

In the previous part of this tutorial we sent a message containing "Hello World!". Now we'll be sending strings that stand for complex tasks. We don't have a real-world task, like images to be resized or pdf files to be rendered, so let's fake it by just pretending we're busy - by using the Thread.sleep() function. We'll take the number of dots in the string as its complexity; every dot will account for one second of "work". For example, a fake task described by Hello... will take three seconds.

在本教程的前一部分中,我们发送了一个包含“Hello World!”的消息。现在我们会发送代表复杂任务的字符串。我们没有实际的任务,比如要调整大小的图像或要渲染的pdf文件,所以让我们假装很忙来模拟它——通过使用Thread.sleep()函数。我们将字符串中点的数量作为它的复杂度;每个点将占一秒的“工作”。例如,Hello…描述的一个假任务。只需要三秒钟。

Fair dispatch调度

You might have noticed that the dispatching still doesn't work exactly as we want. For example in a situation with two workers, when all odd messages are heavy and even messages are light, one worker will be constantly busy and the other one will do hardly any work. Well, RabbitMQ doesn't know anything about that and will still dispatch messages evenly.

您可能已经注意到分派仍然不能完全按照我们的要求工作。例如,如果有两个员工,当所有奇怪的消息都很重,甚至消息都很轻时,一个员工会一直很忙,而另一个人几乎什么工作都不做。好吧,RabbitMQ对此一无所知,它仍然会均匀地分派消息。

This happens because RabbitMQ just dispatches a message when the message enters the queue. It doesn't look at the number of unacknowledged messages for a consumer. It just blindly dispatches every n-th message to the n-th consumer.

这是因为RabbitMQ只在消息进入队列时发送消息。它不查看用户未确认消息的数量。它只是盲目地将每条第n个消息分派给第n个消费者。

.webp)

In order to defeat that we can use the basicQos method with the prefetchCount = 1 setting. This tells RabbitMQ not to give more than one message to a worker at a time. Or, in other words, don't dispatch a new message to a worker until it has processed and acknowledged the previous one. Instead, it will dispatch it to the next worker that is not still busy.

为了克服这个问题,我们可以使用设置为prefetchCount = 1的basicQos方法。这告诉RabbitMQ一次不要给一个worker发送一条以上的消息。或者,换句话说,在worker处理并确认前一个消息之前,不要向它发送新消息。相反,它将把它分派到下一个不繁忙的worker。

Note about queue size

If all the workers are busy, your queue can fill up. You will want to keep an eye on that, and maybe add more workers, or have some other strategy.

你可能感兴趣的