Multiple queue - execution flow of workers

question
,
1

Hi!

I have three queues: “alpha”, “beta” and “default” in SqlServer (exactly like in documentation) and two workers (W1 and W2) .

I enqueue 7 jobs in the same time (at 00:00 for example) and one alpha job at 00:12.

The jobs have the following processing time:

  • alpha: 10 minutes (3 jobs at 00:00 and one at 00:12)
  • beta: 5 minutes (2 jobs at 00:00)
  • default: 1 minute (2 jobs at 00:00)

How will the two workers be distributed and when will the jobs be executed?

From the documentation:

Queues are run in the order that depends on the concrete storage implementation. For example, when we are using Hangfire.SqlServer the order is defined by alphanumeric order and array index is ignored. When using Hangfire.Pro.Redis package, array index is important and queues with a lower index will be processed first. The example above shows a generic approach, where workers will fetch jobs from the alpha queue first, beta second, and then from the default queue, regardless of an implementation.

and especially from this sentence workers will fetch jobs from the alpha queue first I understand the following order of execution:

  • 00:00: W1 takes alpha job until 00:10

  • 00:00: W2 takes alpha job until 00:10 (even there are beta and default jobs, alpha is more important)

  • 00:10: W1 takes alpha job until 00:20

  • 00:10: W2 takes beta job until 00:15 (there are no more alpha jobs)

  • 00:12: NEW job alpha arrived

  • 00:15: W2 takes alpha job until 00:25 (even there are beta and default jobs, alpha is more important)

  • 00:20: W1 takes beta job until 00:25

  • 00:25: W1 takes default job until 00:26 (no more alpha and beta jobs)

  • 00:25: W2 takes default job until 00:26

Is it correct the previous flow of execution?

Thank you.

1 Like
2

When Hangfire retrieves the next job it will favour the alpha queue over beta, beta over default. If there is a job in alpha it will take that one even if there are older jobs in beta. It would be difficult to speak to your exact timing though.

Queues should generally be used to allow priority of jobs over other jobs or to be able to have multiple instances running against the same jobs table. You just have to keep in mind a job could sit in queue for extended periods of time.

If you require specific order of execution for jobs you should use ContinueWith.