Requeue on the same queue, or possibility to have server groups?

question
2

Yes it is. But the answer you will find is not as straightforward as it may sound. I found, and maybe it is just me, that this is always a tricky thing to get right in Hangfire. It’s easy to end up with what you described (jobs going to default queue) and there are a number of ways to go about achieving your goal depending on your situation. There is another (well several) discussions in this forum on this subject.

For my scenario (and this might or might not be needed for what you want to do) I ended up with a number of extensions to the basic functionality to add :

  • Machine specific queues (which I need for jobs continuations that have to execute on the same machine with machine specific file resources).
  • Preservation of the original queue (so requeues later do not get pawned of to a different machine)

I think your most immediate question is around the last one.

For this I have the following JobFilterAttribute (apologies to the original author of this code, it is not mine but was if I recall correctly obtained in a different topic on this forum. I just don’t remember who or where exactly).

public class PreserveOriginalQueueAttribute : JobFilterAttribute, IApplyStateFilter
{
    public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
    {
        // Activating only when enqueueing a background job
        if (!(context.NewState is EnqueuedState enqueuedState)) return;

        // Checking if an original queue is already set
        var originalQueue = JobHelper.FromJson<string>(
            context.Connection.GetJobParameter(
                context.BackgroundJob.Id,
                "OriginalQueue")
        );

        if (originalQueue != null)
        {
            // Override any other queue value that is currently set (by other filters, for example)
            enqueuedState.Queue = originalQueue;
        }
        else
        {
            // Queueing for the first time, we should set the original queue
            context.Connection.SetJobParameter(
                context.BackgroundJob.Id,
                "OriginalQueue",
                JobHelper.ToJson(enqueuedState.Queue));
        }
    }

It does what it suggests, any job decorated with this attribute on the first ‘run’ will store the queuename as a job parameter (for that job instance) and on subsequent runs (requeues) of that same job instance ensure that the same queue is used again. It still means you can add other filters (or attributes) to modify what the queue has to be on the first run but all subsequent runs are forced to the same queue as initially set.

1 Like