Jobs being re-queued in the wrong queue

bug?
1

Hello

My hangfire system consists of a central hangfire server, and multiple hangfire clients, each picking up jobs from a different queue.

Nothing picks up jobs from the default queue.

However, if there is a problem when executing the job, hangfire re-schedules it to the default queue instead of the queue it was originally put on.

Is this a bug?

It has resulted in me having lots of jobs stuck in a queue that will never get read.

2

How do you tell Hangfire what queue should be used for the first time? If you are creating EnqueuedState directly, this behavior is by design.

You can apply the QueueAttribute to your background job method, scheduled jobs will be enqueued to that queue:

[Queue("st_index_population")]
3

I am creating the EnqueuedState directly, because that is the only way I can modify the queue name at runtime.

The queue names have prefixes (st_) in this case that determine which system picks it up and processes it.

Is there any way to have both? Variable queue names and the ability to re-queue to that same one?

4

How do you determine the queue name? Is it based on a method you are calling?

5

here is a code example:

 IBackgroundJobClient client = new BackgroundJobClient();
            IState state = new EnqueuedState
            {
                Queue = Config.HangfireQueuePrefix + "index_population_bg"
            };


            Parallel.ForEach(needToShape, n => client.Create(() => PopulateIndexBG(n), state));
6

Ok, thanks for the code. Target queue can be changed during the state election process, that can be extended by user-defined classes, called filters. You can look at QueueAttribute for example, it is just an extension. You can solve your problems by creating the following filter:

public class UseIndexPopulationQueueAttribute : JobFilterAttribute, IElectStateFilter
{
    public void OnStateElection(ElectStateContext context)
    {
        var enqueuedState = context.CandidateState as EnqueuedState;
        if (enqueuedState != null)
        {
            enqueuedState.Queue = Config.HangfireQueuePrefix + "index_population_bg";
        }
    }
}

And then apply it to your method:

[UseIndexPopulationQueue]
public void PopulateIndexBG() { }

You can also look at properties available in context argument, and use them to determine the queue name:

if (context.Job.Method.Name.StartsWith("PopulateIndex"))
{
    enqueuedState.Queue = "st_index_population_bg";
}

And add a filter globally for all your jobs, so you don’t even need to apply it to your methods as queue name will be determined automatically.

GlobalJobFilters.Filters.Add(new UseIndexPopulationQueue());
7

Excellent.

Thanks for your prompt response!