Dynamic queues?

question
1

I would like to be able to queue a method for background processing, but add it to a queue I name at run time. The various queues would be enabled for parallel processing.

Is this possible with Hangfire? If so, how could I achieve this?

I am trying to prevent a scenario where a shared resource (hardware) that I am controlling is only being worked on by one process at a time. There are multiple places where the hardware is controlled, and so only one should be active at a time (and the other requests queued up).

I know there is a disable parallel processing attribute that can be applied to a method, but this is not quite what I need. I still want parallel processing, but by queues.

There is hardware in various locations, and so the location names would be the queue names. Hardware is added over time and registered with the system, hence the need for the queues names to be defined at runtime.

Thanks for any help/advice,
Richard

2

@rmmcgr, you can use advanced API to specify queue name dynamically:

IBackgroundJobClient client = new BackgroundJobClient();
IState state = new EnqueuedState
{
    Queue = "my-queue"
};

client.Create(() => Console.WriteLine(), state);
1 Like
3

Fantastic, thank you!

4

Is there also an alternative to RecurringJob.AddOrUpdate that allows the specification of a queue?

5

I also have a need for recurring tasks to specific queues. +1 for this feature

6

But the jobs in new queue “my-queue” are not fired. They are always in state Enqueued. So I use Requeue could fire them processing. But got problem job that is created by ContinueWith. There is no way to Requeue after parent job finished and child job is enqueued. Here is my code:

IBackgroundJobClient client = new BackgroundJobClient();
IState state = new EnqueuedState
{
Queue = “my-queue”
};

var hangfireJobId = client.Create(() => Console.WriteLine(“Normal Job”), state);
client.Requeue(hangfireJobId, Hangfire.States.EnqueuedState.StateName);

var childHangfireJobId = client.ContinueWith(latestHangfireJobId, () => Console.WriteLine(“Continue Job”), state, JobContinuationOptions.OnAnyFinishedState);

1 Like
7

When you say advance. Do you mean the pro version

8

I’m finding the same problem, and when I requeue they are going back to the defaul queue

9

@odinserj I have the same problem here as @An_Nguyen where “the jobs in new queue “my-queue” are not fired. They are always in state Enqueued.”.

Is there a fix for this?

10

it seems this:

IState state = new EnqueuedState
{
Queue = “my-queue”
};

only works if “my-queue” is set at the startup of your app,

            .AddHangfireServer(options => options.Queues = new[]
            {
                "default",
                "my-queue"
            })

which isn’t very useful when you need to get queues spun up dynamically

11

Is there no way to do this with Schedule? The ScheduledState class appears to not have the “Queue” property.