RecurringJob being ran on multiple servers

question
1

Hi.

So, I have a setup with load balancing over three servers and also development machines and we need a job to be run on each of the servers once a minute. We use SqlServerStorage which is the same for all servers.

In Program.cs we define a machine specific queue and also default:

    builder.Services.AddHangfireServer(options =>
    {
        options.Queues = new[] { Environment.MachineName.ToLower(), "default" };
    });

We then call the recurringJobManager to setup a recurring job that adds a task in that machine specific queue. We use unique ids to avoid issues.
_recurringJobManager.AddOrUpdate("PerformTask "+Environment.MachineName.ToLower(), Environment.MachineName.ToLower(), () => MethodNameToCall(), "*/1 * * * *");

All looks good. In Servers menu we can see that each server that runs has their own queue it listens to. The recurring jobs are being ran and queues up the jobs for each server queue and it all works well.

Here we noticed an issue though. When I shut my development down, the recurring job are still being run from one of the other machines until the recurring job times out but by then I can have tons of jobs in my specific queue. Ideally I would want to be able to set the recurring job itself to only be run from that same machine that started it.

I have tried to tag the method that does the recurringJobManager.AddOrUpdate() with a [Queue] attribute but it doesn’t help. I even did a huge attribute filter class and set breakpoints on all the OnCreated, OnPerforming and so on but debug never stopped there so I assume setting an attribute on a method that does AddOrUpdate() isn’t possible.

Does anyone have any idea or suggestion?