How to prevent duplicates?

question
1

I have a high frequency call that is being made and I am currently running 3 instances of Hangfire. My problem is that the data coming in can sometimes come in very quickly and a 2nd record that should really just be a patch to the first one will sometimes generate a duplicate row in the database because the 1st record didn’t finish in time. Is there anyway I can specify that a specific service can only run on one of the servers and that server has like a concurrency of 1 so only one job is executing at a time? I think this will then force the 2nd record to wait until the 1st record has finished and an update will be performed instead of another insert. Maybe someone has an idea how to accomplish this?

2

If I’m understanding what you’re trying to do, then I think your best option is to use named queues.

If you’re not familiar, you annotate the job using something like this:

[Queue("your-queue-name")]

This can be applied at the class level or the method level. Then, you would configure each Hangfire host, to specify which queues each will process. Something like:

BackgroundJobServer host; // this is defined at the class level, for me

// this part is inside a bootstrapping method...
GlobalConfiguration.Configuration
    .UseSqlServerStorage(configuration.GetConnectionString("hangfire"), new SqlServerStorageOptions
    {
        CommandBatchMaxTimeout = TimeSpan.FromMinutes(10),
        SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
        QueuePollInterval = TimeSpan.Zero,
        UseRecommendedIsolationLevel = true,
        DisableGlobalLocks = true
    });

host = new BackgroundJobServer(new BackgroundJobServerOptions
{
    Queues = ["your-queue-name"],
    WorkerCount = 2
});

The important part is the BackgroundJobServerOptions - that’s where you can specify the queues for this host.

You can find the documentation for queues here: Configuring Job Queues — Hangfire Documentation

You might also want to look into Configuring the Degree of Parallelism — Hangfire Documentation - that’s where WorkerCount comes into play.