Is there a way to provide Enqueue a unique identifier

question
1

I have a web app that uses Hangfire to enqueue background jobs and set up recurring jobs. It works great.

However, I need to start processing incoming events from an external service. This service sends me events with unique identifiers. If my app doesn’t respond in time, the external service might retry the event.

In order to handle these events, I’d like to do something like this:

BackgroundJob.Enqueue(() => HandleEvent(event), event.UniqueIdentifier);

And let Hangfire handle ensuring that my HandleEvent call is not called twice for the same event. Is there a way to do that?

Right now, it looks like the only way to do that is for my app to keep track of the event IDs and only enqueue a task when the event id is unique.

2

Unfortunately idrmpotent calls aren’t directly supported as for now. But I’m sure it’s possible to write a client filter that acquires a distributed lock, checks existence of some key and:

  1. Exits without doing anything if exists, filter context has the Canceled property to exit without creating a job.
  2. Creates a key with some time to live set, and creates a background job (preferable atomically with key creation, but atomicity will make it more difficult to write this filter).

And then releases the distributed lock. Sorry, this pseudo-code is the only I can give to you currently. And please use slightly different keys for distributed lock and idempotence key (different prefix or suffix) to avoid collisions in storages where common key space is used.

3

@haacked have you found a solution for this?

4

Possible solution: Allow creating any job with a specific job id · Issue #181 · HangfireIO/Hangfire · GitHub