Hangfire job starts twice on same worker after 10 minutes

question
1

Hi all,

I have an issue regarding Hangfire v1.7.17.
I schedule a import by a method call like this:
I have and Admin which sends some data to an API, which then schedules the link to another API which will be ran by Hangfire.
Screenshot_2
The ConsumeApiCall has an attribute “AutomaticRetry=0”

image
image1064×722 38.8 KB

If I run a smaller import, everything works correctly, I get every mail of the enqueued, started and finished states.

But the issue comes when I try to import bigger data, which would take like 20-30 minutes. I saw in the logs, that the import starts (working as expected), then after around ~10 minutes, the same import call basically starts over, but the thing is that in the Hangfire dashboard it’s running on the same worker and server.

And after around 20-21 minutes I get a failed mail with the following message, even tho in the logs it states that both of the imports which where started, the original one and the one after 10 minutes.

The failed message is: "The underlying connection was closed: The connection was closed unexpectedly. " this is thrown by the HttpRequestException

I am using SqlServerStorage, I also tried it without the Options which are given as example of setup in the Hangfire documentation and tried it with setting the timespan of those options to bigger values.

I am clueless what would be the issue. Could somebody help with something? I hope everything is clear.

Thanks in advance!

2

The Job starting multiple times looks like an Invisibility Timeout https://docs.hangfire.io/en/latest/configuration/using-sql-server.html#configuring-the-polling-interval

The failed message looks like your HttpClient is timing out waiting for your API call to complete.

I would definitely think about whether there’s a way to “batch” chunks of your result data rather than asking for the world in what will probably be an ever longer running Job.

3

Hey,
Thanks for the reply.

But I’m not sure with this InvisibilityTimeout thing… Should I increase it or decrease it?

Currently this is how it’s set currently:
SlidingInvisibilityTimeout = TimeSpan.FromMinutes(60)
QueuePollInterval = TimeSpan.Zero

Should these values be less or more? I don’t quite understand it.

Thanks in advance!