What's going on here? Confusing history in the hangfire dashboard

question
,
1

I have a single type of job, which can be time consuming (it can take 2 - 4 hours to run). I have set the retries to be 3.

However, its behaviour in the processing queue is confusing, I have no idea what it’s doing. When I click on the job, the history says the following (so many blocks that all say processing) - what does this mean? Does this mean that it has failed and been retried at some point? If that’s the case, why are there 10 or so retries when I set the attempts to be 3.

Even more confusing is that after a period of time, this job simply vanishes. I would have expected it to then go into either succeeded or failed, or deleted, or retries. But it goes nowhere.

Can anyone let me know what all of this means?

Here is an example of the history of the job (while it’s in the processing queue).

Processing
Server:
BUILD-SERVER:1736
Worker:
#31
+7m 20.973s
Processing
Server:
BUILD-SERVER:1736
Worker:
#24
+30m 1.80s
Processing
Server:
BUILD-SERVER:13000
Worker:
#15
+4h 32m
Processing
Server:
BUILD-SERVER:13000
Worker:
#22
+48m 27.410s
Processing
Server:
BUILD-SERVER:2260
Worker:
#37
+51m 51.440s
Processing
Server:
BUILD-SERVER:8432
Worker:
#34
+37.30s
Processing
Server:
BUILD-SERVER:3120
Worker:
#27
+30m 930ms
Processing
Server:
BUILD-SERVER:14696
Worker:
#13
+64ms
Processing
Server:
BUILD-SERVER:14696
Worker:
#28

  • <1ms
    Enqueued
    Queue:
    DASHBOARD
2

Ended up figuring this out, of anyone comes across this.

What was happening is that the default invisibility timeout was at 30 minutes. The job was taking longer than 30 minutes, so hangfire assumed it had hung, cancelled it, and then requeued it. This would then happen indefinitely.

As explained in.
http://docs.hangfire.io/en/latest/configuration/using-sql-server.html

The fix is to increase this timeout, or recode the job to take less than 30 minutes.