We use Hangfire 1.8.5. We have 3 physical servers, separate for sql server, Hangfire dashboard and windows service for hangfire background server.
Windows service server has 12 Processor and we set 36 background workers.
We have 30 recurring jobs each having different interval. Each recurring job enqueues few more jobs.
It works normally when start (or restart) the service when 36 or more jobs in the queue dashboard shows 36 jobs in “processing” and latency for each job is also very low.
But after couple of hours latency keeps increasing and very less number of jobs are in “processing” while it still shows 36 active workers. And cpu usage goes high.