We are using hangfire in production. It is a cluster of 5 servers. In 1 windows service we start 3 hangfire core’s. Each core works with 1 queue.
If the servers are very busy (CPU and bandwidth) processing jobs sometimes 1 of the queues stops processing jobs.
Not one of the servers will fetch a jobs of that queue while the other queues still process jobs within the same windows service.
If we restart 1 service all the servers start to process jobs from that specific queue again. We use Redis. We get the feeling that there is some kind of semaphore blocking the queue in redis for the servers to fetch jobs.
Can you help me what the problem can be. We already upgraded to the last hangfire and redis versions.