Hi!
I’m having an average on 10.000 of job per hour with 3 servers (one server with 3 applications pools)
I need to increase to 30.000 job per hour
What i shoul do ?
This is dependent on so many factors. How intense are the jobs? How beefy is the server? What bottleneck are you seeing?Depending on the server configuration, sometimes separate boxes perform better than a in parallel. There’s no way for anyone to answer this question without knowing your setup and what issue you are actually trying to solve. Just because you are 3x your job count doesn’t mean there’s any relationship in server scaleout.
In your Startup.cs, you can tweek the number of threads hangfire spins up at a time with this code:
var serverOptions = new BackgroundJobServerOptions { WorkerCount = Environment.ProcessorCount * 10 };
app.UseHangfireServer(serverOptions);
On the hangfire dashboard, I was seeing a max of 20 processes running on a 4 CPU system. The above code doubled the processing count to 40.
Be careful tweaking the worker count. Increasing the number basically increases the number of threads so if they are all really busy and/or long-lived you could actually hurt performance. Context switching can be expensive, so you should try to tune using real-world metrics such that each processor can stay busy without having to switch between too many tasks.