Scaling and Performance (RabbitMQ)

question
,
1

Currently I am configured with SQL server. I am in the process of purchasing the pro version of HangFire

Right now we about 20 servers with 80 agents/workers on each. When I get to that amount of workers, I am seeing a performance degradation and errors occurring. Some of the Hangfire tables get locked and I am seeing errors.

I am getting the following 2 errors:
System.InvalidOperationException: Could not get a value of the job parameterCurrentUICulture. See inner exception for details. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool

and

System.InvalidOperationException: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.

I was thinking of using RabbitMQ with SQL server to help with the database performance. Has anyone have had experience with it and what are some thing I can expect using it.

Thanks,
John

2

It sounds like you are running out of SQL Server pooled connections, the default being 100. The way to solve it is to add the parameter for ‘Max Pool Size’ to your connection string. You could always use the Microsoft Performance Monitor and add the .NET Data Provider number of connections statistic to see how many are being consumed.

Here is a reference–> https://stackoverflow.com/questions/30331955/what-is-the-default-maxpoolsize

FYI - I experimented with MSMQ Hangfire integration and the major improvement was the queue speed as it removes the polling of SQL Server for new jobs. I assume the RabbitMQ implementation would be the same.

3

Thanks Rob. I have already set my max pool to the DB 20% more then the worker threads. I also added the Application Name to keep track of connections to a particular instance.

I am going to do a quick test with Rabbit and see if that helps with performance.

Thanks for the help

4

can you share your tests results ,on this?