I have the following :
2 web servers running in a load balanced setup.
Each of these can enqueue Recurring Jobs with the following code in OwinStartup.cs :
RecurringJob.AddOrUpdate(“Job”, () =>FunctionTrigger(), Cron.Daily(5, 0));
The FunctionTrigger is a simple function using Stateless Transaction session to insert records into the Database.
So when am using one Server the jobs running successfully, But in load balance an error happens and no records inserted into the DB .
This is the log error:
Hangfire.Server.ServerHeartbeat - Error occurred during execution of 'Server Heartbeat' component. Execution will be retried (attempt 2 of 2147483647) in 00:00:03 seconds. System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.Open() at Hangfire.SqlServer.SqlServerStorage.GetConnection() at Hangfire.Server.ServerHeartbeat.Execute(CancellationToken cancellationToken) at Hangfire.Server.AutomaticRetryServerComponentWrapper.ExecuteWithAutomaticRetry(CancellationToken cancellationToken)
I am using Hangfire 1.4.5
**Is this because the job runs tow times on each server ?! how can i avoid that. ?! **