Constant Updating of Quueue

bug?
1

Hi

We are using v1.7.2 of hangfire on ASP.Net

We noticed our database is getting hammered with Update hangfire.queue queries (multiple per second)

We have our server configured like this as per the documentation. If we change the QueuePollInterval to say a minute, it still seems to run every second.

Any ideas?

GlobalConfiguration.Configuration
.SetDataCompatibilityLevel(CompatibilityLevel.Version_170)
.UseSimpleAssemblyNameTypeSerializer()
.UseRecommendedSerializerSettings()
.UseLog4NetLogProvider()
.UseSqlServerStorage(GetConnectionString(), new SqlServerStorageOptions
{
CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
QueuePollInterval = TimeSpan.Zero,
UseRecommendedIsolationLevel = true,
UsePageLocksOnDequeue = true,
DisableGlobalLocks = true
});

2

Any chance the new configuration wasn’t propagated, or for some reason you are running Hangfire.SqlServer of version 1.7.0 on your server? You can increase QueuePollInterval to TimeSpan.FromMilliseconds(500) to have much fewer poll attempts, or use previous value.

3

I upgraded the application to use Hangfire.SqlServer 1.7.3 to see if that helped, but no joy.

The configuration is loaded during Application_Startup.

I had a look at the source code too and I think there is the possibility of an infinite loop with the SlidingInvisibilityTimeout functionality.

I ended up setting this property to null on my config so that I can revert the system to using the old style of polling SQL server and I have set this to one minute. That got me out of trouble but I fear there is something sinister at play with that SlidingInvisibilityTimeout option.

GlobalConfiguration.Configuration
.SetDataCompatibilityLevel(CompatibilityLevel.Version_170)
.UseSimpleAssemblyNameTypeSerializer()
.UseRecommendedSerializerSettings()
.UseLog4NetLogProvider()
.UseSqlServerStorage(GetConnectionString(), new SqlServerStorageOptions
{
CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
SlidingInvisibilityTimeout = null,
QueuePollInterval = TimeSpan.FromMinutes(1),
UseRecommendedIsolationLevel = true,
UsePageLocksOnDequeue = true,
DisableGlobalLocks = true
});

4

Thanks for reporting this, @dmt. Yesterday I’ve released Hangfire 1.7.4 where I’ve fixed one possibility that could end in such behaviour, however I wasn’t able to reproduce it. Now it will throw exceptions that will lead to a delay between loop iterations, but we still need to understand the cause.

Please see the commit description for more information – https://github.com/HangfireIO/Hangfire/commit/591e8abbb189a4691c63669f212653f3c409c8f5.

5

By default, several background workers are going all at once, so the number of DB operations will be grow accordingly. I know this because I wrote and maintain the NHibernate-backed database provider. It’s very chatty!

Try finding the setting to cut down the background thread count. It’s late and I’m not inclined to dig for an example just now, but that may help.