After trying several things, I finally decided to post this issue here.
I am getting the following DistributedLockTimeoutException almost instantly from Hangfire when a job is enqueued. I have ensured that the redis instance has the following policy defined (as per the docs here: http://docs.hangfire.io/en/latest/configuration/using-redis.html)
maxmemory-policy noeviction
What am I missing here?
Hangfire.Storage.DistributedLockTimeoutException: Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'SysOrchestrator.ActivateIndex' resource.
at Hangfire.Pro.Redis.RedisDistributedLock.RetryUntilTrue(Func`1 action, Nullable`1 timeOut)
at Hangfire.Pro.Redis.RedisDistributedLock..ctor(Func`1 databaseFactory, String resource, String lockToken, TimeSpan timeout)
at Hangfire.Pro.Redis.RedisConnection.DistributedLockWrapper..ctor(RedisConnection connection, String resource, TimeSpan timeout)
at Hangfire.Pro.Redis.RedisConnection.AcquireDistributedLock(String resource, TimeSpan timeout)
at Hangfire.DisableConcurrentExecutionAttribute.OnPerforming(PerformingContext filterContext)
at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation)
I am running this in netcoreapp2.1 with the following assemblies
Hangfire.AspNetCore 1.6.20
Hangfire.Pro 2.1.1
Hangfire.Pro.Redis 2.2.2
In ConfigureServices, the Hangfire relevant code is as follows:
services.AddSingleton<HangfireActivator, HangfireActivator>();
var redisStorageOptions = new RedisStorageOptions
{
Prefix = "hangfire:control:",
InvisibilityTimeout = TimeSpan.FromDays(2)
};
services.AddHangfire(cfg => cfg.UseRedisStorage("10.0.0.3,abortConnect=false,allowAdmin=true,syncTimeout=60000,ssl=false", redisStorageOptions).UseFilter(new Utility.JobContext()));
In Configure, the Hangfire relevant code is:
GlobalConfiguration.Configuration.UseActivator(new HangfireActivator(cenv, serviceProvider));
app.UseHangfireServer();
app.UseHangfireDashboard("/hangfire", new DashboardOptions()
{
Authorization = new[] { new HangfireAuthorizationFilter("admin") },
});
GlobalJobFilters.Filters.Add(new AutomaticRetryAttribute { Attempts = 0 });
What am I doing wrong here?
Thanks very much