Unhandled exceptions in a job + Redis as backplane

bug?
, ,
1

Hello!

Im trying out Redis as backplane for our application (.net8) but im having some issues.

Everytime an unhandled exception is thrown from a job it will give me

"System.InvalidOperationException
Transaction was discarded to prevent inconsistency: one or more of active distributed locks were abandoned. Are you hitting Redis memory limits?"

in the dashboard as the reason.

Stack trace:

System.InvalidOperationException:
   at Hangfire.Pro.Redis.RedisTransaction.Commit (Hangfire.Pro.Redis, Version=3.0.9.0, Culture=neutral, PublicKeyToken=null: C:\projects\hangfire-pro-redis\src\Hangfire.Pro.Redis\RedisTransaction.cs:164)
   at Hangfire.States.BackgroundJobStateChanger.ChangeState (Hangfire.Core, Version=1.8.12.0, Culture=neutral, PublicKeyToken=null: C:\projects\hangfire-525\src\Hangfire.Core\States\BackgroundJobStateChanger.cs:159)
   at Hangfire.Server.Worker.TryChangeState (Hangfire.Core, Version=1.8.12.0, Culture=neutral, PublicKeyToken=null: C:\projects\hangfire-525\src\Hangfire.Core\Server\Worker.cs:230)

Im sure my Redis is not hitting any memory limits and I wonder what im missing. It wouldnt be an issue if the dashboard would show the correct exception, now its missleading and im also worried something else is wrong.

My Redis server is an “Azure Cache for Redis” server, and is configured per Using Redis — Hangfire Documentation .

To get this error to happen you only need a basic job that throws an unhandled exception. My “AddHangfireServer” is more or less default, with a few more queues added only.

My RedisStorage looks like this atm:

var redisStorageOptions = new RedisStorageOptions
            {
                Prefix = $"hangfire:{redisPrefix}:",
                InvisibilityTimeout = TimeSpan.FromHours(8),
            };

            return new RedisStorage(connectionstring, redisStorageOptions);

Running the newest version of Hangfire.Pro.Redis and Hangfire.Core.

Anyone have any ideas?

2

Please try setting the DisableJobIntegrityChecks option to true in the RedisStorageOptions’ constructor and let me know if this solves the problem.

3

Thank you. This solves it. I see that its supposed for debugging purposes only?

4

Hmm. Ok, it does not work consistently but seems that it makes it better.

5

So I tried (GitHub - marcoCasamento/Hangfire.Redis.StackExchange: HangFire Redis storage based on original (and now unsupported) Hangfire.Redis but using lovely StackExchange.Redis client) and this package does not have this issue.