After the recent Redis drama we are exploring other options, such as https://www.dragonflydb.io/, which claims to be a drop-in Redis replacement. It seems to function fine for our other caching needs but when using it with Hangfire and doing recurringJobManager.AddOrUpdate() a StackExchange.Redis.RedisServerException is thrown with message ERR EXEC without MULTI.
This suggests that some operations or perhaps some order of operations is accepted by Redis and not by Dragonfly.
Tested with Hangfire.Pro.Redis 3.0.7 and 3.1.0-beta4.
Any ideas on what this could be / will it be investigated to officially make this Redis library more compatible with replacements, perhaps even with the new ValKey?
StackExchange.Redis, either original or our forked one, issue a SELECT command after each EVAL/EVALSHA command in a transaction. And it looks like Dragonfly doesn’t like this kind of database switching, because of its (I suppose) concurrency implementation that doesn’t support (I suppose) commands from different databases inside a single transaction.
So this is a behavior of a client, so I just made a commit to our fork of SE.Redis to change things and send the SELECT command after calling EXEC only, e.g. after executing the entire transaction. This will also prevent other threads of an unexpected database change in a LUA script, but not in the same transaction – I believe transaction’'s command set is a something that can be controlled by a developer, so we shouldn’t overly protect against such a behavior.
As for the Dragonfly, I don’t believe we will support it in the near future officially, but at least tests will work against it after some tuning described in BullMQ | Dragonfly (dragonflydb.io) (steps with allow-undeclared-keys and lock_on_hashtags are the same). Each storage, especially the young one, might contain its own set of bugs related to consistency and concurrency, and it’s not reasonable for us to deep dive into the implementation of each of them when there are troubles.
Hm, tests fail with the ‘Multiple databases inside a transaction are not currently supported’ error, but only on the build server, only on Windows and can’t reproduce this problem locally. Looks like client implementation relies on SELECT commands inside a transaction, so let’s wait for support of Dragonfly and EVAL commands from the original StackExchange.Redis first, since this is a non-easy fix with possible race conditions.