Background job identifier is fetched from its queue using the
RPOPLPUSH that atomically moves background job id from its queue and put it to the pending list. With the behaviour you get it looks like multiple workers were able to pick up that job identifier.
So I’m afraid the question should be asked to KeyDB developers instead why KeyDB allows multiple clients to perform the same
RPOPLPUSH operation concurrently, e.g. why does it allow concurrent access to the same key, an entity, that’s considered to be sequential – because any concurrent access in this case does not make any sense.
I believe this behaviur is due to multi-master configuration. So the full question is “How KeyDB serialise access to a particular key when multiple masters involved?” Because the only answer is “Nohow”, otherwise a Holy Grail of distributed processing belong to them and many patents should be protecting it.
High availability is possible, but with primary/replica scenario (as in Redis Cluster), otherwise transactions will not work or give unexpected results and you’ll end up with inconsistent state, where everything is possible, but nothing is predictable.