Unfortunately yes, synchronization I’ve told you in the previous response and overhead required for batches to work degrade the performance. I slowly improve it, and there will be some changes after Hangfire 1.8 and Hangfire.Pro 3.0 is released, where I will tune some queries, but there will be no magical solution.
Also this is how concurrent data structures work in general – by using locks – and by incurring big penalties when acquiring and releasing them. Even lock-free and wait-free algorithms don’t help when external storages are used in Hangfire and require even more queries. And better data structures for parallelism aren’t implemented yet even in paper, so there’s no silver bullet here.
To avoid LCK_M_X reports, you can only move Hangfire’s database to another server or consider using Redis instead. Those application locks are important to decrease the load on SQL Server and can’t be avoided.
SQL Azure is also slow for writes, and higher plans can help you to increase it. Also I’ve seen that SQL on Virtual Machine gives better throughput, but unfortunately don’t know the actual numbers. And Redis in clouds works much better than RDBMS.