We are already using Hangfire for a while now and it’s proven it’s value in keeping our old and new system in sync.
Now we want to take it a step further. Our new application still makes direct DB calls to our old system and it’s slowing down the application. So my plan is to remove this by taking more advantage of Hangfire.
Currently we have a simple setup. 1 server with 1 (default) queue.
In our new setup we can identify 2 types of jobs.
- Long running
- Fire-and-Forget
The long running jobs are your typical ETL type jobs. Read some data from the old system and transform it to upsert it into the new. (or the other way around)
The fire-and-forget jobs will be short simple actions that are triggered from our application. For example update customer data in the old system when somebody changes it in the new.
My first idea was to create 2 separate queues (fastlane, slowlane) and assign jobs to the queue depending on the type.
I would also use 2 virtual servers to run these. But I’m not sure what the most ideal setup would be.
Run both queues on both servers or use 1 server for 1 queue.
Is there any recommendation that I can follow? Or does anybody have any experience?
For the record, we are running Hangfire in a Windows Service on 1 physical server for the moment.