I have multiple different versions of my application deployed in a server farm (Specifically, different deployment slots on an Azure App). Are there any recommendations for ensuring that jobs will only execute on “one” server (i.e. on the latest/specific version), Ideally with out changing an App-setting (as this would force an app restart which is one of the main reasons I use deployment slots)
I can identify the “ideal” version easily by saving the highest number into the DB (at app startup), but I’m unsure of how to get Hangfire to execute on that server. Should I just manually Start/Stop the respective background servers or is there a more eloquent way of achieve this?
Use the database Prefix method pops to mind. You could probably do something with the version or build number and incorporate that in the table/collection prefix setting for Hangfire so you don’t even have to change a setting anywhere. This would allow complete segregation of the different versions which you will no doubt want. Specifically since the dashboard would get a little upset with different versions of the assembly containing the jobs.
Or you could use the schema approach (it depends a little as to what Storage DB you are using).
Personally I would suggest something like Mongo, reason being it would require absolutely no setup on the database side to accommodate a new version (with a new Prefix). Collections in Mongo are automatically created if they don’t exist (there is no such thing as a table schema to define which means all you need to create a new collection is a name for it). With something like SqlServer you would have to do additional setup work for each version on the DB side.
The different app versions all point the same one database. The difference between the servers is that the application code is different.
e.g. If i find a bug in one of my background jobs, I would fix the code and deploy it to a new server. The goal is to ensure that all future jobs will execute against the new code base (and not the existing ones)
My DB is SQL Azure but I ultimately don’t think it should affect the solution.
Then the Schema approach is the way to go probably. Due to differences in the various storage modules and DB’s there are differences in the implementations. So in that regard yes it does affect the solution. Mongo for instance does not really have the concept of schemas as it exists in SQL or Azure SQL hence why there the solution would be to use a prefix instead of a schema.
Again, the only complication I see is that in the case of Azure SQL you would have to ‘set up’ each new schema, which I think (have not used Hangfire with AzureSQL myself) happens automatically from the SqlServerStorage constructor. I mention this however because even though it is (from the looks of the documentation) automatic it means the credentials you use to connect to the AzureSQL DB do need the proper permissions to do this. Which is not always a given in corporate environments.
Completely separating the different versions, either with schema or prefix or different db, has some problems to it.
One problem is scheduled tasks. If you schedule a task to be executed in a week, you might want to deprecate old app versions by the time to jobb will be executed. Trying to execute a job scheduled in an old version can be problematic, but having the job stored in a ‘deprecated’ database wan’t help you…
Another problem is bugfixes. If you have a job scheduled, and it fails because of a bug, you might wanna deploy a new version with a fix. If the job is stored in a queue only processed by the old app version, there wan’t be much help in retrying the job…
What we ended up with, is to store the highest version number seen. Then we have a jobfilter that will ‘back out’ if the app processing the job has a lower version number by putting the job back into the queue and force a restart of the app. When the app start up again, it checks its own version number against the highest version seen. If it’s an old version, we wan’t connect to hangfire at all. Basically this will render the old versions useless from the point where a newer version is deployed.