Hangfire.Common.JobLoadException: Could not load the job

I am getting an intermittent error in Hangfire around a recurring job:

Hangfire.Common.JobLoadException: Could not load the job. See inner exception for the details. ---> System.TypeLoadException: Could not resolve type 'Company.Hangfire.Jobs.ZoneFiles.IGetCZDSZoneFilesJob' in assembly 'Company.Hangfire, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
   at System.Reflection.TypeNameResolver.GetType(String escapedTypeName, ReadOnlySpan`1 nestedTypeNames, TypeName parsedName)
   at Hangfire.Common.TypeHelper.TypeResolver(Assembly assembly, String typeName, Boolean ignoreCase)
   at System.Reflection.TypeNameResolver.GetType(String escapedTypeName, ReadOnlySpan`1 nestedTypeNames, TypeName parsedName)
   at System.Type.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError)
   at Hangfire.Common.TypeHelper.<>c.<DefaultTypeResolver>b__20_0(String name)
   at Hangfire.Common.TypeHelper.DefaultTypeResolver(String typeName)
   at Hangfire.Storage.InvocationData.<>c.<CachedDeserializeMethod>b__33_0(MethodDeserializerCacheKey key)
   at Hangfire.Storage.InvocationData.CachedDeserializeMethod(Func`2 typeResolver, String typeName, String methodName, String parameterTypes, Type& type, MethodInfo& methodInfo)
   at Hangfire.Storage.InvocationData.DeserializeJob()

   --- End of inner exception stack trace ---
   at Hangfire.Storage.InvocationData.DeserializeJob()
   at Hangfire.RecurringJobExtensions.TriggerRecurringJob(IBackgroundJobFactory factory, JobStorage storage, IStorageConnection connection, IProfiler profiler, RecurringJobEntity recurringJob, DateTime now)
   at Hangfire.Server.RecurringJobScheduler.ScheduleRecurringJob(BackgroundProcessContext context, IStorageConnection connection, String recurringJobId, RecurringJobEntity recurringJob, DateTime now)

However, the assembly is directly referenced in the project and the type exists?

I can still manually trigger the job and it runs fine.

It’s worth pointing out I have multiple job hosts (processes) each with different job remits that process different queues, but all the jobs share a common assembly so they should all know about the job type interfaces.

Would I need to update all job hosts at the same time if I add in a new job type? That’s the only thing I can think it’d be, one of the hosts not having the most recent shared library, even if it doesn’t handle the type.

Are all hosts equally responsible for queueing/scheduling recurring jobs to run by the appropriate worker?

Looks like a type-resolution mismatch between the assembly that enqueued the
job and the one the Hangfire server loads at execution time. “Intermittent” strongly suggests a rolling deployment: Maybe some server instances still run the OLD code that knows the old type, others the new?