I have fixed the below issue by updating the proper values in apphost, but after doing some changes in my code (not related to hangfire). The hangfire is not able to find the Assembly to resume. Are there any specific considerations apart from what document has suggested to fix this? I am using hangfire 1.3.1
{“FailedAt”:“2015-01-21T14:10:08.0184218Z”,“ExceptionType”:“System.IO.FileNotFoundException”,“ExceptionMessage”:“Could not load file or assembly ‘CCIActivityWorkflow, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null’ or one of its dependencies. The system cannot find the file specified.”,“ExceptionDetails”:“System.IO.FileNotFoundException: Could not load file or assembly ‘CCIActivityWorkflow, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null’ or one of its dependencies. The system cannot find the file specified.\r\nFile name: ‘CCIActivityWorkflow, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null’\r\n at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMarkHandle stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName, ObjectHandleOnStack type)\r\n at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName)\r\n at System.Type.GetType(String typeName, Boolean throwOnError, Boolean ignoreCase)\r\n at Hangfire.Storage.InvocationData.Deserialize()\r\n\r\nWRN: Assembly binding logging is turned OFF.\r\nTo enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.\r\nNote: There is some performance penalty associated with assembly bind failure logging.\r\nTo turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].\r\n”}
I am able to fix this issue by not hitting the same hangfire database from my local PC application as part of debug. I think hangfire DB gets confused between the servers when same DB is being hit by same application from different server boxes. This works now but I am still not sure if that was the actual fix. Thanks!
Looks like it is possible only if we’ll add a Troubleshooting page or something like this. I don’t see any other ways to make this more friendly, do you?
Citing @leon’s comment, how is this error any friendly? For scalability reasons, the same db for the same app’s multiple instances should not give these kind of errors. All three of my servers have a heartbeat but, they go off and on saying that they couldn’t find the project’s file… Then magically they work again until they don’t. Really strange case. I do use a loadbalancer, but that shouldn’t make such difference as they all are different instances of the app.
I also use different queues for all the apps that work from the same db, so my A app has a queue named A and B has a queue named B. They don’t care about eachothers’ queues and for further information, I configured the app to keep the original queue if any job fails for some reason.
So this issue does not make much sense to me, do you have any insights?
So what is the fix for this, because I get this error intermittently. The reoccurring job will work one minute and the next, it will say the method doesn’t exist.
Having struggled through this issue recently I thought it worth clarifying:
ALL Hangfire servers connecting to the same database must have the dlls for ALL Jobs stored in that db.
Even if you use different queues to separate the jobs processed by different servers point 1 is still valid. If a job fails during its first execution then any Hangfire server may inspect it to decide if it should be processed. Since queues aren’t part of the Job definition saved in Hangfire db, servers will still try to look at jobs that were originally run on different queues, and then you get a FileNotFoundException if the server doesn’t have the job definition.
Further discussion of the queues problem here and here and here.
Thanks for this - I was suspecting the behavior you note in point #2, based on some issues I’ve been having with split servers and failed jobs. Good to know that I’m not crazy (or any more so, I suppose).