Microsoft Visual Studio Enterprise 2019: v16.8.2
We are using Hangfire for Fire-and-forget jobs, most of the time our jobs are processed very quickly which means that jobs are only in enqued state for a short period of time.
But since 2nd of Nov jobs with status enqueued got stuck for forever. It is showing count of 854\8 and continuously increasing. If we restart our hangfire service when this problem occurs all enqued jobs will be processed, but this time even restart is also not working.
We can see in our logs that Hangfire throws:
2021-11-09 23:54:57,143 [Worker #19] ERROR Hangfire [UpdateFileProcessingStatus] - Exception in FileProcessingStatus: . Error Message: An error occurred while executing the command definition. See the inner exception for details. Inner Exception: Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.