Number Of Servers in the Dashboard Is Incorrect After Stopping The Windows Service

question
,
1

I’m using a Windows Service to process hangfire jobs and SQL Server to storage.

Normall If I stop the service, dashboard displays the number of Servers as 0. But sometimes it displays 1 even after I stopped the service, and when I restart the same service it displays 2 servers.

2

That usually means you’re not properly disposing a BackgroundServer object.

3

I’m disposing it using _backgroundJobServer.Dispose(); method. And out of 10 times, it works 9 times correctly. Just the odd 1 time this issue occurs.

4

Dispose() call waits at most ShutdownTimeout and then returns, regardless of the tasks were really completed. But the bootstrap thread waits for all worker threads to complete before removing the server record, so it may be still waiting when the process terminates and ungracefully kills all background threads.

Try increasing ShutdownTimeout option value to give tasks more time to complete (default is 15 seconds).

3 Likes
5

Indeed, looks like your problem is caused by a timeout during performing a graceful shutdown. It may be caused by a long-running background job. What background jobs do you have? Are you using cancellation tokens?

1 Like
6

No. I’m not using cancellation tokens.

7

That server will be removed automatically after some time (5-10 minutes), when you have at least one Hangfire server running. So that’s not a problem (but we can remove the confusion by highlighting this fact in Dashboard UI). The problem is that some of your background jobs delay the server shutdown, and it’s better to use cancellation tokens to allow them to gracefully exit on server shutdown without adding any delays. After restart, these jobs will be processed again.

1 Like