"Stopping server components" - What's the cause?

question
1

I have a job which runs for longer than 5 minutes and it seems to hang every so often in the final stages.
Right around the time it seems to stall, hangfire shows “Stopping server components” in the log.

What could be causing this and how can I avoid it?

Part of my log is attached below:

2014-09-15 05:33:50.1255 Info HangFire.Server.ServerBootstrapper Stopping server components…
2014-09-15 05:33:50.1411 Info HangFire.SqlServer.ExpirationManager Server component ‘SQL Records Expiration Manager’ stopped.
2014-09-15 05:33:50.1411 Info HangFire.Server.RecurringJobScheduler Server component ‘Recurring Job Scheduler’ stopped.
2014-09-15 05:33:50.1411 Info HangFire.Server.ServerWatchdog Server component ‘Server Watchdog’ stopped.
2014-09-15 05:33:50.1411 Debug HangFire.Server.Worker Server component ‘Worker #3’ stopped.
2014-09-15 05:33:50.1411 Info HangFire.Server.ServerHeartbeat Server component ‘Server Heartbeat’ stopped.
2014-09-15 05:33:50.1411 Debug HangFire.Server.Worker Server component ‘Worker #1’ stopped.
2014-09-15 05:33:50.1411 Info HangFire.Server.SchedulePoller Server component ‘Schedule Poller’ stopped.
2014-09-15 05:33:50.1567 Debug HangFire.Server.Worker Server component ‘Worker #4’ stopped.
2014-09-15 05:33:50.1411 Debug HangFire.Server.Worker Server component ‘Worker #2’ stopped.
2014-09-15 05:33:50.1880 Debug HangFire.Server.Worker Server component ‘Worker #1’ finished successfully.
2014-09-15 05:33:50.2192 Debug HangFire.Server.Worker Server component ‘Worker #2’ finished successfully.
2014-09-15 05:33:50.2192 Debug HangFire.Server.Worker Server component ‘Worker #3’ finished successfully.
2014-09-15 05:33:50.2349 Debug HangFire.Server.Worker Server component ‘Worker #4’ finished successfully.
2014-09-15 05:33:50.2505 Info HangFire.Server.WorkerManager Server component ‘Worker Manager’ stopped.
2014-09-15 05:33:50.2661 Debug HangFire.Server.WorkerManager Server component ‘Worker Manager’ finished successfully.
2014-09-15 05:33:50.2661 Debug HangFire.Server.ServerHeartbeat Server component ‘Server Heartbeat’ finished successfully.
2014-09-15 05:33:50.2818 Debug HangFire.Server.ServerWatchdog Server component ‘Server Watchdog’ finished successfully.
2014-09-15 05:33:50.2974 Debug HangFire.Server.SchedulePoller Server component ‘Schedule Poller’ finished successfully.
2014-09-15 05:33:50.3130 Debug HangFire.Server.RecurringJobScheduler Server component ‘Recurring Job Scheduler’ finished successfully.
2014-09-15 05:33:50.3286 Debug HangFire.SqlServer.ExpirationManager Server component ‘SQL Records Expiration Manager’ finished successfully.
2014-09-15 05:33:50.3442 Info HangFire.Server.ServerBootstrapper Server component ‘Server Core’ stopped.
2014-09-15 05:33:50.3599 Debug HangFire.Server.ServerBootstrapper Server component ‘Server Core’ finished successfully.
2014-09-15 05:33:50.3755 Info HangFire.BackgroundJobServer HangFire Server stopped.
2014-09-15 05:33:53.5161 Info HangFire.SqlServer.SqlServerStorage Start installing HangFire SQL objects…
2014-09-15 05:33:53.6567 Info HangFire.SqlServer.SqlServerStorage HangFire SQL objects installed.
2014-09-15 05:33:53.6724 Debug HangFire.Server.ServerBootstrapper Starting server component ‘Server Core’…
2014-09-15 05:33:53.6724 Info HangFire.BackgroundJobServer Starting HangFire Server…
2014-09-15 05:33:53.6724 Info HangFire.BackgroundJobServer Using job storage: ‘SQL Server: haynes_repadmin’.
2014-09-15 05:33:53.6880 Info HangFire.BackgroundJobServer Using the following options for SQL Server job storage:
2014-09-15 05:33:53.6880 Info HangFire.BackgroundJobServer Queue poll interval: 00:00:15.
2014-09-15 05:33:53.7036 Info HangFire.BackgroundJobServer Invisibility timeout: 00:30:00.
2014-09-15 05:33:53.7192 Info HangFire.BackgroundJobServer Using the following options for HangFire Server:
2014-09-15 05:33:53.7192 Info HangFire.BackgroundJobServer Worker count: 4.
2014-09-15 05:33:53.7349 Info HangFire.BackgroundJobServer Listening queues: ‘default’.
2014-09-15 05:33:53.7349 Info HangFire.BackgroundJobServer Shutdown timeout: 00:00:15.
2014-09-15 05:33:53.7349 Info HangFire.BackgroundJobServer Schedule polling interval: 00:00:15.
2014-09-15 05:33:53.7505 Info HangFire.Server.ServerBootstrapper Server component ‘Server Core’ started.

2

Hello, thanks for the detailed description. Seems like your application pool was recycled, because the following lines show us that SqlServerStorage class instance was re-created.

2014-09-15 05:33:50.3755	Info	HangFire.BackgroundJobServer	HangFire Server stopped.	
2014-09-15 05:33:53.5161	Info	HangFire.SqlServer.SqlServerStorage	Start installing HangFire SQL objects...

You can enable Generate Recycle Event Log Entry settings for an application pool of your application to see what really is going on. After enabling these settings, your event log will include more details about application restarts.

1 Like
3

Thanks for this.

I’m on a shared host so I don’t have access to IIS. I’m trying to get their help to log this better and to better understand why the app pool is restarting so often, but I don’t like my chances of getting much help :frowning:

4

Hey, just an update on this.

I couldn’t get access to the necessary logs on the shared host so I did some local debugging and found the app pool memory limit would have been exceeded on the shared host.

Thanks for the help, I have to re-engineer my hangfire jobs now so they break up the process and recycle the memory better!