Problem with Delayed Job - I am in big trouble

odinserj · December 3, 2014, 2:22pm

Is BackgroundJobServer instance is active (please see the Servers page)? Maybe it is failing to execute, what shows the job details page?

farid_masood · December 3, 2014, 2:29pm

farid_masood · December 3, 2014, 2:31pm

farid_masood · December 3, 2014, 2:33pm

odinserj · December 3, 2014, 2:48pm

Hm. Do you have any logs? Looks like SchedulePoller class have some issues.

farid_masood · December 3, 2014, 2:49pm

I have no idea about logs, where can I get them?

odinserj · December 3, 2014, 2:54pm

Currently here – http://docs.hangfire.io/en/latest/deployment-to-production/adding-logging.html

farid_masood · December 4, 2014, 11:33am

It is very confusing, which versions to use to configure the logger as per the above help article. I am using the Hangfire (.net4) which uses the Common.Logging (2.2.0) which Nlog and Nlog adapters should i use?

We are using the Log4Net and it would be nice if I know which version of Log4Net are compatible. Log4Net adapters are using the Common.Logging > = 2.3 and getting compile error…

odinserj · December 5, 2014, 1:52pm

Answered you in another thread:

odinserj · December 5, 2014, 2:49pm

Can you downgrade the Common.Logging to 2.2.0 at least temporary to see the logs?

kkchan · February 25, 2015, 4:12am

Hi
I am using Hangfire 1.3.4, and cant get schedule working. What mistake could you spot? tq

Remark: WriteX is a static method.

var jobid = BackgroundJob.Schedule(() => WriteX(“GGG”,
DerivedHangfireToken.Null), TimeSpan.FromSeconds(6));

{“EnqueueAt”:“2015-02-25T04:14:05.1189185Z”,“ScheduledAt”:“2015-02-25T04:13:59.1189185Z”}

odinserj · February 25, 2015, 11:56am

Do you have any logs? What displays the dashboard?

kkchan · February 25, 2015, 12:52pm

2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker Manager’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker Manager’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Server Heartbeat’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Server Heartbeat’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Schedule Poller’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Schedule Poller’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Server Watchdog’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Server Watchdog’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Recurring Job Scheduler’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Recurring Job Scheduler’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘SQL Records Expiration Manager’…
.
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘SQL Records Expiration Manager’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker Manager’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Server Heartbeat’…
2015-02-25 08:50:26 [INFO] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker Manager’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Schedule Poller’…
2015-02-25 08:50:26 [INFO] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Server Heartbeat’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Server Watchdog’…
2015-02-25 08:50:26 [INFO] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Schedule Poller’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Recurring Job Scheduler’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #1’…
2015-02-25 08:50:26 [INFO] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Server Watchdog’ started.
2015-02-25 08:50:26 [INFO] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Recurring Job Scheduler’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘SQL Records Expiration Manager’…
.
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #1’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #2’…
2015-02-25 08:50:26 [INFO] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘SQL Records Expiration Manager’ started.
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #2’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #3’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.SqlServer.ExpirationManager) Removing outd
ated records from table ‘Counter’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #3’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #4’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #4’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #5’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #6’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #5’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #6’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #7’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #7’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #8’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #8’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #9’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #9’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #10’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #10’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #11’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #11’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #12’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #12’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #13’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #14’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #13’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #15’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #14’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #16’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #15’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #16’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #17’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #17’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #18’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #19’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #18’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #19’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting a new thread for server component ‘Worker #20’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Starting server component ‘Worker #20’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #1’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #2’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #1’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #3’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #2’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #4’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #3’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #5’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #4’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #6’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #5’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #7’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #6’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #8’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #7’ started.
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #8’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #9’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #10’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #9’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #11’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #10’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #12’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #11’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #13’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #12’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #14’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #13’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #15’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #16’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #17’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #18’…
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #19’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #16’ started.
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #17’ started.
2015-02-25 08:50:26 [INFO] (Hangfire.Server.ServerWatchdog) 1 servers were remo
ved due to timeout
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #19’ started.
2015-02-25 08:50:26 [DEBUG] (Hangfire.SqlServer.ExpirationManager) Removing outd
ated records from table ‘Job’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #18’ started.
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #14’ started.
2015-02-25 08:50:26 [TRACE] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Sending start request for server component ‘Worker #20’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #15’ started.
2015-02-25 08:50:26 [DEBUG] (Hangfire.Server.AutomaticRetryServerComponentWrappe
r) Server component ‘Worker #20’ started.
2015-02-25 08:50:26 [DEBUG] (Hangfire.SqlServer.ExpirationManager) Removing outd
ated records from table ‘List’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.SqlServer.ExpirationManager) Removing outd
ated records from table ‘Set’…
2015-02-25 08:50:26 [DEBUG] (Hangfire.SqlServer.ExpirationManager) Removing outd
ated records from table ‘Hash’…

kkchan · February 25, 2015, 1:09pm

Hi
From dashboard, it shown exception “Can not find the target method.”. for schedule()

However, it I use enqueue, it is working perfectly.

var jobid = BackgroundJob.Enqueue(() => WriteX(“GGG”,
DerivedHangfireToken.Null));

odinserj · February 25, 2015, 2:33pm

Are you using separate application (console, windows service) for processing background jobs or have multiple applications using the same database?

kkchan · February 26, 2015, 2:22am

Hi
I have tried this out in 2 environments, Console, and also Unit Testing.
Both doesn’t work.
And Yes, both shared the same database, however, neither the job scheduled in Console nor Unit Testing are working.

Please advice. thank you

odinserj · February 26, 2015, 10:12am

Can you enumerate all of your applications and instances using Hangfire with shared database in the following way:

Front-end Web Application
Has a reference to assembly that contains WriteX method: Yes
Creates Background Jobs: Yes
Runs Hangfire Server: No

Back-end Web Application
Has a reference to assembly that contains WriteX method: No
Creates Background Jobs: No
Runs Hangfire Server: Yes

odinserj · February 26, 2015, 10:13am

The problem is likely related to a missing assembly reference ensure all your applications have a reference to assembly that contains WriteX method.

kkchan · February 27, 2015, 4:18am

Hi
After more testing, I found the exception only happen in below scenario,
In my unit testing,

Start Server
EnqueueJob
Assert.IsTrue(ManualResetEvent.WaitOne(2000)) which will be set in my method.
Stop Server

The problem here is, as I posted in another thread, SQL Server poll interval is 10 seconds. Before I know about this, I only wait for 2 secs, then will stop the server.

It seem like, if enqueued job not executed and server stopped, program would hit exception as posted above. I even hit some other weird exception either.

After I changed my code to wait more than 10 secs, everything back to normal.
So, my concern is, if that is any pending enqueued Job, and server stopped. Would it throw exception?

odinserj · July 24, 2015, 3:16pm