Recurring Jobs are ending after about 12 minutes

question
1

Hi,
I m testing hangfire with two recurring jobs. Everything is ok.

Then I m testing more than two recurring jobs. Starting tests the jobs are not working anymore after about 12 minutes.

The recurrent jobs are starting every one, two or three minutes with optional delay.

My target is to introduce about 30 recurrent jobs.

What is going wrong ?

We are unsing hangfire with .net console c# client.

Thanks in advance.

Best regards

2

Are there any error messages/logging? When you say they are no longer working do you mean a job itself stops after 12 minutes or Hangfire stops triggering jobs after 12 minutes?

Would it be possible that a job is still running while that same schedule triggers another of that same job to run? Would they interfere with each other?

3

First of all thanks for reply.

Are there any error messages/logging? No

When you say they are no longer working do you mean a job itself stops after 12 minutes or Hangfire stops triggering jobs after 12 minutes? Hangfire is working, the jobs are not working anymore.

Would it be possible that a job is still running while that same schedule triggers another of that same job to run? Yes, one of it.

Would they interfere with each other? It should not.

After setting WorkerCount = Environment.ProcessorCount * 20 it worked.

Then some of the jobs are beginning after 50 Minutes to start multiple at the to same time.

The scheduling is running but the jobs are stopping again at 1 hour 10 minutes.

It is not programmed to start same jobs several times and it is not wanted.

Is there a option to prevent this behaviour ?

My code looks llke this: ( one of five jobs )

recurringJob.AddOrUpdate<Commands.Job1>(“job1”, x => x.run(con.createSession()), “*/2 * * * *”);

Best regards

4

I have stabilized the system. It is running now 2 hours. But after one hour I think the jobs beginning to duplicate.

The setting ```
[DisableConcurrentExecution(timeoutInSeconds: 60)]

effected no improvement. The systems behaviour changed totally and I took it out.

5

Have changed the memory settings into GitHub - HangfireIO/Hangfire.InMemory: In-memory job storage for Hangfire with transactional implementation

Now it runnings almost 4 hours. Then it stopps again What we can do ?

Coincidentally I saw that after 7 hours stopping it began again to start running.

What is going on ? ( cron : “*/2 * * * *”)

6

Hi, I wanted to share my experience. We used the MemoryStorage - solution, no database.

  1. Our .NET console app had delay methods and that caused asynchronity arround hangfire. Hangfire is not liking something like this. That caused that we had to use full worker size. Now we are working with a little worker size after changing this. Worker size has to be the amount of the jobs. If not you have a problem that has to be solved.

  2. Some settings for the InMemory - solution are very important:

         MemoryStorageOptions storageOpts = new MemoryStorageOptions()
     {
         JobExpirationCheckInterval = TimeSpan.FromHours(3),
         FetchNextJobTimeout = TimeSpan.FromDays(2)
     };

GlobalJobFilters.Filters.Add(new AutomaticRetryAttribute { Attempts = 0, LogEvents = false, OnAttemptsExceeded = AttemptsExceededAction.Delete });

The class attribut //[DisableConcurrentExecution(timeoutInSeconds: 60)] has caused many failures. So we do not used that.

  1. Check if you have installed all needed Hangfire Libraries. After checking this the app ran much better.

  2. It could be that your solution does not run continously. So you have to think about a restart for the Hangfire server after a certain period.

Hope above mentioned points helped someone.