Deleting job with OnStateElection & Cancellation token

Tags: #<Tag:0x00007fb505561988>

Hi,
I have implemented a filter on OnStateElection

public void OnStateElection(ElectStateContext context)
{
          var deleteState = context.CandidateState as DeletedState;
            if (deleteState != null)
            {
                DoSomething();
            }
}

When I enqueue my job, I use the following call :

BackgroundJob.Enqueue<T>(job => job.Execute(paramString, cancellationToken)); 

The job is pretty straight forward

 for (var i = 0; i < Int32.MaxValue; i++)
{
     await Task.Delay(1000).ConfigureAwait(false);
     cancellationToken?.ThrowIfCancellationRequested();           
}

When job is deleted using the Dashboard UI Delete button, filter is not called.
Moreover, the cancellationToken is not set to Cancel and job completely stop to execute.

What I am trying to acheive :

  • Delete a job in the UI
  • Do some cleanup, send requests to external systems (potentially queue a new job)
  • Delete the job
1 Like

Isn’t it more correct to do it in OnStateApplied? Your filter might be not last in the Election chain, so it is not reliable to check state there.

Also note that if you’re changing state of already running job, it will reflect changes only on next ThrowIfCancellationRequested() call, so it is possible that your job will be still running when you start your cleanup tasks.

Even with OnStateApplied, my filter is not called (I took the sample from the forum where all hooks are defined).
When dashboard delete a job, not a single breakpoint is hit (there is a breakpoint on each override available)

Good point about the cleanup. What would you suggest ?
thanks !

1 Like

Do you have Dashboard and Server in the same process?

No, they are running on 2 different IIS Website.
Does that make a difference ? I checked the Dashboard code and it looks like it calls a route that does pretty much this

var job = GetJobById(id)
BackgroundJob.Delete(job)

And which one do you have your filter installed at?

The filter is defined in a common project that gets referenced by both sites.
It looks like you have a lot of experience with job deletion & cancellation token. What should I look for to debug that?

Thanks !

I’ve just made a quick test for your case with Hangfire 1.6.6.

Created a simple filter:

public class ApplyStateFilter : IApplyStateFilter
{
    public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
    {
        Console.WriteLine("State applied: {0}", context.NewState.Name);
    }

    public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
    {
        Console.WriteLine("State unapplied: {0}", context.OldStateName);
    }
}

Created a simple job, pretty much like yours:

public class TestTask
{
    public async Task Work(IJobCancellationToken cancellationToken)
    {
        Console.WriteLine("Start of Task");
        try
        {
            for (var i = 0; i < 300; i++)
            {
                await Task.Delay(1000).ConfigureAwait(false);
                cancellationToken?.ThrowIfCancellationRequested();
            }
        }
        finally
        {
            Console.WriteLine("End of Task");
        }
    }
}

Registered a filter on app startup:

GlobalConfiguration.Configuration.UseFilter(new ApplyStateFilter());

Started job:

BackgroundJob.Enqueue<TestTask>(x => x.Work(null));

And got rather expected output:

-- enqueued job --
State unapplied:
State applied: Enqueued
State unapplied: Enqueued
State applied: Processing
Start of Task
-- pressed Delete in Dashboard --
State unapplied: Processing
State applied: Deleted
End of Task
-- the end --

So it basically works, you just need to debug your specific case.

Start with checking whether your filter is really registered and gets called when you enqueue the job.

1 Like

Even by registering the filter in both my dashboard project and my runner project and by using your exact code (except for task enqueuing), I’m still unable to hit the breakpoint for “OnStateApplied” or “OnStateUnapplied” when I hit the delete button.

1 Like

Do you have any other custom filters which may interfere with that?

Are you attaching a debugger to IIS worker? Is IIS hosting Dashboard limited to a single worker process (so it cannot eventually spawn another one to process request)?

Try iisreset, in case it has cached some libraries, as it sometimes doesn’t reload an updated dll (for example, from GAC).

No more ideas at the moment.

After creating a filter, you should apply it to your job, either locally:

[YourFilterAttribute]
public void YourMethod() {}

or globally:

GlobalJobFilters.Filters.Add(new YourFilterAttribute());

Did you perform one of these steps? Can you post here your filter, your method and your Hangfire configuration?

So first of all, I wanted to thank you for your support.

It happens that the web project running the Dashboard was on “Start without debugging” so my breakpoint was never hit. Behavior was “weird” since the breakpoint was hit correctly when the task was created and then enqueued and I though that the dashboard would trigger the filter on the “job server” per ser, thus triggerring the actions.

That being said, I can now hook inside my filter by simply making the site in mode “Start”.
Thanks for your help and advices on “where” I should hook my code to do the cleanup !

1 Like

I don’t know why, but in my case seems like this is not triggered for a recurring job, when deleting from dashboard recurring job page. Rest of methods in my attribute work properly:

    public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
    {
        if (context.NewState is DeletedState) // NOT TRIGGERED
        {
           ...
        }
    }