Enqueue vs Schedule along with ContinueJobWith gives drastically different execution time

question
1

TargetFramework - net5.0
Hangfire package versions:

  • Hangfire 1.7.24
  • Hangfire.AspNetCore 1.7.24
  • Hangfire.Core 1.7.24
  • Hangfire.PostgreSql 1.8.5.4

Executed locally in Debug mode (VS2019) on Windows 10.

There are three APIs which are executing the same scenario but in a different ways uding Hangfire api. Scenario itself is pretty straitforward - it enqueues MainJob and then establishes two continuation jobs for OnlyOnSucceededState of the MainJob.

Sources:

------------ ‘Test - 1’ API ------------

public async Task<ActionResult<JobProcessingDetails>> Test1()
{
    Console.WriteLine("------ Test - 1 ------");
    var execTimer = new Stopwatch();
    execTimer.Start();

    var executionOptions = new ProcessingOptions()
    {
        // some options goes here
    };

    var jobId = BackgroundJob.Enqueue<MainJob>(p => p.Execute(executionOptions.UpdateOptions, executionOptions.ExecutionOptions));
    BackgroundJob.ContinueJobWith<Job1>(jobId, p => p.Execute(executionOptions), JobContinuationOptions.OnlyOnSucceededState);
    BackgroundJob.ContinueJobWith<Job2>(jobId, p => p.Execute(executionOptions), JobContinuationOptions.OnlyOnSucceededState);

    var jobExecutionDetails = new JobProcessingDetails
    {
        JobId = jobId,
        State = JobState.Enqueued
    };

    execTimer.Stop();
    Console.WriteLine($"Execution time, ms  - {execTimer.Elapsed.TotalMilliseconds}");
    return Ok(jobExecutionDetails);
}

------------ ‘Test - 2’ API ------------

Same as above but jobs were settled up differently, main difference is that MainJob has been Scheduled initially instead of Enqueued:

var jobId = BackgroundJob.Schedule<MainJob>(p => p.Execute(executionOptions.UpdateOptions, executionOptions.ExecutionOptions), TimeSpan.FromSeconds(2));
Expression<Action<Job1>> func1 = x => x.Execute(executionOptions);
Expression<Action<Job2>> func2 = x => x.Execute(executionOptions);
BackgroundJob.ContinueJobWith(jobId, func1);
BackgroundJob.ContinueJobWith(jobId, func2);

------------ ‘Test - 3’ API ------------

Same as in ‘Test - 1’ but continuations were established asynchronously in some tricky way as for me (don’t judge me hard).

var jobId = BackgroundJob.Enqueue<MainJob>(p => p.Execute(executionOptions.UpdateOptions, executionOptions.ExecutionOptions));
await SetContinuationsAsync(jobId, executionOptions);

private Task<int> SetContinuationsAsync(string jobId, ProcessingOptions executionOptions)
{
    TaskCompletionSource<int> tcs1 = new TaskCompletionSource<int>();

    Task.Factory.StartNew(() =>
    {
        tcs1.SetResult(15);
        BackgroundJob.ContinueJobWith<Job1>(jobId, p => p.Execute(executionOptions), JobContinuationOptions.OnlyOnSucceededState);
        BackgroundJob.ContinueJobWith<Job2>(jobId, p => p.Execute(executionOptions), JobContinuationOptions.OnlyOnSucceededState);
    });

    return tcs1.Task;
}

Execution logs:

------ Test - 1 ------
MainJob - Start.
MainJob - End.
Job1.Execute()
Execution time, ms - 12152.5929
Job2.Execute()

------ Test - 2 ------
Execution time, ms - 6886.1804
MainJob - Start.
MainJob - End.
Job1.Execute()
Job2.Execute()

------ Test - 3 ------
Execution time, ms - 1235.992
MainJob - Start.
MainJob - End.
Job1.Execute()
Job2.Execute()

As you can see execution time differs drastically, my main concern is why ‘Test - 1’ takes so long and why jobs execution order is so weird as well?

2

I can’t really speak to the execution times. I would suggest trying it many times to see if the results are consistent.

Job 1 & 2 are waiting for the main job. This means they can both run once the main job finishes if you have multiple worker threads. This also means job 2 could complete first if job 1 takes longer. The job executions occurs outside of the test execution so it is possible for the test to complete before the jobs complete and vice versa.

3

Yes, I executed APIs multiple times and the time is average value of multiple executions.

Main advantage that Hangfire provides is to enqueue the job (fire and forget), once JobID is here we do not need to wait for anything, especially for job to be executed, as second Test execution shows, where we not waiting for MainJob to be executed because it is scheduled for later, so this does not prevent Job1 and Job2 to be also established by Hangfire.

I don’t understand why there is a difference in case of Enqueue and Schedule of jobs, what Hangfire does under the hood is serializing necessary stuff and returns job id, so later it could be used to setup continuations. I expected Enqueue as FireAndForget call, but it seems not the case at all.