Speed of job inserts into db

question
1

I generate many jobs and add them like this:

foreach (var job in jobs)
{
    BackgroundJob.Schedule(() => Console.WriteLine(job.id), job.StartTime);
}

Inserting around 300 jobs takes almost 60 seconds. Is there any way to speed this up?

2

You can use Parallel.ForEach as all methods of the BackgroundJob class are thread-safe. This may gain the performance a bit. In my case it gave 1370 jobs/sec instead of 569 jobs/sec.

Parallel.For(0, 10000, i =>
{
    BackgroundJob.Schedule(() => Empty(), TimeSpan.FromSeconds(5));
});

However, your numbers are too low, and looks like you have very slow server or network latency. If you are unable to change this, you can also look at spawning background jobs from other background jobs:

public void CreateJobs(int jobsId)
{
    var jobs = _database.GetByJobsId(jobsId);

    foreach (var job in jobs)
    {
        BackgroundJob.Schedule(() => Console.WriteLine(job.id), job.StartTime);
    }
}

// Somewhere
BackgroundJob.Enqueue(() => CreateJobs(id));
1 Like
3

Hangfire.Pro.Redis on the same machine gives much better numbers ā€“ 5000 jobs/sec.