Hangfire batch/continuations to run on an if-else

feature
1

We have a requirement to provide the end user capability to submit a request through the web application (UI) and receive a file via email containing matching records from DB. Due to the search coming back with a huge number of records and letting the user wait, we decided to implement this solution by submitting the user request in background and use Hangfire pro to implement the background solution. (using batch continuations)

Overall solution to generate a file had following steps:

  • Step 1: Create a Hangfire job to find total count of matching records in the DB.
  • Step 2: Start a new batch of Hangfire child jobs. # of child jobs = # of matching records in Step 1/ Total number of IDs we want each child job to process. For example, IF Step 1 returned 75,000 records and each child job should handle 5000 IDs then number of child jobs = 75000/5000 = 15. Each child job gets a offset and limit representing the chunk of IDs in the total result set they should process,
  • Step 3: Kick off the batch jobs. Each child job does a search in DB to find the matching IDs and then calls various microservices to find the data to include in the output file, upload the output file to an S3 bucket in AWS. Each child job only writes data for the IDs it handled to the output file it created.
    Step 4: When all child batch jobs have finished, then:
    ** If all child jobs succeed, then run a batch continuation job #1 that downloads all output files generated by the child batch job, merges them into a single file and then sends an email to the user with a link to the merged file.
    ** If all or any one child job fails, we send a failure email to the user indicating the output file cannot be generated. Please submit a new request.

Overall logic is like this:e

First step below is the creation of the batch jobs (using Hangfire pro). numberofJobsToCreate is the total number of child batch jobs to create. Each child batch job gets its own share of offset and limit.

Towards only one of the continuations run.

I think the overall solution works as intended but I just want to make sure I am using the batch continuations state as intended and my solution is working to the requirements based on the correcntess as per Hangfire documentation than by fluke.
var batchId = BatchJob.StartNew(x =>
{
int offset = 0;
int limit = 10000;
for (int childJobCounter = 0; childJobCounter < numberOfJobsToCreate; childJobCounter++)
{
x.Enqueue(() => RunCollectDataJobs(s3FolderName, offset + childJobCounter * limit, limit));
}
});

BatchJob.ContinueWith(batchId, x =>
{
x.Enqueue(() => PerformFailureContinuations(batchId));
},
null,
BatchContinuationOptions.OnlyOnCompletedState);

    BatchJob.ContinueWith(batchId, x =>
        {
            x.Enqueue(() => PerformSuccessContinuations(batchId));
        }, 
        null, 
        BatchContinuationOptions.OnlyOnSucceededState);