Hangfire Best Practice - One Big Background task or many small ones

question
1

I need to send emails in a background task. What I’m not sure about is if I should use one big background task or multiple small ones.

One big one:

//One Big Task: call task that will get all subscribers of a post and send a notification email
_jobClient.Enqueue<IDiscussionEmailService>(e => e.Send(post.Id));

//Or, Multiple Small Tasks: Get all Subscribers and then batch mulitple background tasks for each email
public void Send(int postId) { var subscribers = _subSelector.GetSubscribers(topic); var bactchIdNotification = BatchJob.StartNew(x => { foreach (var sub in subscribers) { x.Enqueue<IDiscussionEmailService>(y => y.SendDiscussionNotification(sub.EmailAddress, post.Details)); } }); }

Thanks for the help!

1 Like
2

Depends on what a successful job means for you I guess. What would happen in your example if one job fails? You can always let the jobs retry, so in your example if one send fails it would retry sending all of the mails…

3

If one email fails, I don’t care if other email in the same “group” have successfully been sent, but I will eventually want to retry to send the failed emails. I don’t ever want to send duplicate emails.

I could do one big job, but then I’d have to make sure that if the job fails, I don’t resend emails that were already sent successfully.

I could do one job per email, so if a job fails, it won’t trigger a retry on the other emails/jobs. It will only try to retry itself.

So, I’m guessing there isn’t really a best practice? It’s just however you want to implement it?
For some reason, the thought of having potentially many thousands of one email background jobs sounds strange. So, I just wanted to double check. Thanks!

4

When it comes to sending emails, I would look into the email providers feature set.

Instead of sending thousands of emails one by one - in one or many jobs - I would create the list of recipients and the message and hand it over to the email provider, and then let it be their problem to handle errors as far as possible, and then tell me for which recipients they could not send the message.

If the content of the message is customized for each recipient, most email providers will handle that in a ‘mail merge’ fashion: You give them a message with some mail merge fields, and provide values for each recipient.

I have experience with only SendGrid myself, but have seen that the same functionality is available with other providers.

At least worth to consider :slight_smile: