We have a multi-tenant SaaS application using Hangfire with Redis as the job storage engine. We’re currently running >50k recurring jobs across ~13k tenants and experiencing Redis CPU pressure after adding recent jobs.
Current Setup:
-
Mix of two patterns (seeking to standardize)
-
~10 different job types that need tenant-specific execution
-
Some jobs have tenant-configurable schedules (every 15/30/60 minutes)
-
Others run on fixed schedules (hourly/daily)
Question: Which approach scales better for our scenario?
Option 1: Per-tenant recurring jobs
-
Create individual recurring jobs for each tenant/job-type combination
-
Example:
RecurringJob.AddOrUpdate($"sync-tenant-{tenantId}", ...) -
Currently results in 50k+ recurring jobs
Option 2: Single recurring job with dynamic enqueueing
-
One recurring job per job type that queries active tenants
-
Enqueues individual background jobs per tenant
-
Example: One
RecurringJob→ iterates tenants →BackgroundJob.Enqueue<TenantSync>(tenantId)
Concerns:
-
Redis CPU utilization increasing with job count
-
Need to add more jobs soon, potentially doubling our recurring job count
-
Tenant-specific scheduling requirements (makes Option 2 more complex)
Looking for guidance on:
-
Performance/scalability implications of each pattern at this scale
-
Redis overhead differences between 50k+ recurring jobs vs. frequent background job enqueueing
-
How tenant-specific schedules affect the architecture choice
-
Any Hangfire best practices for multi-tenant applications at scale