Hangfire Multi-tenancy Implementation Feedback

feature
1

Good Morning,

I’m working on a trial run of Hangfire to execute some background jobs for the company I work for. Our application is multi tenant, with separate databases for every client site. We use StructureMap to inject the connection string. I apologize in advance for the long post, but other topics around this haven’t gotten many replies, I suspect because the posters didn’t share what they’ve tried on their own.

Basically here’s how our architecture works

  1. sitename.ourdomain.com is hit
  2. We parse out sitename
  3. Our application uses a master connection string in the web.config to then hit a database that has connection strings for all sites based on sitename as the unique key

We have the following environments
Primarily sharing this so it’s understood that dev’s background server might have code that prod doesn’t. Nothing unique to us, I’m sure most of you face this challenge.

  1. Dev (the entire app/db runs on a dev server, and developers use this for their local machines too)
  2. Preview - Our 2 week publishes after every sprint
  3. Sandbox - every 6 weeks, pre-production
  4. Production - every 6 weeks, 2 weeks trailing sandbox

Here’s my plan for our Web/Mobile Services/API’s queuing background jobs
I think I have a solid understanding of how this should work, but I’m open to feedback.

  1. Hangfire tables will exist on every single database
  2. There will be an IBackgroundJob interface that has a method ExecuteJob(struct jobType, Guid uid), along with one that works similarly for a scheduled job. The Uid will be the Guid based ID in a database table with information about the background job so that the job can start cold.
  3. IBackgroundJob will have a class that’s injected which calls the db initialization at runtime. This is slightly different than the documentation as we cannot call it at app start as it would only initialize for the first client that is hit. We will call BackgroundWorker.Dowork(connectionString, jobType, uid).
    GlobalConfiguration.Configuration.UseSqlServerStorage(“injected connection string goes here”);
    ** We cannot call BusinessLayer.DoWork() like we could if we weren’t using dependency injection because so many of our dependencies are relying on the sitename variable from the architecture overview above.

For the background job worker, I’m a bit lost and need some help

  1. I know at a minimum we need 1 background job worker per environment.
  2. I know we need to load connection strings for every client in that environment. I found a sample of that in this thread Initialization of Hangfire servers for multitenants application

Here’s where I get lost though. Two primary questions

  1. How do we prevent noisy neighbor? What happens if SiteA queues 100,000 request and Site B does 1, can we prevent SiteA from taking over the entire queue?
  2. We have a half dozen servers that are load balanced. Do I need to do anything special to ensure the system service from Server 1 doesn’t step on toes of system service from Server 2?
2

Hi @james.garrett,

We exactly ran into the same scenario, did you get any solution?

Request to share.

Thanks in advance.

3

I have the same situation: a multi tenant environment with a separate database per tenant. How did you implement this in the end?