Hangfire Discussion

One queue for the whole farm, and one queue by server


#1

Hi,
I have the following setup:

  • Multiple web servers running in a load balanced setup.

  • Each of these can enqueue and dequeue jobs with the following code in Startup.cs :

    app.UseHangfire(config =>
    {
    config.UseSqlServerStorage(“mySqlConnectionString”);
    config.UseServer();
    config.UseAuthorizationFilters(new AuthorizationFilter() { /* blah blah… */ });
    });

This works well.
Now i need to periodically execute some non-critical maintenance task on each server, say, for example, cleaning the “temp” dir on each server at nearly 2:00 am each day.

Is there a way to do that ? I didn’t found a way to say Hangfire to process 2 queues : a common one (from the SQL storage), and a server-specific one (from a local storage or whatever…).
Thanks !


#2

Interesting feature, I’ll think how to integrate it to the current architecture. For now you can use simple System.Threading.Timer.


Can we have tasks that can be run on all servers?
#3

Thanks for your reply !
For now I use Hangfire for farm jobs and Quartz.net for server-specific non critical jobs.
Hope to see that feature in Hangfire in the future to have a unique homogeneous solution :smile:


Does the hangfire smart enough not to consume all the resource and hang the hosting server?
#4

I’ve had the same ‘problem’ and came up with the following solution:

  1. when configuring the server add a seperate queue per scheduler instance:
 var serverOptions = new BackgroundJobServerOptions()
                    {
                        ServerName = GetSchedulerInstanceServerName(),
                        Queues = new[] { InstanceName, "Default" } //<==instance name is some static property
                    };

where InstanceName is something unique to the server (e.g. the hostname + port)

  1. Add a jobfilter which extracts the queue from a job parameter:
  internal class UseQueueFromParameterAttribute: JobFilterAttribute, IElectStateFilter
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="QueueAttribute"/> class
        /// using the specified queue name.
        /// </summary>
        /// <param name="queue">Queue name.</param>
        public UseQueueFromParameterAttribute(int parameterIndex)
        {
            this.ParameterIndex = parameterIndex;
        }

        public int ParameterIndex { get; private set; }

        public void OnStateElection(ElectStateContext context)
        {
            var enqueuedState = context.CandidateState as EnqueuedState;
            if (enqueuedState != null)
            {
                enqueuedState.Queue = context.Job.Arguments[ParameterIndex].Replace("\"",string.Empty);
            }
        }
    }

Example usage:

  internal class MaintenanceJob
    {
        [UseQueueFromParameter(0)] //extracts the queuename from parameter with index 0
        public void Execute(string queuename)
        {
            //do some maintenance on the server
        }
    }

//now enqueue the job using the statically configured instance name
 BackgroundJob.Enqueue<MaintenanceJob(x=>x.Execute(InstanceName));

EDIT:
please keep in mind that a queue name may only contain lowercase characters, numbers or underscores and has a maximum length of 20 characters…


Using One hangfire DB for multiple applications
Different Queues having different Worker counts
Can we have tasks that can be run on all servers?
How to handle recurring jobs running each one on multiple servers
#5

I would like to add an amen to the ability to run a task on each server.


#6

Sadly, the Timer solution only works for basic jobs. The true power of Hangfire lies in the ability to pass parameters to tasks. I would love to be able to enqueue a job to run on every server in response to a certain event seen only on one of the servers (e.g. an http request in a web farm), and pass some arguments to this job.


#7

I did something similar, but I didnt use the Job Filter. I am not exactly sure that does. Can you elaborate on UseQueueFromParameterAttribute

also say you have 5 agents. do you need to run the following code 5 times
BackgroundJob.Enqueue<MaintenanceJob(x=>x.Execute(InstanceName));