One queue for the whole farm, and one queue by server

question
1

Hi,
I have the following setup:

  • Multiple web servers running in a load balanced setup.

  • Each of these can enqueue and dequeue jobs with the following code in Startup.cs :

    app.UseHangfire(config =>
    {
    config.UseSqlServerStorage(“mySqlConnectionString”);
    config.UseServer();
    config.UseAuthorizationFilters(new AuthorizationFilter() { /* blah blah… */ });
    });

This works well.
Now i need to periodically execute some non-critical maintenance task on each server, say, for example, cleaning the “temp” dir on each server at nearly 2:00 am each day.

Is there a way to do that ? I didn’t found a way to say Hangfire to process 2 queues : a common one (from the SQL storage), and a server-specific one (from a local storage or whatever…).
Thanks !

6 Likes
2

Interesting feature, I’ll think how to integrate it to the current architecture. For now you can use simple System.Threading.Timer.

3

Thanks for your reply !
For now I use Hangfire for farm jobs and Quartz.net for server-specific non critical jobs.
Hope to see that feature in Hangfire in the future to have a unique homogeneous solution :smile:

1 Like
4

I’ve had the same ‘problem’ and came up with the following solution:

  1. when configuring the server add a seperate queue per scheduler instance:
 var serverOptions = new BackgroundJobServerOptions()
                    {
                        ServerName = GetSchedulerInstanceServerName(),
                        Queues = new[] { InstanceName, "Default" } //<==instance name is some static property
                    };

where InstanceName is something unique to the server (e.g. the hostname + port)

  1. Add a jobfilter which extracts the queue from a job parameter:
  internal class UseQueueFromParameterAttribute: JobFilterAttribute, IElectStateFilter
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="QueueAttribute"/> class
        /// using the specified queue name.
        /// </summary>
        /// <param name="queue">Queue name.</param>
        public UseQueueFromParameterAttribute(int parameterIndex)
        {
            this.ParameterIndex = parameterIndex;
        }

        public int ParameterIndex { get; private set; }

        public void OnStateElection(ElectStateContext context)
        {
            var enqueuedState = context.CandidateState as EnqueuedState;
            if (enqueuedState != null)
            {
                enqueuedState.Queue = context.Job.Arguments[ParameterIndex].Replace("\"",string.Empty);
            }
        }
    }

Example usage:

  internal class MaintenanceJob
    {
        [UseQueueFromParameter(0)] //extracts the queuename from parameter with index 0
        public void Execute(string queuename)
        {
            //do some maintenance on the server
        }
    }

//now enqueue the job using the statically configured instance name
 BackgroundJob.Enqueue<MaintenanceJob(x=>x.Execute(InstanceName));

EDIT:
please keep in mind that a queue name may only contain lowercase characters, numbers or underscores and has a maximum length of 20 characters…

3 Likes
5

I would like to add an amen to the ability to run a task on each server.

1 Like
6

Sadly, the Timer solution only works for basic jobs. The true power of Hangfire lies in the ability to pass parameters to tasks. I would love to be able to enqueue a job to run on every server in response to a certain event seen only on one of the servers (e.g. an http request in a web farm), and pass some arguments to this job.

7

I did something similar, but I didnt use the Job Filter. I am not exactly sure that does. Can you elaborate on UseQueueFromParameterAttribute

also say you have 5 agents. do you need to run the following code 5 times
BackgroundJob.Enqueue<MaintenanceJob(x=>x.Execute(InstanceName));

8

Would love to see this implemented. Have a situation where we refresh a local cache on a schedule but would also like to be able to trigger a refresh on every server via an api without having to implement that logic myself.