Debugging BackgroundJob.Enqueue

question
1

My applications runs in .NET 4.5.1 ASP.NET MVC5 and uses SignalR. I want to implement Hangfire with signalR to submit and monitor an application to Hangfire server, and monitor progress using Hangfire Dashboard.

I have a batch file that runs a job on the server. I run it by executing the batch file through a Windows Process. This works but I want a more robust production environment.

  • I wrote a job progress tracker that uses SignalR to poll app progress from the SignalR.Hub. This works. (I write my own Job Progress details and store it in a MongoDB log collection)

  • I was implementing QuartzNet to schedule the job, to provide a Job Monitoring and Management Dashboard, when I discovered Hangfire. Hangfire is exactly what I was working towards.

  • I got your sample applications working.

  • I got your Job Management Dashboard working.

  • I am now working on using Hangfire with SignalR to submit the background job, using the line of code shown below (I am using SQLServer for job statistics storage). Here’s the line of code that submits the background job.

    BackgroundJob.Enqueue(
                 () =>
                     KESConsoleRunner.Program.ExecuteBackgroundJob(job.RunString));

  • KESConsoleRunner.Program.ExecuteBackgroundJob is a class in the background job which runs the consoleApplication class that runs the background job. Job.RunString is the parameter that was passed through the bat file to run the consoleApplication in the previous architecture, which worked.

  • The line of code above processes with no exception, but I don’t see the job running on the Hangfire Server.
    ** The compute processing should peak when it runs. It doesn’t. And I see no indication that that the job was queued in the Dashboard.

  • I reread your documentation but I am unclear how to set up the code in the server app to start the job and read the jobrun detail. I expect I am overthinking it.
    ** How do I write the code which will asynchronously submits the job to the server, from the front end webapp such that the server application runs outside the webApp context?
    *** I can see the job run details recorded in SQL Server and that it’s status is enqueued.

2

I should have included the Controller. Here’s the code:

[System.Web.Mvc.HttpPost]
    public virtual ActionResult SubmitTransaction(HttpPostedFileBase file)
    {
       if (file != null)
       {
           //Save to safe area
           string exportFilePath =
               Path.Combine(
                   System.Web.HttpContext.Current.Server.MapPath(Startup.SafeBrowserFileArea + file.FileName)
                       .Replace(" ", "%20"));
           file.SaveAs(exportFilePath);
           db.SaveTransactionToDB(Startup.appDatabase.GetCollection<IndexTransaction>("IndexTransactions"),
               exportFilePath);
           ViewBag.Message = db.UploadSuccessful ? "Upload Successful. Transaction Submitted" : "Upload Failed";

           var job = new Job(Guid.NewGuid().ToString());
           job.Source = exportFilePath;

           //sep 23 2014 dbm  bool TransactionJobSubmitted = db.SubmitTransactionJobToServer(exportFilePath, file.FileName);
           //query mongdb for progress report
           /* SignalR Progress Tracker (polling)
       var job = JobManager.Instance.DoJobAsync(j =>
        {
            for (var progress = 0; progress <= 100; progress++)
            {
                j.ReportProgress(progress);
            }
        });

        return Json(new
        {
            JobId = job.Id,
            Progress = job.Progress
        }); 
       */
           //sep 13 2014 dbm 
           job = job ?? new Job(Guid.NewGuid().ToString());
           if (ModelState.IsValid)
           {
               job.RunString =
                   "KESConsoleRunner  \"E:\\Transactions\\|Transaction_Run_20140814_Set-1(RevisedModels)_KES|Transaction_Run_20140814_Set-1(RevisedModels)_DocType\"";
               job.CreatedAt = DateTime.UtcNow;
               db.Jobs = db.Jobs ?? new ConcurrentList<Job>();
               db.Jobs.Add(job);
               db.SaveChanges();

               var sqlServerStorage =
                   new SqlServerStorage(@"Server=BAINMCKAY-PC;Database=master;Trusted_Connection=True;");

              //MSMQ does not work on Windows 7 WOrksation config: sqlServerStorage.UseMsmqQueues(@".\Private$\hangfire{0}", "default", "critical");
               
	   JobStorage.Current = sqlServerStorage;
               
    	   BackgroundJob.Enqueue(
                   () =>
                       KESConsoleRunner.Program.ExecuteBackgroundJob(job.RunString));

               RedirectToAction("Job", new {id = job.Id});
           }
       }
       return RedirectToAction("Index");

   }
3

Got it. I added the following line for out-of -OWIN pipe processing on backend server

               JobStorage.Current =
                    new SqlServerStorage(@"Server=BAINMCKAY-PC;Database=Hangfire;Trusted_Connection=True;");

//here’s the startup code snippet
app.UseHangfire(config =>
{
config.UseAuthorizationFilters();
config.UseSqlServerStorage(“Hangfire”); //MSMQ does not work on Windows 7 workstation. Need to add Windows Server 8 to the dev network: UseMsmqQueues(@".\hangfire-{0}");
config.UseServer();
//added this line for Out of OWIN scope processing on backend server.
JobStorage.Current =
new SqlServerStorage(@“Server=MACHINENAME;Database=Hangfire;Trusted_Connection=True;”);
});

//and the Controller Code snippet
JobStorage.Current = new SqlServerStorage(@“Server=MACHINENAME;Database=Hangfire;Trusted_Connection=True;”);

               BackgroundJob.Enqueue(
                   () =>
                       KESConsoleRunner.Program.ExecuteBackgroundJob(job.RunString));

Now I need to harness n backend servers as a Scaleout Backplane operating in parallel, on a schedule fulfillment basis. SignalR provides a MongoDB Scaleout backplane. I will start with SQLServer. Get that working. Then will get Redis working and then will leverage Redis to implement MongoDB, unless MongoDB is already implemented (I haven’t found it).

4

Below I explain my thinking about how to achieve the level of scalability I need to reach. Any thoughts which can shed some light to steer me in a more productive direction would be well received:

Objective:
Now that I have Hangfire working as a Windows Server on a targeted server, I need to distribute jobs across registered arbitrary backend servers plugged into the SignalR server backplane for optimal scaleout parallel processing

Approach:
I have Hangfire server now running background processes on a sever. I want to create a SignalR Backplane into which you can plug in arbitrary servers transparently to web apps using the service. From what I can see, the server that Hangfire posts jobs to, would be the ScaleOut backplane. On that server, would run HangfireServer, from from a Windows Service (this is working now). To use that server as a HangfireBackplane, I need to have it read jobs from the Jobs DB, and distribute those to run on a set of job queues, where each queue was registered to a physical sever, thus mapping of job queues to physical backend servers for load balancing/distributed parallel processing.

Am I on the right teach? Is there a better way?