Hangfire Discussion

Dashboard API For Monitoring with PRTG

dashboard
Tags: #<Tag:0x00007f84ff998f50>

#1

Hi everyone,
I wanted to monitor the Hangfire clusters we have running without doing any further development. The numbers I wanted to monitor were in the Hangfire dashboard. The totals of the different status. I was able to identify the end point that is used /hangfire/stats

The dashboard sends POST to this end point asking for specific metrics.

So you can test this with POSTMAN or Fiddler.
The post should be application/x-www-form-urlencoded
You will also need to provide Basic Auth if you have turned that on ( I sure hope so) if you are using a different Auth, then you would need to accommodate for that as well.

Then in your Body you would have multiple “metrics[]” Keys, each one has the metric that you want.
Here are a few of the options that I discovered
enqueued:count
servers:count
failed:count
recurring:count
awaiting:count
succeeded:count
processing:count

Now you know where the end point is you can setup your monitoring system to check with hangfire dashboard API to make sure nothing crazy is happening with your system…

Ill post steps on how to do it with PRTG once I have mine working.


#2

Hey j3rm,

Have you had any luck with getting this to work with PRTG? i’m looking to implement the same.


#3

I have had some success. I’m still fine tuning what I have found. I will document it and post it on this thread.
I should have something in a few days.


#4

I think I finally have all the tiny details worked out. Thanks to the support team at PAESSLER, I could not have figured out all of this.
Here is what mine looks like I have not set error thresholds on all of the metrics but that will vary per user.

So in order to get this to work, you first need to setup the template on the PRTG servers.
In this folder
C:\Program Files (x86)\PRTG Network Monitor\Custom Sensors\rest
Create a new text file named hangfire.dashboard.template No TXT on the end.
Then put this as the contents of the file.

{
  "prtg": {
    "result": [ 
      {
        "channel": "Server Count",
        "value": $["servers:count"]["value"]
      },
      {
        "channel": "Failed Count",
        "value": $["failed:count"]["intValue"]
      },
      {
        "channel": "Recurring Count",
        "value": $["recurring:count"]["value"]

      },
      {
        "channel": "Awaiting Count",
        "value": $["awaiting:count"]["intValue"]

      },
      {
        "channel": "Succeeded Count",
        "value": $["succeeded:count"]["intValue"]

      },
      {
        "channel": "Processing Count",
        "value": $["processing:count"]["intValue"]
      },
      {
        "channel": "Processing Count",
        "value": $["processing:count"]["intValue"]

      },
      {
        "channel": "Retries Count",
        "value": $["retries:count"]["intValue"]

      },
      {
        "channel": "Deleted Count",
        "value": $["deleted:count"]["intValue"]

      },

      {
	"channel": "Enqueued Count",
	"value": $( $["enqueued:count-or-null", ""]["value"] )( len(@) == 0 ? 0 : @[0] )
      },
    ]
  }
}

Careful when copying this as Windows some times like to change the quote marks for start quotes and end quotes, which is annoying but it can trip you up and waste a lot of time.

If you are using a PRTG cluster, you will need to copy this file to all of the servers in the clusters, if you have probes that are not part of the cluster I am pretty sure the probes need this file too…

Then in the PRTG Web UI, Create a new Server and set the IPv4 Address/DNS Name to the DNS/ or IP of you hangfire dashboard. Do not include the full url, just the DNS/IP part. Save it, Then add a new Sensor to the server.“Rest Custom BETA” is what it is called right now, I assume the beta will disappear in a few months.

For the new sensor
Give it a name.
Set the Request Mode to POST.

Set the post data to “metrics[]=enqueued:count-or-null&metrics[]=servers:count&metrics[]=failed:count&metrics[]=recurring:count&metrics[]=awaiting:count&metrics[]=succeeded:count&metrics[]=processing:count&metrics[]=retries:count&metrics[]=deleted:count” . -With out the quote marks

Set the Request Protocol should be set based on how you have it setup ( https or not)
Set Authentication to Basic Auth and then provide the login info

Set REST query to “/hangfire/stats” . - again with out the quotes.

Set REST Configuration to hangfire.dashboard.template, if its not in the dropdown something is not right on your server with the file name you created at the top of these instructions.

Save.

Refresh the sensor till it has data. If you see errors in PRTG then your config might be incorrect and may need some slight adjusting.

Please remember you can configure your HF dashboard many different ways HTTPvsHTTPS, different types of auth. If you are doing something outside of the box this may not work, you will have to troubleshoot your own complexities.

For us we use hard coded Basic Auth and HTTPS. I’m sure someone somewhere would debate if thats good or bad but to each their own opinion.

I hope this helps someone out there as PRTG monitoring Hangfire is a no brainer.


#5

Very nice, i’ll give this a go over the next few days & let you know the results.


#6

Can confirm this works perfectly. We use cookie auth for this page, so i had to add custom HTTP headers and disable auth.