Ended up figuring this out, of anyone comes across this.
What was happening is that the default invisibility timeout was at 30 minutes. The job was taking longer than 30 minutes, so hangfire assumed it had hung, cancelled it, and then requeued it. This would then happen indefinitely.
As explained in.
http://docs.hangfire.io/en/latest/configuration/using-sql-server.html
The fix is to increase this timeout, or recode the job to take less than 30 minutes.