CPU cores usage

question
1

Hi,

I would like to know the Usage of CPU cores in Hangfire. I need decide between TPL (Task parallel library) or Hangfire to perform background jobs and as per MSDN, TPL provide two primary benefits:

  1. More efficient and more scalable use of system resources
  2. More programmatic control than is possible with a thread or work item.

source: https://msdn.microsoft.com/en-us/library/dd537609(v=vs.110).aspx

1 Like
2

TPL is not persistent (i.e. app domain recycle or process shutdown will cause loss of all queued/pending tasks), where-as Hangfire supports persistent storage for fault tolerance and recovery.

Just like TPL, you can configure the the degree of parallelism with Hangfire too: http://docs.hangfire.io/en/latest/background-processing/configuring-degree-of-parallelism.html

3

Hi Nick,

Good to hear from your side. Nice response,

After reading from above mentioned link regarding degree of parallelism:
If i am not wrongly understand that Worker Thread excutes on next available core of CPU. Number of Worker thread(s) can be configure using BackgroundJobServerOptions. Another question arise on my mind i.e. number of worker thread will be created on server start event.
Is it possible to create worker thread dynamically (i.e. when i need to perform job only then i create worker thread and on completion of task dispose the worker thread).

Please let me know if i am wrong.

4

Can you give more details about the problem you’re trying to solve ? It would be easier to give more applicable answers with some context.

5

Hi Andrew,

I am not facing any problem right now :smile: , i am just doing fesibility study of Hangfire and other Job scheduler to use in our web / desktop application. Till now, i found Hangfire and TPL is most appropriate candidate. Below mentioned are my checklist:

  1. Should be scalable.

  2. Should be configuration (i.e. Responsibility and Job can be added using config file, number of worker suld be equal to x = (Environment.ProcessorCount * 5) or x = (number of Jobs) configured whichever is less).

  3. High performance.

  4. Robustness (i.e. ESB (Enterprise Service Bus) compatible, Other systems, in an enterprise, can use for background job processing seamlessly)

  5. Testable framework (i.e using of Mocking and IoC for Jobs / Responsibilities).

Hope this will give you more appropriate detail regarding what i am looking for.

Regards

Yogesh Sharma

6

Hi Yogesh,

I would advise that you come up with a series of typical scenarios that you think you will need in your project as I think you will then find it easier to compare different frameworks and libraries.

The reason why I ask is because Hangfire and the TPL are both aimed at solving different problems. Hangfire is designed to handle persisted job queuing and distributed processing while the TPL is more typically aimed at performing ad-hoc parallel processing on the same machine.

I can go through your checklist and give some indication on each point and make some assumptions, but like I say the two are not entirely comparable.

  1. Both are scalable. The TPL parallelises tasks by either executing them sequentially or on a threadpool that is proportional in size to the number of cores - In this case you are ultimately limited by the number of cores that you have. Hangfire processes tasks both across cores within a single machine and also across multiple servers - In this case you are limited by both the number of cores / machines and the performance of the job storage engine / database / Redis. Whether or not one or the other is suitable depends upon what you intend to do.

  2. Number of workers is adjustable for both, but I don’t quite understand why you would want to adjust the number of workers to be related to the number of jobs - Surely this is a variable number ? If you have more workers than jobs, they will just sit idle, periodically checking for new jobs.

  3. This is somewhat covered under point #1, to answer this further you would need to know what your exact requirements are for this. I think the only way to truly know is to benchmark it with a representative workload. On a single machine, the TPL is likely to be more performance but Hangfire can scale out across a server farm if need be.

  4. Hangfire is designed around robustness and guarantees that your jobs will be run at least once. The TPL does not offer persistence or robustness of jobs unless you write this part yourself. You are likely to end up writing something similar to what Hangfire does in order to make TPL job processing robust.

  5. Ultimately both just run the methods you ask, so there’s no reason why your jobs can’t be fully testable using either system. Hangfire supports dependency injection containers out of the box but the TPL would require you to handle this yourself.

Hope that helps.