Excessive SQL Server log file growth?


#1

After leaving my app running all weekend, I’ve now found that it’s creating an awfully large SQL Server log file. Does anyone have any ideas why this might be?

Additionally, what permissions should the Hangfire user have on the database?


#2

Need to deeply investigate this issue. This may be caused by constant update queries that are being used to fetch background jobs. What size of log file are you talking about? If it is too much, there should be recommendation or warning on using Hangfire.SqlServer with databases with full recovery model.


#3

Regarding permissions - I’ve not made a list yet. SqlServerStorage constructor by default executes migration scripts (creates schema, tables, indexes, can be disabled). Daily job uses read/write operations on records and nothing else.


#4

The log file it created was 16gb+, over (about) 2 days. This was with 1.1.0-alpha2. Actually rolled back to 1.0.2 which seemed to address the issue, although we’re still talking 50mb+ over 3 hours. Also this was with using the latest version of Dapper available on NuGet.

As you can appreciate from our private email on my usage of Hangfire, I’m creating A LOT of jobs throughout the day (1000s), and I think that this is part of the problem, because I think that I’m creating one-time jobs constantly throughout the day when in fact I almost probably don’t need to, and I need a slight redesign which more cleverly controls what recurring jobs are created and when.

However, to do that, I think that I’ll need #167, #173 and (possibly) #181 resolving.

What you’ve said about databases in full recovery mode is quite interesting. My app actually uses two databases - one is the main system database, and the other one just contains Hangfire (basically because I wasn’t allowed to install Hangfire into the main system database), and having just checked, the database Hangfire is in is in full recovery mode. It doesn’t have to be, and if you think that will improve the log file usage, I’ll get that changed ASAP.

Separately, 1.1.0-alpha2 has much better CPU usage on the server than 1.0.2 :smile:, so you can understand how I’d like to continue using it.


#5

I am also facing the same issue, how did you solve the issue ?