Job automatically rescheduled after 1 min when using MSMQ with DTC tr

bug?
,
1

I am using Hangfire 1.5.6 with MSMQ and DTC transactions (in order to use remote MSMQ). The problem is that every long running job (> 1 min) is automatically rescheduled after 1 min. This is also done in strange order: job is started again before old job has been canceled.
If I configure Hangfire not to use DTC transactions jobs are running OK.
In the source code I have noticed that MsmqDtcTransaction is opening TransactionScope. Does this scope needs to be open during job execution? What will happen with SQL server transaction log for long running jobs (this particular job is inserting a lot of data into DB)?
I have tried with setting transaction timeout in app.config (this also required changes in machine.config):
<system.transactions> <machineSettings maxTimeout="02:00:00"/> <defaultSettings timeout="02:00:00" /> </system.transactions>
with these changes job was running OK with DTC transactions on.

1 Like
2

I have the same problem. Is this a known issue?

3

We’re encountering the same problem. A long-running job is hitting the default transaction timeout of 1 minute when using MSMQ and DTC.

Could you make the timeout configurable in the code? Or is there some kind of alternative solution?

EDIT: At the moment we are experimenting with our own implementation of the MSMQ interface for Hangfire, one that doesn’t use transactions. Seems to be working better.

4

Guys, I’ve just committed a fix to this, will be released with 1.6.3 soon.

1 Like
5

I have tested version 1.6.5 and there is no more timeout of 1 min, instead there is timeout on 10 minutes, which is windows maximal transaction timeout. Did anyone try running job more than 10 minutes with DTC on?

6

That’s because there is the maxTimeout setting. You should set it using web.config or app.config files as written in the first post of this thread (sometimes you also need to set the allowExeDefinition setting as written here). To check whether it will work for you, you can use the following method. It’s a hack, so it is for testing purposes only.

internal static bool OverrideTransactionManagerMaximumTimeout(TimeSpan timeOut)
{
    try
    {
        var type = typeof(System.Transactions.TransactionManager);

        var cachedMaxTimeout = type.GetField("_cachedMaxTimeout", BindingFlags.NonPublic | BindingFlags.Static);
        var maximumTimeout = type.GetField("_maximumTimeout", BindingFlags.NonPublic | BindingFlags.Static);

        var syncProperty = type.GetProperty("ClassSyncObject", BindingFlags.NonPublic | BindingFlags.Static);
        if (syncProperty == null)
        {
            return false;
        }

        var syncObject = syncProperty.GetValue(null);

        if (syncObject == null || cachedMaxTimeout == null || maximumTimeout == null)
        {
            return false;
        }

        lock (syncObject)
        {
            cachedMaxTimeout.SetValue(null, true);
            maximumTimeout.SetValue(null, timeOut);
        }

        return true;
    }
    catch (Exception)
    {
        return false;
    }
}