Abort transaction on remote MSMQ HF 1.6.6

bug?
1

Hello.

We use HF in our service, we like it wery much, but now we have a problem: we started second server (windows server 2012 r2) with remote MSMQ and on second server many jobs run twice/3 time with such error in log (Transaction aborted), in job itself there is no eror.

I know that there was bug like this until 1.6.3 version, but we use 1.6.6 on both servers.

2016-10-26 11:47:39,850 [Worker #d79d1a61] Hangfire.Server.Worker DEBUG Error occurred during execution of 'Worker #d79d1a61' process. Execution will be retried (attempt 1 of 2147483647) in 00:00:00 seconds. System.Transactions.TransactionAbortedException: Транзакция была прервана. ---> System.TimeoutException: Тайм-аут транзакции --- Конец трассировки внутреннего стека исключений --- в System.Transactions.TransactionStatePromotedAborted.BeginCommit(InternalTransaction tx, Boolean asyncCommit, AsyncCallback asyncCallback, Object asyncState) в System.Transactions.CommittableTransaction.Commit() в System.Transactions.TransactionScope.InternalDispose() в System.Transactions.TransactionScope.Dispose() в Hangfire.Server.Worker.Execute(BackgroundProcessContext context) в Hangfire.Server.AutomaticRetryProcess.Execute(BackgroundProcessContext context)

2

By default all the distributed transactions initiated using System.Transactions have the default timeout of 10 minutes. If we try to override this timeout in the code or in the app.config file it will be adjusted down to 10 minutes. To get around the problem:

  1. Added the following section in the machine.config file to set the maxTimeout to 1 hour :

     <configuration>
  <system.transactions>
    <machineSettings maxTimeout=”01:00:00″ />
  </system.transactions>
 </configuration>

  2. We modified the System.Transactions section in machine.config by setting the allowExeDefinition attribute to “MachineToApplication” (from MachineOnly). After this, we were able to override the machine.config timeout of 10 minutes in the code and that allowed the transaction to complete successfully.

     <sectionGroup name=”system.transactions” type=”System.Transactions.Configuration.TransactionsSectionGroup, System.Transactions, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089, Custom=null”>
     <section name=”defaultSettings” type=”System.Transactions.Configuration.DefaultSettingsSection, System.Transactions, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089, Custom=null” />
     <section name=”machineSettings” type=”System.Transactions.Configuration.MachineSettingsSection, System.Transactions, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089, Custom=null” allowDefinition=”MachineOnly” allowExeDefinition=”MachineToApplication” />
 </sectionGroup>

Or alternatively you can use reflection to change the limit:

public static class TransactionmanagerHelper
{
    public static void OverrideMaximumTimeout(TimeSpan timeout)
    {
        //TransactionScope inherits a *maximum* timeout from Machine.config.  There's no way to override it from
        //code unless you use reflection.  Hence this code!
        //TransactionManager._cachedMaxTimeout
        var type = typeof(TransactionManager);
        var cachedMaxTimeout = type.GetField("_cachedMaxTimeout", BindingFlags.NonPublic | BindingFlags.Static);
        cachedMaxTimeout.SetValue(null, true);

        //TransactionManager._maximumTimeout
        var maximumTimeout = type.GetField("_maximumTimeout", BindingFlags.NonPublic | BindingFlags.Static);
        maximumTimeout.SetValue(null, timeout);
    }
}
3

Thank you very much, we will test it on transaction different time-length, multiple test with 1 minutes, 11 minutes, 61 minutes. Transaction max timeout set in 24 hours (we have really long processing)

4

You even can set it to “00:00:00” or simply “0” to have an infinite max timeout, but this isn’t a good value.

5

31 minutes tests passed successfully many times … and… it seems it is working!!!

1 Like
6

Is it normal, that DTC has so many aborted ops (75366) ?