MSDTC on server 'servername' is unavailable


#1

When MSDTC is not started, an error message is shown when opening /hangfire.axd/queues and you have at least 1 job queued. This seems to be a bug.

The full error message is:

HangFire An exception occurred
MSDTC on server 'servername' is unavailable.

System.Data.SqlClient.SqlException (0x80131904): MSDTC on server 'STA-ETF-003' is unavailable.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.TdsParser.TdsExecuteTransactionManagerRequest(Byte[] buffer, TransactionManagerRequestType request, String transactionName, TransactionManagerIsolationLevel isoLevel, Int32 timeout, SqlInternalTransaction transaction, TdsParserStateObject stateObj, Boolean isDelegateControlRequest)
   at System.Data.SqlClient.TdsParser.GetDTCAddress(Int32 timeout, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlInternalConnectionTds.GetDTCAddress()
   at System.Data.SqlClient.SqlInternalConnection.EnlistNonNull(Transaction tx)
   at System.Data.SqlClient.SqlInternalConnection.Enlist(Transaction tx)
   at System.Data.SqlClient.SqlInternalConnectionTds.Activate(Transaction transaction)
   at System.Data.ProviderBase.DbConnectionInternal.ActivateConnection(Transaction transaction)
   at System.Data.ProviderBase.DbConnectionPool.PrepareConnection(DbConnection owningObject, DbConnectionInternal obj, Transaction transaction)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at HangFire.SqlServer.SqlServerMonitoringApi.UseConnection[T](Func`2 action)
   at HangFire.SqlServer.SqlServerMonitoringApi.EnqueuedJobs(IEnumerable`1 jobIds)
   at HangFire.SqlServer.SqlServerMonitoringApi.<Queues>b__19(SqlConnection connection)
   at HangFire.SqlServer.SqlServerMonitoringApi.UseConnection[T](Func`2 action)
   at HangFire.SqlServer.SqlServerMonitoringApi.Queues()
   at HangFire.Web.Pages.QueuesPage.Execute()
   at HangFire.Web.RazorPage.TransformText(String innerContent)
   at HangFire.Web.RazorPage.ProcessRequest()
ClientConnectionId:74216bb0-529b-42d7-bc7c-99394aa0094e

#2

Thanks you for bug report, @henningst! HangFire does not require MSDTC service, unless you try to use it inside an explicit transaction started from your code using the TransactionScope class. Hm, but even in this case it should not be propagated to the MSDTC, when the same connection string is being used. This requires more investigation.

I’ve already added this bug to GitHub Issues – https://github.com/odinserj/HangFire/issues/120, it is already closed and scheduled for 0.9.1 version that will come this week.


#3

This issue was fixed in HangFire 0.9.1 – https://github.com/odinserj/HangFire/releases/tag/v0.9.1


#4

That fixed it! Thanks for the quick response! :slight_smile:


#5

I still this issue on v1.7.9. I also use Hangfire.Console. It fails when writing message to hangfire dashboard. Below is the stacktrace:

MSDTC on server 'COMPUTER' is unavailable.

System.Data.SqlClient.SqlException (0x80131904): MSDTC on server 'COMPUTER' is unavailable.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.TdsParser.TdsExecuteTransactionManagerRequest(Byte[] buffer, TransactionManagerRequestType request, String transactionName, TransactionManagerIsolationLevel isoLevel, Int32 timeout, SqlInternalTransaction transaction, TdsParserStateObject stateObj, Boolean isDelegateControlRequest)
   at System.Data.SqlClient.TdsParser.GetDTCAddress(Int32 timeout, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlInternalConnectionTds.GetDTCAddress()
   at System.Data.SqlClient.SqlInternalConnection.EnlistNonNull(Transaction tx)
   at System.Data.SqlClient.SqlInternalConnection.Enlist(Transaction tx)
   at System.Data.SqlClient.SqlInternalConnection.EnlistTransaction(Transaction transaction)
   at System.Data.SqlClient.SqlConnection.EnlistTransaction(Transaction transaction)
   at Hangfire.SqlServer.SqlServerStorage.<>c__DisplayClass35_0`1.<UseTransaction>b__0(DbConnection connection)
   at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)
   at Hangfire.SqlServer.SqlServerStorage.UseTransaction[T](DbConnection dedicatedConnection, Func`3 func, Nullable`1 isolationLevel)
   at Hangfire.SqlServer.SqlServerStorage.UseTransaction(DbConnection dedicatedConnection, Action`2 action)
   at Hangfire.SqlServer.SqlServerWriteOnlyTransaction.Commit()
   at Hangfire.Console.Storage.ConsoleStorage.AddLine(ConsoleId consoleId, ConsoleLine line)
   at Hangfire.Console.Server.ConsoleContext.AddLine(ConsoleLine line)
   at PakPos.Job.EmailImportJob.OnProcessedAction(PerformContext context, DateTime startExecutionTime, Int64 batchNumber, ReceivingAccount account, EmailImportError error, List`1 results) in C:\Sources\PakPos\PakPos.WebApi\Job\EmailImportJob.cs:line 208
   at PakPos.WebApi.Job.EmailImportJob.<>c__DisplayClass12_0.<ExecuteAsync>b__1(ReceivingAccount account, EmailImportError error, List`1 results) in C:\Sources\PakPos\PakPos.WebApi\Job\EmailImportJob.cs:line 67
   at PakPos.Domain.EmailImport.EmailImport.<>c__DisplayClass18_1.<<ExecuteAsync>b__3>d.MoveNext() in C:\Sources\PakPos\PakPos.Domain\EmailImport\EmailImport.cs:line 161
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at PakPos.Domain.EmailImport.EmailImport.<ExecuteAsync>d__18.MoveNext() in C:\Sources\PakPos\PakPos.Domain\EmailImport\EmailImport.cs:line 204
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at PakPos.WebApi.Job.EmailImportJob.<ExecuteAsync>d__12.MoveNext() in C:\Sources\PakPos\PakPos.WebApi\Job\EmailImportJob.cs:line 59
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()