MSDTC exception

bug?
1

Hi,

We are getting the below exception referring to MSDTC. I found another post related to MSDTC stating that if we don’t use explicit transactions, we should not need MSDTC. We are using HangFire version 1.2. Can you tell whether we are doing anything wrong?

Thanks,
Vlad

{“FailedAt”:“2014-12-15T10:37:32.0438861Z”,“ExceptionType”:“System.Data.Entity.Core.EntityException”,“ExceptionMessage”:“The underlying provider failed on Open.”,
“ExceptionDetails”:“System.Data.Entity.Core.EntityException: The underlying provider failed on Open. —>
System.Transactions.TransactionManagerCommunicationException:
Network access for Distributed Transaction Manager (MSDTC) has been disabled. Please enable DTC for network access in the security configuration for MSDTC using the Component Services Administrative tool. —>
System.Runtime.InteropServices.COMException: The transaction manager has disabled its support for remote/network transactions. (Exception from HRESULT: 0x8004D024)
at System.Transactions.Oletx.IDtcProxyShimFactory.ReceiveTransaction(UInt32 propgationTokenSize, Byte[] propgationToken, IntPtr managedIdentifier, Guid& transactionIdentifier, OletxTransactionIsolationLevel& isolationLevel, ITransactionShim& transactionShim)
at System.Transactions.TransactionInterop.GetOletxTransactionFromTransmitterPropigationToken(Byte[] propagationToken)
— End of inner exception stack trace —
at System.Transactions.Oletx.OletxTransactionManager.ProxyException(COMException comException)
at System.Transactions.TransactionInterop.GetOletxTransactionFromTransmitterPropigationToken(Byte[] propagationToken)
at System.Transactions.TransactionStatePSPEOperation.PSPEPromote(InternalTransaction tx)
at System.Transactions.TransactionStateDelegatedBase.EnterState(InternalTransaction tx)
at System.Transactions.EnlistableStates.Promote(InternalTransaction tx)
at System.Transactions.Transaction.Promote()
at System.Transactions.TransactionInterop.ConvertToOletxTransaction(Transaction transaction)
at System.Transactions.TransactionInterop.GetExportCookie(Transaction transaction, Byte[] whereabouts)
at System.Data.SqlClient.SqlInternalConnection.EnlistNonNull(Transaction tx)
at System.Data.ProviderBase.DbConnectionInternal.ActivateConnection(Transaction transaction)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource1 retry)
at System.Data.SqlClient.SqlConnection.Open()
at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.<>c__DisplayClass1.b__0()
at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[TResult](Func1 operation) at System.Data.Entity.Core.EntityClient.EntityConnection.Open() --- End of inner exception stack trace --- at System.Data.Entity.Core.EntityClient.EntityConnection.Open() at System.Data.Entity.Core.Objects.ObjectContext.EnsureConnection() at System.Data.Entity.Core.Objects.ObjectContext.ExecuteInTransaction[T](Func1 func, IDbExecutionStrategy executionStrategy, Boolean startLocalTransaction, Boolean releaseConnectionOnSuccess)
at System.Data.Entity.Core.Objects.ObjectQuery1.<>c__DisplayClassb.<GetResults>b__9() at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[TResult](Func1 operation)
at System.Data.Entity.Core.Objects.ObjectQuery1.GetResults(Nullable1 forMergeOption)
at System.Data.Entity.Core.Objects.ObjectQuery1.<System.Collections.Generic.IEnumerable<T>.GetEnumerator>b__0() at System.Lazy1.CreateValue()
at System.Lazy1.LazyInitValue() at System.Data.Entity.Internal.LazyEnumerator1.MoveNext()
at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable1 source) at System.Linq.Queryable.SingleOrDefault[TSource](IQueryable1 source, Expression1 predicate) at CloudConnect.Portal.Services.PartnerMessageService.GetSiteIdFromExternalId(String externalSiteId, Object auditDto, Nullable1 pmpGuid)
at CloudConnect.Portal.Services.PartnerMessageService.ProcessMessage(MeterData meterData, PartnerMessageType messageType, String identifier, String payload, String externalSiteId, String serviceName, Nullable1 serviceId, Nullable1 pmpGuid)
at CloudConnect.Portal.Services.PartnerMessageService.ProcessPartnerMessage(PartnerMessageDto value, String sdkVersion, String userName, Guid pmpGuid)
at CloudConnect.Portal.Services.PartnerMessageService.<>c__DisplayClassb.b__a(PartnerMessageDto v)
at System.Linq.Enumerable.WhereSelectListIterator2.MoveNext() at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer) at Hangfire.States.SucceededState.SerializeData() at Hangfire.SqlServer.SqlServerWriteOnlyTransaction.<>c__DisplayClass8.<SetJobState>b__7(SqlConnection x) at Hangfire.SqlServer.SqlServerWriteOnlyTransaction.Commit() at Hangfire.States.StateChangeProcess.ApplyState(ApplyStateContext context, IEnumerable1 filters)
at Hangfire.States.StateChangeProcess.ChangeState(StateContext context, IState toState, String oldStateName)”}

2

Looks like you are passing an IEnumerable<T> as an argument to your background job and looks like there is a lazy loading enabled for that collection. Instead of enumerating lately, use ToList() or ToArray methods for your collection to load records before background job creation:

// Instead of this:
BackgroundJob.Enqueue(() => SomeMethod(someCollection));
// Do this:
BackgroundJob.Enqueue(() => SomeMethod(someCollection.ToArray()));
3

Big thanks for such a quick reply. Below is the code that is creating the job. It’s WebApi method getting the values parameter from HTTP body. It does not seem to me that it should be lazy loaded. I can add .ToList() to the values parameter if that makes sense. Can you please explain the reasoning behind the lazy loading?

Thanks,
Vlad

    public ApiResponse Batch([FromBody]List<Dto.V3.PartnerMessageDto> values)
    {
        BackgroundJob.Enqueue(() => _partnerMessageService.ProcessPartnerMessage(values, null, User.Identity.Name, null));
4

Looked at the call stack more detailed – this problem may be caused by return value, since the following line is in the call stack.

at Hangfire.States.SucceededState.SerializeData() at

What type are you returning? If it is an entity, it is better to return more simple types, e.g. identifiers or dtos. In this case lazy loading can open a new nested connection in a transaction which lead to transaction escalation which lead to MSDTC.

5

Alright, you found it! I am doing

values.Select(x => something(x))

and returning the IEnumerable<T> result. I now believe that this is causing it. I am going to turn it ToList() at this place. Many thanks for your help!

Vlad

1 Like