No Available Servers - Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'

question
,
1

We are in the process of implementing Hangfire in our application are finding that when clicking on Servers, we are a message that states “there are not active servers. Background tasks will not be processed.” We have also confirmed that there are no servers listed in the Server table. When looking into the Windows event logs, we see the following:

Process information:
Process ID: 5708
Process name: w3wp.exe
Account name: IIS APPPOOL\SpecificAppPool

Exception information:
Exception type: SqlException
Exception message: Login failed for user ‘NT AUTHORITY\ANONYMOUS LOGON’.
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource1 retry)
at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource1 retry) at System.Data.SqlClient.SqlConnection.Open() at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection() at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func2 func)
at Hangfire.SqlServer.SqlServerMonitoringApi.GetStatistics()
at System.Lazy1.CreateValue() at System.Lazy1.LazyInitValue()
at Hangfire.Dashboard.DashboardMetrics.<>c.<.cctor>b__1_0(RazorPage page)
at Hangfire.Dashboard.JsonStats.d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Owin.Mapping.MapMiddleware.d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.d__5.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.d__2.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.StageAsyncResult.End(IAsyncResult ar)
at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

We have deleted all Hangfire schema / tables to verify that the scaffolding does run and re-installs the schema / tables with the current settings. We have tried using the ImpersonationFunc which allows the servers show but all queues that interact with the SQL fail with the same exact error as above. My configuration is below:

public void Configuration(IAppBuilder app)
{
GlobalConfiguration.Configuration
.SetDataCompatibilityLevel(CompatibilityLevel.Version_170)
.UseSimpleAssemblyNameTypeSerializer()
.UseRecommendedSerializerSettings()
.UseSqlServerStorage(“HangfireConnectionString”, new SqlServerStorageOptions
{
CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
QueuePollInterval = TimeSpan.Zero,
UseRecommendedIsolationLevel = true,
DisableGlobalLocks = true,
ImpersonationFunc = HostingEnvironment.Impersonate
});

        app.UseHangfireServer();

        app.UseHangfireDashboard("/hangfire", new DashboardOptions
        {
            AppPath = null,
            Authorization = new[] { new HangireAuthorizationFilter() }
        });
    }

Below are a sample of the database connection strings and our impersonation configurations:

2

Because it won’t let me fix my formatting:

<add name="HangfireConnectionString" connectionString="Initial Catalog=Database;Data Source=Data\DevServer;Integrated Security=SSPI;" providerName="System.Data.SqlClient" />

<identity impersonate="true" userName="username" password="password" />
3

It has taken a couple weeks to really dig into this but it turns out that some settings in the aspnet.config were preventing .NET from allowing impersonation to flow into background threads. Our web server was setup to originally have “legacyImpersonationPolicy” set to true and “alwaysFlowImpersonationPolicy” set to false. When flipping those properties, the impersonation ended up working correctly. I know this isn’t a solution for everybody but it did work for us. Sample below:

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
    <runtime>
        <legacyImpersonationPolicy enabled="false"/>
        <alwaysFlowImpersonationPolicy enabled="true"/>
    </runtime>
</configuration>