We have two environments with different Redis cluster setups. Hangfire works fine with 3 master Node Redis cluster, while on the other environment, we have 3 master nodes and 6 slave nodes, there it says no Servers were found but when looking in Logs, it says everything went fine and the connection was established.
Logs:
Hangfire.Pro.Redis.RedisStorage, Debug: Connecting to Redis server(s) ‘SERVERIP01:6379,SERVERIP01:6380,SERVERIP01:6381’
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6379,SERVERIP01:6380,SERVERIP01:6381,name=Hangfire@SUFYAN-PC,keepAlive=60,syncTimeout=30000,allowAdmin=True,connectTimeout=15000,password=*****,abortConnect=True,connectRetry=0,responseTimeout=300000,heartbeatInterval=0,preferIOCP=False
Hangfire.Pro.Redis.RedisStorage, Debug:
Hangfire.Pro.Redis.RedisStorage, Debug: Connecting SERVERIP01:6379/Interactive…
Hangfire.Pro.Redis.RedisStorage, Debug: BeginConnect: SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: Connecting SERVERIP01:6379/Subscription…
Hangfire.Pro.Redis.RedisStorage, Debug: BeginConnect: SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: Connecting SERVERIP01:6380/Interactive…
Hangfire.Pro.Redis.RedisStorage, Debug: BeginConnect: SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: Connecting SERVERIP01:6380/Subscription…
Hangfire.Pro.Redis.RedisStorage, Debug: BeginConnect: SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: Connecting SERVERIP01:6381/Interactive…
Hangfire.Pro.Redis.RedisStorage, Debug: BeginConnect: SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: Connecting SERVERIP01:6381/Subscription…
Hangfire.Pro.Redis.RedisStorage, Debug: BeginConnect: SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: 3 unique nodes specified
Hangfire.Pro.Redis.RedisStorage, Debug: Allowing endpoints 00:00:15 to respond…
Hangfire.Pro.Redis.RedisStorage, Debug: Waiting for event completion
Hangfire.Pro.Redis.RedisStorage, Debug: EndConnect: SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: EndConnect: SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: EndConnect: SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: Connected Subscription/SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: Connected Interactive/SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: EndConnect: SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: Connected Subscription/SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: EndConnect: SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: Connected Interactive/SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: Server handshake
Hangfire.Pro.Redis.RedisStorage, Debug: Connected Interactive/SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: Server handshake
Hangfire.Pro.Redis.RedisStorage, Debug: Server handshake
Hangfire.Pro.Redis.RedisStorage, Debug: Server handshake
Hangfire.Pro.Redis.RedisStorage, Debug: Server handshake
Hangfire.Pro.Redis.RedisStorage, Debug: Authenticating (password)
Hangfire.Pro.Redis.RedisStorage, Debug: Authenticating (password)
Hangfire.Pro.Redis.RedisStorage, Debug: Authenticating (password)
Hangfire.Pro.Redis.RedisStorage, Debug: Authenticating (password)
Hangfire.Pro.Redis.RedisStorage, Debug: Authenticating (password)
Hangfire.Pro.Redis.RedisStorage, Debug: EndConnect: SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: Connected Subscription/SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: Server handshake
Hangfire.Pro.Redis.RedisStorage, Debug: Authenticating (password)
Hangfire.Pro.Redis.RedisStorage, Debug: Setting client name: Hangfire@SUFYAN-PC
Hangfire.Pro.Redis.RedisStorage, Debug: Setting client name: Hangfire@SUFYAN-PC
Hangfire.Pro.Redis.RedisStorage, Debug: Setting client name: Hangfire@SUFYAN-PC
Hangfire.Pro.Redis.RedisStorage, Debug: Setting client name: Hangfire@SUFYAN-PC
Hangfire.Pro.Redis.RedisStorage, Debug: Setting client name: Hangfire@SUFYAN-PC
Hangfire.Pro.Redis.RedisStorage, Debug: Setting client name: Hangfire@SUFYAN-PC
Hangfire.Pro.Redis.RedisStorage, Debug: Sending critical tracer: Subscription/SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: Auto-configure…
Hangfire.Pro.Redis.RedisStorage, Debug: Auto-configure…
Hangfire.Pro.Redis.RedisStorage, Debug: Auto-configure…
Hangfire.Pro.Redis.RedisStorage, Debug: Sending critical tracer: Subscription/SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: Sending critical tracer: Subscription/SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Subscription/SERVERIP01:6379: ECHO
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Subscription/SERVERIP01:6381: ECHO
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Subscription/SERVERIP01:6380: ECHO
Hangfire.Pro.Redis.RedisStorage, Debug: Flushing outbound buffer
Hangfire.Pro.Redis.RedisStorage, Debug: Flushing outbound buffer
Hangfire.Pro.Redis.RedisStorage, Debug: Flushing outbound buffer
Hangfire.Pro.Redis.RedisStorage, Debug: Sending critical tracer: Interactive/SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: Sending critical tracer: Interactive/SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: Sending critical tracer: Interactive/SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP01:6379: ECHO
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP01:6381: ECHO
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP01:6380: ECHO
Hangfire.Pro.Redis.RedisStorage, Debug: Flushing outbound buffer
Hangfire.Pro.Redis.RedisStorage, Debug: Flushing outbound buffer
Hangfire.Pro.Redis.RedisStorage, Debug: Flushing outbound buffer
Hangfire.Pro.Redis.RedisStorage, Debug: Starting reader thread
Hangfire.Pro.Redis.RedisStorage, Debug: Starting reader thread
Hangfire.Pro.Redis.RedisStorage, Debug: Starting reader thread
Hangfire.Pro.Redis.RedisStorage, Debug: Starting reader thread
Hangfire.Pro.Redis.RedisStorage, Debug: Starting reader thread
Hangfire.Pro.Redis.RedisStorage, Debug: Starting reader thread
Hangfire.Pro.Redis.RedisStorage, Debug: Connect complete: SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: Connect complete: SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: Connect complete: SERVERIP01:6379
Hangfire.Pro.Redis.RedisStorage, Debug: Connect complete: SERVERIP01:6380
Hangfire.Pro.Redis.RedisStorage, Debug: Connect complete: SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: Connect complete: SERVERIP01:6381
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Subscription/SERVERIP01:6379 / ECHO: BulkString: 16 bytes
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Subscription/SERVERIP01:6381 / ECHO: BulkString: 16 bytes
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Subscription/SERVERIP01:6380 / ECHO: BulkString: 16 bytes
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP01:6380 / ECHO: BulkString: 16 bytes
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP01:6380: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP01:6381 / ECHO: BulkString: 16 bytes
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP01:6381: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP01:6379 / ECHO: BulkString: 16 bytes
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP01:6379: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP01:6380 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP01:6381 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP01:6379 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6379 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6380 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6381 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP01:6379: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP02:6379: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP03:6379: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP01:6380: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Allowing endpoints 00:00:13.8760000 to respond…
Hangfire.Pro.Redis.RedisStorage, Debug: Waiting for event completion
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP01:6381: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP02:6380: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP03:6380: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP02:6379 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP01:6380 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP01:6379 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP03:6379 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP01:6381 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP02:6380 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP03:6380 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP02:6381: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Writing to Interactive/SERVERIP03:6381: PING
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP02:6381 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: Response from Interactive/SERVERIP03:6381 / PING: SimpleString: PONG
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6379 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP03:6380 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP02:6379 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP03:6379 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP02:6380 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6380 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP03:6381 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6381 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP02:6381 returned with success
Hangfire.Pro.Redis.RedisStorage, Debug: Cluster: 16384 of 16384 slots covered
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6379: Cluster v4.0.9, master; 16 databases; keep-alive: 00:01:00; int: ConnectedEstablished; sub: ConnectedEstablished, 0 active
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6379: int ops=15, qu=0, qs=0, qc=0, wr=0, sync=15, socks=1; sub ops=3, qu=0, qs=0, qc=0, wr=0, sync=3, socks=1
Hangfire.Pro.Redis.RedisStorage, Debug: Circular op-count snapshot; int: 0+15=15 (1.50 ops/s; spans 10s); sub: 0+3=3 (0.30 ops/s; spans 10s)
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6380: Cluster v4.0.9, slave; 16 databases; keep-alive: 00:01:00; int: ConnectedEstablished; sub: ConnectedEstablished, 0 active
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6380: int ops=13, qu=0, qs=0, qc=0, wr=0, sync=13, socks=1; sub ops=3, qu=0, qs=0, qc=0, wr=0, sync=3, socks=1
Hangfire.Pro.Redis.RedisStorage, Debug: Circular op-count snapshot; int: 0+13=13 (1.30 ops/s; spans 10s); sub: 0+3=3 (0.30 ops/s; spans 10s)
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6381: Cluster v4.0.9, slave; 16 databases; keep-alive: 00:01:00; int: ConnectedEstablished; sub: ConnectedEstablished, 0 active
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP01:6381: int ops=13, qu=0, qs=0, qc=0, wr=0, sync=13, socks=1; sub ops=3, qu=0, qs=0, qc=0, wr=0, sync=3, socks=1
Hangfire.Pro.Redis.RedisStorage, Debug: Circular op-count snapshot; int: 0+13=13 (1.30 ops/s; spans 10s); sub: 0+3=3 (0.30 ops/s; spans 10s)
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP02:6379: Cluster v4.0.9, master; 16 databases; keep-alive: 00:01:00; int: ConnectedEstablished; sub: ConnectedEstablished, 0 active
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP02:6379: int ops=9, qu=0, qs=0, qc=0, wr=0, sync=9, socks=1; sub ops=3, qu=0, qs=0, qc=0, wr=0, sync=3, socks=1
Hangfire.Pro.Redis.RedisStorage, Debug: Circular op-count snapshot; int: 0+9=9 (0.90 ops/s; spans 10s); sub: 0+3=3 (0.30 ops/s; spans 10s)
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP03:6379: Cluster v4.0.9, master; 16 databases; keep-alive: 00:01:00; int: ConnectedEstablished; sub: ConnectedEstablished, 0 active
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP03:6379: int ops=9, qu=0, qs=0, qc=0, wr=0, sync=9, socks=1; sub ops=3, qu=0, qs=0, qc=0, wr=0, sync=3, socks=1
Hangfire.Pro.Redis.RedisStorage, Debug: Circular op-count snapshot; int: 0+9=9 (0.90 ops/s; spans 10s); sub: 0+3=3 (0.30 ops/s; spans 10s)
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP03:6380: Cluster v4.0.9, slave; 16 databases; keep-alive: 00:01:00; int: ConnectedEstablished; sub: ConnectedEstablished, 0 active
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP03:6380: int ops=10, qu=0, qs=0, qc=0, wr=0, sync=10, socks=1; sub ops=3, qu=0, qs=0, qc=0, wr=0, sync=3, socks=1
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP02:6380: Cluster v4.0.9, slave; 16 databases; keep-alive: 00:01:00; int: ConnectedEstablished; sub: ConnectedEstablished, 0 active
Hangfire.Pro.Redis.RedisStorage, Debug: Circular op-count snapshot; int: 0+10=10 (1.00 ops/s; spans 10s); sub: 0+3=3 (0.30 ops/s; spans 10s)
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP02:6380: int ops=10, qu=0, qs=0, qc=0, wr=0, sync=10, socks=1; sub ops=3, qu=0, qs=0, qc=0, wr=0, sync=3, socks=1
Hangfire.Pro.Redis.RedisStorage, Debug: Circular op-count snapshot; int: 0+10=10 (1.00 ops/s; spans 10s); sub: 0+3=3 (0.30 ops/s; spans 10s)
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP02:6381: Cluster v4.0.9, slave; 16 databases; keep-alive: 00:01:00; int: ConnectedEstablished; sub: ConnectedEstablished, 0 active
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP02:6381: int ops=10, qu=0, qs=0, qc=0, wr=0, sync=10, socks=1; sub ops=3, qu=0, qs=0, qc=0, wr=0, sync=3, socks=1
Hangfire.Pro.Redis.RedisStorage, Debug: Circular op-count snapshot; int: 0+10=10 (1.00 ops/s; spans 10s); sub: 0+3=3 (0.30 ops/s; spans 10s)
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP03:6381: Cluster v4.0.9, slave; 16 databases; keep-alive: 00:01:00; int: ConnectedEstablished; sub: ConnectedEstablished, 0 active
Hangfire.Pro.Redis.RedisStorage, Debug: SERVERIP03:6381: int ops=10, qu=0, qs=0, qc=0, wr=0, sync=10, socks=1; sub ops=3, qu=0, qs=0, qc=0, wr=0, sync=3, socks=1
Hangfire.Pro.Redis.RedisStorage, Debug: Circular op-count snapshot; int: 0+10=10 (1.00 ops/s; spans 10s); sub: 0+3=3 (0.30 ops/s; spans 10s)
Hangfire.Pro.Redis.RedisStorage, Debug: Sync timeouts: 0; fire and forget: 0; last heartbeat: 1s ago
Hangfire.Pro.Redis.RedisStorage, Information: Connection to Redis server(s) ‘main: SERVERIP03:6379; connected: SERVERIP01:6379,SERVERIP01:6380,SERVERIP01:6381,SERVERIP02:6379,SERVERIP03:6379,SERVERIP03:6380,SERVERIP02:6380,SERVERIP02:6381,SERVERIP03:6381’ was successfully established.
Hangfire.BackgroundJobServer, Information: Starting Hangfire Server using job storage: ‘redis://SERVERIP03:6379/0’
Hangfire.BackgroundJobServer, Information: Using the following options for Hangfire Server:
Worker count: 40
Listening queues: ‘default’
Shutdown timeout: 00:00:15
Schedule polling interval: 00:00:15
Hangfire.Processing.BackgroundExecution, Debug: Execution loop BackgroundServerProcess:61fe7caf has started in 2.9059 ms
Hangfire.Server.BackgroundServerProcess, Trace: Server sufyan-pc.cc93f88c-47b5-407f-8f91-1f6849e0d4f4:81368:b9d523c9 is announcing itself…
Hangfire.Server.BackgroundServerProcess, Information: Server sufyan-pc.cc93f88c-47b5-407f-8f91-1f6849e0d4f4:81368:b9d523c9 successfully announced in 840.7575 ms
Hangfire.Server.BackgroundServerProcess, Information: Server sufyan-pc.cc93f88c-47b5-407f-8f91-1f6849e0d4f4:81368:b9d523c9 is starting the registered dispatchers: ServerWatchdog, ServerJobCancellationWatcher, FetchedJobsWatcher, Worker, DelayedJobScheduler, RecurringJobScheduler…
Hangfire.Server.ServerJobCancellationWatcher, Trace: Checking for aborted jobs…
Hangfire.Server.ServerJobCancellationWatcher, Trace: No newly aborted jobs found.
Hangfire.Server.BackgroundServerProcess, Information: Server sufyan-pc.cc93f88c-47b5-407f-8f91-1f6849e0d4f4:81368:b9d523c9 all the dispatchers started
Hangfire.Pro.Redis.FetchedJobsWatcher, Debug: Acquiring the lock for the fetched list of the ‘default’ queue…
Hangfire.Pro.Redis.FetchedJobsWatcher, Debug: Looking for timed out jobs in the ‘default’ queue…
Hangfire.Pro.Redis.FetchedJobsWatcher, Debug: No timed out jobs were found in the ‘default’ queue
Hangfire.Server.ServerJobCancellationWatcher, Trace: Checking for aborted jobs…
Hangfire.Server.ServerJobCancellationWatcher, Trace: No newly aborted jobs found.
Hangfire.Server.ServerJobCancellationWatcher, Trace: Checking for aborted jobs…
Hangfire.Server.ServerJobCancellationWatcher, Trace: No newly aborted jobs found.
Hangfire.Server.ServerJobCancellationWatcher, Trace: Checking for aborted jobs…
Hangfire.Server.ServerJobCancellationWatcher, Trace: No newly aborted jobs found.
Hangfire.Server.ServerJobCancellationWatcher, Trace: Checking for aborted jobs…
Hangfire.Server.ServerJobCancellationWatcher, Trace: No newly aborted jobs found.