Fix RedisGrainDirectory uninitialize exception

[miguelhasse]

Updated the Uninitialize method to replace CloseAsync and synchronous Dispose calls with await _redis.DisposeAsync(), ensuring proper asynchronous disposal. Cleanup of _redis and _database fields is now handled in a finally block to guarantee resource release even if disposal fails.

This change is a fix to avoid NullReferenceException being thrown during calls to RedisGrainDirectory .Uninitialize

Microsoft Reviewers: Open in CodeFlow

[Copilot]

Pull request overview

Updates the Redis grain directory shutdown path to dispose the Redis connection asynchronously and ensure _redis/_database fields are cleared even if disposal throws, addressing a NullReferenceException during RedisGrainDirectory.Uninitialize.

Changes:

• Replace CloseAsync() + synchronous Dispose() with await _redis.DisposeAsync().
• Move cleanup of _redis and _database into a finally block to guarantee field cleanup.

[Copilot]

This change targets a shutdown-time exception path, but there’s no coverage ensuring Uninitialize is safe to call when initialization is partial/failed or when it’s invoked multiple times. Consider adding a Redis grain directory test which exercises the shutdown path (e.g., invoke Uninitialize via reflection or lifecycle and assert it doesn’t throw).

[ReubenBond]

What is the source of the NullReferenceException in this case, as in: what is null here? From your exception, it looks like it's null on line 199 (_redis.Dispose();). If that's the case, are there overlapping calls to Uninitialize that are setting _redis to null? If that's an issue (it might be, I'm not sure yet), then it would likely be good to guard against that in code (eg, capture _redis and set it to null atomically, possibly via an interlocked operation or using a lock statement)

[miguelhasse]

What is the source of the NullReferenceException in this case, as in: what is null here? From your exception, it looks like it's null on line 199 (_redis.Dispose();). If that's the case, are there overlapping calls to Uninitialize that are setting _redis to null? If that's an issue (it might be, I'm not sure yet), then it would likely be good to guard against that in code (eg, capture _redis and set it to null atomically, possibly via an interlocked operation or using a lock statement)

From the logged exception it look like it it thrown from inside StackExchange's implementation, but you may be right about the possible race condition. Using an interlocked operation might be a good idea.

[ReubenBond]

@miguelhasse did this get resolved? It is not clear to me that this is an Orleans issue yet.

[miguelhasse]

@miguelhasse did this get resolved? It is not clear to me that this is an Orleans issue yet.

It's not resolved. Logging shows frequent exceptions like the one I posted here.

[miguelhasse]

@ReubenBond logs keep showing frequent exceptions due to this issue. Orleans running on AKS across regions with a single Redis distributed grain directory.

[ReubenBond]

@miguelhasse I believe the core issue here is related to ownership of the Redis instance and hence the disposal of it. Fixing it will result in a small breaking change to how Redis is configured.
I made a change to RedisStreamingOptions to account for shared ownership, changing the CreateMultiplexer signature to return an IsShared value:

    /// <summary>
    /// The delegate used to create a Redis connection multiplexer and indicate whether it is shared.
    /// </summary>
    public Func<RedisStreamingOptions, Task<(IConnectionMultiplexer Multiplexer, bool IsShared)>> CreateMultiplexer { get; set; } = DefaultCreateMultiplexer;

We need to make the changes to the other Redis providers.

[ReubenBond]

I believe this was fixed by #10146. I'll close this for now and we can revisit if it's still an issue.