Deadlock when accessing StackExchange.Redis

These are the workarounds I've found to this deadlock problem:

Workaround #1

By default StackExchange.Redis will ensure that commands are completed in the same order that result messages are received. This could cause a deadlock as described in this question.

Disable that behavior by setting PreserveAsyncOrder to false.

ConnectionMultiplexer connection = ...;
connection.PreserveAsyncOrder = false;

This will avoid deadlocks and could also improve performance.

I encourage anyone that run into to deadlock problems to try this workaround, since it's so clean and simple.

You'll loose the guarantee that async continuations are invoked in the same order as the underlying Redis operations are completed. However, I don't really see why that is something you would rely on.

---

Workaround #2

The deadlock occur when the active async worker thread in StackExchange.Redis completes a command and when the completion task is executed inline.

One can prevent a task from being executed inline by using a custom TaskScheduler and ensure that TryExecuteTaskInline returns false.

public class MyScheduler : TaskScheduler
{
    public override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
    {
        return false; // Never allow inlining.
    }

    // TODO: Rest of TaskScheduler implementation goes here...
}

Implementing a good task scheduler may be a complex task. There are, however, existing implementations in the ParallelExtensionExtras library (NuGet package) that you can use or draw inspiration from.

If your task scheduler would use its own threads (not from the thread pool), then it might be a good idea to allow inlining unless the current thread is from the thread pool. This will work because the active async worker thread in StackExchange.Redis is always a thread pool thread.

public override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
    // Don't allow inlining on a thread pool thread.
    return !Thread.CurrentThread.IsThreadPoolThread && this.TryExecuteTask(task);
}

Another idea would be to attach your scheduler to all of its threads, using thread-local storage.

private static ThreadLocal<TaskScheduler> __attachedScheduler 
                   = new ThreadLocal<TaskScheduler>();

Ensure that this field is assigned when the thread starts running and cleared as it completes:

private void ThreadProc()
{
    // Attach scheduler to thread
    __attachedScheduler.Value = this;

    try
    {
        // TODO: Actual thread proc goes here...
    }
    finally
    {
        // Detach scheduler from thread
        __attachedScheduler.Value = null;
    }
}

Then you can allow inlining of tasks as long as its done on a thread that is "owned" by the custom scheduler:

public override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
    // Allow inlining on our own threads.
    return __attachedScheduler.Value == this && this.TryExecuteTask(task);
}