[clr-interp] Runtime async support

docs/design/coreclr/botr/README.md

@@ -30,6 +30,7 @@ Below is a table of contents.
 - [Mixed Mode Assemblies](mixed-mode.md)
 - [Guide For Porting](guide-for-porting.md)
 - [Vectors and Intrinsics](vectors-and-intrinsics.md)
+- [Runtime Async Codegen](runtime-async-codegen.md)
 
 
 It may be possible that this table is not complete.  You can get a complete list

docs/design/coreclr/botr/clr-abi.md

@@ -678,3 +678,19 @@ MyStruct Test2()
 	return default;
 }
 ```
+
+# Interpreter ABI details
+
+The interpreter data stack is separately allocated from the normal "thread" stack, and it grows UP. The interpreter execution control stack is allocated on the "thread" stack, as a series of `InterpMethodContextFrame` values that are linked in a singly linked list onto an `InterpreterFrame` which is placed onto the Frame chain of the thread. `InterpMethodContextFrame` structures are always allocated in descending order so that a callee method's associated `InterpMethodContextFrame` is always located lower in memory compared to its caller or the containing `InterpreterFrame`.
+
+The base stack pointer within a method never changes, but when a function is called in the interpreter it will have a stack pointer which is associated with the set of arguments passed. In effect argument passing is done by giving a portion of the temporary args space of the caller function to the callee.
+
+All instructions and GC that address the stack pointer are relative to the current stack pointer, which does not move. This requires that implementations of the localloc instruction actually allocate the memory on the heap, and localloc'd memory is not actually tied to the data stack in any way.
+
+The stack pointer in all interpreter functions is always aligned on a `INTERP_STACK_ALIGNMENT` boundary. Currently this is a 16 byte alignment requirement.
+
+The stack elements are always aligned to at least `INTERP_STACK_SLOT_SIZE` and never more than `INTERP_STACK_ALIGNMENT` Given that today's implementation sets `INTERP_STACK_SLOT_SIZE` to 8 and `INTERP_STACK_ALIGNMENT` to 16, this implies all data on the stack is either aligned at an 8 or 16 byte alignment.
+
+Primitive types smaller than 4 bytes are always zero or sign extended to 4 bytes when on the stack.
+
+When a function is async it will have a continuation return. This return is not done using the data stack, but instead is done by setting the Continuation field in the `InterpreterFrame`. Thunks are responsible for setting/resetting this value as we enter/leave code compiled by the JIT.

docs/design/coreclr/botr/runtime-async-codegen.md

@@ -0,0 +1,212 @@
+\## Responsibilities of a code generator for implementing the Runtime Async feature
+
+
+
+This document describes the behaviors that a code generator must conform to to correctly make the runtime async feature work correctly.
+
+
+
+This document is NOT intended to describe the runtime-async feature. That is better described in the runtime-async specification. See (https://github.com/dotnet/runtime/blob/main/docs/design/specs/runtime-async.md).
+
+
+
+The general responsibilities of the runtime-async code generator
+
+1\. Wrap the body of Task and ValueTask returning functions in try/finally blocks which set/reset the `ExecutionContext` and `SynchronizationContext`.
+
+2\. Allow the async thunk logic to work.
+
+3\. Generate Async Debug info (Not yet described in this document)f
+
+
+
+\# Identifying calls to Runtime-Async methods that can be handled by runtime-async
+
+When compiling a call to a method that might be called in the optimized fashion, recognize the following sequence.
+
+```
+
+call\[virt] <Method>
+
+\[ OPTIONAL ]
+
+{
+
+&nbsp; \[ OPTIONAL - Used for ValueTask based ConfigureAwait ]
+
+&nbsp; {
+
+&nbsp;   stloc X;
+
+&nbsp;   ldloca X
+
+&nbsp; }
+
+&nbsp; ldc.i4.0 / ldc.i4.1
+
+&nbsp; call\[virt] <ConfigureAwait> (The virt instruction is used for ConfigureAwait on a Task based runtime async function) NI\_System\_Threading\_Tasks\_Task\_ConfigureAwait
+
+}
+
+call       <Await> One of the functions which matches NI\_System\_Runtime\_CompilerServices\_AsyncHelpers\_Await
+
+```
+
+A search for this sequence is done if Method is known to be async.
+
+
+
+If the pattern is recognized, this is a PREFIX\_IS\_TASK\_AWAIT async call, and if the flag passed to ConfigureAwait is 1, or there is no call to ConfigureAwait, it is also a PREFIX\_TASK\_AWAIT\_CONTINUE\_ON\_CAPTURED\_CONTEXT call. Calls to these functions will save/restore the execution context.
+
+
+
+The dispatch to these functions will save and restore the execution context only on async dispatch.
+
+If PREFIX\_TASK\_AWAIT\_CONTINUE\_ON\_CAPTURED\_CONTEXT, then continuation mode shall be ContinuationContextHandling::ContinueOnCapturedContext otherwise ContinuationContextHandling::ContinueOnThreadPool.
+
+
+
+\# Non-optimized pattern
+
+It is also legal for code to have a simple direct usage of NI\_System\_Runtime\_CompilerServices\_AsyncHelpers\_Await or NI\_System\_Runtime\_CompilerServices\_AsyncHelpers\_AwaitAwaiter. To support this, the Await and AwaitAwaiter functions are marked as async even though they do not return a Task/ValueTask. 
+
+
+
+The dispatch to these functions will save and restore the execution context only on async dispatch.
+
+The dispatch to these functions will set continuation mode to ContinuationContextHandling::None
+
+SaveAndRestoreSynchronizationContextField is disabled
+
+
+
+\# Calli of an async function
+
+The dispatch to these functions will save and restore the execution context only on async dispatch.
+
+
+
+\# The System.Runtime.CompilerServices.AsyncHelpers::AsyncSuspend intrinsic
+
+When encountered, triggers the function to suspend immediately, and return the passed in Continuation.
+
+
+
+\# Saving and restoring of contexts
+
+Capture the execution context before the suspension, and when the function resumes, call `AsyncHelpers.RestoreExecutionContext`. The context should be stored into the Continuation. The context may be captured by calling `AsyncHelpers.CaptureExecutionContext` or it may be captured directly off of the Thread object.
+
+
+
+\# ABI for async function handling
+
+
+
+There is an additional argument which is the Continuation. When calling a function normally, this is always set to 0. When resuming, this is set to the Continuation object. There is also an extra return argument. It is either 0 or a Continuation. If it is a continuation, then the calling function needs to suspend (if it is an async function), or generate a Task/ValueTask (if it is a async function wrapper).
+
+
+
+\## Suspension path
+
+This is what is used in calls to async functions made from async functions.
+
+
+
+```
+
+bool didSuspend = false; // Needed for the context restore
+
+
+
+(result, continuation) = call func(NULL /\* Continuation argument \*/, args)
+
+if (continuation != NULL)
+
+{
+
+&nbsp;   // Allocate new continuation
+
+&nbsp;   // Capture Locals
+
+&nbsp;   // Copy resumption details into continuation (Do things like call AsyncHelpers.CaptureContinuationContext or AsyncHelpers.CaptureExecutionContext as needed)
+
+&nbsp;   // Chain to continuation returned from called function
+
+&nbsp;   // IF in a function which saves the exec and sync contexts, and we haven't yet suspended, restore the old values.
+
+&nbsp;   // return.
+
+
+
+&nbsp;   // Resumption point
+
+
+
+&nbsp;   // Copy values out of continuation (including captured sync context and execution context locals)
+
+&nbsp;   // If the continuation may have an exception, check to see if its there, and if it is, throw it. Do this if CORINFO\_CONTINUATION\_HAS\_EXCEPTION is set.
+
+&nbsp;   // If the continuation has a return value, copy it out of the continuation. (CORINFO\_CONTINUATION\_HAS\_RESULT is set)
+
+}
+
+```
+
+\## Thunks path
+
+This is what is used in non-async functions when calling an async function. Generally used in the AsyncResumptionStub and in 
+
+```
+
+(result, continuation) = call func(NULL /\* Continuation argument \*/, args)
+
+place result onto IL evaluation stack
+
+Place continuation into a local for access using the StubHelpers.AsyncCallContinuation() helper function.
+
+```
+
+Implement an intrinsic for StubHelpers.AsyncCallContinuation() which will load the most recent value stored into the continuation local.
+
+
+
+\# Behavior of ContinuationContextHandling
+
+This only applies to calls which where 
+
+
+
+If set to ContinuationContextHandling::ContinueOnCapturedContext
+
+\- The Continuation shall have an allocated data member for the captured context, and the CORINFO\_CONTINUATION\_HAS\_CONTINUATION\_CONTEXT flag shall be set on the continuation.
+
+\- The Continuation will store the captured synchronization context. This is done by calling `AsyncHelpers.CaptureContinuationContext(syncContextFromBeforeCall, ref newContinuation.ContinuationContext, ref newContinuation.Flags)` while filling in the `Continuation`.
+
+
+
+If set to ContinuationContextHandling::ContinueOnThreadPool
+
+\- The Continuation shall have the CORINFO\_CONTINUATION\_CONTINUE\_ON\_THREAD\_POOL flag set
+
+
+
+\# Exception handling behavior
+
+If an async function is called within a try block (In the jit hasTryIndex return true), set the CORINFO\_CONTINUATION\_HAS\_EXCEPTION bit on the Continuation and make it large enough.
+
+
+
+\# Locals handling
+
+ByRef locals must not be captured. In fact, we should NULL out any locals which are ByRefs or ByRef-like. Currently we do not do this on synchronous execution, but logically possibly we should.
+
+
+
+\# Saving and restoring the synchronization and execution contexts
+
+The code generator must save/restore the sync and execution contexts around the body of all Task/ValueTask methods when directly called with a null continuation context.
+
+
+
+
+

src/coreclr/vm/wasm/helpers.cpp

@@ -715,14 +715,14 @@ void* GetUnmanagedCallersOnlyThunk(MethodDesc* pMD)
     return value->EntryPoint;
 }
 
-void InvokeManagedMethod(MethodDesc *pMD, int8_t *pArgs, int8_t *pRet, PCODE target)
+void InvokeManagedMethod(MethodDesc *pMD, int8_t *pArgs, int8_t *pRet, PCODE target, PTR_PTR_Object pContinuationRet)
 {
     MetaSig sig(pMD);
     void* cookie = GetCookieForCalliSig(sig, false);
 
     _ASSERTE(cookie != NULL);
 
-    InvokeCalliStub(target == NULL ? pMD->GetMultiCallableAddrOfCode(CORINFO_ACCESS_ANY) : target, cookie, pArgs, pRet);
+    InvokeCalliStub(target == NULL ? pMD->GetMultiCallableAddrOfCode(CORINFO_ACCESS_ANY) : target, cookie, pArgs, pRet, pContinuationRet);
 }
 
 void InvokeUnmanagedMethod(MethodDesc *targetMethod, int8_t *pArgs, int8_t *pRet, PCODE callTarget)