-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RFC] Async I/O with structural control flow (a.k.a. Enforced Awaits) #2
Comments
Both From my experience, there no need on var (a, b, c) = await(afut, bfut, cfut) because it used very rare in real world, most of the time you filling array of tasks and waiting for any of them or for all of them. Also you will not find many lines of code which uses |
The dynamic interface in Python allows for working with heterogeneous result types, while in Nim you have to resort to having named The key difference in the new proposal is that the result variables have their proper type in all places and the explicit use of futures is hidden. Otherwise, I've considered the case of requesting a sequence of homogenous operations. I think this should be a separate request throttling API. (EDIT: this use case is now covered by the I'd also say that the need for heterogeneous parallel I/O is quite common in web development for example where it is typical for a web request to lead to several sub-requests to different micro-services, databases and so on. |
|
I don't argue about the low-level details. Nothing about I don't think there is an issue with exceptions. The semantics of With |
Sorry but i dont see any optimizations or safe code. Everything i see is proposal to include some more primitives to existing primitives, and add even more to resolve all possible issues. What is proposed await wait(abc, bcd, ALL_COMPLETED) What is proposed await wait(abc, bcd, cde, ALL_COMPLETED) Most of people do not understand asynchronous programming with current API, so why do you think adding more API will help it understand better? |
@cheatfate, if it's possible to design the system in a way that avoids heap allocations and reliance on the GC, it would be silly not to take advantage of it. The specific optimization being discussed here is placing the Future objects within the stack of the caller. At the moment this is still the heap-allocated environment of the closure iterator, but this can be attacked with further optimizations in the compiler (escape analysis for the closure objects, better support for closure/iterator monomorphism in the run-time code). In the end it may be possible to store all the required state within the single value associated with the registered I/O operation. Rust achieves this. Also, many async procs don't actually need to be compiled as iterators: proc foo: Future[int] {.async.} =
code code code
...
let v = await bar()
...
return v is equivalent to the sync proc: proc foo: int =
code code code
...
let v = waitFor bar()
...
return v
I'd rather have an API that lets me experiment more easily with such changes in the compilation strategy. Do you have any alternative proposal that can ensure all of the above? For the safety, I've given specific examples of problematic code that will be eliminated under the new APIs. And finally, here is the full equivalent example under your alternative: var
googlePageFut = httpRequest(&"http://google.com/?q={keyword}")
jsonDataFut = jsonRpcCall("localhost/myApi")
await wait(googlePageFut, jsonDataFut)
let
googlePage = googlePageFut.read
jsonData = jsonDataFut.read
return googlePage.status == 200 and not jsonData.hasKey("error") Why shouldn't we strive to eliminate this boilerplate? |
@zah, i think you missing not something but alot, or maybe somebody misled you with this optimizations. We already talked with @yglukhov, that if variable passes through state, it become part of closure environment, not stack, so all optimizations will not give you so many benefits as you expect. Also i think you need to read more about read(x).after(read(y).after(read(z))) |
@cheatfate, I know precisely how the compiler works, but you are not paying enough attention to what I'm saying:
The style of the Rust API is irrelevant, it's the underlying run-time approach that's interesting. |
@zah you know precisely how the compiler works, and i know precisely how async works. |
I haven't read all of the discussion so forgive me if I'm missing something, but I already have a few things I would like to note about the whole reasoning for this issue: "Current Problems".
I am against any mixing of pointers and async. If you're doing that then you are on your own and should bear the brunt of the consequences. Separately I do recognise the problem and it similar to the problem of users not knowing better and using Once that works, the above code should fail with an exception. I'll read the rest of this as soon as I can as I am open to alternative solutions. But from what skimming this has seemingly revealed, this is suggesting really big changes to fix this one relatively minor issue. |
So I read the article as well and I must say that the "nursery" idea proposed can be easily implemented on top of async await. I don't see a reason to restrict async await when these nurseries can be implemented with no breaking changes (and very easily too from what I can tell). |
@dom96, what result is expected in example you showed?
(3) will be happen on Unix, but on Windows there is a big chance of (1) and less chance for (3), everything depends on number of messages in sending queue. Also asyncdispatch2 already has procedures which hiding pointer/size usage. While it possible to avoid pointers for writing, its almost impossible to avoid pointers for reading. And about destructors. We already trained and find ways to cheat/workaround GC. Destructors is almost equal thing, we will need to find ways to workaround/cheat destructors too. The only right way (in my opinion) is to introduce manual memory management with a little help from compiler (for example compiler can calculate max size of single allocated object). With this information you can implement/use more fastest memory managers (like memory pools, please not to be confused with memory regions, because memory pools using constant memory blocks). So i'm not waiting destructors, because this is one more crutch. |
Why would (3) be a possibility? Does Windows just ignore a closed socket and report that it sent a message even though it didn't? |
@dom96, |
In an offline discussion, I promised to explain how an async accept loop may work under the new scheme. It was pointed out that if the code is written in the style of the old APIs, enforced awaits will lead to accepting and processing only one connection at a time: while true:
var conn = await socket.accept()
await conn.process() # while this is awaited, we are not accepting new connections The solve this problem, you must use the new async support enabled in parallel:
while true:
var conn = await socket.accept()
spawn conn.process() # this gets added to the list of outstanding async ops
# of the block and the processing continues immediately The parallel block won't be exited until all outstanding operations are complete. In this sense, it can be argued that the new API provides an additional safety benefit over the ad-hoc processing used in the old async APIs. With the old APIs, a shutdown procedure won't do anything about the outstanding async tasks by default, while here we clearly decide whether we want to wait for all the operations to finish or whether we wan to cancel them by using Please also note that all the optimization benefits of the new scheme are still realized, because any local variable declared before the parallel or within the parallel block is guaranteed to outlive all the processing of the async operations started in the block and this allows us to pass pointers to such variables to the said async operations (enabling efficient Besides using |
Ignorant, fly-by question: how much of this could be implemented as a package without change to stdlib? I'm interested in using a system like the Python Trio system, because I think (but don't know) that it would simplify some of my async code. Anything I could do to help? |
A certain version of this can be implemented without any changes to the compiler. Depending on exactly how much integration with parallel blocks is desired (in other words whether you want to mix async I/O with computational tasks backed by a thread pool), some compiler support can improve the system (and it's more about enforcing safety than about expressiveness). If you want to invest time in this, I can help you with some guidance. I would suggest finding me on Discord (https://discord.gg/XRxWahP) or Gitter. |
# This is the 1st commit message: futures: sinkify and lentify this avoids copies here and there throughout the pipeline - ie `copyString` and friends can often be avoided when moving things into and out of futures annoyingly, one has to sprinkle the codebase liberally with `sink` and `move` for the pipeline to work well - sink stuff _generally_ works better in orc/arc # This is the commit message #2: sink test
EDIT: 2020-02-12 Expanded the introduction section with more details explaining the existing problems in non-structural async.
Abstract
This is a proposal to change the semantics of the
async
procs in a way that enforces a more structural control flow. The goal of the new APIs is to force you toawait
your async operations, while still allowing you to easily execute multiple operations in parallel. The proposal eliminates a large category of usage errors with the old APIs and enables some additional optimisations such as storing yourFuture[T]
results on the stack and creating async procs consuming stack-basedopenarray[T]
inputs andvar
parameters.For a more comprehensive set of rationales for enforcing the structural control flow proposed here, please read the following article:
https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
Acknowledgements: some of the key ideas in this proposal were first suggested by @yglukhov in private conversations.
Current problems:
P1) All non-ref input parameters to async procs must be copied
Consider the following async proc:
If this wasn't an async proc, Nim would pass the supplied input sequence as a read-only reference (please note that I'm using C++ terminology here). This relies on the fact that the lifetime of the sequence at the call-site will surely extend until the moment the function delivers its result.
Unfortunately, in the async world this is no longer the case. The caller of
checkBrokenLinks
is free to use it like this:If the
uris
sequence was passed by reference, the async proc may be resumed afterbrokenCode
returns which may result in accessing the now dangling reference. To avoid this problem, the Nim compiler makes sure to copy all input parameters of the async proc to corresponding fields in the "environment" struct associated with the async proc's closure iterator. This copying may be quite expensive for value types such asstring
andseqs
and the users are advised to avoid using such type in async procs and to prefer usingref
parameters where only a pointer must be copied.P2)
var
andopenarray
parameters are not supportedAs a corollary from the previous problem, it becomes impossible to use
var
andopenarray
parameters with async procs, because these require the input data to be passed by reference.P3) The async/await syntax has easily-accessible degrees of freedom that may be dangerous for novice users
Consider the following simple async proc:
It showcases two critical problems triggered by a simple omission of
await
in the code:send
operation will be working with bogus data.This proposal argues that the default behavior should be completely safe and impossible to misuse, while the more advanced concerns such as enabling parallel execution could be handled with a more specialized syntax.
The Proposed Solution:
We create a new set of APIs that hide the explicit use of
Future
values in the user code and enforce awaiting of all async operations. If all operations are awaited, it becomes possible to store the inputs of the said operations in the "pseudo stack" associated with the async proc, which in turn enables the use of the reference types such aslent
,var
andopenarray
providing much better safety than the currentpointer/len
inputs.So, here is the full set of new APIs:
1. Allow
await
to be given multiple arguments or a tupleThis form of
await
just performs the I/O operations in parallel returning a tuple of the final results. It is similar to usingvar r1 = request(); var r2 = request(); await all(r1, r2)
in the current design.For convenience
await (foo, bar)
is considered the same asawait(foo,bar)
.2. Introduce a new
select
API (EDIT: this point is made partially obsolete by point 4)select
is a new API that is given a number of I/O operations that should be started in parallel. The key difference fromawait
is that the handlers installed for each operation will be executed as soon as any of the results are ready. Control flow keywords such asreturn
andbreak
can be used to cancel some of the outstanding operations:The execution after the
select
block continues when all of the handlers have been executed, although there must be a way to mark some of them as optional (here, I've used*
for this).The named results are considered in scope after the
select
statement. You can choose to only name a particular result without providing a handling block.3. Introduce a new
safeasync
pragma (EDIT: this may well be the default mode)The
safeasync
pragma is responsible for inserting theawait
keyword in automatic way. It also has the role of the currentmultisync
pragma in the sense that it allows you to compile the same code for both sync and async usage:How does this work? It inserts a call to a template called
implicitAwait
on each expression within the proc's body.implicitAwait
is defined as an identity for all non-future types and as a regularawait
statement for all futures:Please note that the body of a
safeasync
will work in synchronous mode by executing each operation in turn. It's also possible to compile the code for implicit off-loading to a background thread pool in programs that don't feature an asynchronous event loop.Appending 3.A
Please note that using the
await
statement may still be supported insidesafeasync
procs. One may use it to improve the code clarity. It's also possible to implementsafeasync
in an alternative way that requires the use ofawait
and signals any omission as an error, but the arguments for this are not very strong - in all code there might be significant differences between operations that are algorithmically cheaper or heavier. It's usually the names of the operations that reveal where the I/O waits will happen.4. Support async operations in
parallel
blocksI'm extending the proposal to also enhance Nim's
parallel
construct with additional support for async operations. This proposal can replace the need for a separateselect
API, although it could still exist as a simple high-level helper. The new features are the following:Within
parallel
blocks:4.1) Allow
spawn
to be followed by ado
block that will be executed with the result of the operation, once complete.4.2) Allow
spawn
to be used with procs returningFuture[T]
results.spawn
immediately starts the async operation and it adds theFuture
to a list of tasks to be awaited just before the exit of the parallel block. This enforces the structural handling of the async operations, but one can still work with the returned futures in the familiar fashion - passing them to helper procs, setting up callbacks and so on. It is guaranteed that the callbacks will be executed in the same thread that has entered the parallel block.4.3) Add a new call called
spawnOptional
that launches non-critical parallel operations. If theparallel
block is able to complete before all such operations have completed, they are simply cancelled.4.4) Support
break
andreturn
in parallel blocks by cancelling all outstanding operations.With such an API, the
select
example above becomes:Please note that such a
parallel
block will be more powerful than theselect
construct, because it enables you to add multiple tasks to be awaited from a loop.The use of
parallel
blocks andspawn
comes at a cost. All parameters passed in thespawn
expression must be copied inside the spawned task. Please note that this matches precisely the behavior ofspawn
when it comes to sending computational tasks to a thread pool as well.4.5) Introduce an underlying object representing the "parallel block" and create an accessor for it (e.g. a
thisParallelBlock
magic valid only inside the block). This object will feature operations such asaddThreadJob
,addAsyncIO
,addOptionalAsyncIO
. It's the equivalent to thenursery
object described in the article linked in the abstract. Its goal is to enable the creation of helper libraries that perform something with the parallel block context.parallel: addJobs(thisParallelBlock)
4.6) Define the exception-handling semantics inside parallel blocks - if an exception is thrown by a spawned task inside a parallel block, this exception will be re-raised in the thread that has entered the block. All other spawned tasks are cancelled.
5. Support async tasks in
spawn
outside ofparallel
blocks.This is an escape hatch that will replace the current usages of
asyncCheck
andtraceAsyncErrors
. Semantically, it spawns a new "async thread" of execution. Just like when spawning a regular thread, all parameters passed to thespawn
expression must be copied or moved in the new thread. The spawned function must annotated withraises: [Defect]
. If it terminates with aDefect
, the whole process is also terminated.6. Migration path for the current
async
A semi backwards-compatible
async
pragma can be added to serve as a drop-in replacement for the existingasync
pragma. It will differ in only one way. All started async operations will be added to a list that is awaited at the end of their scope. This is not strictly backwards-compatible, but most of the existing async code should not be affected by the change in semantics.The text was updated successfully, but these errors were encountered: