Asynchronous Promise Cancellation with Cancellable Promises, AbortController and Generic Timer

[CMCDragonkai]

Specification

There are a number of places where we have asynchronous operations where there is a timeout applied. When timedout, certain cancellation operations must be done. Sometimes this means cancelling a promise which may involve IO, and other cases it means breaking a while (true) loop.

These places are:

• src/utils.ts - the poll utility function is being used by a number of tests to testing if an asynchronous function returns what we want using the condition lambda, this is inherently loopy due to its naming of poll, this means cancellation occurs by breaking the loop, and no setTimeout is being used here, when the loop is boken, an exception is thrown as ErrorUtilsPollTimeout
• src/network/ForwardProxy.ts and src/network/ReverseProxy.ts - these make use of the Timer object type, that is used to allow timer inheritance. Unlike poll, here we are using Promise.race to cancel awaiting for a promise that will never resolve. In this situation, the problem is that we want to cancel waiting for a promise, the promise operation itself does not need to be cancelled, see Promise.race usage in src/network/ConnectionForward.ts and src/network/ConnectionReverse.ts
• src/grpc/GRCPServer.ts same situation as the proxies, here it's a promise we want to stop waiting for, the promise operation itself does not need to be cancelled.
• src/identities/providers/github/GitHubProvider.ts - this is the oldest code making use of polling, as such it has not been using the poll or timerStart and timerStop utilities, it should be brought into the fold
• src/status/Status.ts - this uses polling in waitFor
• src/discovery/Discovery.ts - we need to be able to abort the current task (discovery of a node/identity) rather than awaiting for it to finish when we stop the discovery domain CLI and Client & Agent Service test splitting #311 (comment)

Note that "timer" inheritance needs to be considered here.

So there are properties we need to consider when it comes to universal asynchronous "cancellation" due to timeout:

1. The ability to inherit a "timer" object that is used by the networking proxies, this is because a single timeout timer may need to be used by multiple asynchronous operations
2. The ability to cancel asynchronous operations in 3 ways:
   1. Breaking a while (true) loop
   2. Racing a promise and then dropping the awaiting of it
   3. Actually cancelling the promise operation - this is not currently done at all
3. An undefined version of the timeout, perhaps like an infinite version, this is so that function signatures can take timer object, which can defaulted to new Timer(Infinity)

My initial idea for this would be flesh out the Timer type that we are using, include methods that allow one to start and stop.

For 2.3, we actually have to integrate AbortController. This will be important for cancelling async IO operations. We can then integrate this into our timeout system itself.

Imagine:

const timer = new Timer(timeout);

timer.run(
  async (abortSignal) => { 
    // use the abort signal
    // fetch has a signal option
    // it can take the abortSignal
  }
);

In a way it can be an abstraction on top of the AbortController.

General usage summary of the abort controller.

// Creating an abort controller.
const abortController = new AbortController();

// it is pretty standard to pass an abort controller to methods in this way.
async function someFunction(arg1: any, arg2: any, options?: {signal?: AbortSignal}){
  // short way of getting the optional signal
  const { signal } = { ...options };

  for await (const item of longRunningGenerator()) {
    // Doing things here
    if (signal?.aborted){
      // Aborted functions/methods need to throw some kind of abort error to end.
      throw Error('AbortError')
    }
    // passing the signal down
    someOtherFunction(arg2, {signal});
    // Or we can use an event
    signal?.addEventListener('abort', async () => {
      // Do something to abort
    })
  }
}

// using a function that supports the abort signal
const prom = someFunction(1,2, {signal: abortController.signal});
// aborting
abortController.abort();
// will throw the abort error.
await prom;

More prototyping has to be done here.

For while loops, we can check if the signal is true, and if so cancel the loop.

---

There's 3 portions to this issue:

1. The timer/timeout problem
2. The cancellable wrapper higher order function/decorator
3. Refactoring everywhere where we have a "cancellable" asynchronous thing to use the cancellable wrapper and optionally the timer/timeout mechanism

Timer/Timeout problem

Many third party libraries and core core has a timeout convenience parameter that allows an asynchronous function to automatically cancel once a timeout has finished.

The core problem is that timeout is not easily "composable".

This is why the Timer abstraction was created.

/**
 * Use Timer to control timers
 */
type Timer = {
  timer: ReturnType<typeof setTimeout>;
  timedOut: boolean;
  timerP: Promise<void>;
};

This allowed functions to propagate the timer object down to low level functions while also reacting to to this timer. This means each function can figure out if their operations have exceeded the timer. But this is of course advisory, there's no way to force timeout/cancel a particular asynchronous function execution unlike OS schedulers that can force SIGTERM/SIGKILL processes/threads.

This is fine, but the major problem here is how to interact with functions that only support a timeout parameter. This could be solved with an additional method to the Timer object that gives you the remaining time at the point the method is called.

For example:

function f (timer?: Timer | number) {
  // Checks if already Timer, if not, will construct a timer with timeout being undefined or a number
  // An `undefined` should result in some default timeout
  timer = timer instanceof Timer ?? new Timer(timer);

  // This takes some time
  await doSomething();

  // This can inherit the timer
  await doAnotherThing(timer);

  // This uses timeout number soley, and `timer.getTimeout()` gets the remaining time
  // Which can be 0 if the timeout has run out
  await doLegacyThing(timer.getTimeout());

  while (true) {
    if (timer.timedOut) throw new ErrorTimedOut();
    await doSomething();
    break;
  }

}

It is up the function internally to deal with the timer running out and throwing whatever function is appropriate at that point. Remember it's purely advisory.

All we have to do here is to create a new class Timer that supports relevant methods:

class Timer {

  protected timer: ReturnType<typeof setTimeout>;
  public readonly timerP: Promise<void>;
  protected _timedOut: boolean;

  public constructor(time?: number) {
    // construct the timer
  }

  get timedOut(): boolean {
    return this._timedOut;
  }

  public start() {
  }

  public stop() {
  }

  public getTimeout(): number {
    // subtract elapsed time, from total timeout
    // return in ms how much time is left
  }

}

This would replace the timerStart and timerStop functions that we currently have in our src/utils.

Functions that take a Timer should also support a number as well as convenience, and this can be made more convenient just be enabling the time?: number parameter above. Note that by default if there's no time set for the Timer, we can either default in 2 ways:

1. 0 meaning the timer immediately times out
2. Infinity meaning the timer never times out

Certain functions may want to default to infinity, certain functions may wish to default to 0. In most cases defaulting to Infinity is probably the correct behaviour. However we would need to decide what getTimeout() means then, probably return the largest possible number in JS.

Because this is fairly general, if we want to apply this to our other libraries like js-async-locks, then this would need to be a separate package like js-timer that can be imported.

Cancellable Wrapper HOF and/or Decorator

The timer idea is just a specific notion of a general concept of asynchronous cancellation. In fact it should be possible to impement timers on top of general asynchronous cancellation.

There may be a need to cancel things based on business logic, and not just due to the expiry of some time.

In these cases, we want to make use of 2 things: AbortController and CancellablePromise.

The AbortController gives us an EventTarget that we can use to signal to asynchronous operations that we want them to cancel. Just like the timer, it's also advisory.

The CancellablePromise gives us a way to cancel promises without having to keep around a separate AbortController object. The same concept can apply to AsyncGenerator as well, so one may also have CancellableAsyncGenerator. Note that AsyncGenerator already has things like return and throw methods that stop async generator iteration, however they are not capable of terminating any asynchronous side-effects, so they are not usable in this situation.

The idea is to create a cancellable higher order function wrapper that works similar to promisify and callbackify.

The expectation is that a lower order function takes in options?: { signal?: AbortSignal } as the last parameter, and cancellable will automate the usage the signal, and return a new function that returns CancellablePromise.

This higher order cancellable has to function has to:

1. Preserve the input/output types of the lower order function - see the new promisify how to do this (note that overloaded functions will need to be using conditional types to be supported)
2. If the wrapped function is given an abort signal, this signal is preserved and passed through to the lower order function.
3. If the wrapped function is not given an abort signal, the wrapper must create the AbortSignal and connect it to the cancel* methods of the returned promise.
4. The returned CancellablePromise must be infectious, any chained operations after that promise must still be a CancellablePromise. Basically pC.then(...).then(...) must still return a CancellablePromise, even if then chained function returns a normal Promise.
5. The CancellablePromise has 2 additional methods: cancel and cancelAsync, the difference between the 2 is equivalent to our EventBus.emit and EventBus.emitAsync. You would generally use await pC.cancelAsync().
6. Inside the lower order function, if the signal is emitted, a cancel exception must be thrown. This exception will be expected to be caught in the cancelAsync or cancel methods of the cancellable promise. If no exception is thrown, then the method is successful because it is assumed that the promise is already settled. If a different exeption is thrown, then it is considered an error and will be rethrown up.
7. What is the cancel exception? This should be specifiable with an exception array in the higher order function. A default cancel exception can be made available if none is specified.
8. Be usable as a decorator, in some cases, such as class methods, it is best to have this as a @cancellable wrapper similar to our @ready decorator used in js-async-init.
9. Be applicable to AsyncGenerator as well, in which case the @cancellable would return a CancellableAsyncGenerator instead.
10. Be composable with promisify and callbackify.

An example of this being used:

// Assume that cancellable can take a variable number of args first
// If you don't pass any exceptions, it should default on particular exception
const pC = cancellable(
  ErrorCancellation,
  async (options?: { signal?: AbortSignal }) => {
    // Once you wrap it in a signal, then the signal is always passed in
    // however you don't know this in the lower order function
    if (signal?.aborted === true) {
      throw new ErrorCancellation();
    }
  }
);

await pC.cancelAsync();

Or with a decorator:

class C {
  @cancellable(ErrorCancellation)
  public async (options?: { signal?: AbortSignal }) {
    // ...
  }
}

You should see that it would be possible to implement the timer using this concept.

const pC = cancellable(
  ErrorCancellation,
  async (options?: { signal?: AbortSignal }) => {
    // Once you wrap it in a signal, then the signal is always passed in
    // however you don't know this in the lower order function
    if (signal?.aborted === true) {
      throw new ErrorCancellation();
    }
  }
);

setTimeout(() => {
  pC.cancel();
}, 5000);

The timer parameter would just be a convenience parameter to automate a common usecase.

In some of our functions, I can imagine that end up supporting both timer and signal:

class C {
  @cancellable(ErrorCancellation)
  public async (timer?: Timer | number, options?: { signal?: AbortSignal }) {
    timer = timer instanceof Timer ?? new Timer(timer);
    if (timer.timedOut) {
      // Maybe you want to differentiate that this got cancelled by the timer and not by signal?
      throw new ErrorCancellation();
    }
  }
}

If a cancellation error is thrown without actually be cancelled, then this is just a normal exception.

Refactoring Everywhere

Lots of places to refactor. To spec out here TBD.

Additional context

• Create IPC locking library @matrixai/js-file-locks to introduce RWlocks in IPC #290 - Locking operations can make use of this
• Implementing BIP39 recovery, Unattended Bootstrap and Agent Start and Status #283 (comment) - discussion about poll usage and the difference between timers

Tasks

1. Create Timer class into our src/utils
2. Integrate AbortController into Timer
3. Prototype Timer integration into poll utility
4. Prototype Timer integration in network proxies
5. Prototype Timer integration into identities
6. Prototype Timer into our tests and ensure that it works with jest mock timers

[CMCDragonkai]

Another place where this can be applied is the authenticate AsyncGenerator. We can cancel async generators by doing https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Generator/return or https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Generator/throw.

But the underlying pollAccessToken() will not be cancelled immediately. Instead it will loop until one of the exit conditions which includes a timeout.

So having a consistent way of aborting these sorts of operations would be useful.

Being able to abort when we cancel CTRL + C the pk identities authenticate github.com joshuakarp call would be useful.

[CMCDragonkai]

Note that we have a usecase in js-async-init that could make use of a general way of aborting asynchronous operations. It should be optional/advisory so that decorated classes can choose to make use of the abort signal indicated by a stop or destroy operation. If they ignore, that's fine, but if they take it into account, they should then also stop what they are doing and cancel whatever async operation they are doing.

This can be quite useful for blocking async generator methods, so they can cancel their async generator context when the stop signal is raised.

[CMCDragonkai]

It could also be 2 separate libraries, one for in-memory ("in/intra-process") locking, another for IPC locking.

[CMCDragonkai]

• https://github.com/MatrixAI/js-async-locks
• https://github.com/MatrixAI/js-file-locks

js-async-locks
@matrixai/async-locks - async-mutex, RWLock, and variants on that

js-file-locks
@matrixai/file-locks - fd-lock (needs to use C++ and NAPI to get posix or whatever)

[CMCDragonkai]

Some more interesting libraries to try:

• https://github.com/alkemics/CancelablePromise
• https://github.com/srmagura/real-cancellable-promise
• https://github.com/DigitalBrainJS/c-promise
• https://github.com/DigitalBrainJS/use-async-effect