Do notation again

[vegansk]

What do you think about this method of simulating the do notation? https://medium.com/@dhruvrajvanshi/simulating-haskells-do-notation-in-typescript-e48a9501751c

[gcanti]

@vegansk not sure what's the benefit

with assign (excerpt from the linked post)

console.log(
    Maybe.Just({})
    .assign("x", positive(23))
    .assign("y", positive(23))
    .assign("z", (scope) => positive(scope.x - scope.y))
    .then(scope => Maybe.Just(scope.z))
    .match({
        Just:result => `Just(${result})`,
        Nothing:() => "Nothing"
    })
) // Just(0)

with chain (what you can do right now)

console.log(
  positive(23)
    .chain(x => positive(23).chain(y => positive(x - y)))
    .fold(() => 'Nothing', a => `Just(${a})`)
) // Just(0)

[vegansk]

Well, with do notation I don't need to count the closing parenthesis like here :-)

a.chain(_ => b.chain(_ => c.chain(_ => d.chain(_ => e.chain( ... )))))

Seriously speaking, all these nested chain calls get annoying when there are a lot of them. As for me, it looks like choosing between callbacks or async code. For example, in scala code, I use flatMap when there are at most two inner calls, and for-comprehension in other cases.

[raveclassic]

@vegansk @gcanti
Take a look on a version based on generators: https://gist.github.com/raveclassic/f7df7276467d6fcd548248b53d585b4b
It's much more handy than chaining with calling methods and manual 'assigns' to context.

Unfortunately TS lacks correct typings for generators microsoft/TypeScript#2983 (and similar issues) though such trick is possible in flow.

I'm working on a tslint rule to force explicit type on the left side of yield expression and which will also check lhs and rhs types for assignability (described in microsoft/TypeScript#19602) but I'm lacking one important helper in TS checker core - checkTypeAssignableTo. It is expected to land with microsoft/TypeScript#9879 and microsoft/TypeScript#9943 but their status is unknown.

When it's possible to detect inconsistency in lhs and rhs types then it's possible to provide a quickfix for tslint which will automatically and explicitly set the type from the right side on the left side. For example on file save in vscode. (Of course extracting the inner type from the wrapping monad)

Example:

const a = yield some(2)
     ^^^ - error from tslint rule - implicit any
//quickfix
const a: number = yield some(2);
//number is extracted from the right side of `Option<number>`

Any inconsistency will be again caught by tslint and quickfixed.

[raveclassic]

Linking a simpler change microsoft/TypeScript#11728

[vegansk]

@raveclassic , I saw your version. Of course it would be great if we could use generators for do notation, they have much nicer syntax. But I'm not sure if I want to be dependent on tslint rule in my code that rewrites it. I can agree with the use of macros for these purposes, furthermore I did the same thing (do notation) using macros for the Nim language (https://github.com/vegansk/nimfp/blob/master/tests/fp/test_forcomp.nim#L48). But unfortunately we have no macros in TS :-( And that's the reasons why I like the version proposed in the article.

[gcanti]

@vegansk I see, yeah nesting is not nice. So let's see some options

1. Lightweight solution (just chain and an helper function)

function getScoped<F extends HKTS>(
  F: Functor<F>
): <N extends string, A>(name: N, fa: HKTAs<F, A>) => <O>(scope: O) => HKTAs<F, O & { [K in N]: A }>
function getScoped<F>(
  F: Functor<F>
): <N extends string, A>(name: N, fa: HKT<F, A>) => <O>(scope: O) => HKT<F, O & { [K in N]: A }>
function getScoped<F>(
  F: Functor<F>
): <N extends string, A>(name: N, fa: HKT<F, A>) => <O>(scope: O) => HKT<F, O & { [K in N]: A }> {
  return (name, fa) => scope => F.map(a => ({ ...(scope as any), [name]: a }), fa)
}

import * as option from 'fp-ts/lib/Option'

const scoped = getScoped(option)

const result1 = some({})
  .chain(scoped('x', positive(23)))
  .chain(scoped('y', positive(23)))
  .chain(scope => positive(scope.x - scope.y))

// or

const result2 = some({})
  .chain(scoped('x', positive(23)))
  .chain(scoped('y', positive(23)))
  .chain(scope => scoped('z', positive(scope.x - scope.y))(scope))
  .map(scope => scope.z)

2. Module augmentation (additional chain_ method)

(then if it happens to work well we could think to add chain_ (the name is temporary) to all monads)

declare module 'fp-ts/lib/Option' {
  interface None<A> {
    chain_<N extends string, B>(name: N, other: Option<B> | ((a: A) => Option<B>)): Option<A & { [K in N]: B }>
  }
  interface Some<A> {
    chain_<N extends string, B>(name: N, other: Option<B> | ((a: A) => Option<B>)): Option<A & { [K in N]: B }>
  }
}

None.prototype.chain_ = function() {
  return this
}

Some.prototype.chain_ = function(name, other) {
  const fb = typeof other === 'function' ? other(this.value) : other
  return fb.map(b => ({ ...this.value, [name]: b }))
}

const result3 = some({})
  .chain_('x', positive(23))
  .chain_('y', positive(23))
  .chain_('z', scope => positive(scope.x - scope.y))
  .map(scope => scope.z)

// or simply (mixing chain_ and chain)

const result4 = some({})
  .chain_('x', positive(23))
  .chain_('y', positive(23))
  .chain(scope => positive(scope.x - scope.y))  

Others?

[vegansk]

IMHO, first version is good to have it as workaround, outside the library. Second version can be proposed as the standard way to chain monadic operations in fp-ts :-)

[gcanti]

@vegansk have you got some real world examples to test these implementations against?

[vegansk]

@gcanti, sorry, can't show the code, it's closed source. Also it depends on another services, so it can't be used as the test.

I used the synthetic tests while implemented do notation for Nim :-)

[raveclassic]

Looks like there's some movement in microsoft/TypeScript#9943 :)
When it lands I think we could publish a separate experimental package fp-ts-do dependent on fp-ts with tslint rule bundled in. Then when TS fully supports type inference and assertions for generators (I hope we'll see that day) this package could be merged into the core fp-ts.

[vegansk]

@raveclassic , 👍

[pierremarc]

As of real world example, I make use of (2) in a project we're soon to deliver (which makes it rather virtually real world atm). I've seen this issue when looking for something in the vein of clojure's let blocks so I kept the name for it rather than chain_.
It looks like this

  scopeOption()
            .let('lat', fromNullable(getNumber(r[1])))
            .let('lon', fromNullable(getNumber(r[2])))
            .let('zoom', fromNullable(getNumber(r[3])))
            .map(({ lat, lon, zoom }) => {
                viewEvents.updateMapView({
                    dirty: 'geo',
                    center: [lat, lon],
                    zoom,
                })
})

or this (with intermediate use of scope)

const render =
    () => scopeOption()
        .let('row', fromNullable(getSelectedMetadataRow()))
        .let('md', ({ row }) => getDatasetMetadata(row.from as string))
        .fold(
        () => DIV({ className: 'inspire-wrapper empty' }),
       ({ md }) => renderInspireMD(md));

The code above is not yet as clean as I'd like to because there's still a mix of old and new functions due to hoping in the fp-ts train mid project. But still I'm super happy with it as far as I'm concerned. Thank you for the tip.

[edit]
Drifting away from do notation, it occured to me that I was using it to get one final value like

scopeOption()
    .let('a', getA())
    .let('b', ({a}) => getB(a))
    .map(({b}) => b)

Decided to add a pick method, resulting in:

scopeOption()
    .let('a', getA())
    .let('b', ({a}) => getB(a))
    .pick('b')

[gcanti]

FYI there's ts-do by @teves-castro

[RPallas92]

@josete89 and I have have implemented do notation using generators like this:

  it('do notation', () => {

    function taskDo<B>(g: IterableIterator<Task<B>>) {
      const next = (x: B | null): Task<B> => {
        let { value, done } = g.next(x)
        return done
          ? value
          : value.chain(next)
      }
      return next(null)
    }

    const result = taskDo(function* () {
      let x: number = yield task.of(2)
      let y: number = yield task.of(3)
      return task.of(x + y)
    }())

    return result
      .run()
      .then(x => assert.strictEqual(x, 5))
  })

@gcanti, do you think it is worth to implement it for all fp-ts' monads? We can create a PR

[raveclassic]

@RPallas92 Generators are still typeunsafe in TS (as of 3.0). Check #261 (comment)

[RPallas92]

can we do the ts-lint workaround for that ?

[raveclassic]

@RPallas92 Necessary API in TS typechecker is still not published so I guess it's impossible to safely use do-notation based on generators currently.

[RPallas92]

Other option would be to "hack" TS' async/await and make it work with all fp-ts monads.

https://medium.com/@joshuakgoldberg/hacking-typescripts-async-await-awaiter-for-jquery-2-s-promises-60612e293c4b

Then await will be type safe.

[raveclassic]

@RPallas92 async/await is not a complete solution because it is implemented on top of generators which lack some necessary functionality - they cannot be "rerun" from some specific yield point, only resumed (AFAIK). This leads to a situation when you cannot fully use async/await for monads "calling" chain multiple times (Array, Observable etc.) (again I may be wrong).

Do(array)(function*() {
  const a = yield [1, 2, 3];
  console.log(a);  //will this be run 3 times?
})
Do(rxjs)(function*() {
  const a = yield Observable.interval(1000); 
  console.log(a); //will this be run infinite number of times?
}).subscribe()

EDIT: updated the code with logging

[raveclassic]

@RPallas92 It would be helpful if you could provide some POC-repo with this async/await patching and make sure the example above works correctly.

[bmmin]

@RPallas92 Generators are still typeunsafe in TS (as of 3.0). Check #261 (comment)

Now that this is not the case anymore, does it change anything?

[raveclassic]

@bmmin Latest TS improvements for generators do make life better a bit but that's still not enough to properly type both sides of a yield-epxression. So we're still not there yet.

[bmmin]

yeah, I'm guessing if your return type is the same for every yield it might work, but for different return types this seems to be still an issue. Is there any plan or idea if and how this could ever work in a nice way?

[raveclassic]

The only thing coming to my mind is a custom TS transformer plugin. I remember this one https://github.com/funkia/go-notation

[nythrox]

It's possible to get properly typed return values from yield* expressions by adding an iterator into the monads, I was able to implement a type safe version using Purify-ts, but it doesn't work with arrays or observables because of generator's limitations

class Future<L, R> {

  [Symbol.iterator]: () => Iterator<
    Future<L, R>,
    R,
    any
  > = function*() {
     return (yield this) as R
  }

  static for<L, R>(
    fun: () => Generator<
      Future< L, any>,
      R,
      any
    >
  ): Future<L, R> {
       /* implementation details */
  }

I'm able to use it like this

const findGroup: (groupId: string) => Either<Error, Group>
const findTeacher: (teacherId: string) => Either<Error, Teacher>
const findStudent: (studentId: string) => Either<Error, Student>
const addStudentToGroupDb: (studentId: string, groupId: string) => Either<Error, void>;
const incrementTeachersScore : (teacherId: string, incr: any) => Future<Error, void>


const addStudentToGroup = (
  studentId: string,
  groupId: string,
  teacherId: string
) =>  // Future<Error, { group: Group, teacher: Teacher, student: Student  }>
  Future.for(function* () {
    const group = yield* findGroup(groupId);  // Group
    const teacher = yield* findTeacher(teacherId); // Teacher
    const student = yield* findStudent(studentId); // Student

    if (group.players.includes(studentId)) {
      yield* Future(Left(new Error("Already in group"))); // void
    }

    yield* addStudentToGroupDb(student.id, group.id); // void

    yield* incrementTeachersScore(teacher.id, 1); // void


     yield* Future(Left("error!")); // ... Type 'Future<string, never>' is not assignable to type 'Future<Error, any>'.

    const test = yield* Future(Right("hello")) // string

    return {
      group,
      teacher,
      student,
    };
  });

Working sandbox here
https://codesandbox.io/s/spring-http-ngxts?file=/src/index.ts

The only limitations I've found is that it will not work with monads like arrays and streams, but the type inference works incredibly well.

[finkef]

Building on @nythrox' solution, here's a quick shot at using yield* with TaskEither without abstracting the type, so all functions operating on TaskEither are still working:

import * as TE from "fp-ts/lib/TaskEither";
import { sequenceS } from "fp-ts/lib/Apply";

class TEG {
  static *gen<E, A>(te: TE.TaskEither<E, A>) {
    return (yield te) as A;
  }

  static for<E, A>(
    fun: () => Generator<TE.TaskEither<E, any>, A, unknown>
  ): TE.TaskEither<E, A> {
    const iterator = fun();

    const state = iterator.next();

    function run(state: IteratorResult<TE.TaskEither<any, any>, A>): any {
      if (state.done) {
        return TE.right(state.value);
      }
      return TE.chain((val) => {
        return run(iterator.next(val));
      })(state.value as TE.TaskEither<any, any>);
    }

    return run(state);
  }
}

function* test() {
  const one = yield* TEG.gen(TE.right(42));
  const two = yield* TEG.gen(
    sequenceS(TE.taskEither)({ one: TE.right(1), two: TE.right("two") })
  );
  const three = yield* TEG.gen(TE.right(two.one));
  const maybeLeft = yield* TEG.gen(
    Math.random() > 0.5 ? TE.left("string") : TE.right(4)
  );

  // Next line errors because `E` is assumed to be `string` from `maybeLeft`.
  // If line 10 is typed as `TE.TaskEither<any, any>` instead of `TE.TaskEither<E,any>`,
  // the error disappears, but `E` is lost in the result of `for` and typed as `unknown`.
  // const maybeLeftWithDifferentType = yield* TEG.gen(Math.random() > 0.5 ? TE.left(false) : TE.right(0))

  return one + three + maybeLeft;
}

// Typechecks to TE.TaskEither<string, number>
const te = TEG.for(test);

// Logs Right<47> or Left<"string">
te().then(console.log);

Sandbox: https://codesandbox.io/s/elastic-shaw-q1zd2?file=/src/index.ts

[mikearnaldi]

While being very exited about the idea of @nythrox and after having revised it to work without the need to include a generator in the data-types (using a wrapper that contains a generator) I realised this is actually not a "Do".

The code I wrote on the concrete data-type of effect:
https://github.com/Matechs-Garage/matechs-effect/blob/master/packages/system/src/Effect/gen.ts#L63

It does allow a very nice syntax like:

const program = T.gen(function* (_) {
  const a = yield* _(T.access((_: A) => _.a))
  const b = yield* _(T.access((_: B) => _.b))

  const c = a + b

  if (c > 10) {
    yield* _(T.fail(`${c} should be lower then x`))
  }

  return c
})

But the same trick cannot logically work with every "monad", for example it cannot work on "Stream" (and for example in any type that "emits" multiple A like observable), in general every non yielded operation will execute outside the "monad" (it will execute as a result of calling iterator.next(val) and this will make advance the "syntax" iterator).

[mikearnaldi]

Actually @mattiamanzati came up with:

export function gen<Eff extends GenStream<any, any, any>, AEff>(
  f: (i: {
    <R, E, A>(_: Stream<R, E, A>): GenStream<R, E, A>
  }) => Generator<Eff, AEff, any>
): Stream<_R<Eff>, _E<Eff>, AEff> {
  return suspend(() => {
    function run(replayStack: any[]): Stream<any, any, AEff> {
      const iterator = f(adapter as any)
      let state = iterator.next()
      replayStack.forEach((v) => {
        state = iterator.next(v)
      })
      if (state.done) {
        return succeed(state.value)
      }
      return chain_(state.value["effect"], (val) => {
        return run(replayStack.concat([val]))
      })
    }
    return run([])
  })
}

That accounts for the case of Stream so I will have to reconsider my statement that this can in theory be an alternative to Do under the assumption that any non yielded computation is either pure and immutable or mutable but changing only variables binded to the current scope (inside the closure of the generator). (an assumption that, in my opinion, is in line with fp standards).

[mattiamanzati]

One of the beauty of this approach is also that you can detect some edge cases where the code provided by the developer is'nt pure at runtime, see: https://github.com/Matechs-Garage/matechs-effect/blob/455984b68746ca5471fc9e874adf3f568c0ff33c/packages/system/test/gen.test.ts#L216-L230

[mikearnaldi]

confirmed working on HKT:
Effect-TS/effect@c51e745