Improve CLI error messages

[mmkal]

Fixes #17
Closes #36 (by documenting that toMatchTypeOf ~= toExtend)
Closes #37 (by documenting that helper types are not part of the API surface)
Closes #32
Closes #4
Closes #6 (by documenting that you can't use .not.toBeCallableWith(...))
Closes #38

Use generics to give better error messages than "Arguments for the rest parameter 'MISMATCH' were not provided" for most toEqualTypeOf cases and many toMatchTypeOf cases. This trades off some implementation complexity for better end-user error messages. Essentially, write a special <Expected extends ...> constraint for each overload of each method, which does some crazy conditional logic, which boil down to:

• <Expected extends Actual> for toEqualTypeOf (when we aren't in a .not context)
• <Expected extends Satisfies<Actual>> for toMatchTypeOf

Anyone interested, have a look at the snapshot updates in errors.test.ts.snap to see what the real world difference is.

Each of these constraints apply only when we know it's going to "fail" - i.e. Satisfies<...> is a fairly meaningless helper type that is used to try to show errors at the type-constraint level rather than the ...MISMATCH: MismatchArgs<...> level which won't give good error messages.

When using .not, the constraint just becomes extends unknown, and you'll have to squint as before.

See also: #14 for the better long-term solution, if the TypeScript team decide to merge the throw types PR.
See also: #13 for a hopefully-complementary improvement to the information on hover, which will improve the cases this doesn't cover.

TODO:

• See if the expectTypeOf({a: 1}).toMatchTypeOf({a: 'one'}) case can also be improved.
• Document. The constraints are a bit different to what most users would be used to, so it's worth highlighting the best way to read error messages and clarify when they might default to "Arguments for the rest parameter 'MISMATCH' were not provided"

Note: I have publish v0.17.0-1 based on this PR and will hopefully be able to use that version in vitest as a test before merging.

[trevorade]

Nice! Happy to have some better error messages in the command line.

I'll take a closer look tomorrow.

[mmkal]

Update: looks like it's possible to get rid of ...MISMATCH completely. But some more tests/handling of edge cases around never/any are needed. Might also want to automatically snapshot test all the // @ts-expect-error cases in the main test file. Will push commits for that soon.

[trevorade]

Here's something kind of weird...

With these changes in the dx-generics branch, both the following compile:

  expectTypeOf({a: 1}).toMatchTypeOf<{a: number; b?: string}>()
  expectTypeOf({a: 1}).not.toMatchTypeOf<{a: number; b?: string}>()

In main, only the former compiles.

[mmkal]

Here's something kind of weird...

With these changes in the dx-generics branch, both the following compile:

  expectTypeOf({a: 1}).toMatchTypeOf<{a: number; b?: string}>()
  expectTypeOf({a: 1}).not.toMatchTypeOf<{a: number; b?: string}>()

In main, only the former compiles.

Good catch, thank you. I've converted this to a draft since there are a few things up in the air and it isn't ready to go in.

[trevorade]

Spent some time talking about this with @shicks and @12wrigja and we came up with some good ideas for some more dx.

Succinctly, it's a change like this:

const error = Symbol('error')

type ExpectedToBeNull<T> = {[error]: T}
type ExpectedNotToBeNull<T> = {[error]: T}

type IsASuperTypeOf<T> = {[error]: T}
type IsNotASuperTypeOf<T> = {[error]: T}

export type ExpectTypeOf<Actual, B extends boolean> = {
  toBeNull:
     Extends<Actual, null> extends B ?
         () => true :
         B extends true ? ExpectedToBeNull<Actual> : ExpectedToNotBeNull<Actual>

  toMatchTypeOf: {
    <Expected extends (
      B extends true ?
        (Extends<Actual, Expected> extends true ? unknown : IsASuperTypeOf<Actual>) :
        (Extends<Actual, Expected> extends false ? unknown : IsNotASuperTypeOf<Actual>)
    )>(): true
    <Expected extends (
      B extends true ?
          (Extends<Actual, Expected> extends true ? unknown : IsASuperTypeOf<Actual>) :
          (Extends<Actual, Expected> extends false ? unknown : IsNotASuperTypeOf<Actual>)
     )>(
      expected: Expected,
    ): true
  }

For toBeNull, you get errors like:

test/usage.test.ts:97:19 - error TS2349: This expression is not callable.
  Type 'ExpectedToBeNull<number>' has no call signatures.

97   expectTypeOf(1).toBeNull()
                     ~~~~~~~~

test/usage.test.ts:96:26 - error TS2349: This expression is not callable.
  Type 'ExpectedNotToBeNull<null>' has no call signatures.

96   expectTypeOf(null).not.toBeNull()
                            ~~~~~~~~

For toMatchTypeOf, you get an error message like:

test/usage.test.ts:119:54 - error TS2344: Type '{ a: string; }' does not satisfy the constraint 'IsASuperTypeOf<{ a: string | number; }>'.
  Property '[error]' is missing in type '{ a: string; }' but required in type 'IsASuperTypeOf<{ a: string | number; }>'.

119   expectTypeOf<{a: string | number}>().toMatchTypeOf<{a: string}>()
                                                         ~~~~~~~~~~~

  src/index.ts:108:27
    108 type IsASuperTypeOf<T> = {[error]: T}
                                  ~~~~~~~
    '[error]' is declared here.

and for .not:

test/usage.test.ts:27:49 - error TS2344: Type '{ a: string | number; }' does not satisfy the constraint 'IsNotASuperTypeOf<{ a: number; }>'.
  Property '[error]' is missing in type '{ a: string | number; }' but required in type 'IsNotASuperTypeOf<{ a: number; }>'.

27   expectTypeOf<{a: number}>().not.toMatchTypeOf<{a: number|string}>()
                                                   ~~~~~~~~~~~~~~~~~~

  src/index.ts:108:30
    108 type IsNotASuperTypeOf<T> = {[error]: T}
                                     ~~~~~~~
    '[error]' is declared here.

This loses tsc telling you specifically what wasn't matching between expected and actual as having the previous comparison be backwards was a bit misleading but it does print out the types so you're aware of what types you were dealing with. Unfortunately, there's quite a bit of noise in these.

I recognize that this snippet doesn't handle unknown and never correctly like it did before so you'd probably want to adjust for that as well.

[papb]

Hello, I didn't really look well at the PR, but just wanted to mention that I've noticed that the message that appears when hovering on an IDE can change significantly between TS versions. In particular, the expansions of generic expressions. Some versions may show Foo<Bar> while others may show things like { resultOfExpandingFoo: Bar }. Especially with complex types (with many layers of generics). Sometimes, the result of expanding the expression was very simple, but still TS would show things like X extends infer Y ? NonNullable<DeepPartial<Y>> : Something<X> instead of resolving into { a: 1 }. Just a reminder that this may happen. Maybe you put lots of effort in giving users good messages on hovering, but then a new TS version comes (every month!) and it could break...

[shicks]

Seems like the easy solution here is to just make sure any possible expansion is readable. In particular,

type IsASupertypeOf<T> = {[error]: T};

is not particularly readable if it's expanded, but if the internal symbol were instead

type IsASupertypeOf<T> = {[isASuperTypeOf]: T};

then it shouldn't really matter whether it's expanded or not - I can't imagine any reasonable way TypeScript could format that badly.

[mmkal]

Spent some time talking about this with @shicks and @12wrigja and we came up with some good ideas for some more dx.

Succinctly, it's a change like this:
...

@trevorade @shicks @12wrigja I did something like this and I think it's a big improvement - thank you!! I ended up doing lots of refactoring, including separating out positive from negative assertions and reduce .branded down to just the two methods that it can apply to. This reduced the number of crazy ternaries in the code, and allowed for better error messages generally by guiding typescript into choosing specific overloads. If you want to take a look feel free.

[mmkal]

I'm merging this and will release as v0.17.0 soon.