Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix custom async matcher stack traces #7652

Merged
merged 10 commits into from
Feb 13, 2019
Merged
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
### Fixes

- `[jest-cli]` Refactor `-o` and `--coverage` combined ([#7611](https://github.com/facebook/jest/pull/7611))
- `[expect]` Fix custom async matcher stack trace ([#7652](https://github.com/facebook/jest/pull/7652))

### Chore & Maintenance

Expand Down
33 changes: 26 additions & 7 deletions e2e/__tests__/__snapshots__/customMatcherStackTrace.test.js.snap
Original file line number Diff line number Diff line change
@@ -1,7 +1,26 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`custom async matchers 1`] = `
FAIL __tests__/asynchronous.test.js
✕ showing the stack trace for an async matcher

● showing the stack trace for an async matcher

We expect the stack trace and code fence for this matcher to be shown in the console.

9 |
10 | test('showing the stack trace for an async matcher', async () => {
> 11 | await expect(true).toThrowCustomAsyncMatcherError();
| ^
12 | });
13 |
14 | async function toThrowCustomAsyncMatcherError() {

at Object.toThrowCustomAsyncMatcherError (__tests__/asynchronous.test.js:11:22)
`;

exports[`works with custom matchers 1`] = `
FAIL __tests__/customMatcher.test.js
FAIL __tests__/sync.test.js
Custom matcher
✓ passes
✓ fails
Expand All @@ -19,10 +38,10 @@ FAIL __tests__/customMatcher.test.js
47 |
48 | // This expecation fails due to an error we throw (intentionally)

at Error (__tests__/customMatcher.test.js:45:13)
at baz (__tests__/customMatcher.test.js:43:23)
at bar (__tests__/customMatcher.test.js:42:23)
at foo (__tests__/customMatcher.test.js:52:7)
at Object.callback (__tests__/customMatcher.test.js:11:18)
at Object.toCustomMatch (__tests__/customMatcher.test.js:53:8)
at Error (__tests__/sync.test.js:45:13)
at baz (__tests__/sync.test.js:43:23)
at bar (__tests__/sync.test.js:42:23)
at foo (__tests__/sync.test.js:52:7)
at Object.callback (__tests__/sync.test.js:11:18)
at Object.toCustomMatch (__tests__/sync.test.js:53:8)
`;
80 changes: 45 additions & 35 deletions e2e/__tests__/__snapshots__/expectAsyncMatcher.test.js.snap
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,40 @@ FAIL __tests__/failure.test.js
● fail with expected non promise values

Expected value to have length:
2
Received:
1
received.length:
1

11 |
12 | it('fail with expected non promise values', () =>
> 13 | expect([1]).toHaveLengthAsync(Promise.resolve(2)));
| ^
14 |
15 | it('fail with expected non promise values and not', () =>
16 | expect([1, 2]).not.toHaveLengthAsync(Promise.resolve(2)));

2
Received:
1
received.length:
1
at Object.toHaveLengthAsync (__tests__/failure.test.js:13:15)

● fail with expected non promise values and not

Expected value to not have length:
2
Received:
1,2
received.length:
2

14 |
15 | it('fail with expected non promise values and not', () =>
> 16 | expect([1, 2]).not.toHaveLengthAsync(Promise.resolve(2)));
| ^
17 |
18 | it('fail with expected promise values', () =>
19 | expect(Promise.resolve([1])).resolves.toHaveLengthAsync(Promise.resolve(2)));

2
Received:
1,2
received.length:
2
at Object.toHaveLengthAsync (__tests__/failure.test.js:16:22)

● fail with expected promise values

Expand All @@ -36,19 +54,15 @@ FAIL __tests__/failure.test.js
received.length:
1

22 |
23 | it('fail with expected promise values', async () => {
> 24 | await (expect(Promise.resolve([1])): any).resolves.toHaveLengthAsync(
| ^
25 | Promise.resolve(2)
26 | );
27 | });
17 |
18 | it('fail with expected promise values', () =>
> 19 | expect(Promise.resolve([1])).resolves.toHaveLengthAsync(Promise.resolve(2)));
| ^
20 |
21 | it('fail with expected promise values and not', () =>
22 | expect(Promise.resolve([1, 2])).resolves.not.toHaveLengthAsync(

at Object.toHaveLengthAsync (__tests__/failure.test.js:24:54)
at asyncGeneratorStep (__tests__/failure.test.js:11:103)
at _next (__tests__/failure.test.js:13:194)
at __tests__/failure.test.js:13:364
at Object.<anonymous> (__tests__/failure.test.js:13:97)
at Object.toHaveLengthAsync (__tests__/failure.test.js:19:41)

● fail with expected promise values and not

Expand All @@ -59,17 +73,13 @@ FAIL __tests__/failure.test.js
received.length:
2

28 |
29 | it('fail with expected promise values and not', async () => {
> 30 | await (expect(Promise.resolve([1, 2])).resolves.not: any).toHaveLengthAsync(
| ^
31 | Promise.resolve(2)
32 | );
33 | });

at Object.toHaveLengthAsync (__tests__/failure.test.js:30:61)
at asyncGeneratorStep (__tests__/failure.test.js:11:103)
at _next (__tests__/failure.test.js:13:194)
at __tests__/failure.test.js:13:364
at Object.<anonymous> (__tests__/failure.test.js:13:97)
20 |
21 | it('fail with expected promise values and not', () =>
> 22 | expect(Promise.resolve([1, 2])).resolves.not.toHaveLengthAsync(
| ^
23 | Promise.resolve(2)
24 | ));
25 |

at Object.toHaveLengthAsync (__tests__/failure.test.js:22:48)
`;
12 changes: 11 additions & 1 deletion e2e/__tests__/customMatcherStackTrace.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import {extractSummary} from '../Utils';
import {wrap} from 'jest-snapshot-serializer-raw';

test('works with custom matchers', () => {
const {stderr} = runJest('custom-matcher-stack-trace');
const {stderr} = runJest('custom-matcher-stack-trace', ['sync.test.js']);

let {rest} = extractSummary(stderr);

Expand All @@ -23,3 +23,13 @@ test('works with custom matchers', () => {

expect(wrap(rest)).toMatchSnapshot();
});

test('custom async matchers', () => {
const {stderr} = runJest('custom-matcher-stack-trace', [
'asynchronous.test.js',
]);

const {rest} = extractSummary(stderr);

expect(wrap(rest)).toMatchSnapshot();
});
18 changes: 18 additions & 0 deletions e2e/custom-matcher-stack-trace/__tests__/asynchronous.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/**
* Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

expect.extend({toThrowCustomAsyncMatcherError});

test('showing the stack trace for an async matcher', async () => {
await expect(true).toThrowCustomAsyncMatcherError();
});

async function toThrowCustomAsyncMatcherError() {
const message = () =>
'We expect the stack trace and code fence for this matcher to be shown in the console.';
return {message, pass: false};
}
29 changes: 10 additions & 19 deletions e2e/expect-async-matcher/__tests__/failure.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,21 @@
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
'use strict';

import {toHaveLengthAsync} from '../matchers';

expect.extend({
toHaveLengthAsync,
});
expect.extend({toHaveLengthAsync});

it('fail with expected non promise values', async () => {
await (expect([1]): any).toHaveLengthAsync(Promise.resolve(2));
});
it('fail with expected non promise values', () =>
expect([1]).toHaveLengthAsync(Promise.resolve(2)));

it('fail with expected non promise values and not', async () => {
await (expect([1, 2]): any).not.toHaveLengthAsync(Promise.resolve(2));
});
it('fail with expected non promise values and not', () =>
expect([1, 2]).not.toHaveLengthAsync(Promise.resolve(2)));

it('fail with expected promise values', async () => {
await (expect(Promise.resolve([1])): any).resolves.toHaveLengthAsync(
Promise.resolve(2)
);
});
it('fail with expected promise values', () =>
expect(Promise.resolve([1])).resolves.toHaveLengthAsync(Promise.resolve(2)));

it('fail with expected promise values and not', async () => {
await (expect(Promise.resolve([1, 2])).resolves.not: any).toHaveLengthAsync(
it('fail with expected promise values and not', () =>
expect(Promise.resolve([1, 2])).resolves.not.toHaveLengthAsync(
Promise.resolve(2)
);
});
));
29 changes: 10 additions & 19 deletions e2e/expect-async-matcher/__tests__/success.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,21 @@
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
'use strict';

import {toHaveLengthAsync} from '../matchers';

expect.extend({
toHaveLengthAsync,
});
expect.extend({toHaveLengthAsync});

it('works with expected non promise values', async () => {
await (expect([1]): any).toHaveLengthAsync(Promise.resolve(1));
});
it('works with expected non promise values', () =>
expect([1]).toHaveLengthAsync(Promise.resolve(1)));

it('works with expected non promise values and not', async () => {
await (expect([1, 2]): any).not.toHaveLengthAsync(Promise.resolve(1));
});
it('works with expected non promise values and not', () =>
expect([1, 2]).not.toHaveLengthAsync(Promise.resolve(1)));

it('works with expected promise values', async () => {
await (expect(Promise.resolve([1])).resolves: any).toHaveLengthAsync(
Promise.resolve(1)
);
});
it('works with expected promise values', () =>
expect(Promise.resolve([1])).resolves.toHaveLengthAsync(Promise.resolve(1)));

it('works with expected promise values and not', async () => {
await (expect(Promise.resolve([1, 2])).resolves.not: any).toHaveLengthAsync(
it('works with expected promise values and not', () =>
expect(Promise.resolve([1, 2])).resolves.not.toHaveLengthAsync(
Promise.resolve(1)
);
});
));
14 changes: 12 additions & 2 deletions packages/expect/src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,10 @@ const makeThrowingMatcher = (
utils,
};

const processResult = (result: SyncExpectationResult) => {
const processResult = (
result: SyncExpectationResult,
asyncError?: JestAssertionError,
) => {
_validateResult(result);

getState().assertionCalls++;
Expand All @@ -264,6 +267,9 @@ const makeThrowingMatcher = (
if (err) {
error = err;
error.message = message;
} else if (asyncError) {
error = asyncError;
error.message = message;
} else {
error = new JestAssertionError(message);

Expand Down Expand Up @@ -307,9 +313,13 @@ const makeThrowingMatcher = (

if (isPromise((potentialResult: any))) {
const asyncResult = ((potentialResult: any): AsyncExpectationResult);
const asyncError = new JestAssertionError();
if (Error.captureStackTrace) {
Error.captureStackTrace(asyncError, throwingMatcher);
}

return asyncResult
.then(aResult => processResult(aResult))
.then(aResult => processResult(aResult, asyncError))
Copy link

@erikpuk erikpuk Oct 21, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm having some issues with this code, 8 months on. I'm almost certainly not understanding correctly, but it seems to me....

  1. asyncError is always truthy here. Even though this is the normal codepath that any async matcher will follow

  2. In processResult we just test the truthiness of asyncError to decide whether an error was thrown.

  3. That means we end up throwing a normal { message, pass } assertion result, which is not what is supposed to happen.

But.... what that suggests to me is that async custom would just be fully broken all the time, which.... I mean, I assume they're tested? So I assume I'm missing something?

But for me, they are in fact broken all the time.

Copy link
Contributor Author

@theotherdon theotherdon Oct 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What issue are you having? The same one that I fixed in the description or a different one? It's been a while since I worked on this, so I don't remember what we're supposed to be throwing for point 3. Can you remind me what we're supposed to be throwing instead?

asyncError will indeed always be defined if we're using async matchers. However, the only time we do something with the asyncError is when the assertion fails because of this conditional here.

A little bit of context that I'm not sure you're aware of, but I think might be helpful. We always need to pass in an error to preserve the stack trace when we're working with async code. If we don't, then the stack trace is lost when we hit the next tick in the event loop. That's why the async error is always being initialized and passed in. It's just not used unless we hit an error state. Unless I'm misunderstanding what you're saying?

.catch(error => handlError(error));
} else {
const syncResult = ((potentialResult: any): SyncExpectationResult);
Expand Down