Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests pass but vitest fails non-deterministically and sometimes causes Unhandled Rejection #1692

Closed
6 tasks done
prabhatsharma opened this issue Jul 20, 2022 · 10 comments · Fixed by #2253
Closed
6 tasks done

Comments

@prabhatsharma
Copy link

Describe the bug

While test cases results themselves are deterministic but the runs are not. Some of the runs produce unhandled rejection. This seems to be similar to #1191 but not same. My test specs are fairly minimal at this time, like just checking for component rendering and not much.

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

Vitest caught 2 unhandled errors during the test run. This might cause false positive tests.
Please, resolve all the errors to make sure your tests are not affected.

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Rejection ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: 
 ❯ post node_modules/vitest/dist/worker.mjs:70:14
 ❯ node_modules/vitest/dist/chunk-vite-node-utils.8077cd3c.mjs:1376:11
 ❯ sendCall node_modules/vitest/dist/chunk-vite-node-utils.8077cd3c.mjs:1373:16
 ❯ node_modules/vitest/dist/chunk-runtime-rpc.9d1f4c48.mjs:7:55
      5|   return new Proxy(rpc2, {
      6|     get(target, p, handler) {
      7|       const sendCall = Reflect.get(target, p, handler);
       |                                                       ^
      8|       const safeSendCall = (...args) => withSafeTimers(() => sendCall(...args));
      9|       safeSendCall.asEvent = sendCall.asEvent;
 ❯ withSafeTimers node_modules/vitest/dist/chunk-utils-global.0a7416cf.mjs:472:20
 ❯ Proxy.safeSendCall node_modules/vitest/dist/chunk-runtime-rpc.9d1f4c48.mjs:7:41
 ❯ process.<anonymous> node_modules/vitest/dist/worker.mjs:34:11
 ❯ process.emit node:events:527:28
 ❯ emit node:internal/process/promises:140:20

Reproduction

You should be able to reproduce it using this repo.

https://github.com/zinclabs/zinc/tree/38303f59af5f329c62068a320eecd4f53c17b107

steps.

git clone https://github.com/zinclabs/zinc
cd zinc
git checkout 38303f59af5f329c62068a320eecd4f53c17b107
cd web
npm i 
npm run test-once

run the test a couple of times. You will notice that sometimes tests are successful however sometimes there is unhandled rejection.

System Info

System:
    OS: macOS 12.4
    CPU: (8) arm64 Apple M2
    Memory: 372.69 MB / 24.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.16.0 - /usr/local/bin/node
    npm: 8.11.0 - /usr/local/bin/npm
  Browsers:
    Brave Browser: 103.1.41.96
    Chrome: 103.0.5060.134
    Safari: 15.5
  npmPackages:
    @vitejs/plugin-vue: ^2.3.3 => 2.3.3 
    vite: ^2.9.14 => 2.9.14 
    vitest: ^0.18.1 => 0.18.1

Used Package Manager

npm

Validations

@nieyuyao
Copy link
Contributor

nieyuyao commented Jul 24, 2022

This should be caused by some promises not catching errors. For example, the templateService.list() in the getTemplates needs to add a catch.

@rogerleung0411
Copy link

May i ask there is any possible solution? same issue here :(

@calvinf
Copy link

calvinf commented Oct 27, 2022

If you have a vitest setup file via --setupFiles (link), inside of it you can add a very loud failure for unhandledRejection.

// FAIL LOUDLY on unhandled promise rejections / errors
process.on('unhandledRejection', (reason) => {
  // eslint-disable-next-line no-console
  console.log(`FAILED TO HANDLE PROMISE REJECTION`);
  throw reason;
});

This helped me to find the specific file that I was failing to await a Promise that needed to be handled (which was difficult due to hundreds of files/tests).

@sheremet-va
Copy link
Member

@calvinf thank you for pointing it out, Vitest didn't serialise errors thrown in unhandledRejection, so some errors might've slipped as Unknown Error: undefined. #2253 should fix it.

@hugw
Copy link

hugw commented Nov 8, 2022

@sheremet-va could you tell if the issue fixed relates to my scenario? Facing the same problem right now and I'm using version 0.24.5.
I am randomly getting unhandled rejection errors.
Like 30% of the time the tests pass but get this unhanded rejection that doesn't show exactly which test caused it.

Screen Shot 2022-11-08 at 12 34 40 PM

@sheremet-va
Copy link
Member

@sheremet-va could you confirm if the issue was fixed? Facing the same problem right now and I'm using version 0.24.5.

I am randomly getting unhandled rejection errors.

Like 30% of the time the tests pass but get this unhanded rejection that doesn't show exactly which test caused it.

Screen Shot 2022-11-08 at 12 34 40 PM

Yes, mentioned in this PR error is fixed.

You probably have an asynchronous process that's executed after all tests have passed. Vitest cannot track it, it's unhandled.

@hugw
Copy link

hugw commented Nov 8, 2022

I see, thanks for getting back.

@IlyaEremin
Copy link

@sheremet-va I'm facing smae issue with unhandled error. What's the strategies to catch it and find a source?

@jtilford
Copy link

jtilford commented Dec 28, 2022

@sheremet-va I'm facing smae issue with unhandled error. What's the strategies to catch it and find a source?

UPDATE:
I caught additional errors, but this new fix to LogRocket seems to have worked.

Seems to have worked

One of the errors seemed to be related to LogRocket. The error was TypeError: Cannot set properties of undefined (setting '_LRLogger').

So I ended up refactoring my LogRocket import like this:

async function importLogRocket(): Promise<any> {
  if (process.env.VITEST === 'true') return Promise.resolve(null);
  return await import('logrocket');
}

let LogRocket: any = null;

if (process.env.VITEST !== 'true') {
  console.debug('LogRocket init');
  importLogRocket().then((imported_logrocket) => {
    LogRocket = imported_logrocket.default;
    LogRocket.init(LR_ACCOUNT_ID);
  });
}

I'm sure I can refactor further, and probably get rid of the explicit any types for something more specific.


Didn't work on its own

I followed the steps above in #1692 (comment)

My vitest has a config file, vitest.config.ts in the root of my project.

The config contains this (truncated example, I removed environment, include, and plugins):

// vitest.config.ts

export default defineConfig({
  test: {
    setupFiles: ['test/vitest/setup-file.ts'],
  }
});

The setup file now has the snippet mentioned above:

// test/vitest/setup-file.ts

// FAIL LOUDLY on unhandled promise rejections / errors
process.on('unhandledRejection', (reason) => {
  // eslint-disable-next-line no-console
  console.log('FAILED TO HANDLE PROMISE REJECTION');
  throw reason;
});

export default {};

(Note the export default {}; at the end, to make it a valid TS file.)

All I can say is, while I don't yet understand why, the non-deterministic failures seem to have disappeared, and I'm also not getting the console log from setup-file.ts.

@kb-0311
Copy link

kb-0311 commented Feb 9, 2023

@sheremet-va If I do have some async process running in a test suite and it passes only when all the other tests have passes, else throws this unhandled error as mentioned #1692 (comment) what might be the best way to handle it?

@thoriumdesign did your solution work? did it stop throwing errors non-determinstically ?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants