Add Async Test Environment APIs #4506

xfumihiro · 2017-09-19T02:13:39Z

Summary

Adding Async Test Environment APIs based on @cpojer's comments on #3420 & #3832
Fixes #3420

Test plan

Example Puppeteer Environment

class PuppeteerEnvironment extends NodeEnvironment {
  constructor(config) {
    super(config)
  }

  async setup() {
    await super.setup()
    this.global.__BROWSER__ = await puppeteer.connect({
      browserWSEndpoint: this.global.browserWSEndpoint
    })
  }

  async teardown() {
    await this.global.__BROWSER__.close()
    await super.teardown()
  }

  runScript(script) {
    return super.runScript(script)
  }
}

SimenB · 2017-09-19T06:11:35Z

packages/jest-runner/src/run_test.js

@@ -100,7 +100,11 @@ export default function runTest(
    mapCoverage: globalConfig.mapCoverage,
  });
  const start = Date.now();
-  return testFramework(globalConfig, config, environment, runtime, path)
+  return (environment.setup ? environment.setup() : (async () => ({}))())


return (environment.setup ? environment.setup() : Promise.resolve({}))

can be used 2 other places as well

Thanks! TIL :)

cpojer · 2017-09-19T07:51:27Z

packages/jest-runner/src/run_test.js

-          if (globalConfig.logHeapUsage) {
-            if (global.gc) {
-              global.gc();
+        Promise.resolve()


Could we instead use the async modifier on the promise callback and use await? I think that would be tons clearer for this code!

codecov-io · 2017-09-20T12:00:01Z

Codecov Report

Merging #4506 into master will decrease coverage by 0.02%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #4506      +/-   ##
==========================================
- Coverage   57.13%   57.11%   -0.03%     
==========================================
  Files         195      194       -1     
  Lines        6569     6568       -1     
  Branches        3        3              
==========================================
- Hits         3753     3751       -2     
- Misses       2816     2817       +1

Impacted Files	Coverage Δ
packages/jest-editor-support/src/Snapshot.js	`98.07% <ø> (ø)`	⬆️
packages/jest-runner/src/run_test.js	`2.27% <0%> (+0.05%)`	⬆️
packages/jest-environment-node/src/index.js	`57.69% <0%> (-4.81%)`	⬇️
packages/jest-environment-jsdom/src/index.js	`41.37% <0%> (-3.07%)`	⬇️
packages/jest-snapshot/src/plugins.js	`100% <0%> (ø)`	⬆️
packages/jest-snapshot/src/mock_serializer.js

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8a427eb...a259379. Read the comment docs.

xfumihiro · 2017-09-24T21:30:43Z

@cpojer any thing to improve?

cpojer · 2017-09-25T02:28:29Z

packages/jest-runner/src/run_test.js

-    .then((result: TestResult) => {
+  return (environment.setup
+    ? environment.setup()
+    : Promise.resolve({})).then(async testSetup => {


This is really hard to read. Can we not use a ternary here? Thanks!

cpojer · 2017-09-25T02:29:10Z

packages/jest-runner/src/run_test.js

-          throw err;
-        }),
-    );
+      if (environment.teardown) await environment.teardown();


this needs curly braces, wondering why CI didn't catch that :O

https://github.com/prettier/eslint-config-prettier/blob/0bfb266df652850857dc0340af41e6bc2a43434f/README.md#curly

cpojer · 2017-09-25T02:29:18Z

packages/jest-runner/src/run_test.js

+      // Delay the resolution to allow log messages to be output.
+      return new Promise(resolve => setImmediate(() => resolve(result)));
+    } catch (err) {
+      if (environment.teardown) await environment.teardown();


curly braces as well pls

xfumihiro · 2017-09-30T03:01:24Z

@cpojer any thing else?

cpojer · 2017-09-30T06:34:39Z

packages/jest-runner/src/run_test.js

-  return testFramework(globalConfig, config, environment, runtime, path)
-    .then((result: TestResult) => {
+  return environment.setup().then(async testSetup => {
+    setGlobal(environment.global, 'setup', testSetup);


This shouldn't be here, we don't want to expose this function on the global object.

For some use case(i.e, the Test Plan of this PR) it's necessary to set global variables in the setup function for later usage.

maybe

return environment.setup().then(async setupGlobal => { setGlobal(environment.global, 'setup', setupGlobal);

would be better

I prefer not to do this. The change in this PR should not automatically add things to the global scope. The setup function in the environment has access to the global scope and we can leave it to the responsibility of the user.

cpojer · 2017-09-30T06:36:28Z

types/Environment.js

@@ -31,6 +31,8 @@ declare class $JestEnvironment {
  },
  testFilePath: string,
  moduleMocker: ModuleMocker,
+  setup(): Promise<Global>,


Can you change the signature to Promise? I don't think we need to return the global here.

cpojer

Two more changes to clean up this feature please. It would also be great if you could add an integration test for this, with a custom test framework.

xfumihiro · 2017-10-04T01:58:09Z

@cpojer can you rebulid the failing ci-tests? It should be working.

SimenB · 2017-10-04T19:01:08Z

@xfumihiro rebuild didn't help. Have you tested on node 4 and/or 6? Node 8 passes CI, so if you've just tested on latest node locally that might explain it.

devsnek · 2017-10-04T20:45:44Z

wouldn't you want runScript to be async as well? for the puppeteer example you would want scripts to run in the browser (page.evaluate) which is async.

SimenB · 2017-10-04T20:50:40Z

integration_tests/__tests__/test_environment.test.js

@@ -14,10 +14,9 @@ const runJest = require('../runJest');
 const testFixturePackage = require('../test-environment/package.json');

 it('respects testEnvironment docblock', () => {
-  expect(testFixturePackage.jest.testEnvironment).toEqual('node');


can you create a new test instead of modifying an existing one?

xfumihiro · 2017-10-05T01:48:25Z

@devsnek not sure if this should be an issue as I've ran puppeteer tests in my own app successfully.
Once having the browser instance from the setup hook, I thought it would be working just fine in Jest async test case.🤔

devsnek · 2017-10-05T03:08:08Z

@xfumihiro right but it would be cooler to just do that at the environment level, allowing a better test flow for future environments whatever they might be. you could also make it so runScript takes a string intead of vm.Script, for even more flexibility, although that might be out of the scope of this pr.

SimenB · 2017-10-13T07:15:10Z

packages/jest-runner/src/run_test.js

+      result.memoryUsage = process.memoryUsage().heapUsed;
+    }
+    // Delay the resolution to allow log messages to be output.
+    return new Promise(resolve => setImmediate(() => resolve(result)));


this can probably be

await new Promise(resolve => setImmediate(resolve)); return result;

No idea if it affects the stacktraces of matchers, though.

Feel free to ignore!

cpojer · 2017-10-13T09:51:35Z

Thank you.

cpojer · 2017-10-13T09:54:45Z

Published 21.3.0-beta.2.

rotemmiz · 2017-10-13T12:18:52Z

This is the last piece of the puzzle , we can now integrate Jest support in Detox!
Thank you guys!

orta · 2017-10-13T12:49:22Z

Great PR 👍

delgermurun · 2017-10-26T09:19:06Z

It still hasn't released to stable version, then why it is documented?

http://facebook.github.io/jest/docs/en/configuration.html#testenvironment-string

class CustomEnvironment extends NodeEnvironment {
  constructor(config) {
    super(config);
  }

  async setup() {
    await super.setup();
    await someSetupTasks();
  }

  async teardown() {
    await someTeardownTasks();
    await super.teardown();
  }

  runScript(script) {
    return super.runScript(script);
  }
}

You guys should've merge document's change after releasing it.

SimenB · 2017-10-26T09:54:29Z

It should include a note that's it's not available until 21.3.0. PR welcome!

xfumihiro · 2017-10-26T10:00:08Z

Done #4765

rdeltour · 2017-10-28T21:07:36Z

I'm experimenting with an async environment (great PR! 👍), but apparently it doesn't run on Node v6:

   /home/travis/build/daisy/ace/tests/jest-env-puppeteer.js:10
      async setup() {
            ^^^^^   
    SyntaxError: Unexpected identifier    
      at node_modules/jest-runner/build/run_test.js:60:29

Will async environments be limited to versions of Node supporting async/await?

SimenB · 2017-10-28T21:11:20Z

Just do return setup().then... instead of async/await

rdeltour · 2017-10-28T21:27:23Z

Just do return setup.then... instead of async/await

Doh. Of course, makes sense. Thanks!

vajahath · 2017-11-24T12:05:03Z

I'm facing some issues with this async environment. But can't determine if the hooks are properly working.
here is the abstract:

reset.js

require('ts-node/register'); // cz, the below required file is written in typescript
const resetDB = require('./reset-database');

module.exports = function() {
	return resetDB(); // returns Promise
};
/* I tried running the below code to verify this file is working. It is working */
// resetDB()
// 	.then(() => console.log('ok'))
// 	.catch(err => console.log(err));

jest-env.js

const NodeEnvironment = require('jest-environment-node');
const resetDatabase = require('./reset');

class CustomEnvironment extends NodeEnvironment {
	constructor(config) {
		super(config);
	}

	async setup() {
		await super.setup();
		console.log('setup running') // not prinitng
		await resetDatabase();
	}

	async teardown() {
		await resetDatabase();
		await super.teardown();
	}

	runScript(script) {
		return super.runScript(script);
	}
}

module.exports = CustomEnvironment;

these hooks are not 'properly' working for me. I can't see any consoles or database resets either. How do I debug this? Is it a bug?

package.json abstract

	"jest": {
		"globals": {
			"ts-jest": {
				"tsConfigFile": "tsconfig.json"
			}
		},
		"moduleFileExtensions": [
			"ts",
			"js"
		],
		"transform": {
			"^.+\\.(ts)$": "ts-jest"
		},
		"testMatch": [
			"**/tests/**/*.spec.ts"
		],
		"testEnvironment": "<path to jest-env.js>" // <--
	}

SimenB · 2017-11-24T16:08:16Z

Are you using the beta of jest?

imballinst · 2017-11-30T04:28:36Z

Hello, first of all, thanks for the great PR! I want to ask a question, do constructor, setup, teardown and runScript run only once across all test files or once between test files? Because in my setup (I'm not sure though if I'm doing this right):

const NodeEnvironment = require('jest-environment-node');
const MongoDriver = require('../mongo/driver');

class CustomEnvironment extends NodeEnvironment {
  constructor(config) {
    super(config);

    console.log('halo');
  }

  setup() {
    return super.setup()
      .then(() => {
        // setup tasks
        console.log('halo1');
      });
  }

  teardown() {
    const teardownTask = new Promise((resolve, reject) => {
      console.log('halo2');
      resolve((MongoDriver.openDBConnection()));
    });

    return teardownTask.then(() => super.teardown());
  }

  runScript(script) {
    return super.runScript(script);
  }
}

module.exports = CustomEnvironment;

When the tests are running, these halos are always printed. I thought it should run only once for all tests. Am I missing something here? Using "jest": "^21.3.0-beta.2", in my package.json with Node v6.11.5. Thank you for explanation.

xfumihiro · 2017-11-30T05:08:45Z

@imballinst Jest's test environments are "sandboxed" so this is normal.
I've submit a PR for global setup/teardown.

imballinst · 2017-11-30T05:14:17Z

@xfumihiro alright, thanks man!

github-actions · 2021-05-13T02:30:37Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

facebook-github-bot added the CLA Signed ✔️ label Sep 19, 2017

SimenB reviewed Sep 19, 2017

View reviewed changes

xfumihiro force-pushed the async_test_environment branch from a49cb57 to e7ea47c Compare September 19, 2017 07:42

cpojer reviewed Sep 19, 2017

View reviewed changes

xfumihiro force-pushed the async_test_environment branch 4 times, most recently from c9989c9 to f7e61f9 Compare September 20, 2017 11:50

cpojer reviewed Sep 25, 2017

View reviewed changes

xfumihiro force-pushed the async_test_environment branch from f7e61f9 to 2645683 Compare September 25, 2017 03:14

cpojer mentioned this pull request Sep 29, 2017

Global beforeAll / Execute some code only once before ALL test files #4565

Closed

tivac mentioned this pull request Sep 29, 2017

userContext not shared from beforeEach #3673

Closed

cpojer reviewed Sep 30, 2017

View reviewed changes

cpojer previously requested changes Sep 30, 2017

View reviewed changes

xfumihiro force-pushed the async_test_environment branch 2 times, most recently from 5fb8947 to 8a50998 Compare October 3, 2017 08:50

SimenB reviewed Oct 4, 2017

View reviewed changes

xfumihiro force-pushed the async_test_environment branch 2 times, most recently from 49bee97 to 71fda69 Compare October 5, 2017 03:05

xfumihiro force-pushed the async_test_environment branch from bfd8559 to bae1802 Compare October 13, 2017 07:10

Add Async Test Environment APIs

93d9640

xfumihiro force-pushed the async_test_environment branch from bae1802 to 93d9640 Compare October 13, 2017 07:12

SimenB reviewed Oct 13, 2017

View reviewed changes

cpojer merged commit d1cb395 into jestjs:master Oct 13, 2017

xfumihiro mentioned this pull request Oct 13, 2017

Update Configuration.md #4689

Merged

rzane mentioned this pull request Oct 14, 2017

Add example to testEnvironment in Configuration.md #4690

Merged

vajahath mentioned this pull request Nov 30, 2017

Add Global Setup/Teardown APIs #4716

Merged

SimenB mentioned this pull request Dec 13, 2017

global beforeAll #3832

Closed

xfumihiro mentioned this pull request Jan 4, 2018

Extend NodeEnvironment to avoid side effect methods NimaSoroush/differencify#62

Open

renovate-tests mentioned this pull request Feb 6, 2018

Update dependency jest to v22 - autoclosed renovate-tests/changelog-md#3

Closed

renovate bot mentioned this pull request Feb 14, 2018

fix(deps): update dependency jest to ^22.0.0 manekinekko/devoxx-assistant#3

Open

Glavin001 mentioned this pull request Feb 27, 2018

Does this work with Atom 1.24? keplersj/atom-test-runner-jest#1

Closed

github-actions bot locked as resolved and limited conversation to collaborators May 13, 2021

Add Async Test Environment APIs #4506

Add Async Test Environment APIs #4506

Conversation

xfumihiro commented Sep 19, 2017 • edited Loading

SimenB Sep 19, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Sep 20, 2017 • edited Loading

Codecov Report

xfumihiro commented Sep 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xfumihiro commented Sep 30, 2017

Choose a reason for hiding this comment

xfumihiro Sep 30, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cpojer left a comment

Choose a reason for hiding this comment

xfumihiro commented Oct 4, 2017

SimenB commented Oct 4, 2017

devsnek commented Oct 4, 2017

Choose a reason for hiding this comment

xfumihiro commented Oct 5, 2017

devsnek commented Oct 5, 2017 • edited Loading

Choose a reason for hiding this comment

cpojer commented Oct 13, 2017

cpojer commented Oct 13, 2017

rotemmiz commented Oct 13, 2017

orta commented Oct 13, 2017

delgermurun commented Oct 26, 2017

SimenB commented Oct 26, 2017

xfumihiro commented Oct 26, 2017

rdeltour commented Oct 28, 2017

SimenB commented Oct 28, 2017 • edited Loading

rdeltour commented Oct 28, 2017

vajahath commented Nov 24, 2017

SimenB commented Nov 24, 2017

imballinst commented Nov 30, 2017 • edited Loading

xfumihiro commented Nov 30, 2017

imballinst commented Nov 30, 2017

github-actions bot commented May 13, 2021

xfumihiro commented Sep 19, 2017 •

edited

Loading

SimenB Sep 19, 2017 •

edited

Loading

codecov-io commented Sep 20, 2017 •

edited

Loading

xfumihiro Sep 30, 2017 •

edited

Loading

devsnek commented Oct 5, 2017 •

edited

Loading

SimenB commented Oct 28, 2017 •

edited

Loading

imballinst commented Nov 30, 2017 •

edited

Loading