Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transient "cannot find module" when running gulp test #14166

Closed
jonkeller opened this issue Mar 21, 2018 · 18 comments
Closed

Transient "cannot find module" when running gulp test #14166

jonkeller opened this issue Mar 21, 2018 · 18 comments
Assignees

Comments

@jonkeller
Copy link
Contributor

bradfrizzell and I have been experiencing frequent errors like the following when running "gulp test", both locally on our machines as well as on Travis. Restarting the test, or rebuilding and then re-running the test, generally fix the issue.
The module that can't be found seems to vary.

  {
    "message": "Uncaught Error: Cannot find module '/home/travis/build/ampproject/amphtml/test/integration/test-3p-frame.js'\nat /tmp/node_modules/browser-pack/_prelude.js:1:0 <- /tmp/d65e3526f9c22abf2a123e2513182be5.browserify:1:196\n\nError: Cannot find module '/home/travis/build/ampproject/amphtml/test/integration/test-3p-frame.js'\n    at s (/tmp/node_modules/browser-pack/_prelude.js:1:0 <- /tmp/d65e3526f9c22abf2a123e2513182be5.browserify:1:156)\n    at s (/tmp/node_modules/browser-pack/_prelude.js:1:0 <- /tmp/d65e3526f9c22abf2a123e2513182be5.browserify:1:122)\n    at test/integration/test-3p-frame.js:1:34",
    "str": "Uncaught Error: Cannot find module '/home/travis/build/ampproject/amphtml/test/integration/test-3p-frame.js'\nat /tmp/node_modules/browser-pack/_prelude.js:1:0 <- /tmp/d65e3526f9c22abf2a123e2513182be5.browserify:1:196\n\nError: Cannot find module '/home/travis/build/ampproject/amphtml/test/integration/test-3p-frame.js'\n    at s (/tmp/node_modules/browser-pack/_prelude.js:1:0 <- /tmp/d65e3526f9c22abf2a123e2513182be5.browserify:1:156)\n    at s (/tmp/node_modules/browser-pack/_prelude.js:1:0 <- /tmp/d65e3526f9c22abf2a123e2513182be5.browserify:1:122)\n    at test/integration/test-3p-frame.js:1:34"
  }
@rsimha
Copy link
Contributor

rsimha commented Mar 30, 2018

On Travis, I'm fairly certain this has to do with a stale cache. I recently deleted the node_modules cache after upgrading a bunch of dependencies. For local development, you can do the same thing via:

rm -rf node_modules
yarn

Let me know if this eliminates the error for you.

@rsimha rsimha assigned jonkeller and bradfrizzell and unassigned rsimha Mar 30, 2018
@rsimha
Copy link
Contributor

rsimha commented Mar 30, 2018

It would also be useful to understand how you're running these tests. Are you simply running gulp test? Or are you adding flags like --a4a, --watch, or --files? If you're using --watch, are you expecting that the tests are run as a result of an edit / recompile? What files ought to be recompiled? Is that happening? etc.

@rsimha
Copy link
Contributor

rsimha commented Mar 31, 2018

Could it be that you're seeing the same error as described in admc/wd#520? If so, it might be due to a Sauce Labs error :/

@jonkeller
Copy link
Contributor Author

I don't think it's the same as in #520. Will update if it happens again, though FYI I currently have no near-term AMP work on my plate. If nobody else sees this, feel free to just close.

@jonkeller
Copy link
Contributor Author

@rsimha Here is an example. I ran the following commands:

cd ~/workspace
mkdir unskip_tests
cd unskip_tests
git clone [email protected]:jonkeller/amphtml.git
cd amphtml
git remote add upstream [email protected]:ampproject/amphtml.git
git fetch upstream
git checkout -t -b upstream_master upstream/master
git remote -v
git branch
git checkout upstream_master
git pull upstream --ff-only
git checkout master
git branch -u upstream/master
git pull upstream master
yarn
gulp clean
gulp build
git checkout -b unskip_tests_jonkeller
gulp test --files=extensions/amp-analytics/0.1/test/test-amp-analytics.js --nobuild

...and immediately got:

START:
Chrome 65.0.3325 (Mac OS X 10.13.3) ERROR
  {
    "message": "Uncaught Error: Cannot find module '/Users/jonkeller/workspace/unskip_tests/amphtml/extensions/amp-analytics/0.1/test/test-amp-analytics.js'\nat /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/7e1556c39cab1e7aaca343091b581d30.browserify:1:196\n\nError: Cannot find module '/Users/jonkeller/workspace/unskip_tests/amphtml/extensions/amp-analytics/0.1/test/test-amp-analytics.js'\n    at s (/var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/7e1556c39cab1e7aaca343091b581d30.browserify:1:156)\n    at s (/var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/7e1556c39cab1e7aaca343091b581d30.browserify:1:122)\n    at extensions/amp-analytics/0.1/test/test-amp-analytics.js:1:34",
    "str": "Uncaught Error: Cannot find module '/Users/jonkeller/workspace/unskip_tests/amphtml/extensions/amp-analytics/0.1/test/test-amp-analytics.js'\nat /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/7e1556c39cab1e7aaca343091b581d30.browserify:1:196\n\nError: Cannot find module '/Users/jonkeller/workspace/unskip_tests/amphtml/extensions/amp-analytics/0.1/test/test-amp-analytics.js'\n    at s (/var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/7e1556c39cab1e7aaca343091b581d30.browserify:1:156)\n    at s (/var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/7e1556c39cab1e7aaca343091b581d30.browserify:1:122)\n    at extensions/amp-analytics/0.1/test/test-amp-analytics.js:1:34"
  }

I realize I used --nobuild in executing the test. But I had just done "gulp build" immediately prior. Shouldn't this work? It always used to.

@jonkeller
Copy link
Contributor Author

Update: Then I ran the same "gulp test" command, without the "--nobuild", and got the same error:

gulp test --files=extensions/amp-analytics/0.1/test/test-amp-analytics.js
[15:09:48] Using gulpfile ~/workspace/unskip_tests/amphtml/gulpfile.js
[15:09:48] Starting 'update-packages'...
[15:09:48] All packages in node_modules are up to date.
[15:09:48] Finished 'update-packages' after 663 ms
[15:09:48] Starting 'build'...
[15:09:48] Running build. Press Ctrl + C to cancel...
[15:09:48] To skip building during future test runs, use --nobuild with your gulp test command.
[15:09:48] Building the runtime for local testing with the prod AMP config.
[15:09:48] ⤷ Use --config={canary|prod} with your gulp build command to specify which config to apply.
[15:09:48] Building all AMP extensions.
[15:09:48] ⤷ Use --noextensions to skip building extensions.
[15:09:48] ⤷ Use --extensions=amp-foo,amp-bar to choose which extensions to build.
[15:09:48] ⤷ Use --extensions=minimal_set to build just the extensions needed to load article.amp.html.
[15:09:49] Recompiled CSS in amp.css (505 ms)
[15:09:49] Recompiled CSS in amp-access (560 ms)
[15:09:49] Recompiled CSS in amp-access-scroll (559 ms)
[15:09:49] Recompiled CSS in amp-ad (558 ms)
[15:09:49] Recompiled CSS in amp-apester-media (558 ms)
[15:09:49] Recompiled CSS in amp-byside-content (555 ms)
[15:09:49] Recompiled CSS in amp-consent (550 ms)
[15:09:49] Recompiled CSS in amp-document-recommendations (549 ms)
[15:09:49] Recompiled CSS in amp-fit-text (548 ms)
[15:09:49] Recompiled CSS in amp-form (548 ms)
[15:09:49] Recompiled CSS in amp-fx-flying-carpet (547 ms)
[15:09:49] Recompiled CSS in amp-gwd-animation (546 ms)
[15:09:49] Recompiled CSS in amp-image-lightbox (545 ms)
[15:09:49] Recompiled CSS in amp-instagram (543 ms)
[15:09:49] Recompiled CSS in amp-lightbox (542 ms)
[15:09:49] Recompiled CSS in amp-live-list (543 ms)
[15:09:49] Recompiled CSS in amp-mathml (543 ms)
[15:09:49] Recompiled CSS in amp-pinterest (542 ms)
[15:09:49] Recompiled CSS in amp-sidebar (539 ms)
[15:09:49] Recompiled CSS in amp-story-auto-ads (512 ms)
[15:09:49] Recompiled CSS in amp-selector (512 ms)
[15:09:49] Recompiled CSS in amp-web-push (511 ms)
[15:09:49] Recompiled CSS in amp-image-viewer (509 ms)
[15:09:49] Recompiled CSS in amp-user-notification (506 ms)
[15:09:49] Recompiled CSS in amp-viz-vega (506 ms)
[15:09:49] Recompiled CSS in amp-access-laterpay (572 ms)
[15:09:49] Recompiled CSS in amp-app-banner (568 ms)
[15:09:49] Recompiled CSS in amp-carousel (566 ms)
[15:09:49] Recompiled CSS in amp-playbuzz (547 ms)
[15:09:49] Recompiled CSS in amp-sticky-ad (542 ms)
[15:09:49] Recompiled CSS in amp-subscriptions (515 ms)
[15:09:49] Recompiled CSS in amp-lightbox-gallery (555 ms)
[15:09:50] Recompiled CSS in amp-date-picker (712 ms)
[15:09:50] Recompiled CSS in amp-social-share (740 ms)
[15:09:50] Recompiled CSS in amp-story (1.186 s)
[15:09:50] Processed 3p/frame.max.html (025 ms)
[15:09:50] Processed 3p/nameframe.max.html (024 ms)
[15:09:52] Compiled examiner.js (1.790 s)
[15:09:52] Compiled kill.js (1.794 s)
[15:10:01] Compiled core.js (11.068 s)
[15:10:24] Compiled shell.js (33.840 s)
[15:10:26] Compiled ampcontext-lib.js (35.995 s)
[15:10:26] Compiled integration.js (36.022 s)
[15:10:26] Compiled iframe-transport-client-lib.js (36.080 s)
[15:10:26] No configs found in dist.3p/current/integration.js
[15:10:26] Compiled polyfills.js (36.327 s)
[15:10:26] Enabled local development mode in dist.3p/current/integration.js
[15:10:27] Wrote prod AMP config to dist.3p/current/integration.js
[15:10:27] Compiled web-worker.js (36.734 s)
[15:10:29] Compiled amp-viewer-host.js (39.120 s)
[15:10:30] Compiled amp.js (39.552 s)
[15:10:30] Compiled amp-shadow.js (39.544 s)
[15:10:30] Compiled amp-inabox.js (39.573 s)
[15:10:30] Compiled amp-analytics.js (39.564 s)
[15:10:30] No configs found in dist/amp.js
[15:10:30] Compiled install-alp.js (39.958 s)
[15:10:30] Compiled inabox-host.js (39.891 s)
[15:10:30] Enabled local development mode in dist/amp.js
[15:10:30] Wrote prod AMP config to dist/amp.js
[15:10:30] Running npx bundlesize -f "./dist/amp.js" -s "333.28KB"...

 PASS  ./dist/amp.js: 333.27KB < maxSize 333.28KB (gzip)

[15:10:31] Finished 'build' after 43 s
[15:10:31] Starting 'test'...
[15:10:31] Run gulp help to see a list of all test flags.
[15:10:31] ⤷ Use --nohelp to silence these messages.)
[15:10:31] ⤷ Use --headless to run tests in a headless Chrome window.
[15:10:31] Running tests against unminified code.
[15:10:31] --files: Running tests in the file(s): extensions/amp-analytics/0.1/test/test-amp-analytics.js
[15:10:31] Setting the runtime's AMP config to prod
[15:10:31] Removed existing config from dist/amp.js
[15:10:31] Enabled local development mode in dist/amp.js
[15:10:32] Wrote prod AMP config to dist/amp.js
[15:10:32] Webserver started at http://localhost:31862
[15:10:32] Started test responses server on localhost:31862
[15:10:32] Running test. Press Ctrl + C to cancel...
Running tests locally...

START:
Chrome 65.0.3325 (Mac OS X 10.13.3) ERROR
  {
    "message": "Uncaught Error: Cannot find module '/Users/jonkeller/workspace/unskip_tests/amphtml/extensions/amp-analytics/0.1/test/test-amp-analytics.js'\nat /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/79a9552678943b8cd8d7cc5b18b319e2.browserify:1:196\n\nError: Cannot find module '/Users/jonkeller/workspace/unskip_tests/amphtml/extensions/amp-analytics/0.1/test/test-amp-analytics.js'\n    at s (/var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/79a9552678943b8cd8d7cc5b18b319e2.browserify:1:156)\n    at s (/var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/79a9552678943b8cd8d7cc5b18b319e2.browserify:1:122)\n    at extensions/amp-analytics/0.1/test/test-amp-analytics.js:1:34",
    "str": "Uncaught Error: Cannot find module '/Users/jonkeller/workspace/unskip_tests/amphtml/extensions/amp-analytics/0.1/test/test-amp-analytics.js'\nat /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/79a9552678943b8cd8d7cc5b18b319e2.browserify:1:196\n\nError: Cannot find module '/Users/jonkeller/workspace/unskip_tests/amphtml/extensions/amp-analytics/0.1/test/test-amp-analytics.js'\n    at s (/var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/79a9552678943b8cd8d7cc5b18b319e2.browserify:1:156)\n    at s (/var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/node_modules/browser-pack/_prelude.js:1:0 <- /var/folders/93/v772f58d0m5gpp5nq6l1yz2800dbvk/T/79a9552678943b8cd8d7cc5b18b319e2.browserify:1:122)\n    at extensions/amp-analytics/0.1/test/test-amp-analytics.js:1:34"
  }

Finished in 0.886 secs / 0 secs @ 15:10:58 GMT-0400 (EDT)

SUMMARY:
● 0 tests completed
[15:10:59] Shutting down test responses server on localhost:31862
[15:10:59] ERROR: Karma test failed with exit code 1
 ✘ jonkeller@jonkeller-macbookpro  ~/workspace/unskip_tests/amphtml   unskip_tests_jonkeller ● 

@rsimha
Copy link
Contributor

rsimha commented Apr 11, 2018

@jonkeller Yeah, you should be able to run gulp test with --nobuild after having run gulp build. A couple questions:

  1. The error says /Users/jonkeller/workspace/unskip_tests/amphtml/extensions/amp-analytics/0.1/test/test-amp-analytics.js couldn't be found. That path looks weird. Can you confirm that you indeed built the runtime within that directory? Did you run yarn in the directory to generate node_modules? (Most people do development within a fixed directory, like /Users/jonkeller/src/amphtml, while you appear to be creating a separate workspace directory for each branch.)
  2. You're running the unit tests, which don't require gulp build. Any reason why you're not using gulp test --unit --files <file>, which won't run the time-consuming gulp build?

@rsimha
Copy link
Contributor

rsimha commented Apr 11, 2018

@jonkeller I tried running your tests, and was able to do so successfully. See https://gist.github.com/rsimha/92f13fdab40452e9d6a1871276a5ed1e

(I'm still curious about what could be causing the errors your saw, and would love to fix it if we can find a cause.)

@jonkeller
Copy link
Contributor Author

jonkeller commented Apr 11, 2018 via email

@rsimha
Copy link
Contributor

rsimha commented Apr 11, 2018

@jonkeller I suspect this might have to do with you not using --unit. When you run a unit test with --unit, the set of test files includes just the unit tests. When you run a unit test without --unit, the entire build directory is included in the list of test files. This could be messing with your tests due to some logic in some component somewhere.

If we find out what the issue is, we can certainly apply a more general fix.

@jonkeller
Copy link
Contributor Author

jonkeller commented Apr 11, 2018 via email

@rsimha
Copy link
Contributor

rsimha commented May 7, 2018

This is still happening.

According to browserify/browserify#1271, it probably has to do with paths being specified incorrectly.

@rsimha
Copy link
Contributor

rsimha commented May 18, 2018

This is still happening. I have a hunch this is due to gulpjs/gulp#1412 (comment)

Reopening to try a fix.

/cc @choumx

@rsimha rsimha assigned rsimha and unassigned jonkeller and bradfrizzell May 18, 2018
@rsimha
Copy link
Contributor

rsimha commented May 18, 2018

Hmm. We're running into this while running karma-browserify, not gulp.src. This may also be related to the ancient AMP PR #39, which was meant to fix #26. Perhaps the original issue is recurring and we need a larger value for browserify: bundleDelay in karma.conf.js?

/cc @erwinmombay

@dreamofabear
Copy link

Thanks for looking into this. I have a couple PRs where retrying the job doesn't seem to help. The failures seems to have gotten more consistent over the last few days.

@rsimha
Copy link
Contributor

rsimha commented May 18, 2018

Here are some issues that could be related to what we're seeing:

nikku/karma-browserify#114
nikku/karma-browserify#113
nikku/karma-browserify#67
nikku/karma-browserify#49

Will read up some more, and try increasing bundleDelay.

@rsimha rsimha added this to the Sprint H2 May 2018 milestone May 18, 2018
@rsimha
Copy link
Contributor

rsimha commented Aug 9, 2018

@rsimha rsimha reopened this Aug 9, 2018
@rsimha
Copy link
Contributor

rsimha commented Aug 9, 2018

It's quite likely that this is due to nikku/karma-browserify#114. The disk speed on Travis is slow, so we should wait longer than a second for all files to be loaded by the Karma server before the browserify transform is applied. See nikku/karma-browserify#114 (comment) for a solution.

What the bundleDelay does: karma-browserify needs to wait for all files to be loaded by karma before it can actually start bundling. The bundleDelay is the timeout we accept until we assume karma handed over all files to us. If you have a slow disk (that needs more than the default delay to serve subsequent files the plug-in starts bundling without some files (the ones it later prints Cannot find module for).

PR coming up.

/cc @choumx @jridgewell @cvializ

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants