Facing Issue with Parallelism in Jenkins while running Playwright Cucumber tests

[onFilm]

👓 What did you see?

I've encountered an issue while attempting to run Playwright Cucumber tests in parallel on Jenkins. Locally, specifying parallel: 4 in cucumber.mjs works as expected on Windows. However, when running the tests on Jenkins, it seems to be executing in a single worker instead of utilizing parallelism.

✅ What did you expect to see?

Tests should run in parallel, leveraging the specified number of workers.

📦 Which tool/library version are you using?

No response

🔬 How could we reproduce it?

No response

📚 Any additional context?

No response

[davidjgoss]

Are you certain your cucumber.mjs configuration is being picked up when you run in Jenkins?

You could try turning on debug mode which will output the resolved configuration among other things.

Also just to double check, you're not expecting it to spin up 4 executors in Jenkins?

[onFilm]

With DEBUG=cucumber I see below

+ docker run --rm --entrypoint sh -e BUILD_NUMBER -e BUILD_URL -e JOB_NAME -e ENV=stable -e DEBUG=cucumber -v /var/lib/jenkins/workspace/testParallelExecution/reports:/tests/reports -w /tests testparallelexecution -c 'npx cucumber-js --tags "@Smoke" --parallel 4'
Configuration will be loaded from "cucumber.mjs"
No profiles specified; using default profile
Resolved configuration: {
  backtrace: false,
  dryRun: false,
  forceExit: false,
  failFast: false,
  format: [
    'json:reports/cucumber-report.json',
    'html:reports/report.html',
    'rerun:reports/@failedrerun.txt',
    '@cucumber/pretty-formatter'
  ],
  formatOptions: { snippetInterface: 'async-await', printAttachments: false },
  import: [],
  language: 'en',
  name: [],
  order: 'defined',
  paths: [],
  parallel: 4,
  publish: false,
  publishQuiet: true,
  require: [ 'step-definitions/*.ts', 'setup/*.ts', 'utils/*.ts' ],
  requireModule: [ 'ts-node/register' ],
  retry: 1,
  retryTagFilter: '@flaky',
  strict: true,
  tags: '(@Smoke)',
  worldParameters: { foo: 'bar' }
}

During execution, I encounter the error below in jenkins, but I don't observe it in my local run, also despite specifying parallel 4, resulting in at least 2 instances of Chrome launching in local.

VError: Unexpected error on worker.receiveMessage: a BeforeAll hook errored on worker 0, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds
    at exit (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/run_worker.ts:8:38)
    at /tests/node_modules/@cucumber/cucumber/src/runtime/parallel/run_worker.ts:22:9
caused by: VError: a BeforeAll hook errored on worker 0, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds
    at Worker.runTestRunHooks (/tests/node_modules/@cucumber/cucumber/src/runtime/run_test_run_hooks.ts:32:19)
    at Worker.initialize (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/worker.ts:93:5)
    at Worker.receiveMessage (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/worker.ts:110:7)
caused by: Error: function timed out, ensure the promise resolves within 60000 milliseconds
    at Timeout.<anonymous> (/tests/node_modules/@cucumber/cucumber/src/time.ts:52:14)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)
VError: Unexpected error on worker.receiveMessage: a BeforeAll hook errored on worker 1, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds
    at exit (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/run_worker.ts:8:38)
    at /tests/node_modules/@cucumber/cucumber/src/runtime/parallel/run_worker.ts:22:9
caused by: VError: a BeforeAll hook errored on worker 1, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds
    at Worker.runTestRunHooks (/tests/node_modules/@cucumber/cucumber/src/runtime/run_test_run_hooks.ts:32:19)
    at Worker.initialize (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/worker.ts:93:5)
    at Worker.receiveMessage (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/worker.ts:110:7)
caused by: Error: function timed out, ensure the promise resolves within 60000 milliseconds
    at Timeout.<anonymous> (/tests/node_modules/@cucumber/cucumber/src/time.ts:52:14)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)

[davidjgoss]

This is the issue you need to deal with first:

a BeforeAll hook errored on worker 0, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds

Regardless of parallel settings, this will prevent any scenarios from running. Only you will have the context to understand why your BeforeAll hook might be timing out in CI, but it could be connectivity to an external resource, permissions etc.

[onFilm]

I have BeforeAll as below, and still getting this error, I am not able to figure out what is causing. Can you help me figure out issue and solution, Thanks
BeforeAll

Error in console:

[mrsenzy]

@davidjgoss - Similar / Same issue occurring for me, but different stacktrace based on my custom playwright project with cucumber 10.8.0. we were using 10.3.1 and it was working fine and with 10.8.0 upgrade, it works fine for single thread execution but when parallel count is set to 1 or greater -- we get below error

Error: Unexpected error on worker.receiveMessage
at exit (C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\run_worker.ts:7:21)
at C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\run_worker.ts:21:9
at processTicksAndRejections (node:internal/process/task_queues:95:5) {
[cause]: Error: Cannot find module './stepDefinitions/**/*.ts'
Require stack:

• C:\Code\node_modules@cucumber\cucumber\lib\try_require.js
• C:\Code\node_modules@cucumber\cucumber\lib\runtime\parallel\worker.js
• C:\Code\node_modules@cucumber\cucumber\lib\runtime\parallel\run_worker.js
  at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1134:15)
  at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (C:\Code\node_modules@cspotcode\source-map-support\source-map-support.js:811:30)
  at Function.Module._load (node:internal/modules/cjs/loader:975:27)
  at Module.require (node:internal/modules/cjs/loader:1225:19)
  at require (node:internal/modules/helpers:177:18)
  at tryRequire (C:\Code\node_modules@cucumber\cucumber\src\try_require.ts:9:12)
  at C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\worker.ts:72:67
  at Array.map ()
  at Worker.initialize (C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\worker.ts:72:41)
  at Worker.receiveMessage (C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\worker.ts:107:18) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
  'C:\Code\node_modules\@cucumber\cucumber\lib\try_require.js',
  'C:\Code\node_modules\@cucumber\cucumber\lib\runtime\parallel\worker.js',
  'C:\Codee\node_modules\@cucumber\cucumber\lib\runtime\parallel\run_worker.js'
  ]
  }
  }
  Error: Unexpected error on worker.receiveMessage
  at exit (C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\run_worker.ts:7:21)
  at C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\run_worker.ts:21:9
  at processTicksAndRejections (node:internal/process/task_queues:95:5) {
  [cause]: Error: Cannot find module './stepDefinitions/**/*.ts'
  Require stack:
• C:\Code\node_modules@cucumber\cucumber\lib\try_require.js
• C:\Code\node_modules@cucumber\cucumber\lib\runtime\parallel\worker.js
• C:\Code\node_modules@cucumber\cucumber\lib\runtime\parallel\run_worker.js
  at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1134:15)
  at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (C:\Code\node_modules@cspotcode\source-map-support\source-map-support.js:811:30)
  at Function.Module._load (node:internal/modules/cjs/loader:975:27)
  at Module.require (node:internal/modules/cjs/loader:1225:19)
  at require (node:internal/modules/helpers:177:18)
  at tryRequire (C:\Code\node_modules@cucumber\cucumber\src\try_require.ts:9:12)
  at C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\worker.ts:72:67
  at Array.map ()
  at Worker.initialize (C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\worker.ts:72:41)
  at Worker.receiveMessage (C:\Code\node_modules@cucumber\cucumber\src\runtime\parallel\worker.ts:107:18) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
  'C:\Code\node_modules\@cucumber\cucumber\lib\try_require.js',
  'C:\Code\node_modules\@cucumber\cucumber\lib\runtime\parallel\worker.js',
  'C:\Code\node_modules\@cucumber\cucumber\lib\runtime\parallel\run_worker.js'
  ]
  }
  }

[github-actions]

This issue is stale because it has been open for 3 weeks with no activity. Remove the stale label or comment or this will be closed in another 5 days.

[davidjgoss]

@onFilm you have a Playwright issue here which I can't do much to help you with. From a brief scan you have a Address already in use in the error from Playwright when you're trying to spin it up in the BeforeAll. Since this runs for each worker process, that suggests that each worker is trying to use the same port. Perhaps there is some Playwright config you can do to avoid this?

[onFilm]

@davidjgoss Thanks a lot for pointing out the port issue with Playwright. I appreciate your help and will look into configuring it to avoid this conflict.