playwright-test: init at 1.31.1 by teto · Pull Request #227071 · NixOS/nixpkgs

teto · 2023-04-19T14:47:57Z

Adds "playwright test" command.

Description of changes

Raw dump of a package working in an out of tree project, still needs some polish
Fixes #217693

I've tried buildNpmPackage but doesn't seem ready yet to handle this kind of package.

Things done

Adds "playwright test" command.

nixos-discourse · 2023-04-28T08:45:08Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/running-playwright-tests/25655/9

pbek · 2023-04-28T10:59:28Z

I built the PR and run the resulting binary in a (unfortunately private) repository with Playwright tests written in Javascript:

/nix/store/8ck7islaxlgffm584kf91rq05ygigima-_at_playwright_slash_test-1.31.1/bin/playwright test

gave me errors like

Error: Playwright Test did not expect test() to be called here.
Most common reasons include:
- You are calling test() in a configuration file.
- You are calling test() in a file that is imported by the configuration file.
- You have two different versions of @playwright/test. This usually happens
  when one of the dependencies in your package.json depends on @playwright/test.

   at mono/mono.spec.js:7

   5 |     await page.goto('https://url.domain');
   6 |     await expect(page).toHaveURL(/auth\/realms/);
>  7 |     await page.locator('#username').fill(process.env.STUDENT_USERNAME ?? '');
     | ^
   8 |     await page.locator('#password').fill(process.env.STUDENT_PASSWORD ?? '');
   9 |     await page.locator('button[type="submit"]').click()
  10 |     await expect(page).toHaveURL(/mono/);

    at TestTypeImpl._currentSuite (/home/user/dev/e2e-tests-ng/node_modules/@playwright/test/lib/common/testType.js:70:13)
    at TestTypeImpl._createTest (/home/user/dev/e2e-tests-ng/node_modules/@playwright/test/lib/common/testType.js:76:24)
    at /home/user/dev/e2e-tests-ng/node_modules/@playwright/test/lib/common/transform.js:226:12
    at Object.<anonymous> (/home/user/dev/e2e-tests-ng/tests/mono/mono.spec.js:7:1)

Do I need to do something to my test project first?

teto · 2023-04-28T13:46:39Z

You have two different versions of @playwright/test.

Ha I have seen that error at some point. I was told that javascript has a peculiar singleton system based on path that even symlinks could throw off. This package is a port of a package we use internally with success and that shouldn't suffer from the problem.
Why is it that the path is a local one ? /home/user/dev/e2e-tests-ng/node_modules/@playwright/test/lib/common/testType.js:76:24
it should come from the nix store

So maybe it's a plawyright error. Have you tried clearning up NODE_PATH in case it does load an alternative playwright ?

pbek · 2023-04-28T15:04:32Z

My NODE_PATH is NODE_PATH=/nix/store/njh6p007ymbmcgas5l7sqqxfmm394zx1-nodejs-18.15.0/lib/node_modules.
Setting it to an empty string didn't help. 🤔

Only when I was manually removing node_modules/@playwright/ (note that it will be installed again by npm install) I managed to use the test successfully in Chromium!

But not in Firefox, there I got:

browserType.launch: Executable doesn't exist at /nix/store/dkpc25ivrhx19d5v9px74qnsafn65hlr-playwright-browsers-chromium/firefox-1378/firefox/firefox

I don't know where it gets the path from, /nix/store/dkpc25ivrhx19d5v9px74qnsafn65hlr-playwright-browsers-chromium is the path for the chromium browser. Any chance to make it work in firefox too?

My shell.nix is:

{ pkgs ? import <nixpkgs> {} }:
  pkgs.mkShell {
    buildInputs = [
      pkgs.nodejs-18_x
      pkgs.playwright
    ];

    PLAYWRIGHT_BROWSERS_PATH="${pkgs.playwright-driver.browsers}";
  }

env said:
PLAYWRIGHT_BROWSERS_PATH=/nix/store/dkpc25ivrhx19d5v9px74qnsafn65hlr-playwright-browsers-chromium

And is there a chance to make it work inside VS-Code with the Playwright extension (where you are supposed to do the actual development of the tests)?

aszenz · 2023-04-28T15:37:48Z

And is there a chance to make it work inside VS-Code with the Playwright extension (where you are supposed to do the actual development of the tests)?

Add this to your vscode settings

"playwright.env": {
    "PLAYWRIGHT_BROWSERS_PATH": "/nix/store/mlyamflq8icq0lwfircr03aiba0dbzg3-chromium-playwright",
    "PLAYWRIGHT_SKIP_VALIDATE_HOST_REQUIREMENTS": 1
  },

Unfortunately u have to hard code the nix store path, maybe there's a way to read it from env var in vscode

pbek · 2023-04-28T16:06:02Z

Hm, I tried to do that, but vs-code doesn't even seem to detect my JS tests any more. It want to show python tests. 😅

pbek · 2023-05-03T12:27:25Z

I created https://github.com/pbek/playwright-example with an example playwright test to test this PR as follow-up from #215450 (comment).

https://github.com/pbek/playwright-example/blob/main/shell.nix uses this PR.

teto · 2023-05-04T13:36:53Z

awesome, maybe we can integrate your repository as part of the testsuite ?

Seems like it works for the only supported browser aka chrome:

Running 3 tests using 3 workers
  1) [webkit] › example.spec.js:4:1 › has title ────────────────────────────────────────────────────

    browserType.launch: Executable doesn't exist at /nix/store/fpibn255xp5iazcdnghbf06qgbacrqv3-playwright-browsers-chromium/webkit-1792/pw_run.sh
    ╔═════════════════════════════════════════════════════════════════════════╗
    ║ Looks like Playwright Test or Playwright was just installed or updated. ║
    ║ Please run the following command to download new browsers:              ║
    ║                                                                         ║
    ║     npx playwright install                                              ║
    ║                                                                         ║
    ║ <3 Playwright Team                                                      ║
    ╚═════════════════════════════════════════════════════════════════════════╝

  2) [firefox] › example.spec.js:4:1 › has title ───────────────────────────────────────────────────

    browserType.launch: Executable doesn't exist at /nix/store/fpibn255xp5iazcdnghbf06qgbacrqv3-playwright-browsers-chromium/firefox-1378/firefox/firefox
    ╔═════════════════════════════════════════════════════════════════════════╗
    ║ Looks like Playwright Test or Playwright was just installed or updated. ║
    ║ Please run the following command to download new browsers:              ║
    ║                                                                         ║
    ║     npx playwright install                                              ║
    ║                                                                         ║
    ║ <3 Playwright Team                                                      ║
    ╚═════════════════════════════════════════════════════════════════════════╝

  2 failed
    [firefox] › example.spec.js:4:1 › has title ────────────────────────────────────────────────────
    [webkit] › example.spec.js:4:1 › has title ─────────────────────────────────────────────────────
  1 passed (6.0s)

  Serving HTML report at http://localhost:9323. Press Ctrl+C to quit.

I am tempted to merge as is and discuss the test afterwards. There might even be a native testsuite that could be run

pbek · 2023-05-04T13:38:53Z

awesome, maybe we can integrate your repository as part of the testsuite ?

fine with me! 🚀

teto · 2023-05-04T13:42:52Z

Let's go !

pbek · 2023-05-05T10:52:02Z

@teto, do you maybe know why there is no --ui parameter, as in playwright test --ui? (see https://playwright.dev/docs/test-cli)

teto · 2023-05-05T13:12:34Z

nope sry no idea, documentation is not clear to me either.

samhh · 2023-05-09T09:06:41Z

@pbek That was added in 1.32.0.

pbek · 2023-05-09T12:29:19Z

Ah, I see, thank you :)
So we'll need to wait for @toto. 😅

samhh · 2023-05-09T13:28:41Z

@teto We're also keen on a newer version at work. What would be the process of updating? Do we need to think about the driver/browsers as well?

toto · 2023-05-09T13:52:08Z

I think you mean @teto not @toto ;)

teto · 2023-05-09T13:58:57Z

I am sure you could sell @toto for a shitton of money, if only to https://en.wikipedia.org/wiki/Toto_Ltd.

@samhh Absolutely, the playwright-driver version must match the package version ! we could add a warning to check that
the playwright python package has this script to update the driver that I might have broken ?!
https://github.com/teto/nixpkgs/blob/3beded5aa4b3e20e510b2eea292c3f49fbe44341/pkgs/development/python-modules/playwright/update.sh#L24
I dont know if nix-update can replace that script (probably not) but I guess the last lines should be moved to pkgs/development/web/playwright/

I have spent my plawyright FOSS credits but I am happy to review PRs if you wanna update the package.

pkgs/development/web/playwright-test/wrapped.nix was written manually but depends on the driver version. Updating the driver and running pkgs/development/web/playwright-test $ node2nix -i package.json (from memory) should work

pbek · 2023-05-09T14:33:08Z

I think you mean @teto not @toto ;)

I'm so sorry! 😆

kalekseev · 2023-06-04T15:53:13Z

@samhh pr with latest playwright version #235914

teto force-pushed the playwright-test branch from 2de564b to ed27884 Compare April 19, 2023 16:34

playwright-test: init at 1.31.1

9665f56

Adds "playwright test" command.

teto force-pushed the playwright-test branch from ed27884 to 9665f56 Compare April 27, 2023 15:36

teto mentioned this pull request Apr 27, 2023

Build failure: playwright #215450

Closed

teto marked this pull request as ready for review April 27, 2023 19:36

pbek added a commit to pbek/playwright-example that referenced this pull request May 3, 2023

NixOS/nixpkgs#227071 test: use playwright-test

4478ad7

teto merged commit 6a8cb45 into NixOS:master May 4, 2023

teto deleted the playwright-test branch May 4, 2023 13:43

kachick mentioned this pull request Jul 25, 2023

Integrate playwright with Nix kachick/emobu#385

Closed

Uh oh!

Conversation

teto commented Apr 19, 2023

Description of changes

Things done

Uh oh!

nixos-discourse commented Apr 28, 2023

Uh oh!

pbek commented Apr 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

teto commented Apr 28, 2023

Uh oh!

pbek commented Apr 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aszenz commented Apr 28, 2023

Uh oh!

pbek commented Apr 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pbek commented May 3, 2023

Uh oh!

teto commented May 4, 2023

Uh oh!

pbek commented May 4, 2023

Uh oh!

teto commented May 4, 2023

Uh oh!

pbek commented May 5, 2023

Uh oh!

teto commented May 5, 2023

Uh oh!

samhh commented May 9, 2023

Uh oh!

pbek commented May 9, 2023

Uh oh!

samhh commented May 9, 2023

Uh oh!

toto commented May 9, 2023

Uh oh!

teto commented May 9, 2023

Uh oh!

pbek commented May 9, 2023

Uh oh!

kalekseev commented Jun 4, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

pbek commented Apr 28, 2023 •

edited

Loading

pbek commented Apr 28, 2023 •

edited

Loading

pbek commented Apr 28, 2023 •

edited

Loading