playwright: package with buildNpmPackage

[kalekseev]

Description of changes

• @playwright/test and playwright-core packages built from source
• python3Packages.playwright: 1.44.0 -> 1.45.1
• playwright: 1.40.0 -> 1.45.3
• updated update script
• renamed playwright-driver into playwright-core to match original name https://www.npmjs.com/package/playwright-core but we can left it as driver to not break things for users
• removed playwright-test, it's now replaced by playwright https://www.npmjs.com/package/@playwright/test
• made playwright js package a top level package (python version used previously is just a wrapper around js package)

ref #298944
ref #229475

/nix/store/nxawidis47cx9v15j2smbhnrlzjg498i-playwright-core-1.43.0/cli.js
Usage: npx playwright [options] [command]

Options:
  -V, --version                          output the version number
  -h, --help                             display help for command

Commands:
  open [options] [url]                   open page in browser specified via -b, --browser
  codegen [options] [url]                open page and generate code for user actions
  install [options] [browser...]         ensure browsers necessary for this version of Playwright are installed
  uninstall [options]                    Removes browsers used by this installation of Playwright from the system
                                         (chromium, firefox, webkit, ffmpeg). This does not include branded channels.
  install-deps [options] [browser...]    install dependencies necessary to run browsers (will ask for sudo permissions)
  cr [options] [url]                     open page in Chromium
  ff [options] [url]                     open page in Firefox
  wk [options] [url]                     open page in WebKit
  screenshot [options] <url> <filename>  capture a page screenshot
  pdf [options] <url> <filename>         save page as pdf
  show-trace [options] [trace...]        show trace viewer
  test                                   Run tests with Playwright Test. Available in @playwright/test package.
  show-report                            Show Playwright Test HTML report. Available in @playwright/test package.
  merge-reports                          Merge Playwright Test Blob reports Available in @playwright/test package.
  help [command]                         display help for command

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[phaer]

@kalekseev Nice, thanks for working on this!

The current implementation, using node2nix has the benefit of automatically updating the hashes. Could we do the same here? i.e. update them in updateScript?

[kalekseev]

@kalekseev Nice, thanks for working on this!

The current implementation, using node2nix has the benefit of automatically updating the hashes. Could we do the same here? i.e. update them in updateScript?

we have to write a bunch of bash in update.sh

[phaer]

@kalekseev Nice, thanks for working on this!
The current implementation, using node2nix has the benefit of automatically updating the hashes. Could we do the same here? i.e. update them in updateScript?

we have to write a bunch of bash in update.sh

Are you going to do so? Otherwise, I could take a look at it next week.
Ideally, version should be kept in sync with the driver via update.sh as well.

[kalekseev]

Are you going to do so? Otherwise, I could take a look at it next week. Ideally, version should be kept in sync with the driver via update.sh as well.

@phaer don't know if I will have time to work on this PR this week, feel free to work on it.

BTW After recent changes it looks like playwright, playwright-test, playwright-driver were unified as single package playwright so we should review these options:

• replace playwright-test and playwright-driver with playwright
• move python version of playwright out of global packages into pythonPackages and replace it with this package.
• playwright-python should use ${playwright}/cli.js as driver
• update.sh should use browser.json from original repository

[kalekseev]

@phaer I think I'm done with this one, updated update.sh with chatgpt help, not the best code but good enough for start.

[teto]

same

[teto]

Result of nixpkgs-review pr 302759 run on x86_64-linux 1

52 packages built:

• changedetection-io
• changedetection-io.dist
• lacus
• lacus.dist
• pentestgpt
• pentestgpt.dist
• playwright (python311Packages.playwright)
• playwright-driver
• playwright-test
• playwright.dist (python311Packages.playwright.dist)
• python311Packages.batchspawner
• python311Packages.batchspawner.dist
• python311Packages.dockerspawner
• python311Packages.dockerspawner.dist
• python311Packages.jupyterhub
• python311Packages.jupyterhub-ldapauthenticator
• python311Packages.jupyterhub-ldapauthenticator.dist
• python311Packages.jupyterhub-systemdspawner
• python311Packages.jupyterhub-systemdspawner.dist
• python311Packages.jupyterhub-tmpauthenticator
• python311Packages.jupyterhub-tmpauthenticator.dist
• python311Packages.jupyterhub.dist
• python311Packages.lacuscore
• python311Packages.lacuscore.dist
• python311Packages.oauthenticator
• python311Packages.oauthenticator.dist
• python311Packages.playwright-stealth
• python311Packages.playwright-stealth.dist
• python311Packages.playwrightcapture
• python311Packages.playwrightcapture.dist
• python311Packages.pycookiecheat
• python311Packages.pycookiecheat.dist
• python311Packages.pytest-playwright
• python311Packages.pytest-playwright.dist
• python312Packages.lacuscore
• python312Packages.lacuscore.dist
• python312Packages.playwright
• python312Packages.playwright-stealth
• python312Packages.playwright-stealth.dist
• python312Packages.playwright.dist
• python312Packages.playwrightcapture
• python312Packages.playwrightcapture.dist
• python312Packages.pycookiecheat
• python312Packages.pycookiecheat.dist
• python312Packages.pytest-playwright
• python312Packages.pytest-playwright.dist
• shot-scraper
• shot-scraper.dist
• theharvester
• theharvester.dist
• urlwatch
• urlwatch.dist

[phaer]

• I built this successfully on aarch64-darwin, with a disabled sandbox - but that's already been needed before that PR on darwin.
• Some smaller style-related comments regarding update.sh. I also think that the update script could maybe be moved to pkgs/development/web/playwright, as it's mostly affecting the driver.nix file there?
• I think the bundling solution works well, Thanks :)
• As this PR is more up-to-date now, I'll re-base & update #298944 once this one got merged.

[phaer]

That's not checked yet, I believe?

[phaer]

Suggested change

	update_hash() {
	update_npm_hash() {

or something like this? Just to show that it's about npm/nodejs packages here, we are in ./pkgs/development/python-modules here.

[phaer]

@kalekseev is there anything one could do to to help pushing it forward? Or is it just waiting to a merge by a commuter.

None of my comments should be blocking anything, this is definitely progress in my view!

[kalekseev]

@phaer I believe it's ready to be merged, if someone sees any blockers I'm ready to fix them

[phaer]

@teto This might be good to merge then, if you don't have any objections? I think you might be the only one with commit bit among the current reviewers :)

[teto]

@phaer yes I am interested in merging this. I want to test it first, most likely next monday. Could you rebase to fix the conflict please ?

[kalekseev]

@phaer yes I am interested in merging this. I want to test it first, most likely next monday. Could you rebase to fix the conflict please ?

@teto rebased

[ibrahimsag]

Can we update the playwright to 1.46?

[teto]

@phaer yes I am interested in merging this. I want to test it first, most likely next monday. Could you rebase to fix the conflict please ?

Thanks for rebasing. I am sorry it takes so long. Playwright is something I help with on the side and before merging the PR, I would like to test it on our infrastructure, which needs a patch in staging-next. Once my patch reaches nixos-unstable, I can check this with our playwright code and if all goes well I will merge. So this may take a few more days. Other nixpkgs contributors may merge it before me. I will rebase if need be so you dont spend more time on this.

[SuperSandro2000]

I couldn't find any removed deps in this commit

[kalekseev]

deps were removed in master, updated commit message

[SuperSandro2000]

Suggested change

	sourceRoot = "${src.name}"; # update.sh depends on sourceRoot presence
	sourceRoot = src.name;

[kalekseev]

update.sh script depends on this form.

[kalekseev]

Updated to 1.46.0 and rebased.

[teto]

tested here. Looks good. I think we might want to expose more but this can be done later. Merging.

[humemm]

@kalekseev @teto

Just pulled and ran this PR. The build passes fine but the runtime blows up with an error

node:internal/modules/cjs/loader:1148
  throw err;
  ^

Error: Cannot find module '/nix/store/mfiy2g44i2lxmj9rdxns3zhvqdf4cv81-playwright-core-1.46.0/package/cli.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:1145:15)
    at Module._load (node:internal/modules/cjs/loader:986:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:174:12)
    at node:internal/main/run_main_module:28:49 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Just reviewed the locations and code and it seems like cli.js is no longer found in /package/cli.js but instead it's jut in the top level directory.

The following is the fix in playwright/default.nix:

# Set the correct driver path with the help of a patch in patches
substituteInPlace playwright/_impl/_driver.py \
  --replace-fail "@node@" "${lib.getExe nodejs}" \
-  --replace-fail "@driver@" "${driver}/package/cli.js"
+  --replace-fail "@driver@" "${driver}/cli.js"

maybe in the future we should add a simple test that runs playwright --version after build.

[kalekseev]

maybe in the future we should add a simple test that runs playwright --version after build.

this is python package not playwright itself so playwright --version won't catch this, I'll create PR with fix in a couple of hours

[humemm]

@kalekseev

maybe in the future we should add a simple test that runs playwright --version after build.

this is python package not playwright itself so playwright --version won't catch this, I'll create PR with fix in a couple of hours

Not sure what you're referring to. The error above was triggered by simply doing playwright --version. This "python" package also exposes the playwright executable/cli when installed.

[phaer]

@humemm now that this pr is merged, I just need to find time for a rebase of #298944 which includes a NixOS test