From 9b8796bc3fd0f0c6f2975fe72f2eaf1073b93459 Mon Sep 17 00:00:00 2001 From: Jarred Sumner Date: Thu, 13 Aug 2026 19:29:35 -0700 Subject: [PATCH 01/25] install: pnpm-parity commands and workspace fixes New commands: bun dedupe [--check], bun prune [--production] [--dry-run], bun pm licenses [--json] [--prod], bun audit fix. New flags: bun add/remove --filter (also honored by the bun install alias, which previously dropped the filter and could install the filter argument as a package), bun add --catalog[=name]. Fixes: catalog: peer dependencies skipped the hoisting satisfies check; --frozen-lockfile failed on turbo-pruned monorepos; pnpm-lock.yaml v9 migration (bare-hash patchedDependencies, non-semver alias dep-paths, catalog:default, recorded tarball URLs, git path: entries); a project's bunfig.toml is no longer overridden by ~/.npmrc for the same key; dependency::Version::eql treated any two catalog: specifiers as equal. --- docs/docs.json | 10 +- docs/pm/catalogs.mdx | 14 + docs/pm/cli/add.mdx | 62 + docs/pm/cli/audit.mdx | 37 +- docs/pm/cli/dedupe.mdx | 46 + docs/pm/cli/install.mdx | 6 + docs/pm/cli/pm.mdx | 45 + docs/pm/cli/prune.mdx | 61 + docs/pm/npmrc.mdx | 2 + docs/snippets/cli/add.mdx | 5 + src/install/PackageManager.rs | 247 ++- .../PackageManager/CommandLineArguments.rs | 155 +- .../PackageManager/PackageJSONEditor.rs | 196 +- .../PackageManager/PackageManagerOptions.rs | 7 +- .../PackageManager/PopulateManifestCache.rs | 35 + src/install/PackageManager/add_catalog.rs | 381 ++++ .../PackageManager/add_remove_with_filter.rs | 659 +++++++ .../PackageManager/install_with_manager.rs | 18 + .../updatePackageJSONAndInstall.rs | 355 ++-- src/install/audit_fix.rs | 641 +++++++ src/install/dedupe.rs | 448 +++++ src/install/dependency.rs | 19 +- src/install/lib.rs | 3 + src/install/lockfile.rs | 299 +-- src/install/lockfile/CatalogMap.rs | 114 +- src/install/lockfile/Package.rs | 46 +- src/install/lockfile/Package/Meta.rs | 2 +- src/install/lockfile/Package/WorkspaceMap.rs | 4 + src/install/lockfile/Tree.rs | 12 +- src/install/lockfile/bun.lock.rs | 134 +- src/install/lockfile/pruned_workspaces.rs | 18 + src/install/migration.rs | 19 +- src/install/pnpm.rs | 880 ++++++--- src/install/prune.rs | 943 +++++++++ src/install/resolution.rs | 9 +- src/options_types/command_tag.rs | 16 + src/runtime/cli/audit_command.rs | 257 ++- src/runtime/cli/dedupe_command.rs | 57 + src/runtime/cli/install_command.rs | 2 +- src/runtime/cli/mod.rs | 25 +- src/runtime/cli/pack_command.rs | 44 +- src/runtime/cli/package_manager_command.rs | 9 + src/runtime/cli/pm_licenses_command.rs | 678 +++++++ src/runtime/cli/prune_command.rs | 42 + src/semver/SemverQuery.rs | 18 + test/cli/install/bun-add-catalog.test.ts | 974 ++++++++++ test/cli/install/bun-add-filter.test.ts | 1014 ++++++++++ test/cli/install/bun-audit-fix.test.ts | 1020 ++++++++++ test/cli/install/bun-dedupe.test.ts | 927 +++++++++ test/cli/install/bun-pm-licenses.test.ts | 662 +++++++ test/cli/install/bun-prune.test.ts | 1202 ++++++++++++ test/cli/install/catalog-peer-hoist.test.ts | 466 +++++ test/cli/install/catalogs.test.ts | 32 + test/cli/install/config-precedence.test.ts | 210 ++ .../install/frozen-lockfile-pruned.test.ts | 823 ++++++++ .../__snapshots__/pnpm-lock-v9.test.ts.snap | 51 + .../install/migration/pnpm-lock-v9.test.ts | 819 ++++++++ .../package.json | 7 + .../pnpm-lock.yaml | 36 + .../outer/package.json | 8 + .../package.json | 7 + .../pnpm-lock.yaml | 38 + .../shared/config/package.json | 4 + .../pnpm/v9-catalog-default/package.json | 7 + .../pnpm/v9-catalog-default/pnpm-lock.yaml | 28 + .../v9-catalog-default/pnpm-workspace.yaml | 2 + .../pnpm/v9-codeload-tarballs/package.json | 8 + .../pnpm/v9-codeload-tarballs/pnpm-lock.yaml | 33 + .../pnpm/v9-file-directory/package.json | 7 + .../pnpm/v9-file-directory/pnpm-lock.yaml | 33 + .../sub-dep/child/package.json | 4 + .../v9-file-directory/sub-dep/package.json | 7 + .../pnpm/v9-git-references/package.json | 9 + .../pnpm/v9-git-references/pnpm-lock.yaml | 51 + .../pnpm/v9-git-subdirectory/package.json | 7 + .../pnpm/v9-git-subdirectory/pnpm-lock.yaml | 23 + .../pnpm/v9-git-urls-and-orphan/package.json | 8 + .../v9-git-urls-and-orphan/pnpm-lock.yaml | 35 + .../pnpm/v9-injected-workspace/package.json | 11 + .../packages/foo/package.json | 4 + .../pnpm/v9-injected-workspace/pnpm-lock.yaml | 29 + .../apps/web/package.json | 7 + .../v9-link-semver-specifier/package.json | 9 + .../v9-link-semver-specifier/pnpm-lock.yaml | 17 + .../shared/common/package.json | 4 + .../pnpm/v9-local-tarballs/package.json | 8 + .../pnpm/v9-local-tarballs/pnpm-lock.yaml | 32 + .../package.json | 8 + .../pnpm-lock.yaml | 11 + .../package.json | 7 + .../pnpm-lock.yaml | 25 + .../package.json | 8 + .../packages/a/package.json | 7 + .../pnpm-lock.yaml | 15 + .../v9-missing-package-entry/package.json | 7 + .../v9-missing-package-entry/pnpm-lock.yaml | 13 + .../pnpm/v9-multi-document/package.json | 7 + .../pnpm/v9-multi-document/pnpm-lock.yaml | 42 + .../v9-patch-bare-hash-registry/package.json | 7 + .../patches/no-deps.patch | 6 + .../pnpm-lock.yaml | 25 + .../pnpm-workspace.yaml | 2 + .../v9-patched-git-hosted-bare/package.json | 7 + .../patches/is-positive@3.1.0.patch | 9 + .../v9-patched-git-hosted-bare/pnpm-lock.yaml | 27 + .../pnpm-workspace.yaml | 2 + .../v9-patched-git-hosted-legacy/package.json | 7 + .../patches/is-positive@3.1.0.patch | 9 + .../pnpm-lock.yaml | 29 + .../pnpm-workspace.yaml | 2 + .../package.json | 11 + .../packages/peer/package.json | 5 + .../packages/pkg-a/package.json | 8 + .../pnpm-lock.yaml | 41 + .../pnpm/v9-reference-shapes/package.json | 11 + .../pnpm/v9-reference-shapes/pnpm-lock.yaml | 69 + .../pnpm/v9-runtime-entries/package.json | 14 + .../pnpm/v9-runtime-entries/pnpm-lock.yaml | 43 + .../pnpm-all-vulnerabilities-response.json | 1690 +++++++++++++++++ 119 files changed, 17060 insertions(+), 877 deletions(-) create mode 100644 docs/pm/cli/dedupe.mdx create mode 100644 docs/pm/cli/prune.mdx create mode 100644 src/install/PackageManager/add_catalog.rs create mode 100644 src/install/PackageManager/add_remove_with_filter.rs create mode 100644 src/install/audit_fix.rs create mode 100644 src/install/dedupe.rs create mode 100644 src/install/lockfile/pruned_workspaces.rs create mode 100644 src/install/prune.rs create mode 100644 src/runtime/cli/dedupe_command.rs create mode 100644 src/runtime/cli/pm_licenses_command.rs create mode 100644 src/runtime/cli/prune_command.rs create mode 100644 test/cli/install/bun-add-catalog.test.ts create mode 100644 test/cli/install/bun-add-filter.test.ts create mode 100644 test/cli/install/bun-audit-fix.test.ts create mode 100644 test/cli/install/bun-dedupe.test.ts create mode 100644 test/cli/install/bun-pm-licenses.test.ts create mode 100644 test/cli/install/bun-prune.test.ts create mode 100644 test/cli/install/catalog-peer-hoist.test.ts create mode 100644 test/cli/install/config-precedence.test.ts create mode 100644 test/cli/install/frozen-lockfile-pruned.test.ts create mode 100644 test/cli/install/migration/__snapshots__/pnpm-lock-v9.test.ts.snap create mode 100644 test/cli/install/migration/pnpm-lock-v9.test.ts create mode 100644 test/cli/install/migration/pnpm/v9-alias-in-optional-dependencies/package.json create mode 100644 test/cli/install/migration/pnpm/v9-alias-in-optional-dependencies/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/outer/package.json create mode 100644 test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/package.json create mode 100644 test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/shared/config/package.json create mode 100644 test/cli/install/migration/pnpm/v9-catalog-default/package.json create mode 100644 test/cli/install/migration/pnpm/v9-catalog-default/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-catalog-default/pnpm-workspace.yaml create mode 100644 test/cli/install/migration/pnpm/v9-codeload-tarballs/package.json create mode 100644 test/cli/install/migration/pnpm/v9-codeload-tarballs/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-file-directory/package.json create mode 100644 test/cli/install/migration/pnpm/v9-file-directory/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-file-directory/sub-dep/child/package.json create mode 100644 test/cli/install/migration/pnpm/v9-file-directory/sub-dep/package.json create mode 100644 test/cli/install/migration/pnpm/v9-git-references/package.json create mode 100644 test/cli/install/migration/pnpm/v9-git-references/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-git-subdirectory/package.json create mode 100644 test/cli/install/migration/pnpm/v9-git-subdirectory/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-git-urls-and-orphan/package.json create mode 100644 test/cli/install/migration/pnpm/v9-git-urls-and-orphan/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-injected-workspace/package.json create mode 100644 test/cli/install/migration/pnpm/v9-injected-workspace/packages/foo/package.json create mode 100644 test/cli/install/migration/pnpm/v9-injected-workspace/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-link-semver-specifier/apps/web/package.json create mode 100644 test/cli/install/migration/pnpm/v9-link-semver-specifier/package.json create mode 100644 test/cli/install/migration/pnpm/v9-link-semver-specifier/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-link-semver-specifier/shared/common/package.json create mode 100644 test/cli/install/migration/pnpm/v9-local-tarballs/package.json create mode 100644 test/cli/install/migration/pnpm/v9-local-tarballs/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-missing-importer-package-json/package.json create mode 100644 test/cli/install/migration/pnpm/v9-missing-importer-package-json/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-missing-package-entry-transitive/package.json create mode 100644 test/cli/install/migration/pnpm/v9-missing-package-entry-transitive/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/package.json create mode 100644 test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/packages/a/package.json create mode 100644 test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-missing-package-entry/package.json create mode 100644 test/cli/install/migration/pnpm/v9-missing-package-entry/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-multi-document/package.json create mode 100644 test/cli/install/migration/pnpm/v9-multi-document/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/package.json create mode 100644 test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/patches/no-deps.patch create mode 100644 test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/pnpm-workspace.yaml create mode 100644 test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/package.json create mode 100644 test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/patches/is-positive@3.1.0.patch create mode 100644 test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/pnpm-workspace.yaml create mode 100644 test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/package.json create mode 100644 test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/patches/is-positive@3.1.0.patch create mode 100644 test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/pnpm-workspace.yaml create mode 100644 test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/package.json create mode 100644 test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/packages/peer/package.json create mode 100644 test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/packages/pkg-a/package.json create mode 100644 test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-reference-shapes/package.json create mode 100644 test/cli/install/migration/pnpm/v9-reference-shapes/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-runtime-entries/package.json create mode 100644 test/cli/install/migration/pnpm/v9-runtime-entries/pnpm-lock.yaml create mode 100644 test/cli/install/registry/fixtures/audit/pnpm-all-vulnerabilities-response.json diff --git a/docs/docs.json b/docs/docs.json index 54f3545a1c1c..2dca006313a1 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -198,7 +198,15 @@ { "group": "Core Commands", "icon": "terminal", - "pages": ["/pm/cli/install", "/pm/cli/add", "/pm/cli/remove", "/pm/cli/update", "/pm/bunx"] + "pages": [ + "/pm/cli/install", + "/pm/cli/add", + "/pm/cli/remove", + "/pm/cli/update", + "/pm/cli/dedupe", + "/pm/cli/prune", + "/pm/bunx" + ] }, { "group": "Publishing & Analysis", diff --git a/docs/pm/catalogs.mdx b/docs/pm/catalogs.mdx index 2fe7f4a5f0eb..8599d652b81c 100644 --- a/docs/pm/catalogs.mdx +++ b/docs/pm/catalogs.mdx @@ -84,10 +84,15 @@ In your workspace packages, use the `catalog:` protocol to reference versions: "devDependencies": { "jest": "catalog:testing", "testing-library": "catalog:testing" + }, + "peerDependencies": { + "react": "catalog:" } } ``` +`catalog:` also works in `peerDependencies`. A catalog peer is resolved and hoisted exactly as if the catalog's range were written inline: if an ancestor already provides a version that satisfies the catalog range, the workspace package reuses that copy instead of getting its own. + ### 3. Run Bun Install Run `bun install` to install all dependencies according to the catalog versions. @@ -233,6 +238,14 @@ To update versions across all packages, change the version in the root package.j Then run `bun install` to update all packages. +To add a new dependency to the catalog (or refresh an existing entry), run `bun add` with `--catalog` (or `--catalog=` for a named catalog) inside the workspace package: + +```bash terminal icon="terminal" +bun add react --catalog +``` + +Bun writes the resolved version to the catalog in the root `package.json` and `"catalog:"` to the package's `package.json`. See [`bun add --catalog`](/pm/cli/add#--catalog). + ## Lockfile Integration Bun's lockfile tracks catalog versions, so installs are consistent across environments. The lockfile includes: @@ -279,6 +292,7 @@ Bun's lockfile tracks catalog versions, so installs are consistent across enviro - Catalog references must match a dependency defined in either `catalog` or one of the named `catalogs` - Empty strings and whitespace in catalog names are ignored (treated as default catalog) +- `catalog:default` is the same as `catalog:`; both read the `catalog` field, or `catalogs.default` if that is where the entry is defined (as pnpm names it) - Invalid dependency versions in catalogs fail to resolve during `bun install` - Catalogs are only available within workspaces; they cannot be used outside the monorepo diff --git a/docs/pm/cli/add.mdx b/docs/pm/cli/add.mdx index 97cd6031a1c5..7b083903a6b6 100644 --- a/docs/pm/cli/add.mdx +++ b/docs/pm/cli/add.mdx @@ -77,6 +77,68 @@ To view a complete list of options for this command: bun add --help ``` +## `--catalog` + +In a workspace, `bun add react --catalog` resolves the version as usual, but writes it to the root `package.json` [catalog](/pm/catalogs) (`workspaces.catalog`, or `workspaces.catalogs.` with `--catalog=`) and adds `"react": "catalog:"` (or `"catalog:"`) to the current package. An existing catalog entry is updated. Combine with `--exact`, `--dev`, etc. as usual. + +```bash terminal icon="terminal" +bun add react --catalog +bun add vitest --catalog=testing +``` + +```json package.json icon="file-json" +// root package.json +{ + "workspaces": { + "packages": ["packages/*"], + "catalog": { + "react": "^18.2.0" // [!code ++] + } + } +} +``` + +```json packages/app/package.json icon="file-json" +{ + "dependencies": { + "react": "catalog:" // [!code ++] + } +} +``` + +A few things to know, some of which differ from pnpm's `--save-catalog`: + +- If the catalog already has an entry for the package, it is replaced with the newly resolved version (`bun add react --catalog` moves every workspace package that depends on `"catalog:"` to the new version). pnpm leaves an existing entry alone. +- The flag decides which catalog is used. If the current package already depends on `"react": "catalog:legacy"`, `bun add react --catalog` moves it to the default catalog; the old `legacy` entry is left in place. +- `--catalog` and `--catalog=default` are the same catalog: the entry is written to whichever of `catalog` or `catalogs.default` the root `package.json` already defines (`catalog` is created when there is neither), so a repository migrated from pnpm never ends up with the package in both. +- The recorded version is the one that was installed, so an entry in [`overrides`](/pm/overrides) for the package ends up in the catalog too. +- The name must be attached with `=` (`--catalog=testing`); `--catalog testing` adds a package called `testing`. +- Packages must be added by name (`react`, `react@^18`, `alias@npm:react`). Bare URLs, paths and `workspace:` versions are rejected. +- The root `package.json` must have a `workspaces` field. + +## `--filter` + +**Alias** — `-F` + +In a monorepo, `--filter` adds the package to the matching workspace package(s) instead of the package in the current directory. Patterns match workspace names or paths (`./packages/*`), `*` matches every workspace including the root, and `!pattern` excludes. Repeat the flag to combine patterns. Every matching package is edited; if nothing matches, the command fails without touching anything, and a pattern that matches nothing while others do prints a warning. `bun remove` and `bun install ` accept the same flag. See [filtering](/pm/filter). + +```bash terminal icon="terminal" +bun add zod --filter api +bun add -d typescript --filter './packages/*' +bun add ./vendor/logger --filter '*' +bun remove zod --filter '*' --filter '!api' +``` + +A few things to know, some of which differ from pnpm: + +- The workspace root is selected by `*` and by filter sets made only of `!` patterns; exclude it explicitly with `--filter '!'`. To target only the root, use its name or `--filter .` from the root directory. +- A `!` pattern always wins, whatever order the flags are given in: `--filter '!./packages/*' --filter api` selects nothing. +- Name patterns are globs, and `*` does not cross `/`: `*-utils` does not match `@acme/date-utils`; use `*/*-utils` or `@acme/*`. +- Path patterns must match a workspace directory exactly, relative to the current directory: `--filter ./packages` selects nothing, `--filter ./packages/*` selects every package in it, and `--filter .` only works from a workspace's own directory. +- Local paths (`./vendor/logger`, `file:../logger`, `./logger.tgz`) are relative to the current directory and are re-spelled relative to each selected package (`"logger": "../../vendor/logger"` in `packages/api/package.json`). +- The flag chooses which `package.json` files are edited, not what is installed: a single install of the whole workspace runs afterwards. The `package.json` files are written as soon as the lockfile is saved, so they agree with `bun.lock` even if a root `postinstall` script then fails. +- `--dry-run` skips writing; `--filter` cannot be combined with `--global`. + ## `--global` **Alias** — `bun add --global`, `bun add -g`, `bun install --global` and `bun install -g` diff --git a/docs/pm/cli/audit.mdx b/docs/pm/cli/audit.mdx index 0038c4997aa9..e8e4c031bd86 100644 --- a/docs/pm/cli/audit.mdx +++ b/docs/pm/cli/audit.mdx @@ -35,16 +35,16 @@ To update all dependencies to the latest versions (including breaking changes): bun audit --audit-level=high ``` -**`--prod`** - Audit only production dependencies (excludes devDependencies): +**`--prod`** - Audit only production dependencies. A package is production when it is reachable from the root through `dependencies`, `optionalDependencies` or `peerDependencies` edges; packages that are only reached through `devDependencies` or an optional peer dependency are excluded, even when another version of the same package is a production dependency: ```bash terminal icon="terminal" bun audit --prod ``` -**`--ignore `** - Ignore specific CVEs (repeat the flag to ignore several): +**`--ignore `** - Ignore specific advisories by GHSA ID or numeric advisory ID (repeat the flag to ignore several). The registry's advisory data does not include CVE IDs, so a CVE ID does not match anything: ```bash terminal icon="terminal" -bun audit --ignore CVE-2022-25883 --ignore CVE-2023-26136 +bun audit --ignore GHSA-c2qf-rxjj-qqgw --ignore 1112918 ``` ### `--json` @@ -55,6 +55,35 @@ Use the `--json` flag to print the raw JSON response from the registry instead o bun audit --json ``` +### `bun audit fix` + +`bun audit fix` runs the audit and then upgrades each vulnerable package in `bun.lock` to the lowest version that is not affected by any advisory and still satisfies every range that depends on it (including `overrides` and catalog entries), then installs. `package.json` is never modified and unrelated packages are not touched. + +```bash terminal icon="terminal" +bun audit fix +``` + +``` +fixing: + ms@0.7.0 → 0.7.1 + +requires a semver-major update: + minimatch@0.3.0 → 3.0.2 + express@3.21.2 depends on minimatch@0.3.0 + +Fixed 1 vulnerability in 1 package +1 vulnerability remaining +``` + +- Packages whose only safe version falls outside a dependent's range are listed under `requires a semver-major update:` and left alone. Update the dependent, or add an `overrides` entry, and run the command again. A dependency bundled inside another package (`express@3.21.2 bundles minimatch@0.3.0`) can only be fixed by updating the package that bundles it. +- Packages are only ever upgraded, never downgraded, and only stable releases are candidates. `no fix available:` lists packages with no safe newer release, and says so when a safe release exists but is newer than `--minimum-release-age` or the package's manifest could not be fetched. +- Advisories that match none of the installed versions are listed under `not matched to an installed version:` and count as remaining. +- Counts are advisories, as in `bun audit`. A package installed as several versions is upgraded per version, and an advisory only counts as fixed once every version it affects has been upgraded; otherwise it counts as remaining. +- The fix lives in `bun.lock` only. A later dependency that pins the vulnerable version, or regenerating the lockfile, can bring it back; re-run `bun audit`. +- `--dry-run` prints the plan without changing anything. +- `--registry` applies to both the advisory request and the install of the fixed versions; there is no way to query one registry for advisories and install from another. +- `--audit-level`, `--ignore` and `--prod` behave as for `bun audit`, except that `--prod` (like `--frozen-lockfile` and `--no-save`) stops `bun.lock` from being written, so `bun audit fix` rejects it before contacting the registry. + ### Exit code -`bun audit` exits with code `0` if no vulnerabilities are found and `1` if the report lists any, including when `--json` is passed. +`bun audit` exits with code `0` if no vulnerabilities are found and `1` if the report lists any, including when `--json` is passed. `bun audit fix` exits with `0` when nothing vulnerable remains after the fix (or, with `--dry-run`, would remain) and `1` otherwise. diff --git a/docs/pm/cli/dedupe.mdx b/docs/pm/cli/dedupe.mdx new file mode 100644 index 000000000000..732d7d53b9be --- /dev/null +++ b/docs/pm/cli/dedupe.mdx @@ -0,0 +1,46 @@ +--- +title: "bun dedupe" +description: "Remove duplicate versions of packages from bun.lock" +--- + +After bumping or adding a dependency, `bun.lock` can end up with several versions of the same package even though one of them satisfies every range that requests it — for example `esbuild@0.15.10` and `esbuild@0.15.11` when the ranges are `^0.15.7` and `^0.15.8`. `bun dedupe` re-points every dependency range onto the locked version that satisfies the most ranges (the highest version on ties), drops the versions nothing needs any more, saves `bun.lock`, and installs. + +It only uses versions that are already in the lockfile. It never moves a dependency outside its range and never contacts the registry to resolve anything; use `bun update` for that. + +```bash terminal icon="terminal" +bun dedupe +``` + +``` +Removed 2 duplicate versions: esbuild@0.15.10, react@18.2.0 +``` + +If there is nothing to remove, the command prints: + +``` +Already deduplicated. +``` + +### `--check` + +`bun dedupe --check` prints the versions that would be removed and exits with code `1` without writing anything. It exits with code `0` when the lockfile is already deduplicated, which makes it usable in CI. `--dry-run` is an alias. + +```bash terminal icon="terminal" +bun dedupe --check +``` + +``` +1 duplicate version can be removed: esbuild@0.15.10 +``` + +### Notes + +- Overrides and catalogs are honoured: a dependency is re-pointed using its effective range. +- Because it keeps the fewest versions, a direct dependency can be moved to an older version that still satisfies its range when a transitive dependency pins that older version. +- Dependencies pointing at a version listed in `patchedDependencies` are never moved, bundled dependencies stay on the copy inside their tarball, and dependencies specified as a dist-tag (such as `latest`), a git URL, or a tarball keep the version they resolved to. +- Only the ranges of packages that remain installed are counted; a version that the same run removes does not influence where its own dependencies end up, so running the command twice never changes anything the second time. +- It works from the ranges recorded in `bun.lock`; changes made to `package.json` since the last install are applied by the install that follows. +- `bun.lock` and `node_modules` change; `package.json` never does. The result only lives in `bun.lock`: deleting the lockfile and reinstalling can bring the duplicates back. Add an [override](/pm/overrides) to pin a version permanently. +- `--lockfile-only` rewrites `bun.lock` without installing. +- `--frozen-lockfile`, `--production`, and `--no-save` cannot be combined with removing duplicates; the command exits with code `1` and lists the duplicates instead. Use `--check` in CI. +- Works with both the hoisted and isolated linker. It only removes distinct versions of a package; the isolated linker may still keep several copies of one version that differ in their peer dependencies, and those are not reported by `--check`. diff --git a/docs/pm/cli/install.mdx b/docs/pm/cli/install.mdx index ae03186ce5ac..5edba98d9e30 100644 --- a/docs/pm/cli/install.mdx +++ b/docs/pm/cli/install.mdx @@ -170,6 +170,10 @@ For reproducible installs, use `--frozen-lockfile`. Bun installs the exact versi bun install --frozen-lockfile ``` +`--frozen-lockfile` also works on a pruned copy of a monorepo (for example the output of `turbo prune`, or a Docker context that copies `bun.lock` with only some workspace folders): workspaces listed in `bun.lock` whose `package.json` is not on disk are skipped rather than treated as a lockfile change, and packages the pruned lockfile still lists are installed as written. This works whether the root `workspaces` field uses globs or lists each folder explicitly, and Bun prints a `note:` with the number of skipped workspaces (`--verbose` names them), so a `bun.lock` that is stale because a workspace was deleted without re-running `bun install` is still visible in CI logs. A workspace is only skipped when its `package.json` is missing — removing it from the `workspaces` globs while the folder is still present, changing any `package.json` that is on disk, or trimming `overrides` or `catalog` entries from `bun.lock` still fails the install. The skipped workspaces' exclusive dependencies are not downloaded or installed, but they remain in `bun.lock`, so `bun pm ls` and `bun audit` still report them. A package that a surviving workspace lists as an optional peer dependency is installed if the pruned `bun.lock` still contains it, even when only a skipped workspace depended on it. + +`--frozen-lockfile` never writes `bun.lock`, including with `--lockfile-only`; the one exception is the `bun.lockb` to `bun.lock` migration requested with `--save-text-lockfile`. To check a lockfile without installing anything, use `bun install --frozen-lockfile --dry-run`. + See [lockfile](/pm/lockfile) for more on `bun.lock`. --- @@ -516,6 +520,7 @@ The migration process handles: - Preserves package versions and resolution information - Maintains dependency relationships and peer dependencies - Handles patched dependencies with integrity hashes +- Migrates git (`git+ssh://`, `git+https://`), GitHub, tarball URL and `file:` dependencies, including transitive ones and npm aliases (`npm:`) recorded in the lockfile ### Workspace Configuration @@ -582,6 +587,7 @@ Bun migrates the following pnpm configuration from both `pnpm-lock.yaml` and `pn - Requires pnpm lockfile version 7 or higher - Workspace packages must have a `name` field in their `package.json` - All catalog entries referenced by dependencies must exist in the catalogs definition +- Every workspace listed in `pnpm-lock.yaml` must have its `package.json` on disk (when building in Docker, copy each workspace's `package.json` before running `bun install`); otherwise migration reports the missing importer and `bun install` falls back to a fresh resolution After migration, you can safely remove `pnpm-lock.yaml` and `pnpm-workspace.yaml` files. diff --git a/docs/pm/cli/pm.mdx b/docs/pm/cli/pm.mdx index 2fd29fc7a2e1..9a8388df645a 100644 --- a/docs/pm/cli/pm.mdx +++ b/docs/pm/cli/pm.mdx @@ -163,6 +163,51 @@ bun list --trusted └── esbuild@0.21.5 ``` +## licenses + +To list every installed package grouped by its license (read from the `license` or legacy `licenses` field of each package's installed `package.json`; packages declaring neither are listed under `Unknown`, and workspace and `link:` packages are not listed): + +```bash terminal icon="terminal" +bun pm licenses +``` + +```txt +MIT (2) +├── path-parse@1.0.6 +└── resolve@1.9.0 + +Unknown (4) +├── a-dep@1.0.1 +├── no-deps@1.0.0 +├── no-deps@1.0.1 +└── one-dep@1.0.0 +``` + +Pass `--prod` to skip `devDependencies`, and `--json` to get a machine-readable object keyed by license, where each entry has `name`, `versions` (in semver order) and, when present, the `homepage` and `author` of the newest listed version: + +```bash terminal icon="terminal" +bun pm licenses --json --prod +``` + +```json +{ + "MIT": [ + { + "name": "path-parse", + "versions": ["1.0.6"], + "homepage": "https://github.com/jbgutierrez/path-parse#readme", + "author": "Javier Blanco " + } + ] +} +``` + +`--prod` only drops the `devDependencies` of the root package and of workspace packages; `optionalDependencies` and everything a production dependency pulls in are still listed, and from a workspace root every workspace's production dependencies are included. Run it inside a workspace package to list only that package's dependencies, and use [`bun why`](/pm/cli/why) to see which dependency pulls in an unexpected package. + +Packages that are in the lockfile but not in `node_modules` (for example after `bun install --production`) are omitted, and a `warn:` line with the number of omitted packages is printed to stderr. Packages that don't apply to the current platform (`os`/`cpu`) are omitted silently. + +This is the equivalent of `pnpm licenses list`. + ## whoami Print your npm username. Requires you to be logged in (`bunx npm login`) with credentials in either `bunfig.toml` or `.npmrc`: diff --git a/docs/pm/cli/prune.mdx b/docs/pm/cli/prune.mdx new file mode 100644 index 000000000000..f3d2f6c26392 --- /dev/null +++ b/docs/pm/cli/prune.mdx @@ -0,0 +1,61 @@ +--- +title: "bun prune" +description: "Remove packages that are not in bun.lock from node_modules" +--- + +`bun prune` deletes everything in `node_modules` that the current `bun.lock` does not install there — packages left behind after switching branches, editing `bun.lock`, or installing with another package manager. It only reads `bun.lock`; it never contacts the registry and never changes `bun.lock` or `package.json`. + +```bash terminal icon="terminal" +bun prune +``` + +``` +- node_modules/@types/node +- node_modules/left-pad +Removed 2 packages +``` + +If there is nothing to remove, the command prints: + +``` +Nothing to prune. +``` + +### `--production` + +`bun prune --production` (alias `--prod`) additionally removes every package that is only needed by `devDependencies`, leaving exactly what a fresh `bun install --production` would install. `--omit=dev`, `--omit=optional` and `--omit=peer` work the same way as they do for `bun install`. + +This makes it possible to build with `devDependencies` installed and ship without them: + +```dockerfile +COPY package.json bun.lock ./ +RUN bun install --frozen-lockfile +COPY . . +RUN bun run build +RUN bun prune --production +``` + +### `--dry-run` + +`bun prune --dry-run` prints the same list followed by `Would remove N packages` and deletes nothing. + +```bash terminal icon="terminal" +bun prune --production --dry-run +``` + +``` +- node_modules/typescript +Would remove 1 package +``` + +### Notes + +- Works with both linkers. With the hoisted linker it checks every `node_modules` folder `bun install` would install into — including workspace folders and the nested `node_modules` folders of installed packages, so copies left behind by an earlier install are removed too; with the isolated linker it removes unused entries in `node_modules/.bun` and the symlinks that pointed at them. +- The linker is chosen the same way `bun install` chooses it. If `node_modules` was installed with the other linker (for example `bun install --linker hoisted` in a project that defaults to isolated), `bun prune` refuses to run instead of removing packages the other layout needs; pass the same `--linker` you installed with, or run `bun install` to switch layouts. +- Always runs against the workspace root, even when invoked inside a workspace package. +- Workspace symlinks (even `--production` keeps a workspace linked from another workspace's `devDependencies`), `.bin` entries that are still in use, dot-entries such as `.cache`, plain files, and dependencies bundled inside a package's tarball are never removed. `.bin` entries whose package was removed are cleaned up on every platform. +- Nothing outside `node_modules` is ever deleted. A package folder that you replaced with a symlink is left alone, and the `node_modules` folder inside it is not pruned. With `install.globalStore`, only the project's links into the store are removed. +- Packages disabled for the current `os`/`cpu` are removed, just as `bun install` would skip them. Pass `--os` / `--cpu` to prune for another platform. +- Only package names are compared, and only `bun.lock` is consulted: a dependency removed from `package.json` is pruned after the next `bun install` updates `bun.lock`. Run `bun install` if you also want the versions on disk re-verified. +- Lifecycle scripts are never run. +- Equivalent to `pnpm prune` and `npm prune`. diff --git a/docs/pm/npmrc.mdx b/docs/pm/npmrc.mdx index 550366950777..0c2ddd181662 100644 --- a/docs/pm/npmrc.mdx +++ b/docs/pm/npmrc.mdx @@ -5,6 +5,8 @@ description: Bun loads configuration options from [`.npmrc`](https://docs.npmjs.com/cli/v10/configuring-npm/npmrc) files, so you can reuse your existing registry and scope configuration. +Bun reads `~/.npmrc` (or `$XDG_CONFIG_HOME/.npmrc` if it exists), then the project's `./.npmrc`, then `bunfig.toml`. When the same option is set in more than one place, `./.npmrc` overrides `~/.npmrc`, `bunfig.toml` overrides both `.npmrc` files, and command-line flags override everything. Options that only appear in `.npmrc` (such as `///:_authToken`) still apply, including to registries configured in `bunfig.toml`. + We recommend migrating your `.npmrc` file to Bun's [`bunfig.toml`](/runtime/bunfig) format, which supports more options, including Bun-specific ones. diff --git a/docs/snippets/cli/add.mdx b/docs/snippets/cli/add.mdx index b1e1af9327a8..5db517d56400 100644 --- a/docs/snippets/cli/add.mdx +++ b/docs/snippets/cli/add.mdx @@ -38,6 +38,11 @@ bun add <@version> Only add dependencies to package.json if they are not already present + + Add the resolved version to the root package.json catalog and depend on it as catalog:;{" "} + --catalog=NAME targets catalogs.NAME + + ### Project Files & Lockfiles diff --git a/src/install/PackageManager.rs b/src/install/PackageManager.rs index 3aad03b5f197..eb0d78093f2b 100644 --- a/src/install/PackageManager.rs +++ b/src/install/PackageManager.rs @@ -49,6 +49,10 @@ pub mod Command { // Sub-module declarations — explicit #[path] attrs for PascalCase / // camelCase file names. // ────────────────────────────────────────────────────────────────────────── +#[path = "PackageManager/add_catalog.rs"] +pub mod add_catalog; +#[path = "PackageManager/add_remove_with_filter.rs"] +pub mod add_remove_with_filter; #[path = "PackageManager/CommandLineArguments.rs"] pub mod command_line_arguments; #[path = "PackageManager/install_with_manager.rs"] @@ -127,6 +131,9 @@ impl PackageManagerCommand { ├ --all list the entire dependency tree according to the current lockfile └ --trusted list only trusted dependencies bun pm why \ show dependency tree explaining why a package is installed + bun pm licenses list installed packages grouped by license + ├ --json output as JSON + └ --prod omit devDependencies bun pm whoami print the current npm username bun pm view name[@version] view package metadata from the registry (use `bun info` instead) bun pm version [increment] bump the version in package.json and create a git tag @@ -316,6 +323,7 @@ pub struct PackageManager { pub subcommand: Subcommand, pub(crate) update_requests: Box<[UpdateRequest]>, + pub audit_fix_pins: Box<[crate::audit_fix::PlannedFix]>, /// Only set in `bun pm` pub root_package_json_name_at_time_of_init: Box<[u8]>, @@ -411,6 +419,9 @@ pub struct PackageManager { // `bun update -r`/`--filter`: workspaces whose deps update. None = cwd only. pub(crate) update_target_workspaces: Option>, + // `bun add/remove --filter`: package.json edits written as soon as the lockfile is saved. + pub(crate) pending_filtered_write: Option>, + pub(crate) patched_dependencies_to_remove: ArrayHashMap, @@ -460,6 +471,8 @@ pub enum Subcommand { Audit, Info, Why, + Dedupe, + Prune, // bin, // hash, // @"hash-print", @@ -478,9 +491,10 @@ impl Subcommand { } pub(crate) fn supports_workspace_filtering(self) -> bool { - matches!(self, Self::Outdated | Self::Install | Self::Update) - // .pack => true, - // .add => true, + matches!( + self, + Self::Outdated | Self::Install | Self::Update | Self::Add | Self::Remove + ) } pub(crate) fn supports_json_output(self) -> bool { @@ -550,6 +564,38 @@ impl WorkspaceFilter { }) } + pub fn matches_any(filters: &[WorkspaceFilter], name: &[u8], abs_posix_path: &[u8]) -> bool { + let has_positive = filters.iter().any(|f| match f { + WorkspaceFilter::All => true, + WorkspaceFilter::Path(p) | WorkspaceFilter::Name(p) => p.first() != Some(&b'!'), + }); + + let mut matched = !has_positive; + for filter in filters { + let (pattern, subject): (&[u8], &[u8]) = match filter { + WorkspaceFilter::All => { + matched = true; + continue; + } + WorkspaceFilter::Path(pattern) => { + if pattern.is_empty() { + continue; + } + (pattern, abs_posix_path) + } + WorkspaceFilter::Name(pattern) => (pattern, name), + }; + if pattern.first() == Some(&b'!') { + if bun_glob::r#match(&pattern[1..], subject).matches() { + return false; + } + } else if bun_glob::r#match(pattern, subject).matches() { + matched = true; + } + } + matched + } + /// Every workspace (root included), filtered by `filter_patterns` (empty = all). pub fn select_workspaces( lockfile: &crate::Lockfile, @@ -586,52 +632,23 @@ impl WorkspaceFilter { let top_level_dir = FileSystem::instance().top_level_dir(); - let has_positive = converted_filters.iter().any(|f| match f { - WorkspaceFilter::All => true, - WorkspaceFilter::Path(p) | WorkspaceFilter::Name(p) => p.first() != Some(&b'!'), - }); - let mut i = 0; while i < ids.len() { - let pkg_id = ids[i]; - let mut matched = !has_positive; - for filter in &converted_filters { - let (pattern, subject): (&[u8], &[u8]) = match filter { - WorkspaceFilter::All => { - matched = true; - continue; - } - WorkspaceFilter::Path(pattern) => { - if pattern.is_empty() { - continue; - } - let res = &pkg_resolutions[pkg_id as usize]; - let res_path: &[u8] = match res.tag { - crate::resolution::Tag::Workspace => res.workspace().slice(string_buf), - crate::resolution::Tag::Root => top_level_dir, - _ => unreachable!(), - }; - let abs = resolve_path::join_abs_string_buf::( - top_level_dir, - &mut path_buf.0, - &[res_path], - ); - (pattern, strings::without_trailing_slash(abs)) - } - WorkspaceFilter::Name(pattern) => { - (pattern, pkg_names[pkg_id as usize].slice(string_buf)) - } - }; - if pattern.first() == Some(&b'!') { - if bun_glob::r#match(&pattern[1..], subject).matches() { - matched = false; - break; - } - } else if bun_glob::r#match(pattern, subject).matches() { - matched = true; - } - } - if matched { + let pkg_id = ids[i] as usize; + let name = pkg_names[pkg_id].slice(string_buf); + let res = &pkg_resolutions[pkg_id]; + let res_path: &[u8] = match res.tag { + crate::resolution::Tag::Workspace => res.workspace().slice(string_buf), + crate::resolution::Tag::Root => top_level_dir, + _ => unreachable!(), + }; + let abs = resolve_path::join_abs_string_buf::( + top_level_dir, + &mut path_buf.0, + &[res_path], + ); + let abs = strings::without_trailing_slash(abs); + if WorkspaceFilter::matches_any(&converted_filters, name, abs) { i += 1; } else { ids.swap_remove(i); @@ -658,6 +675,8 @@ pub struct CatalogUpdateInfo { pub dep_name: Box<[u8]>, pub original_version_literal: Box<[u8]>, pub is_alias: bool, + /// Set by `Lockfile::clean_with_logger`; `None` leaves the entry as written. + pub new_version_literal: Option>, } pub struct UpdateTargetWorkspace { @@ -1434,6 +1453,106 @@ pub(crate) fn get() -> *mut PackageManager { // init // ────────────────────────────────────────────────────────────────────────── +/// bunfig beats npmrc per field; a registry npmrc left at bunfig's URL is kept since npmrc attached credentials to it. +fn overlay_bunfig_install(install: &mut Api::BunInstall, bunfig: Api::BunInstall) { + let Api::BunInstall { + default_registry, + scoped, + lockfile_path, + save_lockfile_path, + cache_directory, + dry_run, + force, + save_dev, + save_optional, + save_peer, + save_lockfile, + production, + save_yarn_lockfile, + disable_cache, + disable_manifest_cache, + global_dir, + global_bin_dir, + frozen_lockfile, + exact, + concurrent_scripts, + cafile, + save_text_lockfile, + ca, + ignore_scripts, + link_workspace_packages, + node_linker, + global_store, + security_scanner, + minimum_release_age_ms, + minimum_release_age_excludes, + public_hoist_pattern, + hoist_pattern, + hoist, + } = bunfig; + + if let Some(registry) = default_registry { + if install + .default_registry + .as_ref() + .is_none_or(|current| current.url != registry.url) + { + install.default_registry = Some(registry); + } + } + + if let Some(bunfig_scopes) = scoped { + let scopes = &mut install.scoped.get_or_insert_with(Default::default).scopes; + for (name, registry) in bunfig_scopes.scopes.iter() { + if scopes + .get(name) + .is_none_or(|current| current.url != registry.url) + { + scopes.insert(name, registry.clone()); + } + } + } + + macro_rules! overlay { + ($($field:ident),* $(,)?) => { + $( if $field.is_some() { install.$field = $field; } )* + }; + } + overlay!( + lockfile_path, + save_lockfile_path, + cache_directory, + dry_run, + force, + save_dev, + save_optional, + save_peer, + save_lockfile, + production, + save_yarn_lockfile, + disable_cache, + disable_manifest_cache, + global_dir, + global_bin_dir, + frozen_lockfile, + exact, + concurrent_scripts, + cafile, + save_text_lockfile, + ca, + ignore_scripts, + link_workspace_packages, + node_linker, + global_store, + security_scanner, + minimum_release_age_ms, + minimum_release_age_excludes, + public_hoist_pattern, + hoist_pattern, + hoist, + ); +} + /// Returns `&'static mut PackageManager` — the process-singleton (held in /// `holder::RAW_PTR`) is leaked for the process lifetime and `init()` is called /// exactly once on the single CLI dispatch thread. Every @@ -1602,7 +1721,7 @@ pub fn init( } if subcommand == Subcommand::Install { - if cli.positionals.len() > 1 { + if cli.positionals.len() > 1 && cli.filters.is_empty() { // this is `bun add `. // // create the package.json instead of returning an error so that @@ -1732,6 +1851,7 @@ pub fn init( &json_source, prop.loc, None, + true, ) { Ok(v) => v, Err(_) => break, @@ -1869,11 +1989,21 @@ pub fn init( initialize_store(); { - let install_ref = ctx.install.get_or_insert_with(|| { - // `Api::BunInstall` derives `Default` (all fields `None`/empty). - // Own via `Box` — never `Box::leak`. - Box::new(Api::BunInstall::default()) - }); + // npmrc < bunfig < CLI; seeding registries lets npmrc `//host/:_authToken` attach to bunfig registries. + let bunfig_install = ctx + .install + .take() + .map_or_else(Api::BunInstall::default, |b| *b); + let mut install = Api::BunInstall { + default_registry: bunfig_install.default_registry.clone(), + scoped: match &bunfig_install.scoped { + Some(map) => Some(Api::NpmRegistryMap { + scopes: map.scopes.clone()?, + }), + None => None, + }, + ..Default::default() + }; let npmrc_local = ZBox::from_bytes(b".npmrc"); let mut buf = PathBuffer::uninit(); @@ -1900,14 +2030,17 @@ pub fn init( if global_len > 0 { ini::load_npmrc_config( - &mut **install_ref, + &mut install, env, true, &[ZStr::from_buf(&buf[..], global_len), &*npmrc_local], ); } else { - ini::load_npmrc_config(&mut **install_ref, env, true, &[&*npmrc_local]); + ini::load_npmrc_config(&mut install, env, true, &[&*npmrc_local]); } + + overlay_bunfig_install(&mut install, bunfig_install); + ctx.install = Some(Box::new(install)); } let cpu_count: u32 = u32::from(bun_core::get_thread_count()); // Captured before `cli` is moved into `options.load(Some(cli), ...)` below. @@ -2038,6 +2171,7 @@ pub fn init( wr!(root_progress_node, core::ptr::null_mut()); wr!(to_update, false); wr!(update_requests, Box::default()); + wr!(audit_fix_pins, Box::default()); wr!(root_package_id, RootPackageId::default()); wr!(task_batch, thread_pool::Batch::default()); wr!(task_queue, TaskDependencyQueue::default()); @@ -2076,6 +2210,7 @@ pub fn init( wr!(updating_packages, StringArrayHashMap::default()); wr!(updating_catalogs, Vec::new()); wr!(update_target_workspaces, None); + wr!(pending_filtered_write, None); wr!(patched_dependencies_to_remove, ArrayHashMap::default()); wr!(last_reported_slow_lifecycle_script_at, 0); wr!(cached_tick_for_slow_lifecycle_script_logging, 0); @@ -2470,6 +2605,7 @@ fn init_with_runtime_once( wr!(root_progress_node, core::ptr::null_mut()); wr!(to_update, false); wr!(update_requests, Box::default()); + wr!(audit_fix_pins, Box::default()); wr!(root_package_json_name_at_time_of_init, Box::default()); wr!(root_package_id, RootPackageId::default()); wr!(task_batch, thread_pool::Batch::default()); @@ -2515,6 +2651,7 @@ fn init_with_runtime_once( wr!(updating_packages, StringArrayHashMap::default()); wr!(updating_catalogs, Vec::new()); wr!(update_target_workspaces, None); + wr!(pending_filtered_write, None); wr!(patched_dependencies_to_remove, ArrayHashMap::default()); wr!(last_reported_slow_lifecycle_script_at, 0); wr!(cached_tick_for_slow_lifecycle_script_logging, 0); diff --git a/src/install/PackageManager/CommandLineArguments.rs b/src/install/PackageManager/CommandLineArguments.rs index 84b223a973fb..63e2e5260c3c 100644 --- a/src/install/PackageManager/CommandLineArguments.rs +++ b/src/install/PackageManager/CommandLineArguments.rs @@ -133,7 +133,7 @@ pub(crate) static INSTALL_PARAMS: &[ParamType] = concat_params![ clap::param!("--peer Add dependency to \"peerDependencies\""), clap::param!("-E, --exact Add the exact version instead of the ^range"), clap::param!( - "--filter ... Install packages for the matching workspaces" + "-F, --filter ... Install packages for the matching workspaces" ), clap::param!( "-a, --analyze Analyze & install all dependencies of files passed as arguments recursively (using Bun's bundler)" @@ -141,6 +141,9 @@ pub(crate) static INSTALL_PARAMS: &[ParamType] = concat_params![ clap::param!( "--only-missing Only add dependencies to package.json if they are not already present" ), + clap::param!( + "--catalog ? Add the resolved version to the root package.json catalog and depend on it as \"catalog:\" (use --catalog=NAME for a named catalog)" + ), clap::param!(" ... "), ] ]; @@ -203,12 +206,18 @@ pub(crate) static ADD_PARAMS: &[ParamType] = concat_params![ ), clap::param!("--peer Add dependency to \"peerDependencies\""), clap::param!("-E, --exact Add the exact version instead of the ^range"), + clap::param!( + "-F, --filter ... Add the package(s) to the matching workspaces instead of the current package" + ), clap::param!( "-a, --analyze Recursively analyze & install dependencies of files passed as arguments (using Bun's bundler)" ), clap::param!( "--only-missing Only add dependencies to package.json if they are not already present" ), + clap::param!( + "--catalog ? Add the resolved version to the root package.json catalog and depend on it as \"catalog:\" (use --catalog=NAME for a named catalog)" + ), clap::param!( " ... \"name\" or \"name@version\" of package(s) to install" ), @@ -217,9 +226,14 @@ pub(crate) static ADD_PARAMS: &[ParamType] = concat_params![ pub(crate) static REMOVE_PARAMS: &[ParamType] = concat_params![ SHARED_PARAMS, - &[clap::param!( - " ... \"name\" of package(s) to remove from package.json" - ),] + &[ + clap::param!( + "-F, --filter ... Remove the package(s) from the matching workspaces instead of the current package" + ), + clap::param!( + " ... \"name\" of package(s) to remove from package.json" + ), + ] ]; pub(crate) static LINK_PARAMS: &[ParamType] = concat_params![ @@ -345,6 +359,45 @@ static WHY_PARAMS: &[ParamType] = concat_params![ ] ]; +static DEDUPE_PARAMS: &[ParamType] = concat_params![ + SHARED_PARAMS, + &[ + clap::param!( + "--check Exit with code 1 if the lockfile has duplicate versions that can be removed, without changing anything" + ), + clap::param!(" ... "), + ] +]; + +static PRUNE_PARAMS: &[ParamType] = concat_params![ + SHARED_PARAMS, + &[clap::param!(" ... "),] +]; + +const PRUNE_HELP_PARAMS: &[ParamType] = &[ + clap::param!( + "-p, --production Also remove packages that are only needed by devDependencies (alias: --prod)" + ), + clap::param!( + "--omit ... Also remove packages that are only needed by the given dependency types" + ), + clap::param!( + "--dry-run Print what would be removed without deleting anything" + ), + clap::param!( + "--os ... Prune for a different operating system than the current one" + ), + clap::param!( + "--cpu ... Prune for a different CPU architecture than the current one" + ), + clap::param!( + "--linker Prune a node_modules installed with the given linker (one of \"isolated\" or \"hoisted\")" + ), + clap::param!("--silent Don't log anything"), + clap::param!("--cwd Set a specific cwd"), + clap::param!("-h, --help Print this help menu"), +]; + // NOTE: `string` (= `[]const u8`) fields here are slices into process argv (owned by `clap::Args` // which itself lives for the program duration). They are never freed. Mapped to `&'static [u8]` // per PORTING.md (no `deinit`, never `allocator.free`d). An explicit lifetime would only @@ -364,6 +417,7 @@ pub struct CommandLineArguments { pub(crate) backend: Option, pub analyze: bool, pub(crate) only_missing: bool, + pub(crate) add_catalog: Option<&'static [u8]>, pub positionals: &'static [&'static [u8]], pub(crate) yarn: bool, @@ -447,6 +501,7 @@ impl Default for CommandLineArguments { backend: None, analyze: false, only_missing: false, + add_catalog: None, positionals: &[], yarn: false, @@ -699,6 +754,13 @@ Full documentation is available at https://bun.com/docs/install/patchbun add --optional lodash bun add --peer esbuild + Add a dependency to a specific workspace in a monorepo + bun add zod --filter api + + Add to the workspace catalog instead of pinning a version + bun add --catalog react + bun add --catalog=testing vitest + Full documentation is available at https://bun.com/docs/cli/add. "; pretty_help(intro_text); @@ -862,6 +924,7 @@ Full documentation is available at https://bun.com/docs/cli/publish. Usage: bun audit [flags] Check installed packages for vulnerabilities. + bun audit fix upgrades vulnerable packages to the lowest safe version that still satisfies every dependent's range. Flags:"; @@ -874,6 +937,12 @@ Full documentation is available at https://bun.com/docs/cli/publish. Output package vulnerabilities in JSON format. bun audit --json + Upgrade vulnerable packages in place (bun.lock and node_modules; package.json is not modified). + bun audit fix + + Show what bun audit fix would change without changing anything. + bun audit fix --dry-run + Full documentation is available at https://bun.com/docs/install/audit. "; @@ -933,6 +1002,62 @@ Full documentation is available at https://bun.com/docs/cli/why. pretty_help(outro_text); Output::flush(); } + Subcommand::Dedupe => { + let intro_text = r" +Usage: bun dedupe [flags] + + Remove duplicate versions from bun.lock by re-resolving dependency ranges onto versions that are already in the lockfile, then install. + +Flags:"; + + let outro_text = r" + +Examples: + Remove duplicate versions and install + bun dedupe + + Only report removable duplicates; exit code 1 if there are any (for CI) + bun dedupe --check + + Rewrite bun.lock without installing + bun dedupe --lockfile-only + +Full documentation is available at https://bun.com/docs/pm/cli/dedupe. +"; + + pretty_help(intro_text); + clap::simple_help(DEDUPE_PARAMS); + pretty_help(outro_text); + Output::flush(); + } + Subcommand::Prune => { + let intro_text = r" +Usage: bun prune [flags] + + Remove packages from node_modules that are not in bun.lock. With --production, also remove packages that are only needed by devDependencies. + +Flags:"; + + let outro_text = r" + +Examples: + Remove packages that are not in bun.lock from node_modules + bun prune + + Also remove devDependencies, e.g. after the build step in a Dockerfile + bun prune --production + + Show what would be removed without deleting anything + bun prune --dry-run + +Full documentation is available at https://bun.com/docs/pm/cli/prune. +"; + + pretty_help(intro_text); + clap::simple_help(PRUNE_HELP_PARAMS); + pretty_help(outro_text); + Output::flush(); + } } } @@ -953,6 +1078,8 @@ Full documentation is available at https://bun.com/docs/cli/why. Subcommand::Pack => PACK_PARAMS, Subcommand::Publish => PUBLISH_PARAMS, Subcommand::Why => WHY_PARAMS, + Subcommand::Dedupe => DEDUPE_PARAMS, + Subcommand::Prune => PRUNE_PARAMS, // TODO: we will probably want to do this for other *_params. this way extra params // are not included in the help text @@ -1121,6 +1248,10 @@ Full documentation is available at https://bun.com/docs/cli/why. // cli.json_output = args.flag(b"--json"); } + if subcommand == Subcommand::Dedupe && args.flag(b"--check") { + cli.dry_run = true; + } + if matches!( subcommand, Subcommand::Pack | Subcommand::Pm | Subcommand::Publish @@ -1289,6 +1420,9 @@ Full documentation is available at https://bun.com/docs/cli/why. cli.exact = args.flag(b"--exact"); cli.analyze = args.flag(b"--analyze"); cli.only_missing = args.flag(b"--only-missing"); + cli.add_catalog = args + .option(b"--catalog") + .map(|name| strings::trim(name, &strings::WHITESPACE_CHARS)); } if let Some(concurrency) = args.option(b"--concurrent-scripts") { @@ -1409,6 +1543,19 @@ Full documentation is available at https://bun.com/docs/cli/why. Global::crash(); } + if cli.add_catalog.is_some() && cli.global { + Output::err_generic("--catalog cannot be used with --global\n", ()); + Global::crash(); + } + + if cli.add_catalog.is_some() + && subcommand == Subcommand::Install + && cli.positionals.len() < 2 + { + Output::err_generic("--catalog requires at least one package to add\n", ()); + Global::crash(); + } + if subcommand == Subcommand::Pm { // `bun pm version` command options if let Some(git_tag_version) = args.option(b"--git-tag-version") { diff --git a/src/install/PackageManager/PackageJSONEditor.rs b/src/install/PackageManager/PackageJSONEditor.rs index 23322d4adf0c..e55f006f5297 100644 --- a/src/install/PackageManager/PackageJSONEditor.rs +++ b/src/install/PackageManager/PackageJSONEditor.rs @@ -7,9 +7,11 @@ use bun_core::strings; use bun_semver as semver; use bun_install::dependency::{self, TagExt as _}; +use bun_install::lockfile::CatalogMap; use bun_install::lockfile::package::PackageColumns as _; -use bun_install::{Dependency, INVALID_PACKAGE_ID, resolution}; +use bun_install::{Dependency, INVALID_PACKAGE_ID, Lockfile, resolution}; use bun_install_types::{DependencyGroup, PackageNameHash}; +use bun_semver::ExternalString; use super::package_manager_options::{Do, Enable}; use super::{CatalogUpdateInfo, PackageManager, PackageUpdateInfo, Subcommand, UpdateRequest}; @@ -647,6 +649,7 @@ pub(crate) fn edit_catalogs_before_update( dep_name: Box::from(key_str), original_version_literal: Box::from(version_literal), is_alias: alias_at_index.is_some(), + new_version_literal: None, }); if update_to_latest { @@ -676,12 +679,10 @@ pub(crate) fn edit_catalogs_before_update( Ok(!manager.updating_catalogs.is_empty()) } -/// Writes resolved versions back into the recorded catalog entries, preserving -/// the original pin style. Unresolved entries are restored. Returns `changed`. +/// Writes the literals `resolve_catalog_updates` recorded into the root's catalog entries (unresolved ones are restored); returns `changed`. pub(crate) fn edit_catalogs_after_update( manager: &mut PackageManager, root_package_json: &Expr, - options: EditOptions, ) -> Result { // see note in `edit_update_no_args` — always avoid the store let _guard = ExprDisabler::scope(); @@ -692,11 +693,125 @@ pub(crate) fn edit_catalogs_after_update( } let arena = &manager.ast_arena; - let lockfile = &*manager.lockfile; + let mut changed = false; + for_each_catalog_object(root_package_json, |catalog_name, mut catalog_expr| { + if !matches!(catalog_expr.data, bun_ast::ExprData::EObject(_)) { + return Ok(()); + } + for dep in catalog_expr + .data + .e_object_mut() + .expect("infallible: variant checked") + .properties + .slice_mut() + { + let Some(key) = &dep.key else { continue }; + if !matches!(key.data, bun_ast::ExprData::EString(_)) { + continue; + } + let key_str = key + .as_utf8_string_literal() + .unwrap_or_else(|| bun_core::out_of_memory()); + + let Some(info) = infos.iter().find(|info| { + strings::eql_long(&info.dep_name, key_str, true) + && strings::eql_long(&info.catalog_name, catalog_name, true) + }) else { + continue; + }; + + let new_literal: &[u8] = arena_str( + arena, + info.new_version_literal + .as_deref() + .unwrap_or(&info.original_version_literal), + ); + + changed |= !strings::eql_long(new_literal, &info.original_version_literal, true); + + dep.value = Some(Expr::allocate( + arena, + E::EString::init(new_literal), + bun_ast::Loc::EMPTY, + )); + } + Ok(()) + })?; + + Ok(changed) +} + +/// Called from `Lockfile::clean_with_logger` on the cleaned lockfile: records each entry's new literal and writes it into `lockfile.catalogs` so bun.lock matches package.json. +pub(crate) fn resolve_catalog_updates( + lockfile: &mut Lockfile, + manager: &mut PackageManager, + exact_versions: bool, +) -> crate::Result<()> { + let mut infos = core::mem::take(&mut manager.updating_catalogs); + let update_to_latest = manager.options.do_.contains(Do::UPDATE_TO_LATEST); + resolve_catalog_literals(lockfile, &mut infos, update_to_latest, exact_versions); + let result = rewrite_lockfile_catalogs(lockfile, manager, &infos); + manager.updating_catalogs = infos; + result +} + +fn rewrite_lockfile_catalogs( + lockfile: &mut Lockfile, + manager: &mut PackageManager, + infos: &[CatalogUpdateInfo], +) -> crate::Result<()> { + let literals = || { + infos + .iter() + .filter_map(|info| info.new_version_literal.as_deref()) + }; + if literals().next().is_none() { + return Ok(()); + } + + let (mut builder, lf) = lockfile.string_builder_split(); + for literal in literals() { + builder.count(literal); + } + builder.allocate()?; + for info in infos { + let Some(literal) = info.new_version_literal.as_deref() else { + continue; + }; + let external = builder.append::(literal); + let string_bytes = builder.string_bytes.as_slice(); + let sliced = external.value.sliced(string_bytes); + let Some(entry) = lf + .catalogs + .find_mut(string_bytes, &info.catalog_name, &info.dep_name) + else { + continue; + }; + let (entry_name, entry_name_hash) = (entry.name, entry.name_hash); + if let Some(version) = dependency::parse( + entry_name, + entry_name_hash, + sliced.slice, + &sliced, + None, + &mut *manager, + ) { + entry.version = version; + } + } + builder.clamp(); + Ok(()) +} + +fn resolve_catalog_literals( + lockfile: &Lockfile, + infos: &mut [CatalogUpdateInfo], + update_to_latest: bool, + exact_versions: bool, +) { let string_buf = lockfile.buffers.string_bytes.as_slice(); let package_resolutions = lockfile.packages.items_resolution(); - let mut new_literals: Vec>> = vec![None; infos.len()]; debug_assert_eq!( lockfile.buffers.dependencies.len(), lockfile.buffers.resolutions.len() @@ -716,13 +831,16 @@ pub(crate) fn edit_catalogs_after_update( let dep_name = dep.name.slice(string_buf); let catalog_name = dep.version.catalog().slice(string_buf); - let Some(index) = infos.iter().position(|info| { - strings::eql_long(&info.dep_name, dep_name, true) - && strings::eql_long(&info.catalog_name, catalog_name, true) - }) else { + let find = |same_catalog: fn(&[u8], &[u8]) -> bool| { + infos.iter().position(|info| { + strings::eql_long(&info.dep_name, dep_name, true) + && same_catalog(&info.catalog_name, catalog_name) + }) + }; + let Some(index) = find(|a, b| a == b).or_else(|| find(CatalogMap::same_name)) else { continue; }; - if new_literals[index].is_some() { + if infos[index].new_version_literal.is_some() { continue; } @@ -731,7 +849,7 @@ pub(crate) fn edit_catalogs_after_update( continue; } - if !manager.options.do_.contains(Do::UPDATE_TO_LATEST) { + if !update_to_latest { // plain `bun update` does not move an exact pin (matches direct-dep behavior) let resolved_version = lockfile .resolve_catalog_dependency(dep) @@ -746,7 +864,7 @@ pub(crate) fn edit_catalogs_after_update( let info = &infos[index]; let version_fmt = resolution.npm().version.fmt(string_buf); let new_version: Vec = 'new_version: { - if options.exact_versions { + if exact_versions { let mut v = Vec::new(); write!(&mut v, "{}", version_fmt).expect("infallible: in-memory write"); break 'new_version v; @@ -780,7 +898,7 @@ pub(crate) fn edit_catalogs_after_update( v }; - new_literals[index] = Some(if info.is_alias { + let new_literal = if info.is_alias { let dep_literal = &info.original_version_literal; if let Some(at_index) = strings::last_index_of_char(dep_literal, b'@') { let mut v = Vec::new(); @@ -797,55 +915,9 @@ pub(crate) fn edit_catalogs_after_update( } } else { new_version - }); + }; + infos[index].new_version_literal = Some(new_literal.into_boxed_slice()); } - - let mut changed = false; - for_each_catalog_object(root_package_json, |catalog_name, mut catalog_expr| { - if !matches!(catalog_expr.data, bun_ast::ExprData::EObject(_)) { - return Ok(()); - } - for dep in catalog_expr - .data - .e_object_mut() - .expect("infallible: variant checked") - .properties - .slice_mut() - { - let Some(key) = &dep.key else { continue }; - if !matches!(key.data, bun_ast::ExprData::EString(_)) { - continue; - } - let key_str = key - .as_utf8_string_literal() - .unwrap_or_else(|| bun_core::out_of_memory()); - - let Some(index) = infos.iter().position(|info| { - strings::eql_long(&info.dep_name, key_str, true) - && strings::eql_long(&info.catalog_name, catalog_name, true) - }) else { - continue; - }; - - let info = &infos[index]; - let new_literal: &[u8] = match &new_literals[index] { - Some(v) => arena_str(arena, v), - // unresolved: restore the original (may still be the temporary `latest`) - None => arena_dup(arena, &info.original_version_literal), - }; - - changed |= !strings::eql_long(new_literal, &info.original_version_literal, true); - - dep.value = Some(Expr::allocate( - arena, - E::EString::init(new_literal), - bun_ast::Loc::EMPTY, - )); - } - Ok(()) - })?; - - Ok(changed) } /// edits dependencies and trusted dependencies diff --git a/src/install/PackageManager/PackageManagerOptions.rs b/src/install/PackageManager/PackageManagerOptions.rs index 800be1444fb2..e40e7a2d5824 100644 --- a/src/install/PackageManager/PackageManagerOptions.rs +++ b/src/install/PackageManager/PackageManagerOptions.rs @@ -38,6 +38,7 @@ pub struct Options { pub(crate) patch_features: PatchFeatures, pub filter_patterns: &'static [&'static [u8]], + pub add_catalog: Option<&'static [u8]>, pub pack_destination: &'static [u8], pub pack_filename: &'static [u8], pub pack_gzip_level: Option<&'static [u8]>, @@ -89,9 +90,9 @@ pub struct Options { pub minimum_release_age_excludes: Option<&'static [&'static [u8]]>, /// Override CPU architecture for optional dependencies filtering - pub(crate) cpu: Npm::Architecture, + pub cpu: Npm::Architecture, /// Override OS for optional dependencies filtering - pub(crate) os: Npm::OperatingSystem, + pub os: Npm::OperatingSystem, pub(crate) config_version: Option, } @@ -127,6 +128,7 @@ impl Default for Options { }, patch_features: PatchFeatures::Nothing, filter_patterns: &[], + add_catalog: None, pack_destination: b"", pack_filename: b"", pack_gzip_level: None, @@ -731,6 +733,7 @@ impl Options { } self.filter_patterns = cli.filters; + self.add_catalog = cli.add_catalog; self.pack_destination = cli.pack_destination; self.pack_filename = cli.pack_filename; self.pack_gzip_level = cli.pack_gzip_level; diff --git a/src/install/PackageManager/PopulateManifestCache.rs b/src/install/PackageManager/PopulateManifestCache.rs index 79993c5fa0d0..2ca9e5bf423f 100644 --- a/src/install/PackageManager/PopulateManifestCache.rs +++ b/src/install/PackageManager/PopulateManifestCache.rs @@ -101,6 +101,8 @@ fn start_manifest_task( pub enum Packages<'a> { All, Ids(&'a [PackageID]), + /// The manifests of these packages themselves (by name), not of their dependencies. + Exact(&'a [PackageID]), } /// `RunTasksCallbacks` impl for the void-callback `runTasks` call in @@ -269,6 +271,39 @@ pub fn populate_manifest_cache( } } } + Packages::Exact(ids) => { + let placeholder = Dependency::default(); + for &pkg_id in ids { + if pkg_resolutions[pkg_id as usize].tag != ResolutionTag::Npm { + continue; + } + let package_name = pkg_names[pkg_id as usize].slice(string_buf); + let needs_extended_manifest = mgr_ref.options.minimum_release_age_ms.is_some(); + let scope = + bun_ptr::BackRef::new(mgr_ref.options.scope_for_package_name(package_name)); + // SAFETY: `manifests` is disjoint from `options`/`lockfile`; `manager_ptr` is the SRW root. + let cached = unsafe { &mut (*manager_ptr).manifests }.by_name( + cache_ctx, + scope.get(), + package_name, + ManifestLoad::LoadFromMemoryFallbackToDisk, + needs_extended_manifest, + ); + if cached.is_none() { + start_manifest_task( + // SAFETY: SRW root; `start_manifest_task` never mutates `lockfile`, so `package_name` stays valid. + unsafe { &mut *manager_ptr }, + package_name, + &placeholder, + needs_extended_manifest, + )?; + // SAFETY: SRW root; network-queue flush does not mutate `lockfile`. + run_tasks::flush_network_queue(unsafe { &mut *manager_ptr }); + // SAFETY: SRW root; task scheduler does not mutate `lockfile`. + let _ = run_tasks::schedule_tasks(unsafe { &mut *manager_ptr }); + } + } + } } // SAFETY: provenance root; no live shared borrows of `*manager_ptr` remain. diff --git a/src/install/PackageManager/add_catalog.rs b/src/install/PackageManager/add_catalog.rs new file mode 100644 index 000000000000..621f29a95be6 --- /dev/null +++ b/src/install/PackageManager/add_catalog.rs @@ -0,0 +1,381 @@ +use std::io::Write as _; + +use bstr::BStr; +use bun_alloc::AllocError; +use bun_ast::{E, Expr, ExprData, Loc, StoreStr}; +use bun_collections::VecExt as _; +use bun_core::{Global, Output, ZStr}; +use bun_semver::ExternalString; +use bun_sys::{Fd, File}; + +use bun_install::dependency; +use bun_install::lockfile::CatalogMap; +use bun_install::lockfile::package::PackageColumns as _; +use bun_install::{INVALID_PACKAGE_ID, Lockfile, resolution}; + +use super::update_package_json_and_install::print_package_json_into_cache_entry; +use super::workspace_package_json_cache::{GetJSONOptions, GetResult, MapEntry}; +use super::{PackageManager, UpdateRequest}; + +type ExprDisabler = bun_ast::expr::Disabler; + +fn catalog_name(manager: &PackageManager) -> &'static [u8] { + manager + .options + .add_catalog + .expect("add_catalog callers are gated on is_some") +} + +fn estring(arena: &bun_alloc::Arena, bytes: &[u8]) -> Expr { + Expr::allocate( + arena, + E::EString::init(arena.alloc_slice_copy(bytes)), + Loc::EMPTY, + ) +} + +fn reference_literal<'a>(arena: &'a bun_alloc::Arena, name: &[u8]) -> &'a [u8] { + if name.is_empty() { + return b"catalog:"; + } + let mut literal = Vec::with_capacity(b"catalog:".len() + name.len()); + literal.extend_from_slice(b"catalog:"); + literal.extend_from_slice(name); + arena.alloc_slice_copy(&literal) +} + +fn seed_literal(request: &UpdateRequest) -> &[u8] { + if request.version.tag == dependency::Tag::Uninitialized { + return b"latest"; + } + request.version.literal.slice(request.version_buf()) +} + +fn set_property(mut object: Expr, arena: &bun_alloc::Arena, key: &[u8], value: Expr) { + let obj = object + .data + .e_object_mut() + .expect("infallible: caller checked object"); + match obj.as_property(key) { + Some(q) => obj.properties.slice_mut()[q.i as usize].value = Some(value), + None => obj.append_property(estring(arena, key), value), + } +} + +fn object_property( + mut container: Expr, + key: &[u8], + create: Option<&bun_alloc::Arena>, +) -> Option { + let obj = container.data.e_object_mut()?; + let existing = obj.as_property(key); + if let Some(q) = &existing { + if matches!(q.expr.data, ExprData::EObject(_)) { + return Some(q.expr); + } + } + let arena = create?; + let created = Expr::allocate(arena, E::Object::default(), Loc::EMPTY); + match existing { + Some(q) => obj.properties.slice_mut()[q.i as usize].value = Some(created), + None => obj.append_property(estring(arena, key), created), + } + Some(created) +} + +fn entries_object(root: &Expr, name: &[u8], create: Option<&bun_alloc::Arena>) -> Option { + let Some(workspaces) = root.get(b"workspaces") else { + if create.is_some() { + Output::err_generic( + "--catalog requires a \"workspaces\" field in the root package.json", + (), + ); + Global::crash(); + } + return None; + }; + + let has_catalogs = + |expr: &Expr| expr.get(b"catalog").is_some() || expr.get(b"catalogs").is_some(); + let workspaces_is_object = matches!(workspaces.data, ExprData::EObject(_)); + let container = if workspaces_is_object && has_catalogs(&workspaces) { + workspaces + } else if has_catalogs(root) || !workspaces_is_object { + *root + } else { + workspaces + }; + + if !CatalogMap::same_name(name, b"") { + let catalogs = object_property(container, b"catalogs", create)?; + return object_property(catalogs, name, create); + } + let singular = || object_property(container, b"catalog", None); + let in_catalogs = || { + object_property( + object_property(container, b"catalogs", None)?, + b"default", + None, + ) + }; + let existing = if name.is_empty() { + singular().or_else(in_catalogs) + } else { + in_catalogs().or_else(singular) + }; + existing.or_else(|| object_property(container, b"catalog", create)) +} + +/// Call right after `PackageJSONEditor::edit` on the same AST: rewrites only the slot `edit` bound per request. +pub(crate) fn rewrite_references(manager: &PackageManager, updates: &[UpdateRequest]) { + if updates.is_empty() { + return; + } + + for request in updates { + if !request.is_aliased { + Output::err_generic( + "--catalog can only add packages by name, but got \"{s}\"", + (BStr::new( + request.version.literal.slice(request.version_buf()), + ),), + ); + Global::crash(); + } + if request.version.tag == dependency::Tag::Workspace { + Output::err_generic( + "--catalog cannot add a workspace package, but got \"{s}@{s}\"", + ( + BStr::new(request.name), + BStr::new(request.version.literal.slice(request.version_buf())), + ), + ); + Global::crash(); + } + } + + let literal = StoreStr::new(reference_literal(&manager.ast_arena, catalog_name(manager))); + for request in updates { + let Some(e_string) = request.e_string else { + continue; + }; + // SAFETY: same slot `edit` just wrote through (PackageJSONEditor.rs `request.e_string` loop); the tree it points into is still live and no other borrow of it exists here. + unsafe { (*e_string).data = literal }; + } +} + +pub(crate) fn edit_root_before_install( + manager: &PackageManager, + root_package_json: &Expr, + updates: &[UpdateRequest], +) -> Result<(), AllocError> { + if updates.is_empty() { + return Ok(()); + } + let _guard = ExprDisabler::scope(); + + let arena = &manager.ast_arena; + let mut entries = entries_object(root_package_json, catalog_name(manager), Some(arena)) + .expect("infallible: created on demand"); + for request in updates { + set_property( + entries, + arena, + request.name, + estring(arena, seed_literal(request)), + ); + } + let obj = entries + .data + .e_object_mut() + .expect("infallible: entries_object returns objects"); + if obj.properties.len_u32() > 1 { + obj.alphabetize_properties(); + } + Ok(()) +} + +pub(crate) fn edit_root_entry_before_install( + manager: &mut PackageManager, + root_package_json: &mut MapEntry, +) -> Result<(), crate::Error> { + if manager.update_requests.is_empty() { + return Ok(()); + } + let root = root_package_json.root; + edit_root_before_install(&*manager, &root, &manager.update_requests)?; + print_package_json_into_cache_entry(root_package_json, root); + if let Err(err) = root_package_json.reparse_root(manager.log_mut()) { + bun_core::pretty_errorln!("package.json failed to parse due to error {}", err.name()); + Global::crash(); + } + Ok(()) +} + +pub(crate) fn edit_root_after_install( + manager: &PackageManager, + root_package_json: &Expr, + updates: &[UpdateRequest], +) -> Result { + let _guard = ExprDisabler::scope(); + let name = catalog_name(manager); + let Some(mut entries) = entries_object(root_package_json, name, None) else { + return Ok(false); + }; + + let arena = &manager.ast_arena; + let lockfile: &Lockfile = &manager.lockfile; + let string_bytes = lockfile.buffers.string_bytes.as_slice(); + let mut changed = false; + for request in updates { + let Some(dep) = lockfile.catalogs.find(string_bytes, name, request.name) else { + continue; + }; + let new_literal = dep.version.literal.slice(string_bytes); + let obj = entries + .data + .e_object_mut() + .expect("infallible: entries_object returns objects"); + let Some(q) = obj.as_property(request.name) else { + continue; + }; + if q.expr.as_utf8_string_literal() == Some(new_literal) { + continue; + } + obj.properties.slice_mut()[q.i as usize].value = Some(estring(arena, new_literal)); + changed = true; + } + Ok(changed) +} + +pub(crate) fn write_root_after_install( + manager: &mut PackageManager, + root_package_json_path: &ZStr, + updates: &[UpdateRequest], +) -> Result<(), crate::Error> { + if updates.is_empty() { + return Ok(()); + } + let entry_ptr: *mut MapEntry = match manager.workspace_package_json_cache.get_with_path( + manager.log_mut(), + root_package_json_path.as_bytes(), + GetJSONOptions { + guess_indentation: true, + ..Default::default() + }, + ) { + GetResult::ParseErr(err) => { + let _ = manager + .log_mut() + .print(std::ptr::from_mut(Output::error_writer())); + Output::err_generic( + "failed to parse package.json \"{s}\": {s}", + (BStr::new(root_package_json_path.as_bytes()), err.name()), + ); + Global::crash(); + } + GetResult::ReadErr(err) => { + Output::err_generic( + "failed to read package.json \"{s}\": {s}", + (BStr::new(root_package_json_path.as_bytes()), err.name()), + ); + Global::crash(); + } + GetResult::Entry(entry) => core::ptr::from_mut(entry), + }; + // SAFETY: the cache is not touched again while `entry` is live; `edit_root_after_install` only reads disjoint manager fields. + let entry: &mut MapEntry = unsafe { &mut *entry_ptr }; + + let root = entry.root; + if edit_root_after_install(&*manager, &root, updates)? { + print_package_json_into_cache_entry(entry, root); + } + + let file = File::openat(Fd::cwd(), root_package_json_path, bun_sys::O::RDWR, 0) + .map_err(crate::Error::from)?; + file.pwrite_all(&entry.source.contents, 0) + .map_err(crate::Error::from)?; + let _ = bun_sys::ftruncate(file.handle, entry.source.contents.len() as i64); + let _ = file.close(); + Ok(()) +} + +pub(crate) fn rewrite_lockfile_entries( + lockfile: &mut Lockfile, + manager: &mut PackageManager, + updates: &[UpdateRequest], +) -> crate::Result<()> { + let name = catalog_name(manager); + let exact = manager.options.enable.exact_versions(); + + let mut rewrites: Vec<(&[u8], Vec)> = Vec::new(); + { + let buf = lockfile.buffers.string_bytes.as_slice(); + let resolutions = lockfile.packages.items_resolution(); + for request in updates { + if request.version.tag != dependency::Tag::DistTag { + continue; + } + let found = lockfile + .buffers + .dependencies + .iter() + .zip(lockfile.buffers.resolutions.iter()) + .find(|&(dep, &pkg_id)| { + dep.version.tag == dependency::Tag::Catalog + && dep.name_hash == request.name_hash + && CatalogMap::same_name(dep.version.catalog().slice(buf), name) + && pkg_id != INVALID_PACKAGE_ID + && (pkg_id as usize) < resolutions.len() + && resolutions[pkg_id as usize].tag == resolution::Tag::Npm + }); + let Some((_, &pkg_id)) = found else { + continue; + }; + let version = resolutions[pkg_id as usize].npm().version.fmt(buf); + let request_literal = request.version.literal.slice(request.version_buf()); + let mut literal = Vec::new(); + if request_literal.starts_with(b"npm:") { + write!( + &mut literal, + "npm:{}@", + BStr::new(request.version.dist_tag().name.slice(request.version_buf())) + ) + .expect("infallible: in-memory write"); + } + write!(&mut literal, "{}{}", if exact { "" } else { "^" }, version) + .expect("infallible: in-memory write"); + rewrites.push((request.name, literal)); + } + } + if rewrites.is_empty() { + return Ok(()); + } + + let (mut builder, lf) = lockfile.string_builder_split(); + for (_, literal) in &rewrites { + builder.count(literal); + } + builder.allocate()?; + for (dep_name, literal) in &rewrites { + let external = builder.append::(literal); + let string_bytes = builder.string_bytes.as_slice(); + let sliced = external.value.sliced(string_bytes); + let Some(entry) = lf.catalogs.find_mut(string_bytes, name, dep_name) else { + continue; + }; + let (entry_name, entry_name_hash) = (entry.name, entry.name_hash); + if let Some(version) = dependency::parse( + entry_name, + entry_name_hash, + sliced.slice, + &sliced, + None, + &mut *manager, + ) { + entry.version = version; + } + } + builder.clamp(); + Ok(()) +} diff --git a/src/install/PackageManager/add_remove_with_filter.rs b/src/install/PackageManager/add_remove_with_filter.rs new file mode 100644 index 000000000000..9e55dbb74e0d --- /dev/null +++ b/src/install/PackageManager/add_remove_with_filter.rs @@ -0,0 +1,659 @@ +use bstr::BStr; + +use crate::Error; +use crate::bun_fs::FileSystem; +use crate::lockfile_real::package::value_loc_of; +use crate::lockfile_real::package::workspace_map::{NamesArray, WorkspaceMap}; +use bun_core::{Global, Output, strings}; +use bun_install::dependency; +use bun_install::{Lockfile, PackageID, PackageNameHash}; +use bun_paths::path_buffer_pool; +use bun_paths::resolve_path::{self, Platform, join_abs_string_buf, platform}; +use bun_sys::{Fd, File}; + +use super::add_catalog; +use super::install_with_manager::install_with_manager; +use super::options::Do; +use super::package_json_editor::{self as PackageJSONEditor, EditOptions}; +use super::update_package_json_and_install::{ + print_package_json_into_cache_entry, remove_dependencies_from_package_json, + remove_leftover_node_modules, +}; +use super::workspace_package_json_cache::{GetJSONOptions, GetResult, MapEntry}; +use super::{Command, PackageManager, Subcommand, UpdateRequest, WorkspaceFilter}; + +pub(crate) struct WorkspaceTarget { + pub(crate) name: Box<[u8]>, + /// `None` = the workspace root. + pub(crate) name_hash: Option, + pub(crate) package_json_path: Box<[u8]>, +} + +fn root_package_json_path() -> Box<[u8]> { + let top_level = strings::without_trailing_slash(FileSystem::instance().top_level_dir()); + let mut buf = path_buffer_pool::get(); + let path: Box<[u8]> = + join_abs_string_buf::(top_level, &mut buf.0, &[b"package.json"]).into(); + path +} + +fn print_log_and_crash( + manager: &PackageManager, + fmt: &str, + args: impl bun_core::output::FmtTuple, +) -> ! { + let _ = manager + .log_mut() + .print(std::ptr::from_mut(Output::error_writer())); + Output::err_generic(fmt, args); + Global::crash(); +} + +pub(crate) fn select_targets( + manager: &mut PackageManager, + original_cwd: &[u8], +) -> Result, Error> { + debug_assert!(!manager.options.filter_patterns.is_empty()); + let top_level = strings::without_trailing_slash(FileSystem::instance().top_level_dir()); + let root_path = root_package_json_path(); + + let (root_expr, root_source, root_name): (bun_ast::Expr, bun_ast::Source, Box<[u8]>) = { + let log = manager.log_mut(); + match manager.workspace_package_json_cache.get_with_path( + log, + &root_path, + GetJSONOptions { + guess_indentation: true, + ..Default::default() + }, + ) { + GetResult::Entry(entry) => { + let name_expr = entry.root.get(b"name"); + let name: Box<[u8]> = name_expr + .as_ref() + .and_then(|e| e.as_utf8_string_literal()) + .unwrap_or(b"") + .into(); + (entry.root, entry.source.clone(), name) + } + GetResult::ParseErr(err) => print_log_and_crash( + manager, + "failed to parse package.json \"{}\": {}", + (BStr::new(&root_path), err.name()), + ), + GetResult::ReadErr(err) => { + Output::err_generic( + "failed to read package.json \"{}\": {}", + (BStr::new(&root_path), err.name()), + ); + Global::crash(); + } + } + }; + + let mut members = WorkspaceMap::init(); + let workspaces = root_expr.as_property(b"workspaces"); + let packages = workspaces + .as_ref() + .filter(|q| !q.expr.is_array()) + .and_then(|q| q.expr.as_property(b"packages")); + let names: Option<(NamesArray<'_>, bun_ast::Loc)> = match (&workspaces, &packages) { + (Some(q), _) if q.expr.is_array() => Some(( + NamesArray::from_expr(&q.expr, value_loc_of(&root_source, q.loc)) + .expect("is_array was checked above"), + q.loc, + )), + (Some(_), Some(p)) if p.expr.is_array() => Some(( + NamesArray::from_expr(&p.expr, value_loc_of(&root_source, p.loc)) + .expect("is_array was checked above"), + p.loc, + )), + _ => None, + }; + if let Some((arr, loc)) = names { + let log = manager.log_mut(); + if let Err(err) = members.process_names_array( + &mut manager.workspace_package_json_cache, + log, + arr, + &root_source, + loc, + None, + false, + ) { + if log.has_errors() { + let _ = log.print(std::ptr::from_mut(Output::error_writer())); + } else { + Output::err_generic( + "failed to load workspaces from package.json \"{}\": {}", + (BStr::new(&root_path), err.name()), + ); + } + Global::crash(); + } + } + + let mut path_buf = path_buffer_pool::get(); + let patterns = manager.options.filter_patterns; + let filters: Vec = patterns + .iter() + .map(|pattern| { + bun_core::handle_oom(WorkspaceFilter::init( + pattern, + original_cwd, + &mut path_buf.0, + )) + }) + .collect(); + + let root_subject: Box<[u8]> = + strings::without_trailing_slash(join_abs_string_buf::( + top_level, + &mut path_buf.0, + &[b"."], + )) + .into(); + let mut candidates: Vec<(WorkspaceTarget, Box<[u8]>)> = Vec::with_capacity(members.count() + 1); + candidates.push(( + WorkspaceTarget { + name: root_name, + name_hash: None, + package_json_path: root_path, + }, + root_subject, + )); + + let mut package_json_buf = path_buffer_pool::get(); + for (rel, entry) in members.keys().iter().zip(members.values()) { + let rel: &[u8] = rel; + let subject: Box<[u8]> = + strings::without_trailing_slash(join_abs_string_buf::( + top_level, + &mut path_buf.0, + &[rel], + )) + .into(); + candidates.push(( + WorkspaceTarget { + name: entry.name.clone(), + name_hash: Some(bun_semver::string::Builder::string_hash(&entry.name)), + package_json_path: join_abs_string_buf::( + top_level, + &mut package_json_buf.0, + &[rel, b"package.json"], + ) + .into(), + }, + subject, + )); + } + + let unmatched: Vec<&[u8]> = filters + .iter() + .zip(patterns) + .filter(|(filter, _)| { + let negated = match filter { + WorkspaceFilter::All => return false, + WorkspaceFilter::Name(p) | WorkspaceFilter::Path(p) => p.first() == Some(&b'!'), + }; + !negated + && !candidates.iter().any(|(target, subject)| { + WorkspaceFilter::matches_any( + core::slice::from_ref(filter), + &target.name, + subject, + ) + }) + }) + .map(|(_, pattern)| *pattern) + .collect(); + + let targets: Vec = candidates + .into_iter() + .filter(|(target, subject)| WorkspaceFilter::matches_any(&filters, &target.name, subject)) + .map(|(target, _)| target) + .collect(); + + if targets.is_empty() { + Output::err_generic( + "No workspace packages matched the filter {}", + (BStr::new("e_patterns(patterns)),), + ); + Global::crash(); + } + if !unmatched.is_empty() { + bun_core::pretty_errorln!( + "warn: No workspace packages matched the filter {}", + BStr::new("e_patterns(&unmatched)), + ); + } + + Ok(targets) +} + +fn quote_patterns(patterns: &[&[u8]]) -> Vec { + let mut out = Vec::new(); + for (i, pattern) in patterns.iter().enumerate() { + if i > 0 { + out.extend_from_slice(b", "); + } + out.push(b'"'); + out.extend_from_slice(pattern); + out.push(b'"'); + } + out +} + +fn fetch_entry<'a>(manager: &'a mut PackageManager, target: &WorkspaceTarget) -> &'a mut MapEntry { + let log = manager.log_mut(); + match manager.workspace_package_json_cache.get_with_path( + log, + &target.package_json_path, + GetJSONOptions { + init_reset_store: false, + guess_indentation: true, + }, + ) { + GetResult::Entry(entry) => entry, + GetResult::ParseErr(err) | GetResult::ReadErr(err) => { + Output::err_generic( + "failed to read/parse package.json for workspace '{}': {}", + (BStr::new(&target.name), err.name()), + ); + Global::crash(); + } + } +} + +fn fetch_entry_root(manager: &mut PackageManager, target: &WorkspaceTarget) -> bun_ast::Expr { + fetch_entry(manager, target).root +} + +fn store_entry( + manager: &mut PackageManager, + target: &WorkspaceTarget, + root: bun_ast::Expr, + reparse: bool, +) { + let log = manager.log_mut(); + let entry = fetch_entry(manager, target); + print_package_json_into_cache_entry(entry, root); + if reparse { + if let Err(err) = entry.reparse_root(log) { + bun_core::pretty_errorln!("package.json failed to parse due to error {}", err.name()); + Global::crash(); + } + } +} + +fn write_target(manager: &mut PackageManager, target: &WorkspaceTarget) -> bool { + let entry = fetch_entry(manager, target); + let mut zbuf = path_buffer_pool::get(); + let path = resolve_path::z(&target.package_json_path, &mut zbuf); + match File::write_file(Fd::cwd(), path, &entry.source.contents) { + Ok(()) => true, + Err(err) => { + Output::err_generic( + "failed to write package.json for workspace '{}': {}", + (BStr::new(&target.name), BStr::new(err.name())), + ); + false + } + } +} + +fn reset_e_strings(updates: &mut [UpdateRequest]) { + // `e_string` points into the previous target's AST; `edit` skips requests that already have one. + for request in updates.iter_mut() { + request.e_string = None; + } +} + +/// Moves the requests in `wanted` to the front, preserving order; returns how many there are. +fn move_to_front(updates: &mut [UpdateRequest], wanted: &[PackageNameHash]) -> usize { + updates.sort_by_key(|request| !wanted.contains(&request.name_hash)); + updates + .iter() + .take_while(|request| wanted.contains(&request.name_hash)) + .count() +} + +/// The `(prefix, path)` of a positional naming a local path, which is relative to the invoking cwd. +fn local_relative_path(request: &UpdateRequest) -> Option<(&'static [u8], &[u8])> { + let literal = request.version.literal.slice(request.version_buf()); + let (prefix, path): (&'static [u8], &[u8]) = match request.version.tag { + dependency::Tag::Folder | dependency::Tag::Tarball => { + match literal.strip_prefix(b"file:") { + Some(path) => (b"file:", path), + None => (b"", literal), + } + } + dependency::Tag::Symlink => (b"link:", literal.strip_prefix(b"link:")?), + _ => return None, + }; + let is_path = path.starts_with(b".") + || (prefix != b"link:" && !path.is_empty() && !strings::contains(path, b"://")); + (is_path && !path.starts_with(b"//") && !Platform::AUTO.is_absolute(path)) + .then_some((prefix, path)) +} + +fn spell_relative_to( + target: &WorkspaceTarget, + request: &UpdateRequest, + prefix: &[u8], + abs: &[u8], +) -> Vec { + let mut buf = path_buffer_pool::get(); + let target_dir = resolve_path::dirname::(&target.package_json_path); + let rel = + resolve_path::relative_platform_buf::(&mut buf.0, target_dir, abs); + let mut positional = Vec::with_capacity(request.name.len() + prefix.len() + rel.len() + 3); + if request.is_aliased { + positional.extend_from_slice(request.name); + positional.push(b'@'); + } + positional.extend_from_slice(prefix); + let path_start = positional.len(); + let escapes = + rel.starts_with(b"..") && rel.get(2).is_none_or(|&c| Platform::AUTO.is_separator(c)); + if !escapes { + positional.extend_from_slice(b"./"); + } + positional.extend_from_slice(rel); + resolve_path::platform_to_posix_in_place(&mut positional[path_start..]); + positional +} + +/// The requests to install and, per target, the ones it receives; a local path becomes one request per distinct spelling. +fn assign_requests( + manager: &mut PackageManager, + original_cwd: &[u8], + updates: Vec, + targets: &[WorkspaceTarget], +) -> (Vec, Vec>) { + enum Slot { + Shared(PackageNameHash), + PerTarget(Vec>), + } + let mut buf = path_buffer_pool::get(); + let mut requests: Vec = Vec::with_capacity(updates.len()); + let mut slots: Vec = Vec::with_capacity(updates.len()); + for request in updates { + let Some((prefix, path)) = local_relative_path(&request) else { + slots.push(Slot::Shared(request.name_hash)); + requests.push(request); + continue; + }; + let abs: Box<[u8]> = + join_abs_string_buf::(original_cwd, &mut buf.0, &[path]).into(); + slots.push(Slot::PerTarget( + targets + .iter() + .map(|target| spell_relative_to(target, &request, prefix, &abs)) + .collect(), + )); + } + + let positionals: Vec<&[u8]> = slots + .iter() + .filter_map(|slot| match slot { + Slot::PerTarget(spellings) => Some(spellings), + Slot::Shared(_) => None, + }) + .flatten() + .map(Vec::as_slice) + .collect(); + if !positionals.is_empty() { + let log = manager.log_mut(); + let subcommand = manager.subcommand; + UpdateRequest::parse( + Some(&mut *manager), + log, + &positionals, + &mut requests, + subcommand, + ); + } + + let assigned = (0..targets.len()) + .map(|i| { + slots + .iter() + .map(|slot| match slot { + Slot::Shared(name_hash) => *name_hash, + Slot::PerTarget(spellings) => requests + .iter() + .find(|request| request.version_buf() == spellings[i].as_slice()) + .unwrap_or_else(|| { + Output::err_generic( + "\"{}\" is spelled differently relative to each selected workspace; add it to one workspace at a time", + (BStr::new(&spellings[i]),), + ); + Global::crash(); + }) + .name_hash, + }) + .collect() + }) + .collect(); + (requests, assigned) +} + +/// The targets whose package.json changed and, for `add`, the requests each one received. +pub(crate) struct PendingWrite { + targets: Vec<(WorkspaceTarget, Box<[PackageNameHash]>)>, + subcommand: Subcommand, + catalog_mode: bool, + root_target: WorkspaceTarget, +} + +/// Runs once: from `install_with_manager` right after the lockfile is saved, else after it returns. +pub(crate) fn flush_pending_write(manager: &mut PackageManager) -> Result<(), Error> { + let Some(pending) = manager.pending_filtered_write.take() else { + return Ok(()); + }; + if !manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) { + return Ok(()); + } + let mut updates: Box<[UpdateRequest]> = core::mem::take(&mut manager.update_requests); + let result = pending.write(manager, &mut updates); + manager.update_requests = updates; + result +} + +impl PendingWrite { + /// Package ids of the targets that received `request`; `clean_with_logger` resolves it from these. + pub(crate) fn workspace_ids_receiving( + &self, + lockfile: &Lockfile, + request: PackageNameHash, + ) -> Vec { + self.targets + .iter() + .filter(|(_, received)| received.contains(&request)) + .filter_map(|(target, _)| { + let id = lockfile.get_workspace_package_id(target.name_hash); + (target.name_hash.is_none() || id != 0).then_some(id) + }) + .collect() + } + + fn write( + &self, + manager: &mut PackageManager, + updates: &mut [UpdateRequest], + ) -> Result<(), Error> { + let mut any_failed = false; + if self.subcommand == Subcommand::Remove { + for (target, _) in &self.targets { + any_failed |= !write_target(manager, target); + } + if any_failed { + Global::exit(1); + } + return Ok(()); + } + + let dependency_list: &'static [u8] = manager.options.update.prop; + let exact_versions = manager.options.enable.exact_versions(); + let add_trusted_dependencies = manager + .options + .do_ + .contains(Do::TRUST_DEPENDENCIES_FROM_ARGS); + let trusted_snapshot = manager.trusted_deps_to_add_to_package_json.clone(); + let summary_order: Vec = updates.iter().map(|r| r.name_hash).collect(); + + for (target, received) in &self.targets { + let kept = move_to_front(updates, received); + manager.trusted_deps_to_add_to_package_json = trusted_snapshot.clone(); + let mut root = fetch_entry_root(manager, target); + reset_e_strings(updates); + let mut slice: &mut [UpdateRequest] = &mut updates[..kept]; + PackageJSONEditor::edit( + manager, + &mut slice, + &mut root, + dependency_list, + EditOptions { + exact_versions, + add_trusted_dependencies, + ..Default::default() + }, + )?; + if self.catalog_mode { + add_catalog::rewrite_references(manager, &updates[..kept]); + } + let is_catalog_root = self.catalog_mode && target.name_hash.is_none(); + store_entry(manager, target, root, is_catalog_root); + if !is_catalog_root { + any_failed |= !write_target(manager, target); + } + } + updates.sort_by_key(|r| summary_order.iter().position(|&h| h == r.name_hash)); + if any_failed { + Global::exit(1); + } + if self.catalog_mode { + let mut zbuf = path_buffer_pool::get(); + let root_package_json_path = + resolve_path::z(&self.root_target.package_json_path, &mut zbuf); + add_catalog::write_root_after_install(manager, root_package_json_path, updates)?; + } + Ok(()) + } +} + +pub(super) fn update_filtered_workspaces_and_install( + manager: &mut PackageManager, + ctx: Command::Context, + original_cwd: &[u8], + updates: Vec, +) -> Result<(), Error> { + if manager.options.global { + Output::err_generic("--filter cannot be used with --global", ()); + Global::crash(); + } + + let targets = select_targets(manager, original_cwd)?; + let subcommand = manager.subcommand; + let dependency_list: &'static [u8] = manager.options.update.prop; + let exact_versions = manager.options.enable.exact_versions(); + let catalog_mode = subcommand == Subcommand::Add && manager.options.add_catalog.is_some(); + debug_assert!(manager.root_package_id.id.is_none()); + + let root_package_json_path = root_package_json_path(); + let root_target = WorkspaceTarget { + name: Box::default(), + name_hash: None, + package_json_path: root_package_json_path.clone(), + }; + + let (mut updates, assigned) = if subcommand == Subcommand::Remove { + (updates, vec![Vec::new(); targets.len()]) + } else { + assign_requests(manager, original_cwd, updates, &targets) + }; + + let mut changed: Vec<(WorkspaceTarget, Box<[PackageNameHash]>)> = + Vec::with_capacity(targets.len()); + for (target, wanted) in targets.into_iter().zip(assigned) { + let mut root = fetch_entry_root(manager, &target); + let received: Box<[PackageNameHash]> = if subcommand == Subcommand::Remove { + if !remove_dependencies_from_package_json(&mut root, &updates) { + continue; + } + Box::default() + } else { + let wanted_len = move_to_front(&mut updates, &wanted); + reset_e_strings(&mut updates); + let mut slice: &mut [UpdateRequest] = &mut updates[..wanted_len]; + PackageJSONEditor::edit( + manager, + &mut slice, + &mut root, + dependency_list, + EditOptions { + exact_versions, + before_install: true, + ..Default::default() + }, + )?; + let kept = slice.len(); + if kept == 0 { + continue; + } + if catalog_mode { + add_catalog::rewrite_references(manager, &updates[..kept]); + } + updates[..kept].iter().map(|r| r.name_hash).collect() + }; + store_entry(manager, &target, root, true); + changed.push((target, received)); + } + let any_changed = !changed.is_empty(); + + if subcommand != Subcommand::Remove { + updates.retain(|r| { + changed + .iter() + .any(|(_, received)| received.contains(&r.name_hash)) + }); + } + if catalog_mode { + let root = fetch_entry_root(manager, &root_target); + add_catalog::edit_root_before_install(manager, &root, &updates)?; + store_entry(manager, &root_target, root, true); + } + + // The install summary is printed from this workspace's point of view. + let summary_target = changed + .iter() + .find(|(_, received)| received.len() == updates.len()) + .or(changed.first()); + manager.workspace_name_hash = summary_target.and_then(|(target, _)| target.name_hash); + manager.to_update = false; + manager.update_requests = updates.into_boxed_slice(); + manager.pending_filtered_write = Some(Box::new(PendingWrite { + targets: changed, + subcommand, + catalog_mode, + root_target, + })); + + { + let mut zbuf = path_buffer_pool::get(); + let root_package_json_path = resolve_path::z(&root_package_json_path, &mut zbuf); + install_with_manager(manager, ctx, root_package_json_path, original_cwd)?; + } + flush_pending_write(manager)?; + + if subcommand == Subcommand::Remove && manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) { + if !any_changed { + Global::exit(0); + } + let updates: Box<[UpdateRequest]> = core::mem::take(&mut manager.update_requests); + remove_leftover_node_modules(manager, &updates); + } + + Ok(()) +} diff --git a/src/install/PackageManager/install_with_manager.rs b/src/install/PackageManager/install_with_manager.rs index 4cf780fb2d2c..4535882fcc02 100644 --- a/src/install/PackageManager/install_with_manager.rs +++ b/src/install/PackageManager/install_with_manager.rs @@ -109,6 +109,10 @@ pub fn install_with_manager( && manager.options.save_text_lockfile.unwrap_or(false)))), ); + if manager.subcommand == Subcommand::Dedupe { + crate::dedupe::dedupe_before_install(manager, &load_result)?; + } + // this defaults to false // but we force allowing updates to the lockfile when you do bun add let mut had_any_diffs = false; @@ -594,6 +598,10 @@ pub fn install_with_manager( _ => {} } + if !manager.audit_fix_pins.is_empty() && !needs_new_lockfile { + crate::audit_fix::enqueue_planned_fixes(manager)?; + } + if needs_new_lockfile { root = create_new_lockfile_and_enqueue( manager, @@ -894,6 +902,9 @@ pub fn install_with_manager( )?; } + // Before root lifecycle scripts, which exit the process on failure. + super::add_remove_with_filter::flush_pending_write(manager)?; + if needs_new_lockfile { manager.summary.add = manager.lockfile.packages.len() as u32; } @@ -1760,6 +1771,13 @@ fn save_lockfile_only( packages_len_before_install: usize, log_level: Options::LogLevel, ) -> crate::Result<()> { + let migrating_to_text = + load_result.loaded_from_binary_lockfile() && save_format == lockfile::Format::Text; + if manager.options.enable.frozen_lockfile() && !migrating_to_text { + Output::flush(); + return Ok(()); + } + // save the lockfile and exit. make sure metahash is generated for binary lockfile manager.lockfile.meta_hash = manager.lockfile.generate_meta_hash( PackageManager::verbose_install() || manager.options.do_.print_meta_hash_string(), diff --git a/src/install/PackageManager/updatePackageJSONAndInstall.rs b/src/install/PackageManager/updatePackageJSONAndInstall.rs index 850bac548da5..5b61771871cc 100644 --- a/src/install/PackageManager/updatePackageJSONAndInstall.rs +++ b/src/install/PackageManager/updatePackageJSONAndInstall.rs @@ -15,6 +15,7 @@ use bun_js_printer as js_printer; use bun_paths::{self, PathBuffer}; use bun_sys::{self, Fd, File}; +use super::add_catalog; use super::command_line_arguments::CommandLineArguments; use super::package_json_editor as PackageJSONEditor; use super::update_request::Array as UpdateRequestArray; @@ -23,7 +24,7 @@ use super::{ attempt_to_create_package_json, install_with_manager, patch_package, }; -fn print_package_json_into_cache_entry(entry: &mut MapEntry, root: bun_ast::Expr) { +pub(crate) fn print_package_json_into_cache_entry(entry: &mut MapEntry, root: bun_ast::Expr) { let preserve_trailing_newline = entry.source.contents.last() == Some(&b'\n'); let mut buffer_writer = js_printer::BufferWriter::init(); buffer_writer @@ -53,6 +54,81 @@ fn print_package_json_into_cache_entry(entry: &mut MapEntry, root: bun_ast::Expr entry.stale_contents.push(old); } +pub(super) fn remove_dependencies_from_package_json( + package_json: &mut bun_ast::Expr, + updates: &[UpdateRequest], +) -> bool { + let mut any_changes = false; + // if we're removing, they don't have to specify where it is installed in the dependencies list + // they can even put it multiple times and we will just remove all of them + for request in updates.iter() { + const LISTS: [&[u8]; 4] = [ + b"dependencies", + b"devDependencies", + b"optionalDependencies", + b"peerDependencies", + ]; + for list in LISTS { + if let Some(query) = package_json.as_property(list) { + if query.expr.data.is_e_object() { + // reshaped for borrowck — + // `StoreRef` is `Copy` and derefs to a raw arena + // pointer, so taking it once works across writes to both the + // inner list and the parent object. + let mut e_object = query.expr.data.as_e_object(); + let dependencies = e_object.properties.slice_mut(); + let mut i: usize = 0; + let mut new_len = dependencies.len(); + // `G::Property` is not `Copy`, + // so we `swap` instead of copy-from-tail — but the swapped-out + // matched element + // lands in the truncated tail and MUST NOT be revisited (it would + // match again and over-truncate). Bounding by `new_len` yields the + // correct result for the unique-key case package.json guarantees. + while i < new_len { + let key = dependencies[i].key.unwrap(); + if key.data.is_e_string() { + if key.data.as_e_string().unwrap().eql_bytes(request.name) { + if new_len > 1 { + dependencies.swap(i, new_len - 1); + new_len -= 1; + } else { + new_len = 0; + } + + any_changes = true; + } + } + i += 1; + } + + let changed = new_len != dependencies.len(); + if changed { + e_object.properties.truncate(new_len); + + // If the dependencies list is now empty, remove it from the package.json + // since we're swapRemove, we have to re-sort it + if e_object.properties.len_u32() == 0 { + // TODO: Theoretically we could change these two lines to + // `.orderedRemove(query.i)`, but would that change user-facing + // behavior? + let _ = package_json + .data + .as_e_object_mut() + .properties + .swap_remove(query.i as usize); + package_json.data.as_e_object_mut().package_json_sort(); + } else { + e_object.alphabetize_properties(); + } + } + } + } + } + } + any_changes +} + pub fn update_package_json_and_install_with_manager( manager: &mut PackageManager, ctx: Command::Context, @@ -152,6 +228,17 @@ fn update_package_json_and_install_with_manager_with_updates( Global::crash(); } + if matches!(subcommand, Subcommand::Add | Subcommand::Remove) + && !manager.options.filter_patterns.is_empty() + { + return super::add_remove_with_filter::update_filtered_workspaces_and_install( + manager, + ctx, + original_cwd, + updates, + ); + } + if subcommand == Subcommand::Update && updates.is_empty() && (manager.options.do_.recursive() || !manager.options.filter_patterns.is_empty()) @@ -304,76 +391,8 @@ fn update_package_json_and_install_with_manager_with_updates( let mut not_in_workspace_root: Option = None; match subcommand { Subcommand::Remove => { - // if we're removing, they don't have to specify where it is installed in the dependencies list - // they can even put it multiple times and we will just remove all of them - for request in updates.iter() { - const LISTS: [&[u8]; 4] = [ - b"dependencies", - b"devDependencies", - b"optionalDependencies", - b"peerDependencies", - ]; - for list in LISTS { - if let Some(query) = current_package_json_root.as_property(list) { - if query.expr.data.is_e_object() { - // reshaped for borrowck — - // `StoreRef` is `Copy` and derefs to a raw arena - // pointer, so taking it once works across writes to both the - // inner list and the parent object. - let mut e_object = query.expr.data.as_e_object(); - let dependencies = e_object.properties.slice_mut(); - let mut i: usize = 0; - let mut new_len = dependencies.len(); - // `G::Property` is not `Copy`, - // so we `swap` instead of copy-from-tail — but the swapped-out - // matched element - // lands in the truncated tail and MUST NOT be revisited (it would - // match again and over-truncate). Bounding by `new_len` yields the - // correct result for the unique-key case package.json guarantees. - while i < new_len { - let key = dependencies[i].key.unwrap(); - if key.data.is_e_string() { - if key.data.as_e_string().unwrap().eql_bytes(request.name) { - if new_len > 1 { - dependencies.swap(i, new_len - 1); - new_len -= 1; - } else { - new_len = 0; - } - - any_changes = true; - } - } - i += 1; - } - - let changed = new_len != dependencies.len(); - if changed { - e_object.properties.truncate(new_len); - - // If the dependencies list is now empty, remove it from the package.json - // since we're swapRemove, we have to re-sort it - if e_object.properties.len_u32() == 0 { - // TODO: Theoretically we could change these two lines to - // `.orderedRemove(query.i)`, but would that change user-facing - // behavior? - let _ = current_package_json_root - .data - .as_e_object_mut() - .properties - .swap_remove(query.i as usize); - current_package_json_root - .data - .as_e_object_mut() - .package_json_sort(); - } else { - e_object.alphabetize_properties(); - } - } - } - } - } - } + any_changes = + remove_dependencies_from_package_json(&mut current_package_json_root, &updates); } Subcommand::Link | Subcommand::Add | Subcommand::Update => { @@ -396,6 +415,16 @@ fn update_package_json_and_install_with_manager_with_updates( // `edit` may shrink the slice. let new_len = updates_slice.len(); updates.truncate(new_len); + if manager.options.add_catalog.is_some() { + add_catalog::rewrite_references(manager, &updates); + if manager.workspace_name_hash.is_none() { + add_catalog::edit_root_before_install( + manager, + ¤t_package_json_root, + &updates, + )?; + } + } } else if subcommand == Subcommand::Update && manager.update_target_workspaces.is_none() { PackageJSONEditor::edit_update_no_args( @@ -618,6 +647,10 @@ fn update_package_json_and_install_with_manager_with_updates( } } + if manager.options.add_catalog.is_some() && manager.workspace_name_hash.is_some() { + add_catalog::edit_root_entry_before_install(manager, root_package_json)?; + } + // SAFETY: root_package_json_path_buf[root_package_json_path_len] == 0 written above break 'root_package_json_path ZStr::from_buf( &root_package_json_path_buf[..], @@ -677,14 +710,7 @@ fn update_package_json_and_install_with_manager_with_updates( && manager.update_target_workspaces.is_none() { // running from root: catalogs live in this file. - let _ = PackageJSONEditor::edit_catalogs_after_update( - manager, - &new_package_json, - EditOptions { - exact_versions: manager.options.enable.exact_versions(), - ..Default::default() - }, - )?; + let _ = PackageJSONEditor::edit_catalogs_after_update(manager, &new_package_json)?; } } else { let mut updates_slice: &mut [UpdateRequest] = &mut updates[..]; @@ -703,6 +729,13 @@ fn update_package_json_and_install_with_manager_with_updates( }, )?; } + if manager.options.add_catalog.is_some() { + add_catalog::rewrite_references(manager, &updates[..]); + if manager.workspace_name_hash.is_none() { + let _ = + add_catalog::edit_root_after_install(manager, &new_package_json, &updates[..])?; + } + } let mut buffer_writer_two = js_printer::BufferWriter::init(); buffer_writer_two.buffer.list.reserve( (source.contents.len() + 1).saturating_sub(buffer_writer_two.buffer.list.len()), @@ -772,14 +805,8 @@ fn update_package_json_and_install_with_manager_with_updates( let root_package_json: &mut MapEntry = unsafe { &mut *root_package_json_ptr }; let root_package_json_root: bun_ast::Expr = root_package_json.root; - let root_catalogs_changed = PackageJSONEditor::edit_catalogs_after_update( - manager, - &root_package_json_root, - EditOptions { - exact_versions: manager.options.enable.exact_versions(), - ..Default::default() - }, - )?; + let root_catalogs_changed = + PackageJSONEditor::edit_catalogs_after_update(manager, &root_package_json_root)?; if root_catalogs_changed { print_package_json_into_cache_entry(root_package_json, root_package_json_root); @@ -801,6 +828,13 @@ fn update_package_json_and_install_with_manager_with_updates( } } + if manager.options.add_catalog.is_some() + && manager.workspace_name_hash.is_some() + && manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) + { + add_catalog::write_root_after_install(manager, root_package_json_path, &updates[..])?; + } + let _ = written; if let Some(targets) = manager.update_target_workspaces.take() { @@ -861,78 +895,7 @@ fn update_package_json_and_install_with_manager_with_updates( if !any_changes { Global::exit(0); } - - let cwd = bun_sys::Dir::cwd(); - // This is not exactly correct - let mut node_modules_buf = PathBuffer::uninit(); - node_modules_buf[..b"node_modules".len()].copy_from_slice(b"node_modules"); - node_modules_buf[b"node_modules".len()] = bun_paths::SEP; - let name_hashes = manager.lockfile.packages.items_name_hash(); - for request in updates.iter() { - // If the package no longer exists in the updated lockfile, delete the directory - // This is not thorough. - // It does not handle nested dependencies - // This is a quick & dirty cleanup intended for when deleting top-level dependencies - if !name_hashes - .iter() - .any(|h| *h == bun_semver::semver_string::Builder::string_hash(request.name)) - { - let offset_buf = &mut node_modules_buf[b"node_modules/".len()..]; - offset_buf[..request.name.len()].copy_from_slice(request.name); - let _ = cwd.delete_tree( - &node_modules_buf[..b"node_modules/".len() + request.name.len()], - ); - } - } - - // This is where we clean dangling symlinks - // This could be slow if there are a lot of symlinks - match bun_sys::open_dir_for_iteration(cwd.fd(), manager.options.bin_path.as_bytes()) { - Ok(node_modules_bin) => { - // `defer node_modules_bin.close()` — explicit close below (Fd is Copy, no Drop). - let mut iter = bun_sys::iterate_dir(node_modules_bin); - 'iterator: loop { - let Ok(Some(entry)) = iter.next() else { break }; - match entry.kind { - bun_sys::EntryKind::SymLink => { - // any symlinks which we are unable to open are assumed to be dangling - // note that using access won't work here, because access doesn't resolve symlinks - let name = entry.name.slice_u8(); - node_modules_buf[..name.len()].copy_from_slice(name); - node_modules_buf[name.len()] = 0; - let buf: &ZStr = ZStr::from_buf(&node_modules_buf, name.len()); - - match bun_sys::File::openat( - node_modules_bin, - buf, - bun_sys::O::RDONLY, - 0, - ) { - Ok(file) => { - let _ = file.close(); - } - Err(_) => { - let _ = bun_sys::unlinkat(node_modules_bin, buf); - continue 'iterator; - } - } - } - _ => {} - } - } - let _ = bun_sys::close(node_modules_bin); - } - Err(err) => { - if err.get_errno() != bun_sys::E::ENOENT { - Output::err( - crate::Error::from(err), - "while reading node_modules/.bin", - (), - ); - Global::crash(); - } - } - } + remove_leftover_node_modules(manager, &updates); } } @@ -1039,6 +1002,77 @@ fn write_resolved_versions_to_targets( Ok(()) } +pub(super) fn remove_leftover_node_modules( + manager: &mut PackageManager, + updates: &[UpdateRequest], +) { + let cwd = bun_sys::Dir::cwd(); + // This is not exactly correct + let mut node_modules_buf = PathBuffer::uninit(); + node_modules_buf[..b"node_modules".len()].copy_from_slice(b"node_modules"); + node_modules_buf[b"node_modules".len()] = bun_paths::SEP; + let name_hashes = manager.lockfile.packages.items_name_hash(); + for request in updates.iter() { + // If the package no longer exists in the updated lockfile, delete the directory + // This is not thorough. + // It does not handle nested dependencies + // This is a quick & dirty cleanup intended for when deleting top-level dependencies + if !name_hashes + .iter() + .any(|h| *h == bun_semver::semver_string::Builder::string_hash(request.name)) + { + let offset_buf = &mut node_modules_buf[b"node_modules/".len()..]; + offset_buf[..request.name.len()].copy_from_slice(request.name); + let _ = + cwd.delete_tree(&node_modules_buf[..b"node_modules/".len() + request.name.len()]); + } + } + + // This is where we clean dangling symlinks + // This could be slow if there are a lot of symlinks + match bun_sys::open_dir_for_iteration(cwd.fd(), manager.options.bin_path.as_bytes()) { + Ok(node_modules_bin) => { + // `defer node_modules_bin.close()` — explicit close below (Fd is Copy, no Drop). + let mut iter = bun_sys::iterate_dir(node_modules_bin); + 'iterator: loop { + let Ok(Some(entry)) = iter.next() else { break }; + match entry.kind { + bun_sys::EntryKind::SymLink => { + // any symlinks which we are unable to open are assumed to be dangling + // note that using access won't work here, because access doesn't resolve symlinks + let name = entry.name.slice_u8(); + node_modules_buf[..name.len()].copy_from_slice(name); + node_modules_buf[name.len()] = 0; + let buf: &ZStr = ZStr::from_buf(&node_modules_buf, name.len()); + + match bun_sys::File::openat(node_modules_bin, buf, bun_sys::O::RDONLY, 0) { + Ok(file) => { + let _ = file.close(); + } + Err(_) => { + let _ = bun_sys::unlinkat(node_modules_bin, buf); + continue 'iterator; + } + } + } + _ => {} + } + } + let _ = bun_sys::close(node_modules_bin); + } + Err(err) => { + if err.get_errno() != bun_sys::E::ENOENT { + Output::err( + crate::Error::from(err), + "while reading node_modules/.bin", + (), + ); + Global::crash(); + } + } + } +} + pub fn update_package_json_and_install_and_cli( ctx: Command::Context, subcommand: Subcommand, @@ -1062,10 +1096,11 @@ pub fn update_package_json_and_install_and_cli( bun_core::pretty_errorln!("No package.json, so nothing to patch"); Global::crash(); } - _ => { + _ if cli.filters.is_empty() => { attempt_to_create_package_json()?; break 'brk super::init(ctx, cli, subcommand)?; } + _ => {} } } diff --git a/src/install/audit_fix.rs b/src/install/audit_fix.rs new file mode 100644 index 000000000000..24db66761409 --- /dev/null +++ b/src/install/audit_fix.rs @@ -0,0 +1,641 @@ +use core::cmp::Ordering; +use core::mem::ManuallyDrop; +use std::io::Write as _; + +use bstr::BStr; +use bun_collections::HashMap; +use bun_core::{Global, Output, prettyln, strings}; +use bun_semver::query::Group; +use bun_semver::{self as Semver, SlicedString}; + +use crate::dependency::Behavior; +use crate::lockfile::package::PackageColumns as _; +use crate::npm::PackageManifest; +use crate::package_manager::Options::{Enable, LogLevel}; +use crate::package_manager_real::enqueue_dependency_with_main; +use crate::package_manager_real::populate_manifest_cache::{self, Packages}; +use crate::{ + Dependency, DependencyID, DependencyVersionTag, ManifestLoad, PackageID, PackageManager, + PackageNameHash, ResolutionTag, dependency, invalid_package_id, +}; + +pub struct Advisory { + pub package_name: Box<[u8]>, + pub vulnerable_versions: Box<[u8]>, +} + +#[derive(Clone)] +pub struct PlannedFix { + pub name: Box<[u8]>, + pub name_hash: PackageNameHash, + pub from: Box<[u8]>, + pub to: Box<[u8]>, + pub to_version: Semver::Version, +} + +pub struct Blocker { + pub dependent: Box<[u8]>, + pub range: Box<[u8]>, + pub bundled: bool, +} + +pub struct BlockedFix { + pub name: Box<[u8]>, + pub from: Box<[u8]>, + pub needs: Box<[u8]>, + pub blockers: Vec, +} + +pub enum UnfixableReason { + NoSafeRelease, + TooRecent(Box<[u8]>), + ManifestUnavailable, +} + +pub struct UnfixableFix { + pub name: Box<[u8]>, + pub from: Box<[u8]>, + pub reason: UnfixableReason, +} + +pub struct UnmatchedAdvisory { + pub name: Box<[u8]>, + pub range: Box<[u8]>, +} + +pub struct FixPlan { + pub fixes: Vec, + pub blocked: Vec, + pub unfixable: Vec, + pub unmatched: Vec, + pub fixed_vulnerabilities: u32, + pub remaining_vulnerabilities: u32, +} + +struct Edge { + range: Option, + literal: Box<[u8]>, + dependent: Box<[u8]>, + bundled: bool, +} + +struct Instance { + pkg_id: PackageID, + name: Box<[u8]>, + name_hash: PackageNameHash, + from: Box<[u8]>, + current: Semver::Version, + advisories: Vec, + edges: Vec, +} + +fn fmt_version(version: Semver::Version, buf: &[u8]) -> Box<[u8]> { + let mut out: Vec = Vec::new(); + let _ = write!(out, "{}", version.fmt(buf)); + out.into_boxed_slice() +} + +fn vuln_word(n: u32) -> &'static str { + if n == 1 { + "vulnerability" + } else { + "vulnerabilities" + } +} + +fn pkg_word(n: usize) -> &'static str { + if n == 1 { "package" } else { "packages" } +} + +fn order_name_from(a_name: &[u8], a_from: &[u8], b_name: &[u8], b_from: &[u8]) -> Ordering { + strings::order(a_name, b_name).then_with(|| strings::order(a_from, b_from)) +} + +pub fn exit_unless_lockfile_writable(manager: &PackageManager) { + if manager.options.dry_run || manager.options.do_.save_lockfile() { + return; + } + if manager.options.log_level != LogLevel::Silent { + if manager.options.enable.frozen_lockfile() { + Output::err_generic( + "bun audit fix needs to write bun.lock, but the lockfile is frozen", + (), + ); + bun_core::note!( + "remove --frozen-lockfile / --production (or the bunfig.toml equivalent) and run again" + ); + } else { + Output::err_generic( + "bun audit fix needs to write bun.lock, but saving the lockfile is disabled", + (), + ); + bun_core::note!( + "remove --no-save (or install.lockfile.save = false in bunfig.toml) and run again" + ); + } + Output::flush(); + } + Global::exit(1); +} + +pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crate::Result { + let mut range_buf: Vec = Vec::new(); + let mut range_spans: Vec<(usize, usize)> = Vec::with_capacity(advisories.len()); + for advisory in advisories { + let range: &[u8] = if advisory.vulnerable_versions.is_empty() { + b"*" + } else { + &advisory.vulnerable_versions + }; + range_spans.push((range_buf.len(), range.len())); + range_buf.extend_from_slice(range); + } + let advisory_groups: Vec> = range_spans + .iter() + .map(|&(start, len)| { + let input = &range_buf[start..start + len]; + Semver::query::parse(input, SlicedString::init(&range_buf, input)) + .ok() + .filter(|group| !group.is_empty()) + }) + .collect(); + let mut by_name: HashMap> = HashMap::new(); + for (i, advisory) in advisories.iter().enumerate() { + by_name + .entry(Semver::string::Builder::string_hash(&advisory.package_name)) + .or_default() + .push(i); + } + + let mut advisory_matched: Vec = vec![false; advisories.len()]; + let mut instances: Vec = Vec::new(); + { + let lockfile = &*manager.lockfile; + let buf = lockfile.buffers.string_bytes.as_slice(); + let names = lockfile.packages.items_name(); + let name_hashes = lockfile.packages.items_name_hash(); + let res = lockfile.packages.items_resolution(); + let dep_slices = lockfile.packages.items_dependencies(); + let deps = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + + let mut instance_of: Vec = vec![u32::MAX; res.len()]; + for pkg_id in 0..res.len() { + if res[pkg_id].tag != ResolutionTag::Npm { + continue; + } + let Some(candidates) = by_name.get(&name_hashes[pkg_id]) else { + continue; + }; + let current = res[pkg_id].npm().version; + let matched: Vec = candidates + .iter() + .copied() + .filter(|&i| { + advisory_groups[i].as_ref().is_some_and(|group| { + group.satisfies_including_prerelease(current, &range_buf, buf) + }) + }) + .collect(); + if matched.is_empty() { + continue; + } + for &i in &matched { + advisory_matched[i] = true; + } + instance_of[pkg_id] = instances.len() as u32; + instances.push(Instance { + pkg_id: pkg_id as PackageID, + name: Box::from(names[pkg_id].slice(buf)), + name_hash: name_hashes[pkg_id], + from: fmt_version(current, buf), + current, + advisories: matched, + edges: Vec::new(), + }); + } + + if !instances.is_empty() { + let mut parent_of: Vec = vec![invalid_package_id; deps.len()]; + for (pkg_id, slice) in dep_slices.iter().enumerate() { + let end = (slice.end() as usize).min(deps.len()); + for slot in &mut parent_of[(slice.begin() as usize).min(end)..end] { + *slot = pkg_id as PackageID; + } + } + + for (dep_id, &target) in resolutions.iter().enumerate() { + if target == invalid_package_id { + continue; + } + let Some(&instance) = instance_of.get(target as usize) else { + continue; + }; + if instance == u32::MAX || deps[dep_id].behavior.is_optional_peer() { + continue; + } + let dep = &deps[dep_id]; + let parent = parent_of[dep_id]; + let mut dependent: Vec = Vec::new(); + if parent != invalid_package_id { + let parent = parent as usize; + if res[parent].tag == ResolutionTag::Npm { + let _ = write!( + dependent, + "{}@{}", + BStr::new(names[parent].slice(buf)), + res[parent].npm().version.fmt(buf) + ); + } else { + dependent.extend_from_slice(names[parent].slice(buf)); + } + } + if dependent.is_empty() { + dependent.extend_from_slice(b"package.json"); + } + instances[instance as usize].edges.push(Edge { + range: crate::dedupe::effective_npm_range(lockfile, dep), + literal: Box::from(dep.version.literal.slice(buf)), + dependent: dependent.into_boxed_slice(), + bundled: dep.behavior.is_bundled(), + }); + } + } + } + + let mut unmatched: Vec = advisories + .iter() + .zip(&range_spans) + .zip(&advisory_matched) + .filter(|&(_, &matched)| !matched) + .map(|((advisory, &(start, len)), _)| UnmatchedAdvisory { + name: advisory.package_name.clone(), + range: Box::from(&range_buf[start..start + len]), + }) + .collect(); + unmatched.sort_by(|a, b| order_name_from(&a.name, &a.range, &b.name, &b.range)); + unmatched.dedup_by(|a, b| a.name == b.name && a.range == b.range); + + if instances.is_empty() { + return Ok(FixPlan { + fixes: Vec::new(), + blocked: Vec::new(), + unfixable: Vec::new(), + unmatched, + fixed_vulnerabilities: 0, + remaining_vulnerabilities: advisories.len() as u32, + }); + } + + manager.options.enable.set(Enable::MANIFEST_CACHE, false); + manager + .options + .enable + .set(Enable::MANIFEST_CACHE_CONTROL, false); + let ids: Vec = instances.iter().map(|inst| inst.pkg_id).collect(); + populate_manifest_cache::populate_manifest_cache(manager, Packages::Exact(&ids))?; + manager + .log_mut() + .print(std::ptr::from_mut(Output::error_writer()))?; + manager.log_mut().reset(); + + let cache_ctx = manager.manifest_disk_cache_ctx(); + let min_age = manager.options.minimum_release_age_ms; + let excludes = manager.options.minimum_release_age_excludes; + let buf = manager.lockfile.buffers.string_bytes.as_slice(); + + let mut fixes: Vec = Vec::new(); + let mut blocked: Vec = Vec::new(); + let mut unfixable: Vec = Vec::new(); + let mut advisory_still_present: Vec = advisory_matched.iter().map(|&m| !m).collect(); + + for inst in instances { + let mut expired = false; + let scope = manager.options.scope_for_package_name(&inst.name); + let Some(manifest) = manager.manifests.by_name_allow_expired( + cache_ctx, + scope, + &inst.name, + Some(&mut expired), + ManifestLoad::LoadFromMemoryFallbackToDisk, + min_age.is_some(), + ) else { + for &a in &inst.advisories { + advisory_still_present[a] = true; + } + unfixable.push(UnfixableFix { + name: inst.name, + from: inst.from, + reason: UnfixableReason::ManifestUnavailable, + }); + continue; + }; + let manifest: &PackageManifest = manifest; + let manifest_buf: &[u8] = &manifest.string_buf; + let releases = manifest.pkg.releases.keys.get(&manifest.versions); + let release_pkgs = manifest.pkg.releases.values.get(&manifest.package_versions); + let age_limit = min_age.filter(|_| !manifest.should_exclude_from_age_filter(excludes)); + let name_advisories: &[usize] = match by_name.get(&inst.name_hash) { + Some(indices) => indices, + None => &[], + }; + + let mut needs: Option = None; + let mut too_recent: Option = None; + let mut target: Option = None; + for (i, &v) in releases.iter().enumerate() { + if v.order(inst.current, manifest_buf, buf) != Ordering::Greater { + continue; + } + let still_vulnerable = name_advisories.iter().any(|&a| { + advisory_groups[a].as_ref().is_some_and(|group| { + group.satisfies_including_prerelease(v, &range_buf, manifest_buf) + }) + }); + if still_vulnerable { + continue; + } + if let Some(limit) = age_limit + && PackageManifest::is_package_version_too_recent(&release_pkgs[i], limit) + { + too_recent.get_or_insert(v); + continue; + } + if needs.is_none() { + needs = Some(v); + } + let all_dependents_accept = inst.edges.iter().all(|edge| { + !edge.bundled + && edge + .range + .as_ref() + .is_some_and(|range| range.npm().version.satisfies(v, buf, manifest_buf)) + }); + if all_dependents_accept { + target = Some(v); + break; + } + } + + if let Some(v) = target { + fixes.push(PlannedFix { + name: inst.name, + name_hash: inst.name_hash, + from: inst.from, + to: fmt_version(v, manifest_buf), + to_version: Semver::Version { + major: v.major, + minor: v.minor, + patch: v.patch, + ..Default::default() + }, + }); + continue; + } + + for &a in &inst.advisories { + advisory_still_present[a] = true; + } + let Some(needs) = needs else { + unfixable.push(UnfixableFix { + name: inst.name, + from: inst.from, + reason: match too_recent { + Some(v) => UnfixableReason::TooRecent(fmt_version(v, manifest_buf)), + None => UnfixableReason::NoSafeRelease, + }, + }); + continue; + }; + let blockers: Vec = inst + .edges + .iter() + .filter(|edge| { + edge.bundled + || !edge.range.as_ref().is_some_and(|range| { + range.npm().version.satisfies(needs, buf, manifest_buf) + }) + }) + .map(|edge| Blocker { + dependent: edge.dependent.clone(), + range: if edge.bundled { + inst.from.clone() + } else { + match &edge.range { + Some(range) => Box::from(range.literal.slice(buf)), + None => edge.literal.clone(), + } + }, + bundled: edge.bundled, + }) + .collect(); + blocked.push(BlockedFix { + name: inst.name, + from: inst.from, + needs: fmt_version(needs, manifest_buf), + blockers, + }); + } + + fixes.sort_by(|a, b| order_name_from(&a.name, &a.from, &b.name, &b.from)); + blocked.sort_by(|a, b| order_name_from(&a.name, &a.from, &b.name, &b.from)); + unfixable.sort_by(|a, b| order_name_from(&a.name, &a.from, &b.name, &b.from)); + + let remaining = advisory_still_present + .iter() + .filter(|&&present| present) + .count() as u32; + Ok(FixPlan { + fixes, + blocked, + unfixable, + unmatched, + fixed_vulnerabilities: advisories.len() as u32 - remaining, + remaining_vulnerabilities: remaining, + }) +} + +impl FixPlan { + pub fn print_sections(&self) { + if !self.fixes.is_empty() { + prettyln!("fixing:"); + for fix in &self.fixes { + prettyln!( + " {}@{} → {}", + BStr::new(&fix.name), + BStr::new(&fix.from), + BStr::new(&fix.to) + ); + } + prettyln!(""); + } + if !self.blocked.is_empty() { + prettyln!("requires a semver-major update:"); + for item in &self.blocked { + prettyln!( + " {}@{} → {}", + BStr::new(&item.name), + BStr::new(&item.from), + BStr::new(&item.needs) + ); + for blocker in &item.blockers { + prettyln!( + " {} {} {}@{}", + BStr::new(&blocker.dependent), + if blocker.bundled { + "bundles" + } else { + "depends on" + }, + BStr::new(&item.name), + BStr::new(&blocker.range) + ); + } + } + prettyln!(""); + } + if !self.unfixable.is_empty() { + prettyln!("no fix available:"); + for item in &self.unfixable { + let name = BStr::new(&item.name); + let from = BStr::new(&item.from); + match &item.reason { + UnfixableReason::NoSafeRelease => prettyln!(" {}@{}", name, from), + UnfixableReason::TooRecent(version) => prettyln!( + " {}@{} ({} is newer than --minimum-release-age)", + name, + from, + BStr::new(version) + ), + UnfixableReason::ManifestUnavailable => { + prettyln!(" {}@{} (failed to fetch the manifest)", name, from) + } + } + } + prettyln!(""); + } + if !self.unmatched.is_empty() { + prettyln!("not matched to an installed version:"); + for item in &self.unmatched { + prettyln!(" {}@{}", BStr::new(&item.name), BStr::new(&item.range)); + } + prettyln!(""); + } + Output::flush(); + } + + pub fn print_summary(&self, dry_run: bool) { + if self.fixes.is_empty() { + prettyln!("No fixable vulnerabilities"); + } else { + let packages = self + .fixes + .iter() + .zip(self.fixes.iter().skip(1)) + .filter(|(a, b)| a.name != b.name) + .count() + + 1; + prettyln!( + "{} {} {} in {} {}", + if dry_run { "Would fix" } else { "Fixed" }, + self.fixed_vulnerabilities, + vuln_word(self.fixed_vulnerabilities), + packages, + pkg_word(packages) + ); + } + if self.remaining_vulnerabilities > 0 { + prettyln!( + "{} {} remaining", + self.remaining_vulnerabilities, + vuln_word(self.remaining_vulnerabilities) + ); + } + } + + pub fn exit_code(&self) -> u32 { + u32::from(self.remaining_vulnerabilities > 0) + } + + pub fn pins(&self) -> Box<[PlannedFix]> { + self.fixes.clone().into_boxed_slice() + } +} + +pub fn enqueue_planned_fixes(manager: &mut PackageManager) -> crate::Result<()> { + let pins = core::mem::take(&mut manager.audit_fix_pins); + let _ = manager.get_cache_directory(); + let _ = manager.get_temporary_directory(); + manager + .options + .enable + .set(Enable::FORCE_SAVE_LOCKFILE, true); + + let target_of: Vec = { + let lockfile = &*manager.lockfile; + let buf = lockfile.buffers.string_bytes.as_slice(); + let name_hashes = lockfile.packages.items_name_hash(); + let res = lockfile.packages.items_resolution(); + (0..res.len()) + .map(|pkg_id| { + if res[pkg_id].tag != ResolutionTag::Npm { + return u32::MAX; + } + let mut version: Option> = None; + for (i, pin) in pins.iter().enumerate() { + if pin.name_hash != name_hashes[pkg_id] { + continue; + } + let version = + version.get_or_insert_with(|| fmt_version(res[pkg_id].npm().version, buf)); + if version[..] == pin.from[..] { + return i as u32; + } + } + u32::MAX + }) + .collect() + }; + + let n = manager.lockfile.buffers.resolutions.len(); + for dep_id in 0..n { + let target = manager.lockfile.buffers.resolutions[dep_id]; + if target == invalid_package_id + || target as usize >= target_of.len() + || target_of[target as usize] == u32::MAX + { + continue; + } + let row = manager.lockfile.buffers.dependencies[dep_id].clone(); + if row.behavior.is_optional_peer() || row.behavior.is_bundled() { + continue; + } + let pkg_name = manager.lockfile.packages.items_name()[target as usize]; + let pin = &pins[target_of[target as usize] as usize]; + let pinned = Dependency { + name: row.name, + name_hash: row.name_hash, + behavior: row.behavior.with(Behavior::PEER, false), + version: dependency::Version { + tag: DependencyVersionTag::Npm, + literal: Semver::String::default(), + value: dependency::Value { + npm: ManuallyDrop::new(dependency::NpmInfo { + name: pkg_name, + version: Group::from(pin.to_version), + is_alias: true, + }), + }, + }, + }; + manager.lockfile.buffers.resolutions[dep_id] = invalid_package_id; + enqueue_dependency_with_main( + manager, + dep_id as DependencyID, + &pinned, + invalid_package_id, + false, + )?; + } + Ok(()) +} diff --git a/src/install/dedupe.rs b/src/install/dedupe.rs new file mode 100644 index 000000000000..cbeae2a061a0 --- /dev/null +++ b/src/install/dedupe.rs @@ -0,0 +1,448 @@ +use core::cmp::Ordering; +use std::io::Write as _; + +use bstr::BStr; +use bun_core::{Global, Output, strings}; + +use crate::lockfile::package::PackageColumns as _; +use crate::lockfile::{LoadResult, LoadStep, Lockfile, PackageIndexEntry}; +use crate::package_manager::Options::{Enable, LogLevel}; +use crate::{ + Dependency, DependencyID, DependencyVersionTag, PackageID, PackageManager, ResolutionTag, + dependency, invalid_package_id, +}; + +struct Pass<'a> { + lockfile: &'a Lockfile, + groups: &'a [Vec], + group_of: &'a [u32], + pinned: &'a [bool], + edges: Vec>, + candidates: Vec, + sat: Vec, + movable: Vec, + count: Vec, +} + +impl Pass<'_> { + // Re-points the group edges owned by `voters` onto the best `live` version; also returns who voted. + fn vote( + &mut self, + cur: &[PackageID], + live: &[bool], + voters: &[bool], + ) -> (Vec, Vec) { + let lockfile = self.lockfile; + let buf = lockfile.buffers.string_bytes.as_slice(); + let pkg_res = lockfile.packages.items_resolution(); + let dep_slices = lockfile.packages.items_dependencies(); + let deps = lockfile.buffers.dependencies.as_slice(); + + for edges in &mut self.edges { + edges.clear(); + } + let mut voted = vec![false; dep_slices.len()]; + for (pkg_id, slice) in dep_slices.iter().enumerate() { + if !voters[pkg_id] { + continue; + } + for dep_id in slice.begin() as usize..slice.end() as usize { + let target = cur[dep_id]; + if target == invalid_package_id { + continue; + } + let Some(&g) = self.group_of.get(target as usize) else { + continue; + }; + if g == u32::MAX { + continue; + } + self.edges[g as usize].push(dep_id as DependencyID); + voted[pkg_id] = true; + } + } + + let mut next: Vec = cur.to_vec(); + for (g, group) in self.groups.iter().enumerate() { + let edges = &self.edges[g]; + if edges.is_empty() { + continue; + } + self.candidates.clear(); + self.candidates + .extend(group.iter().copied().filter(|&id| live[id as usize])); + let candidates = self.candidates.as_slice(); + let n = candidates.len(); + if n < 2 { + continue; + } + let m = edges.len(); + self.sat.clear(); + self.sat.resize(m * n, false); + self.movable.clear(); + self.movable.resize(m, false); + self.count.clear(); + self.count.resize(n, 0); + + for (e, &dep_id) in edges.iter().enumerate() { + let dep = &deps[dep_id as usize]; + let target = cur[dep_id as usize]; + let cur_c = candidates + .iter() + .position(|&c| c == target) + .expect("edge target is a live candidate of its group"); + let row = &mut self.sat[e * n..(e + 1) * n]; + + let range = if self.pinned[target as usize] || dep.behavior.is_bundled() { + None + } else { + effective_npm_range(lockfile, dep) + }; + match range { + None => row[cur_c] = true, + Some(range) => { + self.movable[e] = true; + let query = &range.npm().version; + for (c, &id) in candidates.iter().enumerate() { + row[c] = query.satisfies(pkg_res[id as usize].npm().version, buf, buf); + } + } + } + for (c, &ok) in row.iter().enumerate() { + self.count[c] += ok as u32; + } + } + + for (e, &dep_id) in edges.iter().enumerate() { + if !self.movable[e] { + continue; + } + let row = &self.sat[e * n..(e + 1) * n]; + let mut best: Option = None; + for c in 0..n { + if !row[c] { + continue; + } + let Some(b) = best else { + best = Some(c); + continue; + }; + let better = match self.count[c].cmp(&self.count[b]) { + Ordering::Greater => true, + Ordering::Less => false, + Ordering::Equal => { + let vc = pkg_res[candidates[c] as usize].npm().version; + let vb = pkg_res[candidates[b] as usize].npm().version; + match vc.order(vb, buf, buf) { + Ordering::Greater => true, + Ordering::Less => false, + Ordering::Equal => candidates[c] < candidates[b], + } + } + }; + if better { + best = Some(c); + } + } + if let Some(b) = best { + next[dep_id as usize] = candidates[b]; + } + } + } + (next, voted) + } + + // Packages the pass itself removes must not vote (pnpm/pnpm#9213): shrink voters until the outcome agrees. + fn settle(&mut self, cur: &[PackageID], live: &[bool]) -> Vec { + let dep_slices = self.lockfile.packages.items_dependencies(); + let mut voters = live.to_vec(); + let mut best: Option> = None; + loop { + let (next, voted) = self.vote(cur, live, &voters); + let after = reachable(dep_slices, &next); + if (0..after.len()).any(|p| after[p] && !voters[p]) { + return best.unwrap_or(next); + } + let mut died = false; + for p in 0..after.len() { + if voted[p] && !after[p] { + voters[p] = false; + died = true; + } + } + if !died { + return next; + } + best = Some(next); + } + } +} + +fn label(lockfile: &Lockfile, id: PackageID) -> Vec { + let buf = lockfile.buffers.string_bytes.as_slice(); + let mut label = Vec::new(); + let _ = write!( + label, + "{}@{}", + BStr::new(lockfile.packages.items_name()[id as usize].slice(buf)), + lockfile.packages.items_resolution()[id as usize] + .npm() + .version + .fmt(buf) + ); + label +} + +pub fn dedupe_lockfile(lockfile: &mut Lockfile) -> Vec> { + let pkg_res = lockfile.packages.items_resolution(); + let dep_slices = lockfile.packages.items_dependencies(); + let has_patches = lockfile.patched_dependencies.count() > 0; + + let mut groups: Vec> = Vec::new(); + let mut group_of: Vec = vec![u32::MAX; pkg_res.len()]; + let mut pinned: Vec = vec![false; pkg_res.len()]; + + for entry in lockfile.package_index.values() { + let PackageIndexEntry::Ids(ids) = entry else { + continue; + }; + let candidates: Vec = ids + .iter() + .copied() + .filter(|&id| pkg_res[id as usize].tag == ResolutionTag::Npm) + .collect(); + if candidates.len() < 2 { + continue; + } + for &id in &candidates { + group_of[id as usize] = groups.len() as u32; + pinned[id as usize] = has_patches + && lockfile.patched_dependencies.contains( + &bun_semver::string::Builder::string_hash(&label(lockfile, id)), + ); + } + groups.push(candidates); + } + + if groups.is_empty() { + return Vec::new(); + } + + let initial = reachable(dep_slices, &lockfile.buffers.resolutions); + let mut live = initial.clone(); + let mut cur: Vec = lockfile.buffers.resolutions.to_vec(); + { + let mut pass = Pass { + lockfile, + groups: &groups, + group_of: &group_of, + pinned: &pinned, + edges: vec![Vec::new(); groups.len()], + candidates: Vec::new(), + sat: Vec::new(), + movable: Vec::new(), + count: Vec::new(), + }; + loop { + cur = pass.settle(&cur, &live); + let after = reachable(dep_slices, &cur); + if after == live { + break; + } + live = after; + } + } + + let mut removed: Vec = groups + .iter() + .flatten() + .copied() + .filter(|&id| initial[id as usize] && !live[id as usize]) + .collect(); + if removed.is_empty() { + return Vec::new(); + } + + let buf = lockfile.buffers.string_bytes.as_slice(); + let names = lockfile.packages.items_name(); + removed.sort_by(|&a, &b| { + let (a, b) = (a as usize, b as usize); + strings::order(names[a].slice(buf), names[b].slice(buf)).then_with(|| { + pkg_res[a] + .npm() + .version + .order(pkg_res[b].npm().version, buf, buf) + }) + }); + let labels = removed + .iter() + .map(|&id| label(lockfile, id).into_boxed_slice()) + .collect(); + + lockfile.buffers.resolutions = cur; + labels +} + +pub(crate) fn effective_npm_range( + lockfile: &Lockfile, + dep: &Dependency, +) -> Option { + let mut version = if dep.version.tag == DependencyVersionTag::Npm && dep.version.npm().is_alias + { + dep.version.clone() + } else { + lockfile + .overrides + .get(dep.name_hash) + .unwrap_or_else(|| dep.version.clone()) + }; + if version.tag == DependencyVersionTag::Catalog { + version = lockfile + .catalogs + .get(lockfile, *version.catalog(), dep.name)? + .version; + } + (version.tag == DependencyVersionTag::Npm).then_some(version) +} + +// Optional-peer edges are followed too: with an in-sync package.json `clean` runs with `keep_optional_peer_targets`. +fn reachable( + dep_slices: &[crate::lockfile::DependencySlice], + resolutions: &[PackageID], +) -> Vec { + let mut seen = vec![false; dep_slices.len()]; + if seen.is_empty() { + return seen; + } + seen[0] = true; + let mut worklist: Vec = vec![0]; + while let Some(pkg_id) = worklist.pop() { + let slice = dep_slices[pkg_id as usize]; + for i in slice.begin() as usize..slice.end() as usize { + let target = resolutions[i]; + if target == invalid_package_id { + continue; + } + let Some(slot) = seen.get_mut(target as usize) else { + continue; + }; + if !*slot { + *slot = true; + worklist.push(target); + } + } + } + seen +} + +fn load_step_verb(step: LoadStep) -> &'static str { + match step { + LoadStep::OpenFile => "open", + LoadStep::ReadFile => "read", + LoadStep::ParseFile => "parse", + LoadStep::Migrating => "migrate", + } +} + +pub fn dedupe_before_install( + manager: &mut PackageManager, + load_result: &LoadResult<'_>, +) -> crate::Result<()> { + let quiet = manager.options.log_level == LogLevel::Silent; + + match load_result { + LoadResult::NotFound => { + if !quiet { + Output::err_generic("missing lockfile, nothing to dedupe", ()); + } + Global::exit(1); + } + LoadResult::Err(cause) => { + if !quiet { + Output::err_generic( + "failed to {s} lockfile: {s}", + (load_step_verb(cause.step), cause.value.name()), + ); + if manager.log_mut().has_errors() { + let _ = manager + .log_mut() + .print(core::ptr::from_mut(Output::error_writer())); + } + } + Global::crash(); + } + LoadResult::Ok(_) => {} + } + + let removed = dedupe_lockfile(&mut manager.lockfile); + + if removed.is_empty() { + if !quiet { + bun_core::prettyln!("Already deduplicated."); + Output::flush(); + } + if manager.options.dry_run { + Global::exit(0); + } + return Ok(()); + } + + let n = removed.len(); + let plural = if n == 1 { "" } else { "s" }; + let mut list: Vec = Vec::new(); + for (i, label) in removed.iter().enumerate() { + if i > 0 { + list.extend_from_slice(b", "); + } + list.extend_from_slice(label); + } + + if manager.options.dry_run { + if !quiet { + bun_core::prettyln!( + "{} duplicate version{} can be removed: {}", + n, + plural, + BStr::new(&list) + ); + bun_core::note!("run 'bun dedupe' to remove them"); + Output::flush(); + } + Global::exit(1); + } + + if !manager.options.do_.save_lockfile() { + if !quiet { + let why = if manager.options.enable.frozen_lockfile() { + "the lockfile is frozen" + } else { + "saving the lockfile is disabled" + }; + bun_core::pretty_errorln!( + "error: {} duplicate version{} can be removed, but {}: {}", + n, + plural, + why, + BStr::new(&list) + ); + bun_core::note!("run 'bun dedupe --check' to only report duplicates"); + Output::flush(); + } + Global::exit(1); + } + + if !quiet { + bun_core::prettyln!( + "Removed {} duplicate version{}: {}", + n, + plural, + BStr::new(&list) + ); + Output::flush(); + } + manager + .options + .enable + .set(Enable::FORCE_SAVE_LOCKFILE, true); + Ok(()) +} diff --git a/src/install/dependency.rs b/src/install/dependency.rs index 3a2661f63ab6..749904124596 100644 --- a/src/install/dependency.rs +++ b/src/install/dependency.rs @@ -463,24 +463,6 @@ pub mod tarball { pub use super::{TarballInfo, URI as Uri}; } -pub(crate) fn split_version_and_maybe_name(str: &[u8]) -> (&[u8], Option<&[u8]>) { - if let Some(at_index) = strings::index_of_char(str, b'@') { - let at_index = at_index as usize; - if at_index != 0 { - return (&str[at_index + 1..], Some(&str[0..at_index])); - } - - let Some(second) = strings::index_of_char(&str[1..], b'@') else { - return (str, None); - }; - let second_at_index = second as usize + 1; - - return (&str[second_at_index + 1..], Some(&str[0..second_at_index])); - } - - (str, None) -} - /// Turns `foo@1.1.1` into `foo`, `1.1.1`, or `@foo/bar@1.1.1` into `@foo/bar`, `1.1.1`, or `foo` into `foo`, `null`. fn split_name_and_maybe_version(str: &[u8]) -> (&[u8], Option<&[u8]>) { if let Some(at_index) = strings::index_of_char(str, b'@') { @@ -734,6 +716,7 @@ impl VersionExt for Version { Tag::Tarball => self.tarball().eql(rhs.tarball(), lhs_buf, rhs_buf), Tag::Symlink => self.symlink().eql(*rhs.symlink(), lhs_buf, rhs_buf), Tag::Workspace => self.workspace().eql(*rhs.workspace(), lhs_buf, rhs_buf), + Tag::Catalog => self.catalog().eql(*rhs.catalog(), lhs_buf, rhs_buf), _ => true, } } diff --git a/src/install/lib.rs b/src/install/lib.rs index 11258ad52a84..c2e2204be780 100644 --- a/src/install/lib.rs +++ b/src/install/lib.rs @@ -108,8 +108,10 @@ pub mod package_manager_task; #[path = "TarballStream.rs"] pub mod tarball_stream; pub use lockfile_real::{DEFAULT_TRUSTED_DEPENDENCIES_LIST, default_trusted_dependencies}; +pub mod audit_fix; #[path = "bin.rs"] pub mod bin_real; +pub mod dedupe; pub mod hoisted_install; pub mod isolated_install; pub mod lifecycle_script_runner; @@ -120,6 +122,7 @@ pub mod package_install; pub mod package_installer; pub mod patch_install; pub mod pnpm; +pub mod prune; #[path = "repository.rs"] pub mod repository_real; pub mod yarn; diff --git a/src/install/lockfile.rs b/src/install/lockfile.rs index 8fcba19edbf0..0716f97e6f05 100644 --- a/src/install/lockfile.rs +++ b/src/install/lockfile.rs @@ -60,6 +60,8 @@ pub mod lockfile_json_stringify_for_debugging; pub mod override_map; #[path = "lockfile/Package.rs"] pub mod package; +#[path = "lockfile/pruned_workspaces.rs"] +pub(crate) mod pruned_workspaces; #[path = "lockfile/Tree.rs"] pub mod tree; #[path = "lockfile/printer"] @@ -696,141 +698,153 @@ impl Lockfile { updates: &mut [UpdateRequest], exact_versions: bool, ) -> Result<(), BunError> { - let workspace_package_id = manager - .root_package_id - .get(old, manager.workspace_name_hash); - let root_deps_list: DependencySlice = - old.packages.items_dependencies()[workspace_package_id as usize]; + let workspace_ids: Vec> = updates + .iter() + .map(|update| { + let mut ids = old.update_request_workspace_ids(manager, update); + ids.retain(|&id| { + (old.packages.items_dependencies()[id as usize].off as usize) + < old.buffers.dependencies.len() + }); + ids + }) + .collect(); + if workspace_ids.iter().all(Vec::is_empty) { + return Ok(()); + } - if (root_deps_list.off as usize) < old.buffers.dependencies.len() { - // Split-borrow: `string_builder!` only takes - // `old.buffers.string_bytes` + `old.string_pool`, leaving - // `old.packages` / `old.buffers.{dependencies,resolutions}` free. - let mut string_builder = string_builder!(old); + // `bun add x --dev` seeds only the dev slot with a dist-tag; the name's other groups keep their own ranges. + let only_dist_tag_deps = manager.subcommand != crate::Subcommand::Update; - { - let root_deps: &[Dependency] = - root_deps_list.get(old.buffers.dependencies.as_slice()); - let old_resolutions_list = - old.packages.items_resolutions()[workspace_package_id as usize]; - let old_resolutions: &[PackageID] = - old_resolutions_list.get(old.buffers.resolutions.as_slice()); - let resolutions_of_yore: &[Resolution] = old.packages.items_resolution(); - let packages_len = old.packages.len(); - - for update in updates.iter() { - if update.package_id == invalid_package_id { - debug_assert_eq!(root_deps.len(), old_resolutions.len()); - for (dep, &old_resolution) in root_deps.iter().zip(old_resolutions.iter()) { - if dep.name_hash == SemverStringBuilder::string_hash(update.name) { - if old_resolution as usize >= packages_len { - continue; - } - let res = resolutions_of_yore[old_resolution as usize]; - if res.tag != ResolutionTag::Npm - || update.version.tag != dependency::Tag::DistTag - { - continue; - } + // Split-borrow: `string_builder!` takes only `string_bytes` + `string_pool`, leaving `packages`/`dependencies`/`resolutions` free. + let mut string_builder = string_builder!(old); - // TODO(dylan-conway): this will need to handle updating dependencies (exact, ^, or ~) and aliases + { + let resolutions_of_yore: &[Resolution] = old.packages.items_resolution(); + let packages_len = old.packages.len(); - let npm_ver = res.npm().version; - let len = bun_core::fmt::count(format_args!( - "{}{}", - if exact_versions { "" } else { "^" }, - npm_ver.fmt(string_builder.string_bytes.as_slice()), - )); + for (update, workspace_ids) in updates.iter().zip(&workspace_ids) { + if update.package_id != invalid_package_id { + continue; + } + for &workspace_package_id in workspace_ids { + let root_deps: &[Dependency] = old.packages.items_dependencies() + [workspace_package_id as usize] + .get(old.buffers.dependencies.as_slice()); + let old_resolutions: &[PackageID] = old.packages.items_resolutions() + [workspace_package_id as usize] + .get(old.buffers.resolutions.as_slice()); + debug_assert_eq!(root_deps.len(), old_resolutions.len()); + for (dep, &old_resolution) in root_deps.iter().zip(old_resolutions.iter()) { + if dep.name_hash == SemverStringBuilder::string_hash(update.name) { + if old_resolution as usize >= packages_len { + continue; + } + let res = resolutions_of_yore[old_resolution as usize]; + if res.tag != ResolutionTag::Npm + || update.version.tag != dependency::Tag::DistTag + || dep.version.tag == dependency::Tag::Catalog + || (only_dist_tag_deps + && dep.version.tag != dependency::Tag::DistTag) + { + continue; + } - if len >= SemverString::MAX_INLINE_LEN { - string_builder.cap += len; - } + // TODO(dylan-conway): this will need to handle updating dependencies (exact, ^, or ~) and aliases + + let npm_ver = res.npm().version; + let len = bun_core::fmt::count(format_args!( + "{}{}", + if exact_versions { "" } else { "^" }, + npm_ver.fmt(string_builder.string_bytes.as_slice()), + )); + + if len >= SemverString::MAX_INLINE_LEN { + string_builder.cap += len; } } } } } + } - string_builder.allocate()?; - // `string_builder.clamp()` must run once after the entire second - // loop completes. A scopeguard would mutably capture - // `string_builder`, conflicting with the `append` calls below. Call `clamp()` - // explicitly at the end of this block instead (the inner loop has no `?` exits; - // the only fallible call above is `allocate()`, which precedes this point). + string_builder.allocate()?; + // `clamp()` runs once after the second pass; nothing below can `?` out before it. - { - let mut temp_buf = [0u8; 513]; - - let root_deps: &mut [Dependency] = - root_deps_list.mut_(old.buffers.dependencies.as_mut_slice()); - let old_resolutions_list_lists = old.packages.items_resolutions(); - let old_resolutions_list = - old_resolutions_list_lists[workspace_package_id as usize]; - let old_resolutions: &[PackageID] = - old_resolutions_list.get(old.buffers.resolutions.as_slice()); - let resolutions_of_yore: &[Resolution] = old.packages.items_resolution(); - let packages_len = old.packages.len(); - - for update in updates.iter_mut() { - if update.package_id == invalid_package_id { - debug_assert_eq!(root_deps.len(), old_resolutions.len()); - for (dep, &old_resolution) in - root_deps.iter_mut().zip(old_resolutions.iter()) - { - if dep.name_hash == SemverStringBuilder::string_hash(update.name) { - if old_resolution as usize >= packages_len { - continue; - } - let res = resolutions_of_yore[old_resolution as usize]; - if res.tag != ResolutionTag::Npm - || update.version.tag != dependency::Tag::DistTag - { - continue; - } + { + let mut temp_buf = [0u8; 513]; + let packages_len = old.packages.len(); - // TODO(dylan-conway): this will need to handle updating dependencies (exact, ^, or ~) and aliases - - let npm_ver = res.npm().version; - let buf = { - let mut cursor: &mut [u8] = &mut temp_buf[..]; - let start_len = cursor.len(); - if write!( - cursor, - "{}{}", - if exact_versions { "" } else { "^" }, - npm_ver.fmt(string_builder.string_bytes.as_slice()), - ) - .is_err() - { - break; - } - let written = start_len - cursor.len(); - &temp_buf[..written] - }; - - let external_version = string_builder.append::(buf); - let sliced = external_version - .value - .sliced(string_builder.string_bytes.as_slice()); - dep.version = dependency::parse( - dep.name, - dep.name_hash, - sliced.slice, - &sliced, - None, - &mut *manager, - ) - .unwrap_or_default(); + for (update, workspace_ids) in updates.iter_mut().zip(&workspace_ids) { + update.e_string = None; + if update.package_id != invalid_package_id { + continue; + } + for &workspace_package_id in workspace_ids { + let root_deps_list: DependencySlice = + old.packages.items_dependencies()[workspace_package_id as usize]; + let root_deps: &mut [Dependency] = + root_deps_list.mut_(old.buffers.dependencies.as_mut_slice()); + let old_resolutions: &[PackageID] = old.packages.items_resolutions() + [workspace_package_id as usize] + .get(old.buffers.resolutions.as_slice()); + let resolutions_of_yore: &[Resolution] = old.packages.items_resolution(); + debug_assert_eq!(root_deps.len(), old_resolutions.len()); + for (dep, &old_resolution) in root_deps.iter_mut().zip(old_resolutions.iter()) { + if dep.name_hash == SemverStringBuilder::string_hash(update.name) { + if old_resolution as usize >= packages_len { + continue; } + let res = resolutions_of_yore[old_resolution as usize]; + if res.tag != ResolutionTag::Npm + || update.version.tag != dependency::Tag::DistTag + || dep.version.tag == dependency::Tag::Catalog + || (only_dist_tag_deps + && dep.version.tag != dependency::Tag::DistTag) + { + continue; + } + + // TODO(dylan-conway): this will need to handle updating dependencies (exact, ^, or ~) and aliases + + let npm_ver = res.npm().version; + let buf = { + let mut cursor: &mut [u8] = &mut temp_buf[..]; + let start_len = cursor.len(); + if write!( + cursor, + "{}{}", + if exact_versions { "" } else { "^" }, + npm_ver.fmt(string_builder.string_bytes.as_slice()), + ) + .is_err() + { + break; + } + let written = start_len - cursor.len(); + &temp_buf[..written] + }; + + let external_version = string_builder.append::(buf); + let sliced = external_version + .value + .sliced(string_builder.string_bytes.as_slice()); + dep.version = dependency::parse( + dep.name, + dep.name_hash, + sliced.slice, + &sliced, + None, + &mut *manager, + ) + .unwrap_or_default(); } } - - update.e_string = None; } } - - string_builder.clamp(); } + + string_builder.clamp(); Ok(()) } @@ -952,6 +966,22 @@ impl Lockfile { } } + /// Package ids of the workspaces whose package.json received `update`: the `--filter` targets, else the cwd workspace. + fn update_request_workspace_ids( + &self, + manager: &mut PackageManager, + update: &UpdateRequest, + ) -> Vec { + match &manager.pending_filtered_write { + Some(pending) => pending.workspace_ids_receiving(self, update.name_hash), + None => vec![ + manager + .root_package_id + .get(self, manager.workspace_name_hash), + ], + } + } + // `#[inline(never)]` keeps the panic/format machinery from // `bun_core::output` (pulled in by the cold helpers below) out of callers; // the hot copy/remap loop stays in this body while the three cold sections @@ -1033,12 +1063,14 @@ impl Lockfile { let clone_queue_ = PendingResolutions::new(); // Explicit `&mut *` reborrows so `old`/`manager`/`new` are // released back to this scope once `cloner` is dropped. + let keep_optional_peer_targets = !manager.summary.has_diffs(); let mut cloner = Cloner { old: &mut *old, lockfile: &mut *new, mapping: &mut package_id_mapping, clone_queue: clone_queue_, optional_peers: PendingResolutions::new(), + keep_optional_peer_targets, log, old_preinstall_state, manager: &mut *manager, @@ -1157,6 +1189,20 @@ impl Lockfile { clean_migrate_patched_dependencies_cold(old, &mut new)?; } + // Read catalog resolutions from `new`: `old` still holds the stale dependency slots that `bun update` orphaned. + if !updates.is_empty() && manager.options.add_catalog.is_some() { + crate::package_manager_real::add_catalog::rewrite_lockfile_entries( + &mut new, manager, updates, + )?; + } + if !manager.updating_catalogs.is_empty() { + crate::package_manager_real::package_json_editor::resolve_catalog_updates( + &mut new, + manager, + exact_versions, + )?; + } + // Don't allow invalid memory to happen if !updates.is_empty() { // `UpdateRequest.version_buf` is a raw `*const [u8]` (PORTING.md @@ -1174,17 +1220,17 @@ impl Lockfile { // updates might be applied to the root package.json or one // of the workspace package.json files. - let workspace_package_id = manager - .root_package_id - .get(&new, manager.workspace_name_hash); - - let dep_list = slice.items_dependencies()[workspace_package_id as usize]; - let res_list = slice.items_resolutions()[workspace_package_id as usize]; - let workspace_deps: &[Dependency] = dep_list.get(new.buffers.dependencies.as_slice()); - let resolved_ids: &[PackageID] = res_list.get(new.buffers.resolutions.as_slice()); - 'request_updated: for update in updates.iter_mut() { - if update.package_id == invalid_package_id { + if update.package_id != invalid_package_id { + continue; + } + for workspace_package_id in new.update_request_workspace_ids(manager, update) { + let dep_list = slice.items_dependencies()[workspace_package_id as usize]; + let res_list = slice.items_resolutions()[workspace_package_id as usize]; + let workspace_deps: &[Dependency] = + dep_list.get(new.buffers.dependencies.as_slice()); + let resolved_ids: &[PackageID] = + res_list.get(new.buffers.resolutions.as_slice()); debug_assert_eq!(resolved_ids.len(), workspace_deps.len()); for (&package_id, dep) in resolved_ids.iter().zip(workspace_deps.iter()) { if update.matches(dep, string_buf) { @@ -1313,6 +1359,7 @@ pub struct Cloner<'a> { pub(crate) clone_queue: PendingResolutions, /// Bound in `flush`, once `clone_queue` has decided which targets survive. pub(crate) optional_peers: PendingResolutions, + pub(crate) keep_optional_peer_targets: bool, pub lockfile: &'a mut Lockfile, pub(crate) old: &'a mut Lockfile, pub(crate) mapping: &'a mut [PackageID], diff --git a/src/install/lockfile/CatalogMap.rs b/src/install/lockfile/CatalogMap.rs index 2064fcd8edac..1cf5cde36ca7 100644 --- a/src/install/lockfile/CatalogMap.rs +++ b/src/install/lockfile/CatalogMap.rs @@ -5,6 +5,7 @@ use bun_alloc::AllocError; use bun_collections::ArrayHashMap; use bun_collections::array_hash_map::ArrayHashAdapter; use bun_install::dependency::DependencyExt as _; +use bun_install::dependency::{Tag as DependencyVersionTag, Version as DependencyVersion}; use bun_install::lockfile::{Buffers, StringBuilder}; use bun_install::{Dependency, Lockfile, PackageManager}; // Layering: every install-side caller (Package.rs / pnpm.rs) parses JSON/YAML @@ -45,27 +46,114 @@ impl CatalogMap { self.default.count() > 0 || self.groups.count() > 0 } + /// `catalog:` and `catalog:default` name the same catalog. + pub(crate) fn same_name(a: &[u8], b: &[u8]) -> bool { + let is_default = |name: &[u8]| name.is_empty() || name == b"default"; + a == b || (is_default(a) && is_default(b)) + } + + /// `(None, i)` indexes `default`, `(Some(g), i)` indexes `groups`; the default catalog is looked up under both spellings, the one matching `catalog_name` first. + fn locate( + &self, + string_buf: &[u8], + catalog_name: &[u8], + dep_name: &[u8], + ) -> Option<(Option, usize)> { + let dep_key = String::init(dep_name, dep_name); + let dep_ctx = ArrayHashContext { + arg_buf: dep_name, + existing_buf: string_buf, + }; + let in_default = || { + self.default + .get_index_adapted(&dep_key, &dep_ctx) + .map(|i| (None, i)) + }; + let in_group = |name: &[u8]| { + let ctx = ArrayHashContext { + arg_buf: name, + existing_buf: string_buf, + }; + let g = self + .groups + .get_index_adapted(&String::init(name, name), &ctx)?; + let i = self.groups.values()[g].get_index_adapted(&dep_key, &dep_ctx)?; + Some((Some(g), i)) + }; + if catalog_name.is_empty() { + return in_default().or_else(|| in_group(b"default")); + } + in_group(catalog_name).or_else(|| (catalog_name == b"default").then(in_default).flatten()) + } + + pub fn find<'a>( + &'a self, + string_buf: &[u8], + catalog_name: &[u8], + dep_name: &[u8], + ) -> Option<&'a Dependency> { + let (group, i) = self.locate(string_buf, catalog_name, dep_name)?; + let map = match group { + Some(g) => &self.groups.values()[g], + None => &self.default, + }; + Some(&map.values()[i]) + } + + pub(crate) fn find_mut<'a>( + &'a mut self, + string_buf: &[u8], + catalog_name: &[u8], + dep_name: &[u8], + ) -> Option<&'a mut Dependency> { + let (group, i) = self.locate(string_buf, catalog_name, dep_name)?; + let map = match group { + Some(g) => &mut self.groups.values_mut()[g], + None => &mut self.default, + }; + Some(&mut map.values_mut()[i]) + } + + pub(crate) fn get_ref<'a>( + &'a self, + string_buf: &[u8], + catalog_name: String, + dep_name: String, + ) -> Option<&'a Dependency> { + self.find( + string_buf, + catalog_name.slice(string_buf), + dep_name.slice(string_buf), + ) + } + pub(crate) fn get( &self, lockfile: &Lockfile, catalog_name: String, dep_name: String, ) -> Option { - let buf = lockfile.buffers.string_bytes.as_slice(); - if catalog_name.is_empty() { - if self.default.count() == 0 { - return None; - } - return self.default.get_adapted(&dep_name, &ctx(buf)).cloned(); - } - - let group = self.groups.get_adapted(&catalog_name, &ctx(buf))?; + self.get_ref( + lockfile.buffers.string_bytes.as_slice(), + catalog_name, + dep_name, + ) + .cloned() + } - if group.count() == 0 { - return None; + // Falls back to the unresolved `catalog:` version when the entry is missing. + pub(crate) fn resolve_range<'a>( + &'a self, + string_buf: &[u8], + dep: &'a Dependency, + ) -> &'a DependencyVersion { + if dep.version.tag != DependencyVersionTag::Catalog { + return &dep.version; + } + match self.get_ref(string_buf, *dep.version.catalog(), dep.name) { + Some(entry) => &entry.version, + None => &dep.version, } - - group.get_adapted(&dep_name, &ctx(buf)).cloned() } /// Takes `buf: &[u8]` (the lockfile's string buffer, used for the hash diff --git a/src/install/lockfile/Package.rs b/src/install/lockfile/Package.rs index 43b9edf56896..24e3c99efd5c 100644 --- a/src/install/lockfile/Package.rs +++ b/src/install/lockfile/Package.rs @@ -617,7 +617,8 @@ impl Package { }; // Peer slots must not keep their target alive; bound in `Cloner::flush`. - if old_dependencies[i].behavior.is_optional_peer() { + if old_dependencies[i].behavior.is_optional_peer() && !cloner.keep_optional_peer_targets + { cloner.optional_peers.push(pending); continue; } @@ -918,6 +919,8 @@ pub struct DiffSummary { pub(crate) removed_trusted_dependencies: TrustedDependenciesSet, pub(crate) patched_dependencies_changed: bool, + + pub(crate) pruned_workspaces: Vec, } impl DiffSummary { @@ -1334,6 +1337,27 @@ impl Diff { }; if !found { + if is_root + && from_dep.behavior.is_workspace() + && pm.options.enable.frozen_lockfile() + && lockfile::pruned_workspaces::workspace_is_missing_on_disk( + &*from_lockfile, + from_dep.name_hash, + ) + { + if pm.options.log_level.is_verbose() { + bun_core::note!( + "skipping workspace \"{}\": listed in bun.lock but not on disk", + bstr::BStr::new( + from_dep + .name + .slice(from_lockfile.buffers.string_bytes.as_slice()) + ), + ); + } + summary.pruned_workspaces.push(from_dep.name_hash); + continue; + } // We found a removed dependency! // We don't need to remove it // It will be cleaned up later @@ -1526,10 +1550,20 @@ impl Diff { // Use saturating arithmetic here because a migrated // package-lock.json could be out of sync with the package.json, so the // number of from_deps could be greater than to_deps. - summary.add = (to_deps!() - .len() - .saturating_sub(from_deps.len().saturating_sub(summary.remove as usize))) - as u32; + summary.add = (to_deps!().len().saturating_sub( + from_deps + .len() + .saturating_sub(summary.remove as usize + summary.pruned_workspaces.len()), + )) as u32; + + if !summary.pruned_workspaces.is_empty() && !pm.options.log_level.is_silent() { + let count = summary.pruned_workspaces.len(); + bun_core::note!( + "skipped {} workspace{} listed in bun.lock but not on disk", + count, + if count == 1 { "" } else { "s" }, + ); + } if from.resolution.tag != ResolutionTag::Root { for (to_hook, from_hook) in to.scripts.hooks().iter().zip(from.scripts.hooks().iter()) { @@ -2230,6 +2264,7 @@ impl Package { source, dependencies_q.loc, Some(&mut string_builder), + pm.options.enable.frozen_lockfile(), )?; break 'brk; } @@ -2276,6 +2311,7 @@ impl Package { source, packages_loc, Some(&mut string_builder), + pm.options.enable.frozen_lockfile(), )?; } diff --git a/src/install/lockfile/Package/Meta.rs b/src/install/lockfile/Package/Meta.rs index b8cb9eef3ba4..2ac6e612756f 100644 --- a/src/install/lockfile/Package/Meta.rs +++ b/src/install/lockfile/Package/Meta.rs @@ -65,7 +65,7 @@ impl Default for Meta { impl Meta { /// Does the `cpu` arch and `os` match the requirements listed in the package? /// This is completely unrelated to "devDependencies", "peerDependencies", "optionalDependencies" etc - pub(crate) fn is_disabled(&self, cpu: Architecture, os: OperatingSystem) -> bool { + pub fn is_disabled(&self, cpu: Architecture, os: OperatingSystem) -> bool { !self.arch.is_match(cpu) || !self.os.is_match(os) } diff --git a/src/install/lockfile/Package/WorkspaceMap.rs b/src/install/lockfile/Package/WorkspaceMap.rs index 07052f36bafd..4a66ff4711ee 100644 --- a/src/install/lockfile/Package/WorkspaceMap.rs +++ b/src/install/lockfile/Package/WorkspaceMap.rs @@ -175,6 +175,7 @@ impl WorkspaceMap { source: &bun_ast::Source, loc: bun_ast::Loc, mut string_builder: Option<&mut StringBuilder<'_>>, + skip_missing: bool, ) -> crate::Result { let workspace_names = self; let item_count = arr.len(); @@ -239,6 +240,9 @@ impl WorkspaceMap { let (abs_package_json_path, workspace_entry) = match processed { Ok(processed) => processed, Err(err) => { + if skip_missing && err == crate::Error::Sys(bun_errno::SystemErrno::ENOENT) { + continue; + } if err == crate::Error::Sys(bun_errno::SystemErrno::EISDIR) || err == crate::Error::Sys(bun_errno::SystemErrno::EPERM) || err == crate::Error::Sys(bun_errno::SystemErrno::ENOENT) diff --git a/src/install/lockfile/Tree.rs b/src/install/lockfile/Tree.rs index 66d7c3c067ce..129fdb5403cf 100644 --- a/src/install/lockfile/Tree.rs +++ b/src/install/lockfile/Tree.rs @@ -628,6 +628,10 @@ pub(crate) fn is_filtered_dependency_or_workspace( return false; } + if manager.summary.pruned_workspaces.contains(&dep.name_hash) { + return true; + } + let mut workspace_matched = workspace_filters.is_empty(); for filter in workspace_filters { @@ -1065,10 +1069,14 @@ impl Tree { } }; - if dependency.version.tag == crate::dependency::VersionTag::Npm { + let peer_range: &crate::dependency::Version = builder + .lockfile() + .catalogs + .resolve_range(builder.buf(), dependency); + if peer_range.tag == crate::dependency::VersionTag::Npm { let resolution: Resolution = builder.lockfile().packages.items_resolution()[res_id as usize]; - let version = &dependency.version.npm().version; + let version = &peer_range.npm().version; if resolution.tag == crate::resolution::Tag::Npm && version.satisfies(resolution.npm().version, builder.buf(), builder.buf()) { diff --git a/src/install/lockfile/bun.lock.rs b/src/install/lockfile/bun.lock.rs index 379de751af9b..ee6399a19485 100644 --- a/src/install/lockfile/bun.lock.rs +++ b/src/install/lockfile/bun.lock.rs @@ -19,8 +19,7 @@ use crate::{ bin::{Bin, Tag as BinTag}, dependency, dependency::{ - Behavior, Dependency, DependencyExt as _, Value as DependencyVersionValue, - Version as DependencyVersion, + Behavior, Dependency, Value as DependencyVersionValue, Version as DependencyVersion, }, invalid_package_id, resolution::Tag as ResolutionTag, @@ -42,7 +41,7 @@ use bun_install_types::DependencyVersionTag; use super::PackageIDSlice; use super::package::{Meta, PackageColumns as _, value_loc_of}; use super::{ - DependencySlice, LoadResult, Lockfile as BinaryLockfile, OverrideMap, Package, + CatalogMap, DependencySlice, LoadResult, Lockfile as BinaryLockfile, OverrideMap, Package, PackageIndexEntry, PackageIndexMap, PatchedDep, TrustedDependenciesSet, VersionHashMap, tree, }; @@ -2809,6 +2808,7 @@ pub(crate) fn parse_into_binary_lockfile( // tree path (see `resolve_peer_dep_version_based`). let package_index = &lockfile.package_index; let overrides = &lockfile.overrides; + let catalogs: &CatalogMap = &lockfile.catalogs; // Disjoint-field split of `lockfile.buffers` so each loop body can hold // `&mut dependencies[i]` and `&mut resolutions[i]` together with a shared @@ -2824,17 +2824,14 @@ pub(crate) fn parse_into_binary_lockfile( let dep_id: DependencyID = _dep_id; let dep = &mut dependencies[dep_id as usize]; - let peer_res_id = if is_deferred_peer(dep) { - resolve_peer_dep_version_based( - dep, - package_index, - overrides, - pkg_resolutions, - string_buf, - ) - } else { - None - }; + let peer_res_id = resolve_peer_dep_version_based( + dep, + catalogs, + package_index, + overrides, + pkg_resolutions, + string_buf, + ); let Some(res_id) = peer_res_id.or_else(|| pkg_map.get(dep.name.slice(string_buf)).copied()) else { @@ -2909,17 +2906,14 @@ pub(crate) fn parse_into_binary_lockfile( &buf_slice[..needed] }; - let peer_res_id = if is_deferred_peer(dep) { - resolve_peer_dep_version_based( - dep, - package_index, - overrides, - pkg_resolutions, - string_buf, - ) - } else { - None - }; + let peer_res_id = resolve_peer_dep_version_based( + dep, + catalogs, + package_index, + overrides, + pkg_resolutions, + string_buf, + ); let Some(res_id) = peer_res_id.or_else(|| { pkg_map .get(workspace_node_modules) @@ -2978,17 +2972,14 @@ pub(crate) fn parse_into_binary_lockfile( let dep_id: DependencyID = _dep_id; let dep = &mut dependencies[dep_id as usize]; - let peer_res_id = if is_deferred_peer(dep) { - resolve_peer_dep_version_based( - dep, - package_index, - overrides, - pkg_resolutions, - string_buf, - ) - } else { - None - }; + let peer_res_id = resolve_peer_dep_version_based( + dep, + catalogs, + package_index, + overrides, + pkg_resolutions, + string_buf, + ); let res_id = match peer_res_id { Some(id) => id, None => { @@ -3052,17 +3043,26 @@ pub(crate) fn parse_into_binary_lockfile( Ok(()) } -/// True for peer edges the fresh resolver defers to its second phase +/// The catalog-resolved range of a peer edge the fresh resolver defers to its second phase /// (`install_peer`) and binds by version there. Two exemptions, matching /// `enqueue_dependency_with_main_and_success_fn`: optional peers return /// before the deferred phase and are bound to the hoisted-tree sibling by /// `process_subtree` instead, and `*` peers express no version preference /// and bind to whatever sibling pin existed first. Both of those are /// exactly what the printed tree's path walk reproduces, so they keep it. -fn is_deferred_peer(dep: &Dependency) -> bool { - dep.behavior.is_peer() - && !dep.behavior.is_optional_peer() - && !(dep.version.tag == DependencyVersionTag::Npm && dep.version.npm().version.is_star()) +fn deferred_peer_range<'a>( + dep: &'a Dependency, + catalogs: &'a CatalogMap, + string_buf: &[u8], +) -> Option<&'a DependencyVersion> { + if !dep.behavior.is_peer() || dep.behavior.is_optional_peer() { + return None; + } + let range = catalogs.resolve_range(string_buf, dep); + if range.tag == DependencyVersionTag::Npm && range.npm().version.is_star() { + return None; + } + Some(range) } /// Resolve a peer dependency edge the way the fresh resolver's @@ -3076,7 +3076,7 @@ fn is_deferred_peer(dep: &Dependency) -> bool { /// `list[0]` there, and reproducing its choice exactly is the point of /// this helper). Returns `None` when no package with the name exists /// or the fallback is a different kind; the caller then falls back to -/// the path walk. +/// the path walk. Edges `deferred_peer_range` rejects also return `None`. /// /// Peer edges cannot be resolved from the printed tree the way regular /// edges are: a peer never materializes its own `node_modules` path when @@ -3087,11 +3087,6 @@ fn is_deferred_peer(dep: &Dependency) -> bool { /// re-keys isolated-linker store entries (and global-store entry hashes) /// on warm installs. /// -/// `catalog:` peer ranges are left on the path walk: the version scan -/// cannot satisfy them (no catalog branch below), so they resolve exactly -/// as before this helper existed. Closing that residual would mean -/// replicating the catalog rewrite chain here. -/// /// Peers whose name matches a workspace package need no special casing /// even though the fresh resolver binds them to the workspace before any /// deferral (`'resolve_from_workspace`): the version scan below picks an @@ -3101,23 +3096,27 @@ fn is_deferred_peer(dep: &Dependency) -> bool { /// edge value is ever consulted. fn resolve_peer_dep_version_based( dep: &Dependency, + catalogs: &CatalogMap, package_index: &PackageIndexMap, overrides: &OverrideMap, pkg_resolutions: &[Resolution], string_buf: &[u8], ) -> Option { - // `package_index` is keyed by *real* package names while `dep.name_hash` - // may hold an alias, so an `npm:`-aliased peer must be looked up under - // the real package name (`dep.realname()`). Mirrors the realname hashing - // in `enqueue_dependency_with_main_and_success_fn`. - let name_hash = match dep.version.tag { - DependencyVersionTag::DistTag - | DependencyVersionTag::Git - | DependencyVersionTag::Github - | DependencyVersionTag::Npm - | DependencyVersionTag::Tarball - | DependencyVersionTag::Workspace => { - StringBuilder::string_hash(dep.realname().slice(string_buf)) + let range = deferred_peer_range(dep, catalogs, string_buf)?; + // `package_index` is keyed by real package names; `range` (not `dep.name`) carries them for aliases. + let name_hash = match range.tag { + DependencyVersionTag::Npm => StringBuilder::string_hash(range.npm().name.slice(string_buf)), + DependencyVersionTag::DistTag => { + StringBuilder::string_hash(range.dist_tag().name.slice(string_buf)) + } + DependencyVersionTag::Git => { + StringBuilder::string_hash(range.git().package_name.slice(string_buf)) + } + DependencyVersionTag::Github => { + StringBuilder::string_hash(range.github().package_name.slice(string_buf)) + } + DependencyVersionTag::Tarball => { + StringBuilder::string_hash(range.tarball().package_name.slice(string_buf)) } _ => dep.name_hash, }; @@ -3132,7 +3131,13 @@ fn resolve_peer_dep_version_based( // workspace-only edges are never overridden. let overridable = !dep.behavior.is_workspace() && (dep.version.tag != DependencyVersionTag::Npm || !dep.version.npm().is_alias); - if overridable && overrides.get(name_hash).is_some() { + // Overrides are applied before catalog resolution, so a catalog peer is overridden by its own name. + let override_name_hash = if dep.version.tag == DependencyVersionTag::Catalog { + dep.name_hash + } else { + name_hash + }; + if overridable && overrides.get(override_name_hash).is_some() { return None; } @@ -3144,11 +3149,8 @@ fn resolve_peer_dep_version_based( for &id in candidates { if (id as usize) < pkg_resolutions.len() - && pkg_resolutions[id as usize].satisfies_dependency_version( - &dep.version, - string_buf, - string_buf, - ) + && pkg_resolutions[id as usize] + .satisfies_dependency_version(range, string_buf, string_buf) { return Some(id); } @@ -3157,7 +3159,7 @@ fn resolve_peer_dep_version_based( let &first = candidates.first()?; if (first as usize) < pkg_resolutions.len() { let res_tag = pkg_resolutions[first as usize].tag; - let ver_tag = dep.version.tag; + let ver_tag = range.tag; if (res_tag == ResolutionTag::Npm && ver_tag == DependencyVersionTag::Npm) || (res_tag == ResolutionTag::Git && ver_tag == DependencyVersionTag::Git) || (res_tag == ResolutionTag::Github && ver_tag == DependencyVersionTag::Github) diff --git a/src/install/lockfile/pruned_workspaces.rs b/src/install/lockfile/pruned_workspaces.rs new file mode 100644 index 000000000000..96fbab00ed5e --- /dev/null +++ b/src/install/lockfile/pruned_workspaces.rs @@ -0,0 +1,18 @@ +use bun_paths::AutoAbsPath; + +use crate::PackageNameHash; +use crate::lockfile_real::Lockfile; + +pub(crate) fn workspace_is_missing_on_disk( + lockfile: &Lockfile, + workspace_name_hash: PackageNameHash, +) -> bool { + let Some(workspace_path) = lockfile.workspace_paths.get(&workspace_name_hash).copied() else { + return false; + }; + let mut package_json_path: AutoAbsPath = AutoAbsPath::init_top_level_dir(); + let _ = + package_json_path.append(workspace_path.slice(lockfile.buffers.string_bytes.as_slice())); + let _ = package_json_path.append(b"package.json"); + !bun_sys::exists_z(package_json_path.slice_z()) +} diff --git a/src/install/migration.rs b/src/install/migration.rs index d5e30d86ea78..f00f50d900c4 100644 --- a/src/install/migration.rs +++ b/src/install/migration.rs @@ -153,23 +153,11 @@ pub fn detect_and_load_other_lockfile<'a>( } bun_core::warn!("Failed to parse pnpm-lock.yaml."); } - MigratePnpmLockfileError::PnpmLockfileNotObject - | MigratePnpmLockfileError::PnpmLockfileMissingVersion - | MigratePnpmLockfileError::PnpmLockfileVersionInvalid - | MigratePnpmLockfileError::PnpmLockfileMissingImporters - | MigratePnpmLockfileError::PnpmLockfileMissingRootPackage - | MigratePnpmLockfileError::PnpmLockfileInvalidSnapshot - | MigratePnpmLockfileError::PnpmLockfileInvalidDependency - | MigratePnpmLockfileError::PnpmLockfileMissingDependencyVersion - | MigratePnpmLockfileError::PnpmLockfileMissingCatalogEntry - | MigratePnpmLockfileError::PnpmLockfileUnresolvableDependency => { - // These errors are continuable - log the error but don't exit - // The install will continue with a fresh install instead of migration + _ => { if log.has_errors() { let _ = log.print(std::ptr::from_mut(Output::error_writer())); } } - _ => {} } log.reset(); return LoadResult::Err(LoadResultErr { @@ -182,6 +170,10 @@ pub fn detect_and_load_other_lockfile<'a>( }; if matches!(migrate_result, LoadResult::Ok { .. }) { + if log.warnings > 0 && !log.has_errors() { + let _ = log.print(std::ptr::from_mut(Output::error_writer())); + log.reset(); + } Output::print_elapsed(timer.elapsed().as_nanos() as f64 / 1_000_000.0); bun_core::pretty_error!(" "); bun_core::pretty_errorln!("migrated lockfile from pnpm-lock.yaml"); @@ -334,6 +326,7 @@ fn migrate_npm_lockfile<'a>( &json_src, wksp_loc, None, + false, )?; debug!("found {} workspace packages", workspace_packages_count); num_deps += workspace_packages_count; diff --git a/src/install/pnpm.rs b/src/install/pnpm.rs index 790ff81256b0..df6dd8e327de 100644 --- a/src/install/pnpm.rs +++ b/src/install/pnpm.rs @@ -17,6 +17,7 @@ use crate::external_slice::ExternalSlice; use crate::integrity::Integrity; use crate::lockfile::{self, LoadResult, LoadResultOk, Lockfile}; use crate::npm::{self}; +use crate::repository::Repository; use crate::resolution::{self, Resolution, TaggedValue}; use crate::{DependencyID, INVALID_PACKAGE_ID, PackageID, PackageManager}; @@ -90,6 +91,206 @@ fn remove_suffix(path: &[u8]) -> &[u8] { path } +/// pnpm dependency-path refToRelative +fn pnpm_reference_is_dep_path(reference: &[u8]) -> bool { + if reference.first() == Some(&b'@') { + return true; + } + let Some(at) = strings::index_of_char_usize(reference, b'@') else { + return false; + }; + if strings::index_of_char_usize(reference, b':').is_some_and(|colon| colon < at) { + return false; + } + if strings::index_of_char_usize(reference, b'(').is_some_and(|paren| paren < at) { + return false; + } + true +} + +fn write_pnpm_dep_path( + out: &mut Vec, + dep_name: &[u8], + reference: &[u8], +) -> Result<(), AllocError> { + out.clear(); + if pnpm_reference_is_dep_path(reference) { + out.extend_from_slice(reference); + return Ok(()); + } + write!( + out, + "{}@{}", + bstr::BStr::new(dep_name), + bstr::BStr::new(reference) + ) + .map_err(|_| AllocError) +} + +fn missing_package_entry( + log: &mut bun_ast::Log, + dep_path: &[u8], + dep_name: &[u8], + parent: core::fmt::Arguments<'_>, +) -> MigratePnpmLockfileError { + log.add_error_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml has no package entry '{}' for dependency '{}' of {}", + bstr::BStr::new(dep_path), + bstr::BStr::new(dep_name), + parent + ), + ); + MigratePnpmLockfileError::PnpmLockfileUnresolvableDependency +} + +fn collect_patch_paths( + obj: &Expr, + out: &mut StringArrayHashMap>, +) -> Result<(), AllocError> { + for prop in e_object(obj).properties.slice() { + let key = prop.key.as_ref().expect("infallible: prop has key"); + let value = prop.value.as_ref().expect("infallible: prop has value"); + if let (Some(key_str), Some(path_str)) = (as_string(key), as_string(value)) { + out.put(key_str, Box::from(path_str))?; + } + } + Ok(()) +} + +/// Current pnpm records only the patch hash in the lockfile; the patch file path lives in the config. +fn read_config_patch_paths( + manager: &mut PackageManager, + log: &mut bun_ast::Log, +) -> Result>, AllocError> { + let mut paths: StringArrayHashMap> = StringArrayHashMap::new(); + + let mut pkg_json_path = bun_paths::AutoAbsPath::init_top_level_dir(); + let _ = pkg_json_path.append(b"package.json"); + if let crate::GetJsonResult::Entry(pkg_json) = manager + .workspace_package_json_cache + .get_with_path(log, pkg_json_path.slice(), Default::default()) + { + if let Some(patched) = pkg_json + .root + .get(b"pnpm") + .and_then(|pnpm| pnpm.get_object(b"patchedDependencies")) + { + collect_patch_paths(&patched, &mut paths)?; + } + } + + if let Ok(contents) = sys::File::read_from(Fd::cwd(), b"pnpm-workspace.yaml") { + let contents: &'static [u8] = js_ast::data_store_dupe_str(&contents); + let source = bun_ast::Source::init_path_string(b"pnpm-workspace.yaml", contents); + let arena = bun_alloc::Arena::new(); + if let Ok(ws_root) = bun_parsers::yaml::YAML::parse( + &source, + log, + &arena, + bun_parsers::yaml::CyclicAliases::Reject, + ) { + if let Some(patched) = ws_root.get_object(b"patchedDependencies") { + collect_patch_paths(&patched, &mut paths)?; + } + } + } + + Ok(paths) +} + +/// `work:1.0.0` -> `1.0.0` for pnpm's registry-qualified dep paths (pnpm11/deps/path parseRegistryQualifiedVersion). +fn split_registry_qualified_version(res_str: &[u8]) -> Option<(&[u8], &[u8])> { + let colon = strings::index_of_char_usize(res_str, b':')?; + let (registry, version) = (&res_str[..colon], &res_str[colon + 1..]); + if registry.is_empty() + || !registry[0].is_ascii_alphabetic() + || !registry + .iter() + .all(|c| c.is_ascii_alphanumeric() || matches!(c, b'_' | b'.' | b'-')) + || !version.first().is_some_and(u8::is_ascii_digit) + || matches!( + registry, + b"file" | b"link" | b"npm" | b"runtime" | b"workspace" | b"catalog" | b"git" + ) + { + return None; + } + Some((registry, version)) +} + +fn resolution_from_package_entry( + res_str: &[u8], + resolution_expr: Option<&Expr>, + string_buf: &mut semver::string::Buf<'_>, +) -> Result { + let tarball = resolution_expr.and_then(|obj| get_string(obj, b"tarball").map(|(url, _)| url)); + + if let Some(obj) = resolution_expr { + let ty = get_string(obj, b"type").map(|(s, _)| s).unwrap_or_default(); + + // `path:` is pnpm's `repo#commit&path:sub/dir` on git and git-hosted tarball resolutions. + if get_string(obj, b"path").is_some() { + return Err(MigratePnpmLockfileError::PnpmLockfileGitSubdirectory); + } + + if ty == b"git" { + if let Some((repo, _)) = get_string(obj, b"repo") { + let commit = get_string(obj, b"commit") + .map(|(s, _)| s) + .unwrap_or_default(); + return Ok(Resolution::init(TaggedValue::Git(Repository { + repo: string_buf.append(strings::without_prefix(repo, b"git+"))?, + committish: string_buf.append(commit)?, + ..Default::default() + }))); + } + } + + if ty == b"directory" { + if let Some((dir, _)) = get_string(obj, b"directory") { + return Ok(Resolution::init(TaggedValue::Folder( + string_buf.append(dir)?, + ))); + } + } + + if let Some(path) = + tarball.and_then(|url| strings::without_prefix_if_possible_comptime(url, b"file:")) + { + return Ok(Resolution::init(TaggedValue::LocalTarball( + string_buf.append(path)?, + ))); + } + } + + // Registry packages: the caller decides whether a recorded `tarball:` is trusted. + if res_str.first().is_some_and(u8::is_ascii_digit) { + return Ok(Resolution::from_pnpm_lockfile(res_str, string_buf)?); + } + + if let Some(url) = tarball { + if strings::has_prefix_comptime(url, b"https://codeload.github.com/") { + return Ok(Resolution::from_pnpm_lockfile(url, string_buf)?); + } + return Ok(Resolution::init(TaggedValue::RemoteTarball( + string_buf.append(url)?, + ))); + } + + if let Some(path) = strings::without_prefix_if_possible_comptime(res_str, b"file:") { + if Dependency::is_tarball(path) { + return Ok(Resolution::init(TaggedValue::LocalTarball( + string_buf.append(path)?, + ))); + } + } + + Ok(Resolution::from_pnpm_lockfile(res_str, string_buf)?) +} + #[derive(thiserror::Error, Debug, strum::IntoStaticStr)] pub enum MigratePnpmLockfileError { #[error("out of memory")] @@ -128,6 +329,8 @@ pub enum MigratePnpmLockfileError { PnpmLockfileMissingCatalogEntry, #[error("PnpmLockfileUnresolvableDependency")] PnpmLockfileUnresolvableDependency, + #[error("PnpmLockfileGitSubdirectory")] + PnpmLockfileGitSubdirectory, } bun_core::oom_from_alloc!(MigratePnpmLockfileError); @@ -244,7 +447,14 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( Ok(r) => r, Err(_) => return Err(MigratePnpmLockfileError::YamlParseError), }; - let root: Expr = bun_core::handle_oom(_root.deep_clone(&yaml_arena)); + let mut root: Expr = bun_core::handle_oom(_root.deep_clone(&yaml_arena)); + + // pnpm 11 writes `------`; the last document is the lockfile. + if let Some(mut documents) = root.as_array() { + while let Some(document) = documents.next() { + root = document; + } + } if !root.is_object() { log.add_error_fmt( @@ -305,7 +515,19 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( return Err(MigratePnpmLockfileError::PnpmLockfileTooOld); } + if lockfile_version_num >= 10.0 { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml lockfileVersion {} is newer than the supported 9.0, migrating as 9.0", + lockfile_version_num + ), + ); + } + let mut found_patches: StringArrayHashMap> = StringArrayHashMap::new(); + let mut snapshot_dep_paths = SnapshotDepPaths::new(); let (pkg_map, importer_dep_res_versions, workspace_pkgs_off, workspace_pkgs_end) = 'build: { if let Some(mut catalogs_expr) = root.get_object(b"catalogs") { @@ -327,32 +549,73 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( let Some(name_str) = as_string(key) else { return Err(invalid_pnpm_lockfile()); }; - let name_hash = semver::string::Builder::string_hash(name_str); - let name = sbuf!(lockfile).append_with_hash(name_str, name_hash)?; - let Some(version_str) = as_string(value) else { - // TODO: + log.add_error_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml override '{}' must be a string", + bstr::BStr::new(name_str) + ), + ); return Err(invalid_pnpm_lockfile()); }; + if version_str == b"-" { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml override '{}' removes the dependency ('-'), which bun does not support", + bstr::BStr::new(name_str) + ), + ); + } else if Dependency::split_name_and_maybe_version(name_str) + .1 + .is_some() + || strings::contains_char(name_str, b'>') + { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml override '{}' is scoped to a version range or parent package, which bun does not support; it will not apply", + bstr::BStr::new(name_str) + ), + ); + } + + let name_hash = semver::string::Builder::string_hash(name_str); + let name = sbuf!(lockfile).append_with_hash(name_str, name_hash)?; + let version_hash = semver::string::Builder::string_hash(version_str); let version = sbuf!(lockfile).append_with_hash(version_str, version_hash)?; let version_sliced = version.sliced(string_bytes!(lockfile)); + let Some(version) = Dependency::parse( + name, + name_hash, + version_sliced.slice, + &version_sliced, + Some(&mut *log), + Some(&mut *manager), + ) else { + log.add_error_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml override '{}' has an invalid value '{}'", + bstr::BStr::new(name_str), + bstr::BStr::new(version_str) + ), + ); + return Err(invalid_pnpm_lockfile()); + }; + let dep = Dependency { name, name_hash, - version: match Dependency::parse( - name, - name_hash, - version_sliced.slice, - &version_sliced, - Some(&mut *log), - Some(&mut *manager), - ) { - Some(v) => v, - None => return Err(invalid_pnpm_lockfile()), - }, + version, ..Default::default() }; @@ -362,44 +625,58 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( struct Patch { path: String, - dep_name: Box<[u8]>, - } - impl Default for Patch { - fn default() -> Self { - Self { - path: String::default(), - dep_name: Box::from(b"" as &[u8]), - } - } + key: Box<[u8]>, } - let mut patches: StringArrayHashMap = StringArrayHashMap::new(); + // patch hash -> every patchedDependencies key using that patch file + let mut patches: StringArrayHashMap> = StringArrayHashMap::new(); let mut patch_join_buf: Vec = Vec::new(); if let Some(patched_dependencies_expr) = root.get_object(b"patchedDependencies") { + let mut config_patch_paths: Option>> = None; + for prop in e_object(&patched_dependencies_expr).properties.slice() { - let dep_name_expr = prop.key.as_ref().expect("infallible: prop has key"); + let key_expr = prop.key.as_ref().expect("infallible: prop has key"); let value = prop.value.as_ref().expect("infallible: prop has value"); - let Some(dep_name_str) = as_string(dep_name_expr) else { + let Some(key_str) = as_string(key_expr) else { return Err(invalid_pnpm_lockfile()); }; - let Some((path_str, _)) = get_string(value, b"path") else { - return Err(invalid_pnpm_lockfile()); - }; - - let Some((hash_str, _)) = get_string(value, b"hash") else { - return Err(invalid_pnpm_lockfile()); + let (hash_str, path_str) = if let Some(hash_str) = as_string(value) { + if config_patch_paths.is_none() { + config_patch_paths = Some(read_config_patch_paths(manager, log)?); + } + let config = config_patch_paths.as_ref().expect("set above"); + let path = config.get(key_str).or_else(|| { + config.get(Dependency::split_name_and_maybe_version(key_str).0) + }); + let Some(path) = path else { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml patch for '{}' is not in patchedDependencies of package.json or pnpm-workspace.yaml, skipping", + bstr::BStr::new(key_str) + ), + ); + continue; + }; + (hash_str, &**path) + } else { + let Some((path_str, _)) = get_string(value, b"path") else { + return Err(invalid_pnpm_lockfile()); + }; + let Some((hash_str, _)) = get_string(value, b"hash") else { + return Err(invalid_pnpm_lockfile()); + }; + (hash_str, path_str) }; - let entry = patches.get_or_put(hash_str)?; - if entry.found_existing { - return Err(invalid_pnpm_lockfile()); - } - *entry.value_ptr = Patch { - path: sbuf!(lockfile).append(path_str)?, - dep_name: Box::<[u8]>::from(dep_name_str), - }; + let path = sbuf!(lockfile).append(path_str)?; + patches.get_or_put(hash_str)?.value_ptr.push(Patch { + path, + key: Box::from(key_str), + }); } } @@ -434,13 +711,25 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( let _ = pkg_json_path.append(importer_path); // OOM/capacity error is non-actionable here let _ = pkg_json_path.append(b"package.json"); // OOM/capacity error is non-actionable here - let importer_pkg_json = match manager - .workspace_package_json_cache - .get_with_path(log, pkg_json_path.slice(), Default::default()) - .unwrap() - { - Ok(j) => j, - Err(_) => return Err(invalid_pnpm_lockfile()), + let importer_pkg_json = match manager.workspace_package_json_cache.get_with_path( + log, + pkg_json_path.slice(), + Default::default(), + ) { + crate::GetJsonResult::Entry(j) => j, + crate::GetJsonResult::ReadErr(_) => { + log.add_error_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml lists importer '{}' but '{}/package.json' does not exist", + bstr::BStr::new(importer_path), + bstr::BStr::new(importer_path) + ), + ); + return Err(invalid_pnpm_lockfile()); + } + crate::GetJsonResult::ParseErr(_) => return Err(invalid_pnpm_lockfile()), }; let workspace_root = &importer_pkg_json.root; @@ -746,10 +1035,14 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( struct SnapshotEntry { obj: Expr, + patch_hash: Option<&'static [u8]>, } impl Default for SnapshotEntry { fn default() -> Self { - Self { obj: Expr::EMPTY } + Self { + obj: Expr::EMPTY, + patch_hash: None, + } } } let mut snapshots: StringArrayHashMap = StringArrayHashMap::new(); @@ -790,50 +1083,27 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( key_str }; - 'try_patch: { - let Some(idx) = patch_hash_idx else { - break 'try_patch; - }; - let patch_hash_str = &key_str[idx + b"(patch_hash=".len()..]; - let Some(end_idx) = strings::index_of_char(patch_hash_str, b')') else { - return Err(invalid_pnpm_lockfile()); - }; - let Some(patch) = - patches.fetch_swap_remove(&patch_hash_str[0..end_idx as usize]) - else { - break 'try_patch; - }; - - let Ok((_, res_str)) = - dependency::split_name_and_version(key_str_without_suffix) - else { - return Err(invalid_pnpm_lockfile()); - }; - - found_patches.put(&patch.value.dep_name, Box::from(res_str))?; - - patch_join_buf.clear(); - write!( - &mut patch_join_buf, - "{}@{}", - bstr::BStr::new(&patch.value.dep_name), - bstr::BStr::new(res_str) - ) - .map_err(|_| AllocError)?; - - let patch_hash = semver::string::Builder::string_hash(&patch_join_buf); - lockfile.patched_dependencies.put( - patch_hash, - crate::lockfile_real::PatchedDep::with_path(patch.value.path), - )?; - } + let patch_hash = match patch_hash_idx { + Some(idx) => { + let patch_hash_str = &key_str[idx + b"(patch_hash=".len()..]; + let Some(end_idx) = strings::index_of_char_usize(patch_hash_str, b')') + else { + return Err(invalid_pnpm_lockfile()); + }; + Some(&patch_hash_str[..end_idx]) + } + None => None, + }; let entry = snapshots.get_or_put(key_str_without_suffix)?; if entry.found_existing { continue; } - *entry.value_ptr = SnapshotEntry { obj: *value }; + *entry.value_ptr = SnapshotEntry { + obj: *value, + patch_hash, + }; } for packages_prop in e_object(&packages_obj).properties.slice() { @@ -854,37 +1124,152 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( return Err(invalid_pnpm_lockfile()); } + // Pruned lockfiles (`turbo prune`) can leave peer-suffixed keys in `packages:`. + let key_str = remove_suffix(key_str); + if pkg_map.contains(key_str) { + continue; + } + + // Like pnpm, a `packages:` entry without a snapshot is unreachable and ignored. let Some(snapshot) = snapshots.get(key_str) else { - log.add_error_fmt( - None, - bun_ast::Loc::EMPTY, - format_args!( - "pnpm-lock.yaml package '{}' missing corresponding snapshot entry", - bstr::BStr::new(key_str) - ), - ); - return Err(MigratePnpmLockfileError::PnpmLockfileInvalidSnapshot); + continue; }; let snapshot_obj = snapshot.obj; + let snapshot_patch_hash = snapshot.patch_hash; let Ok((name_str, res_str)) = dependency::split_name_and_version(key_str) else { return Err(invalid_pnpm_lockfile()); }; + if strings::has_prefix_comptime(res_str, b"runtime:") { + continue; + } + + let res_str = match split_registry_qualified_version(res_str) { + Some((registry, version)) => { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml package '{}' is from pnpm registry '{}', resolving it from the configured registry instead", + bstr::BStr::new(key_str), + bstr::BStr::new(registry) + ), + ); + version + } + None => res_str, + }; + + let resolution_expr: Option = package_obj.get(b"resolution"); + if let Some(r) = &resolution_expr { + if !r.is_object() { + return Err(invalid_pnpm_lockfile()); + } + } + + let mut res = match resolution_from_package_entry( + res_str, + resolution_expr.as_ref(), + &mut sbuf!(lockfile), + ) { + Ok(res) => res, + Err(MigratePnpmLockfileError::InvalidPnpmLockfile) => { + log.add_error_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml package '{}' has an unsupported resolution", + bstr::BStr::new(key_str) + ), + ); + return Err(invalid_pnpm_lockfile()); + } + Err(MigratePnpmLockfileError::PnpmLockfileGitSubdirectory) => { + log.add_error_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml package '{}' is a git sub-directory dependency (resolution.path), which bun does not support", + bstr::BStr::new(key_str) + ), + ); + return Err(invalid_pnpm_lockfile()); + } + Err(err) => return Err(err), + }; + + // pnpm records injected workspace packages as `name@file:`. + if res.tag == resolution::Tag::Folder { + let mut path_buf = bun_paths::AutoAbsPath::init_top_level_dir(); + let _ = path_buf.join(&[res.folder().slice(string_bytes!(lockfile))]); + if let Some(workspace_pkg_id) = pkg_map + .get(path_buf.slice()) + .copied() + .filter(|id| (*id as usize) < workspace_pkgs_end) + { + let entry = pkg_map.get_or_put(key_str)?; + if entry.found_existing { + return Err(invalid_pnpm_lockfile()); + } + *entry.value_ptr = workspace_pkg_id; + continue; + } + } + let name_hash = semver::string::Builder::string_hash(name_str); let name = sbuf!(lockfile).append_with_hash(name_str, name_hash)?; - let mut res = Resolution::from_pnpm_lockfile(res_str, &mut sbuf!(lockfile))?; + if let Some(patch_list) = snapshot_patch_hash.and_then(|hash| patches.get(hash)) { + if let Some(patch) = patch_list.iter().find(|patch| { + Dependency::split_name_and_maybe_version(&patch.key).0 == name_str + }) { + patch_join_buf.clear(); + write!( + &mut patch_join_buf, + "{}@{}", + bstr::BStr::new(name_str), + res.fmt(string_bytes!(lockfile), bun_core::fmt::PathSep::Posix) + ) + .map_err(|_| AllocError)?; + lockfile.patched_dependencies.put( + semver::string::Builder::string_hash(&patch_join_buf), + crate::lockfile_real::PatchedDep::with_path(patch.path), + )?; + if Dependency::split_name_and_maybe_version(&patch.key) + .1 + .is_none() + { + found_patches.put( + &patch.key, + Box::from(&patch_join_buf[name_str.len() + 1..]), + )?; + } + } + } if res.tag == resolution::Tag::Npm { - let scope = manager.scope_for_package_name(name_str); - let url = crate::extract_tarball::build_url( - scope.url.href(), - &strings::StringOrTinyString::init(name.slice(string_bytes!(lockfile))), - res.npm().version, - string_bytes!(lockfile), - )?; - res.npm_mut().url = sbuf!(lockfile).append(url)?; + let registry = manager.scope_for_package_name(name_str).url.href(); + // Registries like GitHub Packages serve tarballs off the canonical `/-/` path (pnpm/pnpm#13534). + let recorded = resolution_expr + .as_ref() + .and_then(|r| get_string(r, b"tarball")) + .map(|(url, _)| url) + .filter(|url| lockfile::bun_lock::url_is_under_registry(url, registry)); + res.npm_mut().url = match recorded { + Some(url) => sbuf!(lockfile).append(url)?, + None => { + let url = crate::extract_tarball::build_url( + registry, + &strings::StringOrTinyString::init( + name.slice(string_bytes!(lockfile)), + ), + res.npm().version, + string_bytes!(lockfile), + )?; + sbuf!(lockfile).append(url)? + } + }; } let mut pkg = lockfile::Package { @@ -893,18 +1278,14 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( ..Default::default() }; - if let Some(res_expr) = package_obj.get(b"resolution") { - if !res_expr.is_object() { + if let Some(integrity_expr) = + resolution_expr.as_ref().and_then(|r| r.get(b"integrity")) + { + let Some(integrity_str) = as_string(&integrity_expr) else { return Err(invalid_pnpm_lockfile()); - } - - if let Some(integrity_expr) = res_expr.get(b"integrity") { - let Some(integrity_str) = as_string(&integrity_expr) else { - return Err(invalid_pnpm_lockfile()); - }; + }; - pkg.meta.integrity = Integrity::parse(integrity_str); - } + pkg.meta.integrity = Integrity::parse(integrity_str); } if let Some(os_expr) = package_obj.get(b"os") { @@ -915,8 +1296,13 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( } // TODO: libc - let (off, len) = - parse_append_package_dependencies(lockfile, package_obj, &snapshot_obj, log)?; + let (off, len) = parse_append_package_dependencies( + lockfile, + package_obj, + &snapshot_obj, + log, + &mut snapshot_dep_paths, + )?; pkg.dependencies = ExternalSlice::new(off, len); pkg.resolutions = ExternalSlice::new(off, len); @@ -993,12 +1379,10 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( if strings::has_prefix(version_maybe_alias, b"npm:") { version_maybe_alias = &version_maybe_alias[b"npm:".len()..]; } - let (version, has_alias) = - dependency::split_version_and_maybe_name(version_maybe_alias); - let version_without_suffix = remove_suffix(version); + let reference = remove_suffix(version_maybe_alias); if let Some(maybe_symlink_or_folder_or_workspace_path) = - strings::without_prefix_if_possible_comptime(version_without_suffix, b"link:") + strings::without_prefix_if_possible_comptime(reference, b"link:") { let mut path_buf = bun_paths::AutoAbsPath::init_top_level_dir(); let _ = path_buf.join(&[maybe_symlink_or_folder_or_workspace_path]); // path-buffer overflow unreachable for bounded inputs @@ -1008,17 +1392,15 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( } } - res_buf.clear(); - write!( - &mut res_buf, - "{}@{}", - bstr::BStr::new(has_alias.unwrap_or(dep_name)), - bstr::BStr::new(version_without_suffix) - ) - .map_err(|_| AllocError)?; + write_pnpm_dep_path(&mut res_buf, dep_name, reference)?; let Some(pkg_id) = pkg_map.get(&res_buf) else { - return Err(invalid_pnpm_lockfile()); + return Err(missing_package_entry( + log, + &res_buf, + dep_name, + format_args!("importer '.'"), + )); }; lockfile.buffers.resolutions[dep_id as usize] = *pkg_id; @@ -1058,12 +1440,10 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( if strings::has_prefix(version_maybe_alias, b"npm:") { version_maybe_alias = &version_maybe_alias[b"npm:".len()..]; } - let (version, has_alias) = - dependency::split_version_and_maybe_name(version_maybe_alias); - let version_without_suffix = remove_suffix(version); + let reference = remove_suffix(version_maybe_alias); if let Some(maybe_symlink_or_folder_or_workspace_path) = - strings::without_prefix_if_possible_comptime(version_without_suffix, b"link:") + strings::without_prefix_if_possible_comptime(reference, b"link:") { let mut path_buf = bun_paths::AutoAbsPath::init_top_level_dir(); let _ = path_buf.join(&[workspace_path, maybe_symlink_or_folder_or_workspace_path]); // path-buffer overflow unreachable for bounded inputs @@ -1073,17 +1453,15 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( } } - res_buf.clear(); - write!( - &mut res_buf, - "{}@{}", - bstr::BStr::new(has_alias.unwrap_or(dep_name)), - bstr::BStr::new(version_without_suffix) - ) - .map_err(|_| AllocError)?; + write_pnpm_dep_path(&mut res_buf, dep_name, reference)?; let Some(res_pkg_id) = pkg_map.get(&res_buf) else { - return Err(invalid_pnpm_lockfile()); + return Err(missing_package_entry( + log, + &res_buf, + dep_name, + format_args!("importer '{}'", bstr::BStr::new(workspace_path)), + )); }; lockfile.buffers.resolutions[dep_id as usize] = *res_pkg_id; @@ -1098,41 +1476,44 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( let dep_id: DependencyID = _dep_id; let dep = lockfile.buffers.dependencies[dep_id as usize].clone(); let string_buf = string_bytes!(lockfile); + let dep_name = dep.name.slice(string_buf); let mut version_maybe_alias = dep.version.literal.slice(string_buf); if strings::has_prefix(version_maybe_alias, b"npm:") { version_maybe_alias = &version_maybe_alias[b"npm:".len()..]; } - let (version, has_alias) = - dependency::split_version_and_maybe_name(version_maybe_alias); - let version_without_suffix = remove_suffix(version); - - match dep.version.tag { - dependency::VersionTag::Folder - | dependency::VersionTag::Symlink - | dependency::VersionTag::Workspace => { - let maybe_symlink_or_folder_or_workspace_path = - strings::without_prefix(version_without_suffix, b"link:"); - let mut path_buf = bun_paths::AutoAbsPath::init_top_level_dir(); - let _ = path_buf.join(&[maybe_symlink_or_folder_or_workspace_path]); // path-buffer overflow unreachable for bounded inputs - if let Some(link_pkg_id) = pkg_map.get(path_buf.slice()) { - lockfile.buffers.resolutions[dep_id as usize] = *link_pkg_id; - continue; + let reference = remove_suffix(version_maybe_alias); + + if let Some(dep_path) = snapshot_dep_paths.get(&dep_id) { + res_buf.clear(); + res_buf.extend_from_slice(dep_path); + } else { + match dep.version.tag { + dependency::VersionTag::Folder + | dependency::VersionTag::Symlink + | dependency::VersionTag::Workspace => { + let maybe_symlink_or_folder_or_workspace_path = + strings::without_prefix(reference, b"link:"); + let mut path_buf = bun_paths::AutoAbsPath::init_top_level_dir(); + let _ = path_buf.join(&[maybe_symlink_or_folder_or_workspace_path]); // path-buffer overflow unreachable for bounded inputs + if let Some(link_pkg_id) = pkg_map.get(path_buf.slice()) { + lockfile.buffers.resolutions[dep_id as usize] = *link_pkg_id; + continue; + } } + _ => {} } - _ => {} - } - res_buf.clear(); - write!( - &mut res_buf, - "{}@{}", - bstr::BStr::new(has_alias.unwrap_or_else(|| dep.name.slice(string_buf))), - bstr::BStr::new(version_without_suffix) - ) - .map_err(|_| AllocError)?; + write_pnpm_dep_path(&mut res_buf, dep_name, reference)?; + } let Some(res_pkg_id) = pkg_map.get(&res_buf) else { - return Err(invalid_pnpm_lockfile()); + let pkg_name = lockfile.packages.items_name()[pkg_id as usize].slice(string_buf); + return Err(missing_package_entry( + log, + &res_buf, + dep_name, + format_args!("package '{}'", bstr::BStr::new(pkg_name)), + )); }; lockfile.buffers.resolutions[dep_id as usize] = *res_pkg_id; @@ -1195,13 +1576,41 @@ impl From for MigratePnpmLockfileError { } } +/// dep -> full pnpm dep-path for aliases whose version isn't a registry version (`cfg: hi2@file:x` has no `npm:` spelling) +type SnapshotDepPaths = bun_collections::HashMap>; + +fn append_snapshot_dependency_version( + lockfile: &mut Lockfile, + reference: &[u8], + version_buf: &mut Vec, + aliased_dep_paths: &mut StringArrayHashMap>, + dep_name: &[u8], +) -> Result<(String, Option), AllocError> { + if pnpm_reference_is_dep_path(reference) { + if let Ok((alias_str, version_str)) = dependency::split_name_and_version(reference) { + if !version_str.first().is_some_and(u8::is_ascii_digit) { + aliased_dep_paths.put(dep_name, Box::from(reference))?; + return Ok((sbuf!(lockfile).append(version_str)?, None)); + } + let alias = sbuf!(lockfile).append_external(alias_str)?; + version_buf.clear(); + write!(version_buf, "npm:{}", bstr::BStr::new(reference)).map_err(|_| AllocError)?; + let version = sbuf!(lockfile).append(version_buf.as_slice())?; + return Ok((version, Some(alias))); + } + } + Ok((sbuf!(lockfile).append(reference)?, None)) +} + fn parse_append_package_dependencies( lockfile: &mut Lockfile, package_obj: &Expr, snapshot_obj: &Expr, log: &mut bun_ast::Log, + snapshot_dep_paths: &mut SnapshotDepPaths, ) -> Result<(u32, u32), ParseAppendDependenciesError> { let mut version_buf: Vec = Vec::new(); + let mut aliased_dep_paths: StringArrayHashMap> = StringArrayHashMap::new(); let off = lockfile.buffers.dependencies.len(); @@ -1233,7 +1642,13 @@ fn parse_append_package_dependencies( let version_without_suffix = remove_suffix(version_str); - let version = sbuf!(lockfile).append(version_without_suffix)?; + let (version, alias) = append_snapshot_dependency_version( + lockfile, + version_without_suffix, + &mut version_buf, + &mut aliased_dep_paths, + name_str, + )?; let version_sliced = version.sliced(string_bytes!(lockfile)); let behavior: dependency::Behavior = group_behavior; @@ -1243,8 +1658,8 @@ fn parse_append_package_dependencies( name_hash, behavior, version: match Dependency::parse( - name.value, - name.hash, + alias.map(|a| a.value).unwrap_or(name.value), + alias.map(|a| a.hash).unwrap_or(name.hash), version_sliced.slice, &version_sliced, Some(&mut *log), @@ -1283,23 +1698,13 @@ fn parse_append_package_dependencies( let version_without_suffix = remove_suffix(version_str); - // pnpm-lock.yaml does not prefix aliases with npm: in snapshots - let (_, has_alias) = dependency::split_version_and_maybe_name(version_without_suffix); - - let mut alias: Option = None; - let version: String = if let Some(alias_str) = has_alias { - alias = Some(sbuf!(lockfile).append_external(alias_str)?); - version_buf.clear(); - write!( - &mut version_buf, - "npm:{}", - bstr::BStr::new(version_without_suffix) - ) - .map_err(|_| AllocError)?; - sbuf!(lockfile).append(&version_buf)? - } else { - sbuf!(lockfile).append(version_without_suffix)? - }; + let (version, alias) = append_snapshot_dependency_version( + lockfile, + version_without_suffix, + &mut version_buf, + &mut aliased_dep_paths, + name_str, + )?; let version_sliced = version.sliced(string_bytes!(lockfile)); if let Some(peers) = package_obj.get(b"peerDependencies") { @@ -1406,6 +1811,16 @@ fn parse_append_package_dependencies( lockfile.buffers.dependencies[off..].sort_by(|a, b| Dependency::cmp(bytes, a, b)); } + if aliased_dep_paths.count() > 0 { + let bytes = lockfile.buffers.string_bytes.as_slice(); + for (i, dep) in lockfile.buffers.dependencies[off..end].iter().enumerate() { + if let Some(dep_path) = aliased_dep_paths.get(dep.name.slice(bytes)) { + let dep_id = u32::try_from(off + i).expect("int cast"); + snapshot_dep_paths.put(dep_id, dep_path.clone())?; + } + } + } + Ok(( u32::try_from(off).expect("int cast"), u32::try_from(end - off).expect("int cast"), @@ -1474,6 +1889,19 @@ fn parse_append_importer_dependencies( return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); }; + if strings::has_prefix_comptime(version_str, b"runtime:") { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml runtime dependency '{}@{}' is not migrated", + bstr::BStr::new(name_str), + bstr::BStr::new(version_str) + ), + ); + continue; + } + let entry = importer_versions.get_or_put(name_str)?; if entry.found_existing { continue; @@ -1485,7 +1913,11 @@ fn parse_append_importer_dependencies( }; if strings::has_prefix(specifier_str, b"catalog:") { - let catalog_group_name_str = &specifier_str[b"catalog:".len()..]; + let mut catalog_group_name_str = + specifier_str[b"catalog:".len()..].trim_ascii(); + if catalog_group_name_str == b"default" { + catalog_group_name_str = b""; + } let catalog_group_name = sbuf!(lockfile).append(catalog_group_name_str)?; // `CatalogMap::get` needs both `&mut self.catalogs` and // `&self`; temporarily move catalogs out so the disjoint @@ -1500,7 +1932,7 @@ fn parse_append_importer_dependencies( bun_ast::Loc::EMPTY, format_args!( "pnpm-lock.yaml catalog '{}' missing entry for dependency '{}'", - bstr::BStr::new(catalog_group_name_str), + bstr::BStr::new(specifier_str[b"catalog:".len()..].trim_ascii()), bstr::BStr::new(name_str) ), ); @@ -1601,6 +2033,37 @@ fn parse_append_importer_dependencies( )) } +/// bun.lock keys patches by `name@version`; pnpm also allows a bare `name` key. +fn rewrite_bare_patch_keys( + obj: &mut Expr, + patches: &StringArrayHashMap>, +) -> Result<(), AllocError> { + if patches.count() == 0 { + return Ok(()); + } + let mut join_buf: Vec = Vec::new(); + for prop in e_object_mut(obj).properties.slice_mut() { + let Some(key_str) = as_string(prop.key.as_ref().expect("infallible: prop has key")) else { + continue; + }; + let Some(res_str) = patches.get(key_str) else { + continue; + }; + join_buf.clear(); + write!( + &mut join_buf, + "{}@{}", + bstr::BStr::new(key_str), + bstr::BStr::new(&**res_str) + ) + .map_err(|_| AllocError)?; + // Interned into the DATA_STORE backing the cached package.json Expr tree, which outlives this fn. + let interned: &[u8] = js_ast::data_store_dupe_str(join_buf.as_slice()); + prop.key = Some(Expr::init(E::EString::init(interned), bun_ast::Loc::EMPTY)); + } + Ok(()) +} + /// Updates package.json with workspace and catalog information after migration fn update_package_json_after_migration( manager: &mut PackageManager, @@ -1668,8 +2131,9 @@ fn update_package_json_after_migration( } } - if let Some(patched_field) = pnpm_obj.get(b"patchedDependencies") { + if let Some(mut patched_field) = pnpm_obj.get(b"patchedDependencies") { if patched_field.is_object() { + rewrite_bare_patch_keys(&mut patched_field, patches)?; if let Some(mut existing_prop) = json.as_property(b"patchedDependencies") { if existing_prop.expr.is_object() { let existing_patches = e_object_mut(&mut existing_prop.expr); @@ -1999,36 +2463,8 @@ fn update_package_json_after_migration( // Handle patchedDependencies from pnpm-workspace.yaml if let Some(ws_patched) = &mut workspace_patched_deps_obj { - let mut join_buf: Vec = Vec::new(); - if ws_patched.is_object() { - let props_len = e_object(ws_patched).properties.len_u32() as usize; - for prop_i in 0..props_len { - // convert keys to expected "name@version" instead of only "name" - let prop = &mut e_object_mut(ws_patched).properties.slice_mut()[prop_i]; - let Some(key_str) = as_string(prop.key.as_ref().expect("infallible: prop has key")) - else { - continue; - }; - let Some(res_str) = patches.get(key_str) else { - continue; - }; - join_buf.clear(); - write!( - &mut join_buf, - "{}@{}", - bstr::BStr::new(key_str), - bstr::BStr::new(&**res_str) - ) - .map_err(|_| AllocError)?; - // The rewritten key ends up inside - // `root_pkg_json.root` (Store-backed, cached in - // `workspace_package_json_cache`), so it must outlive this - // function — intern into the thread-local `DATA_STORE` that - // backs the surrounding `Expr` nodes, NOT the local `bump`. - let interned: &[u8] = js_ast::data_store_dupe_str(join_buf.as_slice()); - prop.key = Some(Expr::init(E::EString::init(interned), bun_ast::Loc::EMPTY)); - } + rewrite_bare_patch_keys(ws_patched, patches)?; if let Some(mut existing_prop) = json.as_property(b"patchedDependencies") { if existing_prop.expr.is_object() { let existing_patches = e_object_mut(&mut existing_prop.expr); diff --git a/src/install/prune.rs b/src/install/prune.rs new file mode 100644 index 000000000000..440bacc3f6f5 --- /dev/null +++ b/src/install/prune.rs @@ -0,0 +1,943 @@ +use std::io::Write as _; + +use bstr::BStr; +use bun_core::{Global, Output, ZStr, strings}; +use bun_install_types::NodeLinker::NodeLinker; +use bun_paths::SEP; +use bun_sys::{self as sys, Dir, E, EntryKind}; + +use crate::config_version::ConfigVersion; +use crate::lockfile::package::PackageColumns as _; +use crate::lockfile::tree::is_filtered_dependency_or_workspace; +use crate::lockfile::{LoadResult, Lockfile, tree}; +use crate::package_manager::Options::LogLevel; +use crate::{PackageID, PackageManager, ResolutionTag, invalid_package_id}; + +const STORE_DIR: &[u8] = b"node_modules/.bun"; + +#[derive(Clone, Copy, PartialEq, Eq)] +enum Layout { + Hoisted, + Isolated, +} + +#[derive(Clone, Copy, PartialEq, Eq)] +enum FolderKind { + NodeModules, + Scope { parent: usize }, + Store, +} + +struct Folder { + path: Box<[u8]>, + kind: FolderKind, + dir: Option, + direct: Option>>, + touched: bool, +} + +struct Entry<'a> { + dir: &'a Dir, + alias: &'a [u8], + name: &'a [u8], + kind: EntryKind, +} + +struct Removal { + folder: usize, + name: Box<[u8]>, + kind: EntryKind, + display: Box<[u8]>, +} + +#[derive(Default)] +struct Plan { + folders: Vec, + removals: Vec, +} + +impl Plan { + fn push_folder(&mut self, path: &[u8], kind: FolderKind) -> usize { + self.folders.push(Folder { + path: path.into(), + kind, + dir: None, + direct: None, + touched: false, + }); + self.folders.len() - 1 + } + + fn remove(&mut self, folder: usize, name: &[u8], kind: EntryKind) { + let display = join(&self.folders[folder].path, name); + self.removals.push(Removal { + folder, + name: name.into(), + kind, + display, + }); + self.folders[folder].touched = true; + if let FolderKind::Scope { parent } = self.folders[folder].kind { + self.folders[parent].touched = true; + } + } + + fn retain(&mut self, folder: usize, dir: Dir) { + if self.folders[folder].touched { + self.folders[folder].dir = Some(dir); + } + } + + fn dir(&self, folder: usize) -> &Dir { + self.folders[folder] + .dir + .as_ref() + .expect("touched folders retain their Dir") + } +} + +fn join(dir: &[u8], name: &[u8]) -> Box<[u8]> { + let mut out = Vec::with_capacity(dir.len() + 1 + name.len()); + out.extend_from_slice(dir); + out.push(SEP); + out.extend_from_slice(name); + out.into_boxed_slice() +} + +fn zname(name: &[u8]) -> Vec { + let mut z = Vec::with_capacity(name.len() + 1); + z.extend_from_slice(name); + z.push(0); + z +} + +fn contains(sorted: &[Box<[u8]>], name: &[u8]) -> bool { + sorted + .binary_search_by(|item| item.as_ref().cmp(name)) + .is_ok() +} + +fn plural(n: usize) -> &'static str { + if n == 1 { "" } else { "s" } +} + +pub fn prune(manager: &mut PackageManager) -> crate::Result<()> { + let quiet = manager.options.log_level == LogLevel::Silent; + let dry_run = manager.options.dry_run; + + let load = manager.load_lockfile_from_cwd::(); + let loaded = match &load { + LoadResult::NotFound => Err(None), + LoadResult::Err(cause) => Err(Some(cause.value.name())), + LoadResult::Ok(_) => Ok(load.choose_config_version().0), + }; + drop(load); + let config_version = match loaded { + Ok(config_version) => config_version, + Err(None) => { + if !quiet { + Output::err_generic("missing lockfile, nothing to prune", ()); + bun_core::note!("run 'bun install' first"); + } + Global::exit(1); + } + Err(Some(name)) => { + if !quiet { + Output::err_generic("failed to load lockfile: {s}", (name,)); + if manager.log_mut().has_errors() { + let _ = manager + .log_mut() + .print(core::ptr::from_mut(Output::error_writer())); + } + } + Global::exit(1); + } + }; + + let store_present = match Dir::open(b"node_modules") { + Ok(node_modules) => lstat_kind(&node_modules, b".bun") == EntryKind::Directory, + Err(err) if err.get_errno() == E::ENOENT => { + if !quiet { + bun_core::prettyln!("Nothing to prune."); + Output::flush(); + } + return Ok(()); + } + Err(err) => { + Output::err(err, "failed to open node_modules", ()); + Global::exit(1); + } + }; + + let layout = match manager.options.node_linker { + NodeLinker::Hoisted => Layout::Hoisted, + NodeLinker::Isolated => Layout::Isolated, + NodeLinker::Auto => match config_version { + ConfigVersion::V0 => Layout::Hoisted, + ConfigVersion::V1 => { + if manager.lockfile.workspace_paths.len() > 0 { + Layout::Isolated + } else { + Layout::Hoisted + } + } + }, + }; + + let workspace_names = collect_workspace_names(&manager.lockfile); + + let mut plan = Plan::default(); + match layout { + Layout::Hoisted => plan_hoisted(manager, &workspace_names, &mut plan), + Layout::Isolated => plan_isolated(manager, &workspace_names, &mut plan), + } + + if layout_mismatch(&plan, layout, store_present) { + if !quiet { + let (configured, actual) = match layout { + Layout::Hoisted => ("hoisted", "isolated"), + Layout::Isolated => ("isolated", "hoisted"), + }; + Output::err_generic( + "node_modules was installed with the {s} linker, but bun prune would use the {s} linker", + (actual, configured), + ); + bun_core::note!( + "run 'bun prune --linker {}' to prune it as-is, or 'bun install' to reinstall with the {} linker", + actual, + configured + ); + } + Global::exit(1); + } + + plan.removals + .sort_unstable_by(|a, b| a.display.cmp(&b.display)); + + let n = plan.removals.len(); + if n == 0 { + if !quiet { + bun_core::prettyln!("Nothing to prune."); + Output::flush(); + } + return Ok(()); + } + + if dry_run { + if !quiet { + for removal in &plan.removals { + bun_core::prettyln!("- {}", BStr::new(&removal.display)); + } + bun_core::prettyln!("Would remove {} package{}", n, plural(n)); + Output::flush(); + } + return Ok(()); + } + + let failed = execute(&plan, quiet); + housekeeping(&plan, layout, manager); + + let removed = n - failed; + if !quiet { + bun_core::prettyln!("Removed {} package{}", removed, plural(removed)); + Output::flush(); + } + if failed > 0 { + Global::exit(1); + } + Ok(()) +} + +fn collect_workspace_names(lockfile: &Lockfile) -> Vec> { + let buf = lockfile.buffers.string_bytes.as_slice(); + let names = lockfile.packages.items_name(); + let pkg_res = lockfile.packages.items_resolution(); + let mut out: Vec> = pkg_res + .iter() + .zip(names) + .filter(|(res, _)| res.tag == ResolutionTag::Workspace) + .map(|(_, name)| name.slice(buf).into()) + .collect(); + out.sort_unstable(); + out.dedup(); + out +} + +fn hoist_filtered(manager: &mut PackageManager) { + let pm: *mut PackageManager = manager; + // SAFETY: same split as `PackageManager::load_lockfile_from_cwd` — `lockfile` is its own `Box` allocation and the Filter builder only reads `manager.options`/`subcommand`/`summary`. + let result = unsafe { + let lf: *mut Lockfile = &raw mut *(*pm).lockfile; + let log: *mut bun_ast::Log = (*pm).log; + (*lf).hoist::<{ tree::BuilderMethod::Filter }>(&mut *log, Some(&*pm), true, &[], None) + }; + if result.is_err() { + manager.crash(); + } +} + +fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], plan: &mut Plan) { + let keep_workspaces = |entry: &Entry| contains(workspace_names, entry.alias); + if manager.lockfile.packages.len() == 0 { + if let Ok(dir) = Dir::open(b"node_modules") { + scan_folder(dir, b"node_modules", false, &keep_workspaces, plan); + } + return; + } + + hoist_filtered(manager); + + let lockfile: &Lockfile = &manager.lockfile; + let buf = lockfile.buffers.string_bytes.as_slice(); + let deps = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + let trees = lockfile.buffers.trees.as_slice(); + let pkg_res = lockfile.packages.items_resolution(); + let dep_slices = lockfile.packages.items_dependencies(); + + let has_bundled_deps = |pkg_id: PackageID| { + let slice = dep_slices[pkg_id as usize]; + (slice.begin() as usize..slice.end() as usize).any(|i| deps[i].behavior.is_bundled()) + }; + + let mut nested_trees: Vec<(tree::Id, &[u8])> = trees + .iter() + .skip(1) + .map(|t| (t.parent, t.folder_name(deps, buf))) + .collect(); + nested_trees.sort_unstable(); + + let mut visited = vec![false; pkg_res.len()]; + let mut expected: Vec<(&[u8], PackageID)> = Vec::new(); + let mut it = tree::Iterator::<{ tree::IteratorPathStyle::NodeModules }>::init(lockfile); + while let Some(folder) = it.next(None) { + let owner: PackageID = match trees[folder.tree_id as usize].dependency_id { + tree::ROOT_DEP_ID => 0, + dep_id => resolutions[dep_id as usize], + }; + if owner != invalid_package_id && (owner as usize) < pkg_res.len() { + visited[owner as usize] = true; + let tag = pkg_res[owner as usize].tag; + if tag != ResolutionTag::Root + && tag != ResolutionTag::Workspace + && has_bundled_deps(owner) + { + continue; + } + } + + expected.clear(); + expected.extend(folder.dependencies.iter().map(|&dep_id| { + ( + deps[dep_id as usize].name.slice(buf), + resolutions[dep_id as usize], + ) + })); + expected.sort_unstable(); + expected.dedup_by_key(|(alias, _)| *alias); + + let Some(dir) = open_tree_folder(trees, deps, buf, folder.tree_id, workspace_names) else { + continue; + }; + let folder_path = folder.relative_path.as_bytes(); + + for &(alias, pkg_id) in &expected { + if (pkg_id as usize) >= pkg_res.len() + || nested_trees.binary_search(&(folder.tree_id, alias)).is_ok() + { + continue; + } + let extracted = matches!( + pkg_res[pkg_id as usize].tag, + ResolutionTag::Npm + | ResolutionTag::LocalTarball + | ResolutionTag::RemoteTarball + | ResolutionTag::Git + | ResolutionTag::Github + ); + if !extracted || has_bundled_deps(pkg_id) { + continue; + } + let Some(nested) = descend(&dir, alias, false) + .and_then(|package| open_real_subdir(&package, b"node_modules")) + else { + continue; + }; + let nested_path = join(&join(folder_path, alias), b"node_modules"); + scan_folder(nested, &nested_path, false, &keep_workspaces, plan); + } + + scan_folder( + dir, + folder_path, + false, + &|entry| { + expected + .binary_search_by(|(name, _)| (*name).cmp(entry.alias)) + .is_ok() + || contains(workspace_names, entry.alias) + }, + plan, + ); + } + + if !visited[0] { + if let Ok(dir) = Dir::open(b"node_modules") { + scan_folder(dir, b"node_modules", false, &keep_workspaces, plan); + } + } + for (pkg_id, res) in pkg_res.iter().enumerate() { + if visited[pkg_id] || res.tag != ResolutionTag::Workspace { + continue; + } + let path = strings::without_trailing_slash(res.workspace().slice(buf)); + if path.is_empty() { + continue; + } + let folder_path = join(path, b"node_modules"); + if let Ok(dir) = Dir::open(&folder_path) { + scan_folder(dir, &folder_path, false, &keep_workspaces, plan); + } + } +} + +fn open_tree_folder( + trees: &[tree::Tree], + deps: &[crate::Dependency], + buf: &[u8], + tree_id: tree::Id, + workspace_names: &[Box<[u8]>], +) -> Option { + let mut chain: Vec = Vec::new(); + let mut id = tree_id; + while id != 0 && (id as usize) < trees.len() { + chain.push(id); + id = trees[id as usize].parent; + } + let mut dir = Dir::open(b"node_modules").ok()?; + while let Some(id) = chain.pop() { + let alias = trees[id as usize].folder_name(deps, buf); + let package = descend(&dir, alias, contains(workspace_names, alias))?; + dir = open_real_subdir(&package, b"node_modules")?; + } + Some(dir) +} + +fn descend(dir: &Dir, alias: &[u8], follow: bool) -> Option { + let (scope, name) = match strings::split_once_char(alias, b'/') { + Some(split) if alias.first() == Some(&b'@') => (Some(split.0), split.1), + _ => (None, alias), + }; + let scope_dir = match scope { + Some(scope) => Some(open_real_subdir(dir, scope)?), + None => None, + }; + let parent = scope_dir.as_ref().unwrap_or(dir); + if follow { + parent.open_at(name).ok() + } else { + open_real_subdir(parent, name) + } +} + +fn open_real_subdir(dir: &Dir, name: &[u8]) -> Option { + if lstat_kind(dir, name) != EntryKind::Directory { + return None; + } + dir.open_at(name).ok() +} + +fn lstat_kind(dir: &Dir, name: &[u8]) -> EntryKind { + match sys::lstatat(dir.fd(), ZStr::from_slice_with_nul(&zname(name))) { + Ok(st) => sys::kind_from_mode(st.st_mode as sys::Mode), + Err(_) => EntryKind::Unknown, + } +} + +fn entry_kind(dir: &Dir, name: &[u8], kind: EntryKind) -> EntryKind { + if kind != EntryKind::Unknown { + return kind; + } + lstat_kind(dir, name) +} + +fn read_entries(dir: &Dir) -> Vec<(Box<[u8]>, EntryKind)> { + let mut out = Vec::new(); + let mut iter = sys::iterate_dir(dir.fd()); + while let Ok(Some(entry)) = iter.next() { + let name = entry.name.slice_u8(); + if name.first() == Some(&b'.') { + continue; + } + let kind = entry_kind(dir, name, entry.kind); + if kind == EntryKind::Directory || kind == EntryKind::SymLink { + out.push((name.into(), kind)); + } + } + out +} + +fn scan_folder( + dir: Dir, + folder_path: &[u8], + touched: bool, + keep: &dyn Fn(&Entry) -> bool, + plan: &mut Plan, +) -> usize { + let folder_idx = plan.push_folder(folder_path, FolderKind::NodeModules); + plan.folders[folder_idx].touched = touched; + let mut alias = Vec::new(); + for (name, kind) in read_entries(&dir) { + if name.first() == Some(&b'@') && kind == EntryKind::Directory { + let Ok(scope_dir) = dir.open_at(&name) else { + continue; + }; + let scope_path = join(folder_path, &name); + let scope_idx = plan.push_folder(&scope_path, FolderKind::Scope { parent: folder_idx }); + for (inner, inner_kind) in read_entries(&scope_dir) { + alias.clear(); + alias.extend_from_slice(&name); + alias.push(b'/'); + alias.extend_from_slice(&inner); + let entry = Entry { + dir: &scope_dir, + alias: &alias, + name: &inner, + kind: inner_kind, + }; + if !keep(&entry) { + plan.remove(scope_idx, &inner, inner_kind); + } + } + plan.retain(scope_idx, scope_dir); + continue; + } + let entry = Entry { + dir: &dir, + alias: &name, + name: &name, + kind, + }; + if !keep(&entry) { + plan.remove(folder_idx, &name, kind); + } + } + plan.retain(folder_idx, dir); + folder_idx +} + +fn wanted_packages(manager: &PackageManager) -> Vec { + let lockfile: &Lockfile = &manager.lockfile; + let resolutions = lockfile.buffers.resolutions.as_slice(); + let dep_slices = lockfile.packages.items_dependencies(); + let mut wanted = vec![false; dep_slices.len()]; + if wanted.is_empty() { + return wanted; + } + wanted[0] = true; + let mut worklist: Vec = vec![0]; + while let Some(parent) = worklist.pop() { + let slice = dep_slices[parent as usize]; + for dep_id in slice.begin()..slice.end() { + if is_filtered_dependency_or_workspace( + dep_id, + parent, + &[], + true, + manager, + lockfile, + resolutions, + ) { + continue; + } + let target = resolutions[dep_id as usize]; + if target == invalid_package_id + || (target as usize) >= wanted.len() + || wanted[target as usize] + { + continue; + } + wanted[target as usize] = true; + worklist.push(target); + } + } + wanted +} + +fn store_keys(lockfile: &Lockfile, wanted: &[bool]) -> Vec> { + let buf = lockfile.buffers.string_bytes.as_slice(); + let names = lockfile.packages.items_name(); + let pkg_res = lockfile.packages.items_resolution(); + let mut keys: Vec> = Vec::new(); + let mut key: Vec = Vec::new(); + for pkg_id in 0..wanted.len() { + if !wanted[pkg_id] || pkg_res[pkg_id].tag == ResolutionTag::Workspace { + continue; + } + let name = names[pkg_id]; + let res = &pkg_res[pkg_id]; + key.clear(); + let written = match res.tag { + ResolutionTag::Root => { + if name.is_empty() { + key.extend_from_slice(bun_paths::basename( + crate::bun_fs::FileSystem::instance().top_level_dir(), + )); + Ok(()) + } else { + write!(key, "{}@root", name.fmt_store_path(buf)) + } + } + ResolutionTag::Folder => write!( + key, + "{}@file+{}", + name.fmt_store_path(buf), + res.folder().fmt_store_path(buf) + ), + _ => write!( + key, + "{}@{}", + name.fmt_store_path(buf), + res.fmt_store_path(buf) + ), + }; + if written.is_ok() { + keys.push(key.as_slice().into()); + } + } + keys.sort_unstable(); + keys.dedup(); + keys +} + +fn strip_peer_hash(name: &[u8]) -> Option<&[u8]> { + const SUFFIX_LEN: usize = 1 + 16; + if name.len() <= SUFFIX_LEN { + return None; + } + let (base, suffix) = name.split_at(name.len() - SUFFIX_LEN); + (suffix[0] == b'+' && suffix[1..].iter().all(u8::is_ascii_hexdigit)).then_some(base) +} + +fn direct_aliases(manager: &PackageManager, pkg_id: PackageID, filtered: bool) -> Vec> { + let lockfile: &Lockfile = &manager.lockfile; + let buf = lockfile.buffers.string_bytes.as_slice(); + let deps = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + let slice = lockfile.packages.items_dependencies()[pkg_id as usize]; + let mut direct: Vec> = Vec::new(); + for dep_id in slice.begin()..slice.end() { + if filtered { + if is_filtered_dependency_or_workspace( + dep_id, + pkg_id, + &[], + true, + manager, + lockfile, + resolutions, + ) { + continue; + } + let target = resolutions[dep_id as usize]; + if target == invalid_package_id || (target as usize) >= lockfile.packages.len() { + continue; + } + } + direct.push(deps[dep_id as usize].name.slice(buf).into()); + } + direct.sort_unstable(); + direct.dedup(); + direct +} + +fn public_hoist_matches(manager: &PackageManager, alias: &[u8]) -> bool { + manager + .options + .public_hoist_pattern + .as_ref() + .is_some_and(|pattern| pattern.is_match(alias)) +} + +fn plan_isolated(manager: &PackageManager, workspace_names: &[Box<[u8]>], plan: &mut Plan) { + let wanted = wanted_packages(manager); + let keys = store_keys(&manager.lockfile, &wanted); + let keep_store_entry = |name: &[u8]| { + contains(&keys, name) || strip_peer_hash(name).is_some_and(|base| contains(&keys, base)) + }; + + let mut removed_store: Vec> = Vec::new(); + if let Ok(store) = Dir::open(STORE_DIR) { + let store_idx = plan.push_folder(STORE_DIR, FolderKind::Store); + for (name, kind) in read_entries(&store) { + if &*name == b"node_modules" || keep_store_entry(&name) { + continue; + } + plan.remove(store_idx, &name, kind); + removed_store.push(name); + } + plan.retain(store_idx, store); + } + removed_store.sort_unstable(); + let store_touched = !removed_store.is_empty(); + + let lockfile: &Lockfile = &manager.lockfile; + let buf = lockfile.buffers.string_bytes.as_slice(); + let pkg_res = lockfile.packages.items_resolution(); + for pkg_id in 0..lockfile.packages.len() { + let res = &pkg_res[pkg_id]; + let folder_path: Box<[u8]> = match res.tag { + ResolutionTag::Root if pkg_id == 0 => b"node_modules".as_slice().into(), + ResolutionTag::Workspace => { + let path = strings::without_trailing_slash(res.workspace().slice(buf)); + if path.is_empty() { + continue; + } + join(path, b"node_modules") + } + _ => continue, + }; + let Ok(dir) = Dir::open(&folder_path) else { + continue; + }; + let direct = direct_aliases(manager, pkg_id as PackageID, true); + let declared = direct_aliases(manager, pkg_id as PackageID, false); + let folder_idx = scan_folder( + dir, + &folder_path, + store_touched, + &|entry| { + let known = match entry.kind { + EntryKind::SymLink => { + contains(&declared, entry.alias) + || store_link_target(entry.dir, entry.name) + .is_some_and(|target| contains(&removed_store, &target)) + } + _ => contains(&direct, entry.alias), + }; + known + || contains(workspace_names, entry.alias) + || public_hoist_matches(manager, entry.alias) + }, + plan, + ); + plan.folders[folder_idx].direct = Some(direct); + } +} + +fn layout_mismatch(plan: &Plan, layout: Layout, store_present: bool) -> bool { + match layout { + Layout::Isolated => { + !store_present && plan.removals.iter().any(|r| r.kind == EntryKind::Directory) + } + Layout::Hoisted => plan.removals.iter().any(|r| { + r.kind == EntryKind::SymLink && store_link_target(plan.dir(r.folder), &r.name).is_some() + }), + } +} + +fn store_link_target(dir: &Dir, name: &[u8]) -> Option> { + let z = zname(name); + let mut buf = bun_paths::path_buffer_pool::get(); + let len = sys::readlinkat(dir.fd(), ZStr::from_slice_with_nul(&z), buf.as_mut_slice()).ok()?; + let mut components = strings::tokenize_any(&buf.as_slice()[..len], b"/\\"); + while let Some(component) = components.next() { + if component == b".bun" { + return components + .next() + .filter(|entry| strings::contains_char(entry, b'@')) + .map(Into::into); + } + } + None +} + +fn remove_link(dir: &Dir, name: &[u8]) -> sys::Maybe<()> { + let z = zname(name); + let z = ZStr::from_slice_with_nul(&z); + let result = sys::unlinkat(dir.fd(), z); + #[cfg(windows)] + let result = result.or_else(|_| sys::rmdirat(dir.fd(), z)); + result +} + +fn execute(plan: &Plan, quiet: bool) -> usize { + let mut failed = 0usize; + let mut removed_from = vec![false; plan.folders.len()]; + + for removal in &plan.removals { + let dir = plan.dir(removal.folder); + let result = match removal.kind { + EntryKind::SymLink => remove_link(dir, &removal.name), + _ => dir.delete_tree(&removal.name), + }; + match result { + Ok(()) => {} + Err(err) if err.get_errno() == E::ENOENT => {} + Err(err) => { + Output::err(err, "failed to remove {s}", (BStr::new(&removal.display),)); + failed += 1; + continue; + } + } + if !quiet { + bun_core::prettyln!("- {}", BStr::new(&removal.display)); + } + removed_from[removal.folder] = true; + } + + for (idx, folder) in plan.folders.iter().enumerate() { + let FolderKind::Scope { parent } = folder.kind else { + continue; + }; + if !removed_from[idx] { + continue; + } + let scope_name = &folder.path[plan.folders[parent].path.len() + 1..]; + rmdir(plan.dir(parent), scope_name); + } + + failed +} + +fn rmdir(dir: &Dir, name: &[u8]) { + let z = zname(name); + let _ = sys::rmdirat(dir.fd(), ZStr::from_slice_with_nul(&z)); +} + +fn is_dangling(dir: &Dir, name: &[u8]) -> bool { + let z = zname(name); + match sys::exists_at_type(dir.fd(), ZStr::from_slice_with_nul(&z)) { + Ok(_) => false, + Err(err) => matches!(err.get_errno(), E::ENOENT | E::ENOTDIR), + } +} + +fn unlink_links(dir: &Dir, should_unlink: &dyn Fn(&Dir, &[u8], &[u8]) -> bool) { + let mut alias = Vec::new(); + for (name, kind) in read_entries(dir) { + match kind { + EntryKind::SymLink => { + if should_unlink(dir, &name, &name) { + let _ = remove_link(dir, &name); + } + } + EntryKind::Directory if name.first() == Some(&b'@') => { + let Ok(scope_dir) = dir.open_at(&name) else { + continue; + }; + let mut unlinked = false; + for (inner, inner_kind) in read_entries(&scope_dir) { + if inner_kind != EntryKind::SymLink { + continue; + } + alias.clear(); + alias.extend_from_slice(&name); + alias.push(b'/'); + alias.extend_from_slice(&inner); + if should_unlink(&scope_dir, &alias, &inner) { + unlinked |= remove_link(&scope_dir, &inner).is_ok(); + } + } + drop(scope_dir); + if unlinked { + rmdir(dir, &name); + } + } + _ => {} + } + } +} + +#[cfg(not(windows))] +fn prune_bins(dir: &Dir) { + let Some(bin) = open_real_subdir(dir, b".bin") else { + return; + }; + let mut dangling: Vec> = Vec::new(); + let mut iter = sys::iterate_dir(bin.fd()); + while let Ok(Some(entry)) = iter.next() { + let name = entry.name.slice_u8(); + if entry_kind(&bin, name, entry.kind) == EntryKind::SymLink && is_dangling(&bin, name) { + dangling.push(name.into()); + } + } + drop(iter); + for name in &dangling { + let _ = remove_link(&bin, name); + } +} + +// `.bunx` layout: windows-shim/BinLinkingShim.rs (target path is relative to this node_modules folder). +#[cfg(windows)] +fn prune_bins(dir: &Dir) { + let Some(bin) = open_real_subdir(dir, b".bin") else { + return; + }; + let mut shims: Vec> = Vec::new(); + let mut iter = sys::iterate_dir(bin.fd()); + while let Ok(Some(entry)) = iter.next() { + let name = entry.name.slice_u8(); + if let Some(stem) = name.strip_suffix(b".bunx") { + shims.push(stem.into()); + } + } + drop(iter); + let mut file_name: Vec = Vec::new(); + for stem in &shims { + file_name.clear(); + file_name.extend_from_slice(stem); + file_name.extend_from_slice(b".bunx"); + let Ok(shim) = sys::File::read_from(bin.fd(), &file_name) else { + continue; + }; + let Some(target_len) = shim.chunks_exact(2).position(|unit| unit == [b'"', 0]) else { + continue; + }; + let target = strings::to_utf8_alloc_from_le_bytes(&shim[..target_len * 2]); + if !is_dangling(dir, &target) { + continue; + } + let _ = remove_link(&bin, &file_name); + file_name.truncate(stem.len()); + file_name.extend_from_slice(b".exe"); + let _ = remove_link(&bin, &file_name); + } +} + +fn housekeeping(plan: &Plan, layout: Layout, manager: &PackageManager) { + let workspace_names = collect_workspace_names(&manager.lockfile); + for (idx, folder) in plan.folders.iter().enumerate() { + if !folder.touched { + continue; + } + match folder.kind { + FolderKind::Scope { .. } => {} + FolderKind::Store => { + if layout != Layout::Isolated { + continue; + } + let Some(hidden) = open_real_subdir(plan.dir(idx), b"node_modules") else { + continue; + }; + unlink_links(&hidden, &|dir, _, name| is_dangling(dir, name)); + } + FolderKind::NodeModules => { + if let (Layout::Isolated, Some(direct)) = (layout, &folder.direct) { + if let Ok(dir) = Dir::open(&folder.path) { + unlink_links(&dir, &|dir, alias, name| { + if contains(direct, alias) || contains(&workspace_names, alias) { + return false; + } + if public_hoist_matches(manager, alias) { + return is_dangling(dir, name); + } + true + }); + } + } + prune_bins(plan.dir(idx)); + } + } + } +} diff --git a/src/install/resolution.rs b/src/install/resolution.rs index c3041672be89..5998e39bc87b 100644 --- a/src/install/resolution.rs +++ b/src/install/resolution.rs @@ -461,10 +461,7 @@ impl ResolutionType { } } - pub(crate) fn fmt_store_path<'a>( - &'a self, - string_buf: &'a [u8], - ) -> StorePathFormatter<'a, SemverInt> { + pub fn fmt_store_path<'a>(&'a self, string_buf: &'a [u8]) -> StorePathFormatter<'a, SemverInt> { StorePathFormatter { res: self, string_buf, @@ -929,7 +926,7 @@ impl Tag { pub(crate) const Uninitialized: Tag = Tag(0); pub const Root: Tag = Tag(1); pub const Npm: Tag = Tag(2); - pub(crate) const Folder: Tag = Tag(4); + pub const Folder: Tag = Tag(4); pub(crate) const LocalTarball: Tag = Tag(8); @@ -937,7 +934,7 @@ impl Tag { pub(crate) const Git: Tag = Tag(32); - pub(crate) const Symlink: Tag = Tag(64); + pub const Symlink: Tag = Tag(64); pub const Workspace: Tag = Tag(72); diff --git a/src/options_types/command_tag.rs b/src/options_types/command_tag.rs index 7cdea42b05d5..e424d6925f11 100644 --- a/src/options_types/command_tag.rs +++ b/src/options_types/command_tag.rs @@ -42,6 +42,8 @@ pub enum Tag { PublishCommand, AuditCommand, WhyCommand, + DedupeCommand, + PruneCommand, FuzzilliCommand, } @@ -82,6 +84,8 @@ impl Tag { Tag::PublishCommand => b'k', Tag::AuditCommand => b'A', Tag::WhyCommand => b'W', + Tag::DedupeCommand => b'd', + Tag::PruneCommand => b'N', Tag::FuzzilliCommand => b'F', } } @@ -100,6 +104,8 @@ impl Tag { | Tag::OutdatedCommand | Tag::PublishCommand | Tag::AuditCommand + | Tag::DedupeCommand + | Tag::PruneCommand ) } @@ -119,6 +125,8 @@ impl Tag { | Tag::OutdatedCommand | Tag::PublishCommand | Tag::AuditCommand + | Tag::DedupeCommand + | Tag::PruneCommand ) } @@ -161,6 +169,8 @@ impl Tag { Self::PublishCommand, Self::AuditCommand, Self::WhyCommand, + Self::DedupeCommand, + Self::PruneCommand, Self::FuzzilliCommand, ]; @@ -218,6 +228,8 @@ pub static LOADS_CONFIG: TagTable = TagTable({ a[Tag::UpdateInteractiveCommand as usize] = true; a[Tag::PublishCommand as usize] = true; a[Tag::AuditCommand as usize] = true; + a[Tag::DedupeCommand as usize] = true; + a[Tag::PruneCommand as usize] = true; a }); @@ -237,6 +249,8 @@ pub static ALWAYS_LOADS_CONFIG: TagTable = TagTable({ a[Tag::UpdateInteractiveCommand as usize] = true; a[Tag::PublishCommand as usize] = true; a[Tag::AuditCommand as usize] = true; + a[Tag::DedupeCommand as usize] = true; + a[Tag::PruneCommand as usize] = true; a }); @@ -244,6 +258,8 @@ pub static USES_GLOBAL_OPTIONS: TagTable = TagTable({ let mut a = [true; Tag::COUNT]; a[Tag::AddCommand as usize] = false; a[Tag::AuditCommand as usize] = false; + a[Tag::DedupeCommand as usize] = false; + a[Tag::PruneCommand as usize] = false; a[Tag::BunxCommand as usize] = false; a[Tag::CreateCommand as usize] = false; a[Tag::InfoCommand as usize] = false; diff --git a/src/runtime/cli/audit_command.rs b/src/runtime/cli/audit_command.rs index b24816aff445..d39a45e2974f 100644 --- a/src/runtime/cli/audit_command.rs +++ b/src/runtime/cli/audit_command.rs @@ -6,15 +6,18 @@ use bun_collections::{StringArrayHashMap, StringHashMap}; use bun_core::{Global, Output, pretty, prettyln}; use bun_core::{MutableString, strings}; use bun_http::{self as http, HeaderBuilder}; +use bun_install::audit_fix::{self, Advisory}; use bun_install::lockfile::package::PackageColumns as _; use bun_install::package_manager_real::command_line_arguments::AuditLevel; +use bun_install::package_manager_real::{ROOT_PACKAGE_JSON_PATH, install_with_manager}; use bun_install::resolution::Tag as ResolutionTag; -use bun_install::{CommandLineArguments, PackageManager, Subcommand}; +use bun_install::{CommandLineArguments, Lockfile, PackageID, PackageManager, Subcommand}; use bun_libdeflate_sys::libdeflate; use bun_parsers::json as bun_json; use bun_url::URL; use crate::cli::Command; +use crate::cli::install_command::InstallCommand; use crate::cli::package_manager_command::PackageManagerCommand; // Boxed to avoid a struct lifetime param; the @@ -66,8 +69,13 @@ impl AuditCommand { let audit_level = cli.audit_level; let production = cli.production; let audit_ignore_list = cli.audit_ignore_list; + let fix = cli.positionals.len() > 1 && cli.positionals[1] == b"fix"; + if fix && cli.positionals.len() > 2 { + Output::err_generic("bun audit fix does not take arguments", ()); + Global::exit(1); + } - let (manager, _original_cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Audit) + let (manager, original_cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Audit) { Ok(v) => v, Err(err) => { @@ -90,6 +98,14 @@ impl AuditCommand { }; let json_output = manager.options.json_output; + if fix { + if json_output { + Output::err_generic("--json is not supported by bun audit fix", ()); + Global::exit(1); + } + return Self::audit_fix(ctx, manager, audit_level, audit_ignore_list, &original_cwd); + } + let code = Self::audit( ctx, manager, @@ -183,6 +199,87 @@ impl AuditCommand { return Ok(0); } } + + fn audit_fix( + ctx: Command::Context, + pm: &mut PackageManager, + audit_level: Option, + ignore_list: &[&[u8]], + original_cwd: &[u8], + ) -> crate::Result { + bun_core::pretty_error!( + "bun audit fix v{}\n", + Global::package_json_version_with_sha, + ); + Output::flush(); + + { + let log_level = pm.options.log_level; + let load_lockfile = pm.load_lockfile_from_cwd::(); + PackageManagerCommand::handle_load_lockfile_errors(&load_lockfile, log_level); + } + + audit_fix::exit_unless_lockfile_writable(pm); + + let packages_result = collect_packages_for_audit(pm, false)?; + let response_text = send_audit_request(pm, &packages_result.audit_body)?; + + let vulnerabilities = if response_text.is_empty() { + Vec::new() + } else { + match collect_vulnerabilities(&response_text, audit_level, ignore_list)? { + Some(vulnerabilities) => vulnerabilities, + None => { + let _ = Output::writer().write_all(&response_text); + let _ = Output::writer().write_all(b"\n"); + Output::flush(); + Global::exit(1); + } + } + }; + + print_skipped_packages(&packages_result.skipped_packages); + + if vulnerabilities.is_empty() { + prettyln!("No vulnerabilities found"); + Output::flush(); + Global::exit(0); + } + + let advisories: Vec = vulnerabilities + .iter() + .map(|vulnerability| Advisory { + package_name: vulnerability.package_name.clone(), + vulnerable_versions: vulnerability.vulnerable_versions.clone(), + }) + .collect(); + let dry_run = pm.options.dry_run; + let plan = audit_fix::plan_fixes(pm, &advisories)?; + plan.print_sections(); + + if dry_run || plan.fixes.is_empty() { + plan.print_summary(dry_run); + Output::flush(); + Global::exit(plan.exit_code()); + } + + pm.audit_fix_pins = plan.pins(); + + // SAFETY: `ROOT_PACKAGE_JSON_PATH` is written exactly once inside `PackageManager::init`; only read thereafter. + let root_package_json_path = unsafe { ROOT_PACKAGE_JSON_PATH.read() }; + if let Err(e) = install_with_manager(pm, &mut *ctx, root_package_json_path, original_cwd) { + InstallCommand::handle_error(crate::Error::from(e))?; + Global::exit(1); + } + + if pm.any_failed_to_install { + Global::exit(1); + } + + plan.print_summary(false); + Output::flush(); + Global::exit(plan.exit_code()); + } } fn print_skipped_packages(skipped_packages: &[Box<[u8]>]) { @@ -245,55 +342,39 @@ fn build_dependency_tree( Ok(dependency_tree) } -fn build_production_package_set( - pm: &mut PackageManager, - prod_set: &mut StringHashMap<()>, -) -> Result<(), bun_alloc::AllocError> { - let root_id = pm.root_package_id.get(&pm.lockfile, pm.workspace_name_hash); - - let packages = pm.lockfile.packages.slice(); - let pkg_names = packages.items_name(); +fn build_production_package_set(lockfile: &Lockfile, root_id: PackageID) -> Vec { + let packages = lockfile.packages.slice(); let pkg_dependencies = packages.items_dependencies(); let pkg_resolutions = packages.items_resolutions(); - let buf = pm.lockfile.buffers.string_bytes.as_slice(); - let dependencies = pm.lockfile.buffers.dependencies.as_slice(); - let resolutions = pm.lockfile.buffers.resolutions.as_slice(); - - let mut queue: std::collections::VecDeque = std::collections::VecDeque::new(); - - let root_deps = pkg_dependencies[root_id as usize]; - let root_resolutions = pkg_resolutions[root_id as usize]; - let dep_slice = root_deps.get(dependencies); - let res_slice = root_resolutions.get(resolutions); - - for (dep, &resolved_pkg_id) in dep_slice.iter().zip(res_slice.iter()) { - if !dep.behavior.is_dev() && (resolved_pkg_id as usize) < packages.len() { - let pkg_name = pkg_names[resolved_pkg_id as usize].slice(buf); - prod_set.put(pkg_name, ())?; - queue.push_back(resolved_pkg_id); - } + let dependencies = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + + let mut prod_set = vec![false; packages.len()]; + let mut queue: std::collections::VecDeque = std::collections::VecDeque::new(); + if (root_id as usize) < packages.len() { + prod_set[root_id as usize] = true; + queue.push_back(root_id); } while let Some(current_pkg_id) = queue.pop_front() { - let current_deps = pkg_dependencies[current_pkg_id as usize]; - let current_resolutions = pkg_resolutions[current_pkg_id as usize]; - let current_dep_slice = current_deps.get(dependencies); - let current_res_slice = current_resolutions.get(resolutions); + let dep_slice = pkg_dependencies[current_pkg_id as usize].get(dependencies); + let res_slice = pkg_resolutions[current_pkg_id as usize].get(resolutions); - for (_, &resolved_pkg_id) in current_dep_slice.iter().zip(current_res_slice.iter()) { - if (resolved_pkg_id as usize) >= pkg_names.len() { + for (dep, &resolved_pkg_id) in dep_slice.iter().zip(res_slice.iter()) { + if dep.behavior.is_dev() || dep.behavior.is_optional_peer() { continue; } - - let pkg_name = pkg_names[resolved_pkg_id as usize].slice(buf); - if !prod_set.contains_key(pkg_name) { - prod_set.put(pkg_name, ())?; + let Some(seen) = prod_set.get_mut(resolved_pkg_id as usize) else { + continue; + }; + if !*seen { + *seen = true; queue.push_back(resolved_pkg_id); } } } - Ok(()) + prod_set } struct CollectPackagesResult { @@ -315,17 +396,9 @@ fn collect_packages_for_audit( let mut packages_list: Vec = Vec::new(); let mut skipped_packages: Vec> = Vec::new(); - let mut prod_packages: Option> = None; - if prod_only { - let mut set = StringHashMap::default(); - build_production_package_set(pm, &mut set)?; - prod_packages = Some(set); - } + let prod_packages: Option> = + prod_only.then(|| build_production_package_set(&pm.lockfile, root_id)); - // Note: reshaped for borrowck — column slices borrow `pm.lockfile` - // immutably for the loop, so resolve `root_id` / `prod_packages` (which - // need `&mut pm`) above, and split-borrow `pm.options` for the scope lookup - // (disjoint from `pm.lockfile`). let options = &pm.options; let default_url_hash = options.scope.url_hash; let packages = pm.lockfile.packages.slice(); @@ -341,16 +414,12 @@ fn collect_packages_for_audit( continue; } - let name_slice = name.slice(buf); - - if prod_only { - if let Some(ref prod) = prod_packages { - if !prod.contains_key(name_slice) { - continue; - } - } + if prod_packages.as_ref().is_some_and(|prod| !prod[idx]) { + continue; } + let name_slice = name.slice(buf); + let package_scope = options.scope_for_package_name(name_slice); if package_scope.url_hash != default_url_hash { skipped_packages.push(Box::<[u8]>::from(name_slice)); @@ -714,6 +783,61 @@ fn find_dependency_paths( Ok(paths) } +fn keep_vulnerability( + vulnerability: &VulnerabilityInfo, + audit_level: Option, + ignore_list: &[&[u8]], +) -> bool { + if let Some(level) = audit_level { + if !level.should_include_severity(&vulnerability.severity) { + return false; + } + } + + !ignore_list.iter().any(|ignored_cve| { + strings::eql(&vulnerability.id, ignored_cve) + || strings::index_of(&vulnerability.url, ignored_cve).is_some() + }) +} + +fn collect_vulnerabilities( + response_text: &[u8], + audit_level: Option, + ignore_list: &[&[u8]], +) -> Result>, bun_alloc::AllocError> { + let source = bun_ast::Source::init_path_string(b"audit-response.json", response_text); + let mut log = bun_ast::Log::init(); + + let parsed = match bun_json::ParsedJson::parse_json(&source, &mut log) { + Ok(e) => e, + Err(_) => return Ok(None), + }; + + let ExprData::EObjectJSON(obj) = &parsed.root.data else { + return Ok(None); + }; + + let mut vulnerabilities: Vec = Vec::new(); + for prop in obj.get().properties() { + let package_name: &[u8] = prop.key.slice(); + + let Some(arr) = prop.value.as_array() else { + continue; + }; + for vuln in arr.items() { + let Some(vuln_obj) = vuln.as_object() else { + continue; + }; + let vulnerability = parse_vulnerability(package_name, vuln_obj)?; + if keep_vulnerability(&vulnerability, audit_level, ignore_list) { + vulnerabilities.push(vulnerability); + } + } + } + + Ok(Some(vulnerabilities)) +} + #[derive(Default)] struct VulnCounts { low: u32, @@ -762,25 +886,8 @@ fn print_enhanced_audit_report( if let Some(vuln_obj) = vuln.as_object() { let vulnerability = parse_vulnerability(package_name, vuln_obj)?; - if let Some(level) = audit_level { - if !level.should_include_severity(&vulnerability.severity) { - continue; - } - } - - if !ignore_list.is_empty() { - let mut should_ignore = false; - for ignored_cve in ignore_list { - if strings::eql(&vulnerability.id, ignored_cve) - || strings::index_of(&vulnerability.url, ignored_cve).is_some() - { - should_ignore = true; - break; - } - } - if should_ignore { - continue; - } + if !keep_vulnerability(&vulnerability, audit_level, ignore_list) { + continue; } if vulnerability.severity.as_ref() == b"low" { diff --git a/src/runtime/cli/dedupe_command.rs b/src/runtime/cli/dedupe_command.rs new file mode 100644 index 000000000000..3fe02ce380d7 --- /dev/null +++ b/src/runtime/cli/dedupe_command.rs @@ -0,0 +1,57 @@ +use bun_core::{Global, Output}; +use bun_install::package_manager_real::{ + CommandLineArguments, PackageManager, ROOT_PACKAGE_JSON_PATH, Subcommand, install_with_manager, +}; + +use crate::Command; +use crate::cli::install_command::InstallCommand; + +pub(crate) struct DedupeCommand; + +impl DedupeCommand { + pub(crate) fn exec(ctx: Command::Context) -> crate::Result<()> { + let cli = CommandLineArguments::parse(Subcommand::Dedupe)?; + + // positionals[0] is "dedupe" itself + if cli.positionals.len() > 1 { + Output::err_generic( + "bun dedupe does not take arguments, it always deduplicates the whole lockfile", + (), + ); + bun_core::note!("run 'bun dedupe --help' for more information"); + Global::exit(1); + } + + let (manager, original_cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Dedupe) + { + Ok(v) => v, + Err(err) if err == bun_install::Error::MissingPackageJSON => { + Output::err_generic("missing package.json, nothing to dedupe", ()); + Global::exit(1); + } + Err(err) => return Err(err.into()), + }; + + if manager.options.should_print_command_name() { + bun_core::prettyln!( + "bun dedupe v{}\n\n", + Global::package_json_version_with_sha, + ); + Output::flush(); + } + + // SAFETY: `ROOT_PACKAGE_JSON_PATH` is written exactly once inside `PackageManager::init` above; only read thereafter. + let root_package_json_path = unsafe { ROOT_PACKAGE_JSON_PATH.read() }; + if let Err(e) = + install_with_manager(manager, &mut *ctx, root_package_json_path, &original_cwd) + { + return InstallCommand::handle_error(crate::Error::from(e)); + } + + if manager.any_failed_to_install { + Global::exit(1); + } + + Ok(()) + } +} diff --git a/src/runtime/cli/install_command.rs b/src/runtime/cli/install_command.rs index 523eed71e266..23a3813663cd 100644 --- a/src/runtime/cli/install_command.rs +++ b/src/runtime/cli/install_command.rs @@ -25,7 +25,7 @@ impl InstallCommand { /// touches this code, and demand-paging it in pollutes the startup window). #[cold] #[inline(never)] - fn handle_error(e: Error) -> Result<(), Error> { + pub(crate) fn handle_error(e: Error) -> Result<(), Error> { if matches!( e, crate::Error::InstallFailed diff --git a/src/runtime/cli/mod.rs b/src/runtime/cli/mod.rs index 9bd01c88ea88..a18d1134044d 100644 --- a/src/runtime/cli/mod.rs +++ b/src/runtime/cli/mod.rs @@ -338,6 +338,8 @@ pub mod upgrade_command; pub(crate) mod add_command; #[path = "audit_command.rs"] pub mod audit_command; +#[path = "dedupe_command.rs"] +pub(crate) mod dedupe_command; #[path = "filter_arg.rs"] pub mod filter_arg; #[path = "filter_run.rs"] @@ -354,6 +356,8 @@ pub mod pack_command; pub(crate) mod patch_command; #[path = "patch_commit_command.rs"] pub(crate) mod patch_commit_command; +#[path = "pm_licenses_command.rs"] +pub(crate) mod pm_licenses_command; #[path = "pm_pkg_command.rs"] pub mod pm_pkg_command; #[path = "pm_trusted_command.rs"] @@ -365,6 +369,8 @@ pub mod pm_version_command; pub mod pm_view_command; #[path = "pm_why_command.rs"] pub(crate) mod pm_why_command; +#[path = "prune_command.rs"] +pub(crate) mod prune_command; #[path = "publish_command.rs"] pub mod publish_command; #[path = "remove_command.rs"] @@ -651,6 +657,8 @@ pub mod help_command { remove {:<16} Remove a dependency from package.json (bun rm) update {:<16} Update outdated dependencies audit Check installed packages for vulnerabilities + dedupe Remove duplicate versions from the lockfile + prune Remove packages that are not in the lockfile from node_modules outdated Display latest versions of outdated dependencies link [\\] Register or link a local npm package unlink Unregister a local npm package @@ -1044,6 +1052,12 @@ pub mod command { if x == RootCommandMatcher::case(b"info") { return Tag::InfoCommand; } + if x == RootCommandMatcher::case(b"dedupe") { + return Tag::DedupeCommand; + } + if x == RootCommandMatcher::case(b"prune") { + return Tag::PruneCommand; + } // reserved if x == RootCommandMatcher::case(b"deploy") || x == RootCommandMatcher::case(b"cloud") @@ -1052,7 +1066,6 @@ pub mod command { || x == RootCommandMatcher::case(b"auth") || x == RootCommandMatcher::case(b"login") || x == RootCommandMatcher::case(b"logout") - || x == RootCommandMatcher::case(b"prune") { return Tag::ReservedCommand; } @@ -1296,6 +1309,8 @@ pub mod command { Tag::UpdateInteractiveCommand => exec_update_interactive(log), Tag::PublishCommand => exec_publish(log), Tag::AuditCommand => exec_audit(log), + Tag::DedupeCommand => exec_dedupe(log), + Tag::PruneCommand => exec_prune(log), Tag::WhyCommand => exec_why(log), Tag::BunxCommand => exec_bunx(log), Tag::ReplCommand => exec_repl(log), @@ -1600,6 +1615,8 @@ pub mod command { exec_update_interactive => (UpdateInteractiveCommand, super::update_interactive_command::UpdateInteractiveCommand::exec), exec_publish => (PublishCommand, super::publish_command::PublishCommand::exec), exec_why => (WhyCommand, super::why_command::WhyCommand::exec), + exec_dedupe => (DedupeCommand, super::dedupe_command::DedupeCommand::exec), + exec_prune => (PruneCommand, super::prune_command::PruneCommand::exec), exec_remove => (RemoveCommand, super::remove_command::RemoveCommand::exec), exec_link => (LinkCommand, super::link_command::LinkCommand::exec), exec_unlink => (UnlinkCommand, super::unlink_command::UnlinkCommand::exec), @@ -2157,6 +2174,12 @@ Execute a shell script directly from Bun. Tag::AuditCommand => { pm_print_help(PmSubcommand::Audit); } + Tag::DedupeCommand => { + pm_print_help(PmSubcommand::Dedupe); + } + Tag::PruneCommand => { + pm_print_help(PmSubcommand::Prune); + } Tag::InfoCommand => { pretty!( "\ diff --git a/src/runtime/cli/pack_command.rs b/src/runtime/cli/pack_command.rs index 749980616d6b..a1ce7dd2c0e1 100644 --- a/src/runtime/cli/pack_command.rs +++ b/src/runtime/cli/pack_command.rs @@ -10,7 +10,7 @@ use bun_core::{Global, Output, Progress, fmt as bun_fmt}; use bun_glob as glob; use bun_install::package_manager::LogLevel; use bun_install::package_manager::workspace_package_json_cache as WorkspacePackageJSONCache; -use bun_install::{Dependency, Lockfile, PackageManager}; +use bun_install::{Lockfile, PackageManager}; use bun_parsers::json as JSON; // Note: `WorkspacePackageJSONCache` returns the T2 value-subset // `bun_ast::Expr` (see `bun_install::bun_json`), not the full T4 @@ -3415,45 +3415,10 @@ fn edit_root_package_json( } }; - let catalog_name = Semver::String::init(catalog_name_str, catalog_name_str); let map_buf: &[u8] = lockfile.buffers.string_bytes.as_slice(); - - // Note: `CatalogMap::get_group` takes `&mut self` - // (returns `&mut Map`) but `pack` only needs read - // access via `&Lockfile`; inline an immutable lookup. - let catalog = if catalog_name.is_empty() { - Some(&lockfile.catalogs.default) - } else { - let ctx = Semver::string::ArrayHashContext { - arg_buf: catalog_name_str, - existing_buf: map_buf, - }; - let h = ctx.hash(catalog_name); - lockfile - .catalogs - .groups - .get_index_adapted_raw(h, |k, i| ctx.eql(catalog_name, *k, i)) - .map(|i| &lockfile.catalogs.groups.values()[i]) - }; - let Some(catalog) = catalog else { - Output::err_generic( - "Failed to resolve catalog version for \"{}\" in `{}` (no matching catalog).", - ( - bstr::BStr::new(dep_name_str), - bstr::BStr::new(dependency_group), - ), - ); - Global::crash(); - }; - - let dep_name = Semver::String::init(dep_name_str, dep_name_str); - let dep_ctx = Semver::string::ArrayHashContext { - arg_buf: dep_name_str, - existing_buf: map_buf, - }; - let dep_h = dep_ctx.hash(dep_name); - let Some(dep_idx) = catalog - .get_index_adapted_raw(dep_h, |k, i| dep_ctx.eql(dep_name, *k, i)) + let catalog_name = + strings::trim(catalog_name_str, &strings::WHITESPACE_CHARS); + let Some(dep) = lockfile.catalogs.find(map_buf, catalog_name, dep_name_str) else { Output::err_generic( "Failed to resolve catalog version for \"{}\" in `{}` (no matching catalog dependency).", @@ -3464,7 +3429,6 @@ fn edit_root_package_json( ); Global::crash(); }; - let dep: &Dependency = &catalog.values()[dep_idx]; let literal = pack_bump().alloc_slice_copy(dep.version.literal.slice(map_buf)); diff --git a/src/runtime/cli/package_manager_command.rs b/src/runtime/cli/package_manager_command.rs index 5f31eb8f7ff2..1d267eecb449 100644 --- a/src/runtime/cli/package_manager_command.rs +++ b/src/runtime/cli/package_manager_command.rs @@ -17,6 +17,7 @@ use bun_resolver::fs as Fs; use bun_sys::{self, Dir, Fd, File}; use crate::cli::Command; +use crate::cli::pm_licenses_command::PmLicensesCommand; use crate::cli::pm_pkg_command::PmPkgCommand; use crate::cli::pm_trusted_command::{DefaultTrustedCommand, TrustCommand, UntrustedCommand}; use crate::cli::pm_version_command::PmVersionCommand; @@ -159,6 +160,9 @@ impl PackageManagerCommand { ├ --all list the entire dependency tree according to the current lockfile\n\ └ --trusted list only trusted dependencies\n\ bun pm why \\ show dependency tree explaining why a package is installed\n\ + bun pm licenses list installed packages grouped by license\n\ + ├ --json output as JSON\n\ + └ --prod omit devDependencies\n\ bun pm whoami print the current npm username\n\ bun pm view name[@version] view package metadata from the registry (use `bun info` instead)\n\ bun pm version [increment] bump the version in package.json and create a git tag\n\ @@ -202,6 +206,7 @@ Learn more about these at https://bun.com/docs/cli/pm.\n"; .is_some_and(|arg| strings::eql_comptime(arg.as_bytes(), b"whoami")); let cli = CommandLineArguments::parse(Subcommand::Pm)?; + let production = cli.production; let (pm, cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Pm) { Ok(v) => v, Err(err) => { @@ -701,6 +706,10 @@ Learn more about these at https://bun.com/docs/cli/pm.\n"; let positionals: &[&[u8]] = pm.options.positionals; PmWhyCommand::exec(&&mut *ctx, pm, positionals)?; Global::exit(0); + } else if strings::eql_comptime(subcommand, b"licenses") { + let positionals: &[&[u8]] = pm.options.positionals; + PmLicensesCommand::exec(pm, positionals, production)?; + Global::exit(0); } else if strings::eql_comptime(subcommand, b"pkg") { let positionals: &[&[u8]] = pm.options.positionals; PmPkgCommand::exec(&&mut *ctx, pm, positionals, &cwd)?; diff --git a/src/runtime/cli/pm_licenses_command.rs b/src/runtime/cli/pm_licenses_command.rs new file mode 100644 index 000000000000..d5ff539121d9 --- /dev/null +++ b/src/runtime/cli/pm_licenses_command.rs @@ -0,0 +1,678 @@ +use std::cmp::Ordering; +use std::io::Write as _; + +use bstr::BStr; +use bun_ast::{Expr, Log, Source}; +use bun_collections::StringHashMap; +use bun_core::fmt::PathSep; +use bun_core::{FileKind, Global, Output, strings}; +use bun_install::lockfile::{Lockfile, package::PackageColumns as _, tree}; +use bun_install::npm::{Architecture, OperatingSystem}; +use bun_install::{PackageID, PackageManager, Resolution, ResolutionTag}; +use bun_parsers::json as JSON; +use bun_paths::AutoAbsPath; +use bun_sys::{self, Dir, Fd, File}; + +use crate::cli::package_manager_command::PackageManagerCommand; + +const UNKNOWN_LICENSE: &[u8] = b"Unknown"; +const MAX_SCAN_DEPTH: usize = 64; + +struct PackageInfo { + name: Option>, + version: Option>, + license: Box<[u8]>, + homepage: Option>, + author: Option>, +} + +struct Entry { + license: Box<[u8]>, + name: Box<[u8]>, + version: Box<[u8]>, + semver: Option, + homepage: Option>, + author: Option>, +} + +/// Lazily-scanned, sorted entry names of `node_modules/.bun/`; `None` until first needed. +struct BunStore { + entries: Option>>, +} + +struct DiskIndex { + entries: Option>, +} + +pub(crate) struct PmLicensesCommand; + +impl PmLicensesCommand { + pub(crate) fn exec( + pm: &mut PackageManager, + positionals: &[&[u8]], + production: bool, + ) -> crate::Result<()> { + if positionals.len() > 1 + && !strings::eql_comptime(positionals[1], b"list") + && !strings::eql_comptime(positionals[1], b"ls") + { + Output::err_generic("Unknown subcommand: {s}", (BStr::new(positionals[1]),)); + Global::exit(1); + } + + let log_level = pm.options.log_level; + let load = pm.load_lockfile_from_cwd::(); + PackageManagerCommand::handle_load_lockfile_errors(&load, log_level); + + let json_output = pm.options.json_output; + let root_id = pm.root_package_id.get(&pm.lockfile, pm.workspace_name_hash); + let lockfile: &Lockfile = &pm.lockfile; + + let mut path = AutoAbsPath::init_top_level_dir(); + let top_len = path.len(); + let _ = path.append(b"node_modules"); + if !bun_sys::exists(path.slice()) { + Output::err_generic("node_modules not found. Run \"bun install\" first", ()); + Global::exit(1); + } + path.set_length(top_len); + + let wanted = + reachable_packages(lockfile, root_id, production, pm.options.cpu, pm.options.os); + let locations = tree_locations(lockfile); + + let packages = lockfile.packages.slice(); + let pkg_names = packages.items_name(); + let pkg_resolution = packages.items_resolution(); + let buf = lockfile.buffers.string_bytes.as_slice(); + + let mut log = Log::init(); + let mut store = BunStore { entries: None }; + let mut disk = DiskIndex { entries: None }; + let mut entries: Vec = Vec::new(); + let mut missing: usize = 0; + + for pkg_id in 0..packages.len() { + if !wanted[pkg_id] { + continue; + } + let resolution = &pkg_resolution[pkg_id]; + if matches!( + resolution.tag, + ResolutionTag::Root | ResolutionTag::Workspace | ResolutionTag::Symlink + ) { + continue; + } + + let mut version: Vec = Vec::new(); + let _ = write!(&mut version, "{}", resolution.fmt(buf, PathSep::Posix)); + let is_npm = resolution.tag == ResolutionTag::Npm; + + let mut info = match &locations[pkg_id] { + Some(location) => { + let segments: [&[u8]; 2] = [location, b"package.json"]; + read_package_info_at(&mut path, top_len, &segments, &mut log) + } + None => None, + }; + + if is_npm { + info = info.filter(|info| match &info.version { + Some(installed) => installed[..] == version[..], + None => true, + }); + } + + if info.is_none() { + let store_entry: Option> = store + .lookup(&mut path, top_len, pkg_names[pkg_id], resolution, buf) + .map(Into::into); + if let Some(store_entry) = store_entry { + let segments: [&[u8]; 6] = [ + b"node_modules", + b".bun", + &store_entry, + b"node_modules", + pkg_names[pkg_id].slice(buf), + b"package.json", + ]; + info = read_package_info_at(&mut path, top_len, &segments, &mut log); + } + } + + if info.is_none() && is_npm { + info = disk.take( + &mut path, + top_len, + &mut log, + pkg_names[pkg_id].slice(buf), + &version, + ); + } + + let Some(info) = info else { + missing += 1; + continue; + }; + + entries.push(Entry { + license: info.license, + name: pkg_names[pkg_id].slice(buf).into(), + version: version.into_boxed_slice(), + semver: is_npm.then(|| resolution.npm().version), + homepage: info.homepage, + author: info.author, + }); + } + + if missing > 0 { + Output::warn(format_args!( + "omitted {} {} from the lockfile not found in node_modules", + missing, + if missing == 1 { "package" } else { "packages" }, + )); + } + + entries.sort_by(|a, b| { + sort_key(a) + .cmp(&sort_key(b)) + .then_with(|| match (a.semver, b.semver) { + (Some(x), Some(y)) => x.order(y, buf, buf), + (Some(_), None) => Ordering::Less, + (None, Some(_)) => Ordering::Greater, + (None, None) => Ordering::Equal, + }) + .then_with(|| a.version.cmp(&b.version)) + }); + + if json_output { + print_json(&entries); + } else { + print_text(&entries); + } + + Output::flush(); + Ok(()) + } +} + +fn sort_key(e: &Entry) -> (bool, &[u8], &[u8]) { + ( + &e.license[..] == UNKNOWN_LICENSE, + &e.license[..], + &e.name[..], + ) +} + +fn reachable_packages( + lockfile: &Lockfile, + root_id: PackageID, + production: bool, + cpu: Architecture, + os: OperatingSystem, +) -> Vec { + let packages = lockfile.packages.slice(); + let pkg_resolution = packages.items_resolution(); + let pkg_metas = packages.items_meta(); + let pkg_dependencies = packages.items_dependencies(); + let pkg_resolutions = packages.items_resolutions(); + let dependencies = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + let len = packages.len(); + + let mut marked = vec![false; len]; + if (root_id as usize) >= len { + return marked; + } + marked[root_id as usize] = true; + let mut work: Vec = vec![root_id]; + + while let Some(pkg) = work.pop() { + let skip_dev = production + && matches!( + pkg_resolution[pkg as usize].tag, + ResolutionTag::Root | ResolutionTag::Workspace + ); + let dep_slice = pkg_dependencies[pkg as usize].get(dependencies); + let res_slice = pkg_resolutions[pkg as usize].get(resolutions); + + for (dep, &dep_pkg_id) in dep_slice.iter().zip(res_slice.iter()) { + if (dep_pkg_id as usize) >= len { + continue; + } + if skip_dev && dep.behavior.is_dev() { + continue; + } + if pkg_metas[dep_pkg_id as usize].is_disabled(cpu, os) { + continue; + } + if !marked[dep_pkg_id as usize] { + marked[dep_pkg_id as usize] = true; + work.push(dep_pkg_id); + } + } + } + + marked +} + +fn tree_locations(lockfile: &Lockfile) -> Vec>> { + let len = lockfile.packages.len(); + let mut out: Vec>> = vec![None; len]; + let dependencies = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + let buf = lockfile.buffers.string_bytes.as_slice(); + + let mut it = tree::Iterator::<{ tree::IteratorPathStyle::NodeModules }>::init(lockfile); + while let Some(folder) = it.next(None) { + for &dep_id in folder.dependencies { + let pkg_id = resolutions[dep_id as usize]; + if (pkg_id as usize) >= len || out[pkg_id as usize].is_some() { + continue; + } + let relative_path = folder.relative_path.as_bytes(); + let alias = dependencies[dep_id as usize].name.slice(buf); + let mut location: Vec = Vec::with_capacity(relative_path.len() + 1 + alias.len()); + location.extend_from_slice(relative_path); + location.push(bun_paths::SEP); + location.extend_from_slice(alias); + out[pkg_id as usize] = Some(location.into_boxed_slice()); + } + } + + out +} + +fn read_package_info_at( + path: &mut AutoAbsPath, + base_len: usize, + segments: &[&[u8]], + log: &mut Log, +) -> Option { + path.set_length(base_len); + for &segment in segments { + let _ = path.append(segment); + } + let info = read_package_info(path.slice(), log); + path.set_length(base_len); + info +} + +fn read_package_info(path: &[u8], log: &mut Log) -> Option { + let contents = File::read_from(Fd::cwd(), path).ok()?; + bun_ast::initialize_store_or_reset(); + let source = Source::init_path_string(path, contents.as_slice()); + Some( + match JSON::parse_package_json_utf8(&source, log, crate::cli::cli_arena()) { + Ok(json) => PackageInfo { + name: string_field(&json, b"name"), + version: string_field(&json, b"version"), + license: license_of(&json), + homepage: string_field(&json, b"homepage"), + author: author_of(&json), + }, + Err(_) => PackageInfo { + name: None, + version: None, + license: UNKNOWN_LICENSE.into(), + homepage: None, + author: None, + }, + }, + ) +} + +fn str_of(expr: &Expr) -> Option<&[u8]> { + expr.as_string(crate::cli::cli_arena()) + .filter(|s| !s.is_empty()) +} + +fn string_field(json: &Expr, key: &[u8]) -> Option> { + json.get(key).and_then(|e| str_of(&e).map(Into::into)) +} + +fn string_or_type(expr: &Expr) -> Option> { + match str_of(expr) { + Some(s) => Some(s.into()), + None => string_field(expr, b"type").or_else(|| string_field(expr, b"name")), + } +} + +fn parse_license_field(field: &Expr) -> Option> { + if let Some(value) = string_or_type(field) { + return Some(value); + } + + let mut collected: Vec> = Vec::new(); + let mut items = field.as_array()?; + while let Some(item) = items.next() { + if let Some(value) = string_or_type(&item) { + collected.push(value); + } + } + + match collected.len() { + 0 => None, + 1 => collected.pop(), + _ => { + let mut joined: Vec = Vec::new(); + joined.push(b'('); + for (i, value) in collected.iter().enumerate() { + if i > 0 { + joined.extend_from_slice(b" OR "); + } + joined.extend_from_slice(value); + } + joined.push(b')'); + Some(joined.into_boxed_slice()) + } + } +} + +fn license_of(json: &Expr) -> Box<[u8]> { + json.get(b"license") + .and_then(|field| parse_license_field(&field)) + .or_else(|| { + json.get(b"licenses") + .and_then(|field| parse_license_field(&field)) + }) + .unwrap_or_else(|| UNKNOWN_LICENSE.into()) +} + +fn author_of(json: &Expr) -> Option> { + let author = json.get(b"author")?; + if let Some(s) = str_of(&author) { + return Some(s.into()); + } + + let mut out: Vec = Vec::new(); + if let Some(name) = author.get(b"name").as_ref().and_then(str_of) { + out.extend_from_slice(name); + } + if let Some(email) = author.get(b"email").as_ref().and_then(str_of) { + if !out.is_empty() { + out.push(b' '); + } + out.push(b'<'); + out.extend_from_slice(email); + out.push(b'>'); + } + if let Some(url) = author.get(b"url").as_ref().and_then(str_of) { + if !out.is_empty() { + out.push(b' '); + } + out.push(b'('); + out.extend_from_slice(url); + out.push(b')'); + } + + (!out.is_empty()).then(|| out.into_boxed_slice()) +} + +impl BunStore { + fn lookup( + &mut self, + path: &mut AutoAbsPath, + top_len: usize, + pkg_name: bun_semver::String, + resolution: &Resolution, + buf: &[u8], + ) -> Option<&[u8]> { + if self.entries.is_none() { + self.entries = Some(Self::scan(path, top_len)); + } + let entries = self.entries.as_deref()?; + if entries.is_empty() { + return None; + } + + let mut key: Vec = Vec::new(); + if resolution.tag == ResolutionTag::Folder { + let _ = write!( + &mut key, + "{}@file+{}", + pkg_name.fmt_store_path(buf), + resolution.fmt_store_path(buf) + ); + } else { + let _ = write!( + &mut key, + "{}@{}", + pkg_name.fmt_store_path(buf), + resolution.fmt_store_path(buf) + ); + } + + let i = entries.partition_point(|e| &e[..] < &key[..]); + let entry = entries.get(i)?; + let exact = entry[..] == key[..]; + let peer_suffixed = + entry.len() > key.len() && entry.starts_with(&key) && entry[key.len()] == b'+'; + (exact || peer_suffixed).then_some(&entry[..]) + } + + fn scan(path: &mut AutoAbsPath, top_len: usize) -> Vec> { + path.set_length(top_len); + let _ = path.append(b"node_modules"); + let _ = path.append(b".bun"); + let mut names: Vec> = list_dir(path.slice()) + .into_iter() + .map(|(name, _)| name) + .collect(); + path.set_length(top_len); + names.sort_unstable(); + names + } +} + +fn list_dir(path: &[u8]) -> Vec<(Box<[u8]>, FileKind)> { + let mut out: Vec<(Box<[u8]>, FileKind)> = Vec::new(); + let Ok(dir) = Dir::open(path) else { + return out; + }; + let mut iter = bun_sys::iterate_dir(dir.fd()); + while let Ok(Some(entry)) = iter.next() { + out.push((entry.name.slice_u8().into(), entry.kind)); + } + out +} + +fn disk_key(name: &[u8], version: &[u8]) -> Vec { + let mut key: Vec = Vec::with_capacity(name.len() + 1 + version.len()); + key.extend_from_slice(name); + key.push(b'@'); + key.extend_from_slice(version); + key +} + +impl DiskIndex { + fn take( + &mut self, + path: &mut AutoAbsPath, + top_len: usize, + log: &mut Log, + name: &[u8], + version: &[u8], + ) -> Option { + let entries = self.entries.get_or_insert_with(|| { + let mut entries = StringHashMap::default(); + path.set_length(top_len); + let _ = path.append(b"node_modules"); + Self::scan_node_modules(path, 0, log, &mut entries); + path.set_length(top_len); + entries + }); + entries.remove(&disk_key(name, version)[..]) + } + + fn scan_node_modules( + path: &mut AutoAbsPath, + depth: usize, + log: &mut Log, + out: &mut StringHashMap, + ) { + let nm_len = path.len(); + for (name, kind) in list_dir(path.slice()) { + if name.starts_with(b".") || (depth > 0 && kind == FileKind::SymLink) { + continue; + } + if path.append(&name[..]).is_err() { + continue; + } + if !name.starts_with(b"@") { + Self::scan_package(path, depth, log, out); + } else { + let scope_len = path.len(); + for (scoped_name, scoped_kind) in list_dir(path.slice()) { + if scoped_name.starts_with(b".") + || (depth > 0 && scoped_kind == FileKind::SymLink) + { + continue; + } + if path.append(&scoped_name[..]).is_ok() { + Self::scan_package(path, depth, log, out); + } + path.set_length(scope_len); + } + } + path.set_length(nm_len); + } + } + + fn scan_package( + path: &mut AutoAbsPath, + depth: usize, + log: &mut Log, + out: &mut StringHashMap, + ) { + let pkg_len = path.len(); + if let Some(info) = read_package_info_at(path, pkg_len, &[b"package.json"], log) { + if let (Some(name), Some(version)) = (&info.name, &info.version) { + let key = disk_key(name, version); + if !out.contains_key(&key[..]) { + let _ = out.put(&key, info); + } + } + } + if depth < MAX_SCAN_DEPTH && path.append(b"node_modules").is_ok() { + Self::scan_node_modules(path, depth + 1, log, out); + } + path.set_length(pkg_len); + } +} + +fn print_text(entries: &[Entry]) { + let mut start = 0; + while start < entries.len() { + let license = &entries[start].license; + let mut end = start + 1; + while end < entries.len() && entries[end].license == *license { + end += 1; + } + + if start > 0 { + Output::print(format_args!("\n")); + } + bun_core::prettyln!("{} ({})", BStr::new(license), end - start); + for (i, entry) in entries[start..end].iter().enumerate() { + if start + i + 1 < end { + bun_core::prettyln!( + "├── {}@{}", + BStr::new(&entry.name), + BStr::new(&entry.version) + ); + } else { + bun_core::prettyln!( + "└── {}@{}", + BStr::new(&entry.name), + BStr::new(&entry.version) + ); + } + } + + start = end; + } +} + +fn json_string(out: &mut Vec, s: &[u8]) { + let _ = write!( + out, + "{}", + bun_core::fmt::format_json_string_utf8(s, Default::default()) + ); +} + +fn print_json(entries: &[Entry]) { + let mut out: Vec = Vec::new(); + + if entries.is_empty() { + out.extend_from_slice(b"{}\n"); + let _ = Output::writer().write_all(&out); + return; + } + + out.extend_from_slice(b"{\n"); + let mut start = 0; + while start < entries.len() { + let license = &entries[start].license; + let mut end = start + 1; + while end < entries.len() && entries[end].license == *license { + end += 1; + } + + if start > 0 { + out.extend_from_slice(b",\n"); + } + out.extend_from_slice(b" "); + json_string(&mut out, license); + out.extend_from_slice(b": ["); + + let mut group_start = start; + let mut first_group = true; + while group_start < end { + let first = &entries[group_start]; + let mut group_end = group_start + 1; + while group_end < end && entries[group_end].name == first.name { + group_end += 1; + } + + if !first_group { + out.push(b','); + } + first_group = false; + out.extend_from_slice(b"\n {\n \"name\": "); + json_string(&mut out, &first.name); + out.extend_from_slice(b",\n \"versions\": ["); + let mut previous: Option<&[u8]> = None; + for entry in &entries[group_start..group_end] { + if previous == Some(&entry.version[..]) { + continue; + } + if previous.is_some() { + out.extend_from_slice(b", "); + } + json_string(&mut out, &entry.version); + previous = Some(&entry.version[..]); + } + out.push(b']'); + let newest = &entries[group_end - 1]; + if let Some(homepage) = &newest.homepage { + out.extend_from_slice(b",\n \"homepage\": "); + json_string(&mut out, homepage); + } + if let Some(author) = &newest.author { + out.extend_from_slice(b",\n \"author\": "); + json_string(&mut out, author); + } + out.extend_from_slice(b"\n }"); + + group_start = group_end; + } + + out.extend_from_slice(b"\n ]"); + start = end; + } + out.extend_from_slice(b"\n}\n"); + + let _ = Output::writer().write_all(&out); +} diff --git a/src/runtime/cli/prune_command.rs b/src/runtime/cli/prune_command.rs new file mode 100644 index 000000000000..26a59feaa1ce --- /dev/null +++ b/src/runtime/cli/prune_command.rs @@ -0,0 +1,42 @@ +use bun_core::{Global, Output}; +use bun_install::package_manager_real::{CommandLineArguments, PackageManager, Subcommand}; + +use crate::Command; + +pub(crate) struct PruneCommand; + +impl PruneCommand { + pub(crate) fn exec(ctx: Command::Context) -> crate::Result<()> { + let cli = CommandLineArguments::parse(Subcommand::Prune)?; + + // positionals[0] is "prune" itself + if cli.positionals.len() > 1 { + Output::err_generic( + "bun prune does not take arguments, it always prunes the whole node_modules", + (), + ); + bun_core::note!("run 'bun prune --help' for more information"); + Global::exit(1); + } + + let (manager, _original_cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Prune) + { + Ok(v) => v, + Err(err) if err == bun_install::Error::MissingPackageJSON => { + Output::err_generic("missing package.json, nothing to prune", ()); + Global::exit(1); + } + Err(err) => return Err(err.into()), + }; + + if manager.options.should_print_command_name() { + bun_core::prettyln!( + "bun prune v{}\n", + Global::package_json_version_with_sha, + ); + Output::flush(); + } + + bun_install::prune::prune(manager).map_err(crate::Error::from) + } +} diff --git a/src/semver/SemverQuery.rs b/src/semver/SemverQuery.rs index d1004735c6e2..aeef0e3fb34b 100644 --- a/src/semver/SemverQuery.rs +++ b/src/semver/SemverQuery.rs @@ -560,6 +560,24 @@ impl Group { self.head.satisfies(version, group_buf, version_buf) } } + + /// True when `parse` found no comparator at all (e.g. the input was only unrecognised words). + pub fn is_empty(&self) -> bool { + self.head.next.is_none() + && self.head.head.next.is_none() + && !self.head.head.range.has_left() + } + + /// npm's `includePrerelease`: a prerelease only has to satisfy the comparators. + #[inline] + pub fn satisfies_including_prerelease( + &self, + version: Version, + group_buf: &[u8], + version_buf: &[u8], + ) -> bool { + self.head.satisfies(version, group_buf, version_buf) + } } #[derive(Clone, Copy, Default)] diff --git a/test/cli/install/bun-add-catalog.test.ts b/test/cli/install/bun-add-catalog.test.ts new file mode 100644 index 000000000000..6098c02d4e01 --- /dev/null +++ b/test/cli/install/bun-add-catalog.test.ts @@ -0,0 +1,974 @@ +import { file, write } from "bun"; +import { readTarball } from "bun:internal-for-testing"; +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { VerdaccioRegistry, bunEnv, bunExe, runBunInstall } from "harness"; +import { join } from "path"; + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +const PKG1 = JSON.stringify({ name: "pkg1", version: "1.0.0" }); + +async function createDir( + root: Record | string, + pkg1: Record | string = PKG1, + extraFiles: Record = {}, +) { + const { packageDir } = await registry.createTestDir({ + bunfigOpts: { linker: "hoisted", saveTextLockfile: true }, + files: { + "package.json": typeof root === "string" ? root : JSON.stringify(root), + "packages/pkg1/package.json": typeof pkg1 === "string" ? pkg1 : JSON.stringify(pkg1), + ...extraFiles, + }, + }); + + const rootPath = join(packageDir, "package.json"); + const pkg1Path = join(packageDir, "packages", "pkg1", "package.json"); + const pkg2Path = join(packageDir, "packages", "pkg2", "package.json"); + + return { + packageDir, + pkg1Dir: join(packageDir, "packages", "pkg1"), + pkg2Dir: join(packageDir, "packages", "pkg2"), + rootPath, + pkg1Path, + root: () => file(rootPath).json(), + pkg1: () => file(pkg1Path).json(), + pkg2: () => file(pkg2Path).json(), + rootText: () => file(rootPath).text(), + pkg1Text: () => file(pkg1Path).text(), + pkg2Text: () => file(pkg2Path).text(), + lockText: () => file(join(packageDir, "bun.lock")).text(), + lockExists: () => file(join(packageDir, "bun.lock")).exists(), + lock: async () => Bun.JSONC.parse(await file(join(packageDir, "bun.lock")).text()) as any, + installed: (name: string) => file(join(packageDir, "node_modules", name, "package.json")).json(), + installedExists: (name: string) => file(join(packageDir, "node_modules", name, "package.json")).exists(), + }; +} + +const PKG2 = JSON.stringify({ name: "pkg2", version: "1.0.0" }); +const withPkg2 = (pkg2: Record | string = PKG2) => ({ + "packages/pkg2/package.json": typeof pkg2 === "string" ? pkg2 : JSON.stringify(pkg2), +}); + +async function run(cwd: string, args: string[], env: Record = bunEnv) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...args], + cwd, + env, + stdout: "pipe", + stderr: "pipe", + stdin: "ignore", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +function runAdd(cwd: string, ...args: string[]) { + return run(cwd, ["add", ...args]); +} + +function expectOk({ stderr, exitCode }: { stderr: string; exitCode: number }) { + expect(stderr).not.toContain("error:"); + expect(stderr).not.toContain("panic:"); + expect(exitCode).toBe(0); +} + +const workspacesObject = (catalogs: Record = {}) => ({ + name: "root", + workspaces: { packages: ["packages/*"], ...catalogs }, +}); + +describe.concurrent("bun add --catalog", () => { + test("default catalog from a workspace member", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + + const lock = await dir.lock(); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:" }); + expect(lock.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect(lock.packages["no-deps"][0]).toBe("no-deps@2.0.0"); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("named catalog creates the catalogs object", async () => { + const dir = await createDir(workspacesObject({ catalog: { "no-deps": "1.0.0" } })); + + expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=testing")); + + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:testing" }); + expect((await dir.root()).workspaces).toEqual({ + packages: ["packages/*"], + catalog: { "no-deps": "1.0.0" }, + catalogs: { testing: { "a-dep": "^1.0.10" } }, + }); + expect((await dir.installed("a-dep")).version).toBe("1.0.10"); + + const lock = await dir.lock(); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "a-dep": "catalog:testing" }); + expect(lock.catalogs).toEqual({ testing: { "a-dep": "^1.0.10" } }); + }); + + describe("placement when no catalog is defined yet", () => { + test("workspaces object gets workspaces.catalog", async () => { + const dir = await createDir(workspacesObject()); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect(await dir.root()).toEqual({ + name: "root", + workspaces: { packages: ["packages/*"], catalog: { "no-deps": "^2.0.0" } }, + }); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + }); + + test("workspaces array gets a top-level catalog", async () => { + const dir = await createDir({ name: "root", workspaces: ["packages/*"] }); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect(await dir.root()).toEqual({ + name: "root", + workspaces: ["packages/*"], + catalog: { "no-deps": "^2.0.0" }, + }); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + }); + }); + + describe("existing top-level placement is respected", () => { + const topLevel = { name: "root", catalog: { "no-deps": "1.0.0" }, workspaces: ["packages/*"] }; + + test("default catalog", async () => { + const dir = await createDir(topLevel); + + expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog")); + + const root = await dir.root(); + expect(root).toEqual({ + name: "root", + catalog: { "a-dep": "^1.0.10", "no-deps": "1.0.0" }, + workspaces: ["packages/*"], + }); + expect(Object.keys(root.catalog)).toEqual(["a-dep", "no-deps"]); + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:" }); + }); + + test("named catalog", async () => { + const dir = await createDir(topLevel); + + expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=x")); + + expect(await dir.root()).toEqual({ + name: "root", + catalog: { "no-deps": "1.0.0" }, + catalogs: { x: { "a-dep": "^1.0.10" } }, + workspaces: ["packages/*"], + }); + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:x" }); + }); + }); + + describe("catalog literal", () => { + for (const { args, entry, installed } of [ + { args: ["no-deps@1.0.0", "--catalog"], entry: "1.0.0", installed: "1.0.0" }, + { args: ["no-deps@^1.0.0", "--catalog"], entry: "^1.0.0", installed: "1.1.0" }, + { args: ["no-deps", "--catalog", "--exact"], entry: "2.0.0", installed: "2.0.0" }, + ]) { + test(`bun add ${args.join(" ")} writes ${JSON.stringify(entry)}`, async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + + expectOk(await runAdd(dir.pkg1Dir, ...args)); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": entry }); + expect((await dir.installed("no-deps")).version).toBe(installed); + expect((await dir.lock()).catalog).toEqual({ "no-deps": entry }); + }); + } + }); + + test("refreshes an existing catalog entry", async () => { + const dir = await createDir(workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), { + name: "pkg1", + version: "1.0.0", + dependencies: { "no-deps": "catalog:" }, + }); + + await runBunInstall(bunEnv, dir.packageDir); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + + const lockText = await dir.lockText(); + expect(lockText).not.toContain("no-deps@1.1.0"); + expect(lockText).toContain("no-deps@2.0.0"); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "^2.0.0" }); + }); + + test("run from the workspace root", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + + expectOk(await runAdd(dir.packageDir, "no-deps", "--catalog")); + + expect(await dir.root()).toEqual({ + name: "root", + dependencies: { "no-deps": "catalog:" }, + workspaces: { packages: ["packages/*"], catalog: { "no-deps": "^2.0.0" } }, + }); + expect(await dir.pkg1Text()).toBe(PKG1); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + + const lock = await dir.lock(); + expect(lock.workspaces[""].dependencies).toEqual({ "no-deps": "catalog:" }); + expect(lock.catalog).toEqual({ "no-deps": "^2.0.0" }); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("bun install --catalog --dev", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + + expectOk(await run(dir.pkg1Dir, ["install", "no-deps", "--catalog", "--dev"])); + + const pkg1 = await dir.pkg1(); + expect(pkg1.devDependencies).toEqual({ "no-deps": "catalog:" }); + expect(pkg1.dependencies).toBeUndefined(); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + }); + + test("several packages into a named catalog", async () => { + const dir = await createDir(workspacesObject()); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "a-dep", "--catalog=libs")); + + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:libs", "no-deps": "catalog:libs" }); + const root = await dir.root(); + expect(root.workspaces.catalogs.libs).toEqual({ "a-dep": "^1.0.10", "no-deps": "^2.0.0" }); + expect(Object.keys(root.workspaces.catalogs.libs)).toEqual(["a-dep", "no-deps"]); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + expect((await dir.installed("a-dep")).version).toBe("1.0.10"); + expect((await dir.lock()).catalogs).toEqual({ libs: { "a-dep": "^1.0.10", "no-deps": "^2.0.0" } }); + }); + + for (const from of ["member", "root"] as const) { + test(`--dry-run writes nothing (from ${from})`, async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const cwd = from === "member" ? dir.pkg1Dir : dir.packageDir; + const { stderr, exitCode } = await runAdd(cwd, "no-deps", "--catalog", "--dry-run"); + + expect(stderr).not.toContain("error:"); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await file(join(dir.packageDir, "bun.lock")).exists()).toBeFalse(); + expect(exitCode).toBe(0); + }); + } + + describe("only the dependency group plain add edits is rewritten", () => { + test("name in devDependencies and peerDependencies, adding with --dev", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), { + name: "pkg1", + devDependencies: { "no-deps": "1.0.0" }, + peerDependencies: { "no-deps": ">=1" }, + }); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog", "--dev")); + + expect(await dir.pkg1()).toEqual({ + name: "pkg1", + devDependencies: { "no-deps": "catalog:" }, + peerDependencies: { "no-deps": ">=1" }, + }); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect((await dir.lock()).workspaces["packages/pkg1"]).toEqual({ + name: "pkg1", + devDependencies: { "no-deps": "catalog:" }, + peerDependencies: { "no-deps": ">=1" }, + }); + }); + + test("name only in peerDependencies, adding with --dev, matches plain add", async () => { + const pkg1 = { name: "pkg1", peerDependencies: { "no-deps": ">=1" } }; + const [plain, catalog] = await Promise.all([ + createDir(workspacesObject({ catalog: {} }), pkg1), + createDir(workspacesObject({ catalog: {} }), pkg1), + ]); + + const [plainResult, catalogResult] = await Promise.all([ + runAdd(plain.pkg1Dir, "no-deps", "--dev"), + runAdd(catalog.pkg1Dir, "no-deps", "--dev", "--catalog"), + ]); + expectOk(plainResult); + expectOk(catalogResult); + + const plainPkg1 = await plain.pkg1(); + const catalogPkg1 = await catalog.pkg1(); + expect(Object.keys(catalogPkg1)).toEqual(Object.keys(plainPkg1)); + expect(plainPkg1).toEqual({ name: "pkg1", peerDependencies: { "no-deps": "^2.0.0" } }); + expect(catalogPkg1).toEqual({ name: "pkg1", peerDependencies: { "no-deps": "catalog:" } }); + }); + + test("--peer", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog", "--peer")); + + expect(await dir.pkg1()).toEqual({ name: "pkg1", version: "1.0.0", peerDependencies: { "no-deps": "catalog:" } }); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + }); + }); + + test("target already on a named catalog reference is moved to the flag's catalog", async () => { + const dir = await createDir(workspacesObject({ catalogs: { testing: { "no-deps": "1.0.0" } } }), { + name: "pkg1", + dependencies: { "no-deps": "catalog:testing" }, + }); + await runBunInstall(bunEnv, dir.packageDir); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.root()).workspaces).toEqual({ + packages: ["packages/*"], + catalogs: { testing: { "no-deps": "1.0.0" } }, + catalog: { "no-deps": "^2.0.0" }, + }); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + + const lock = await dir.lock(); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:" }); + expect(lock.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect(lock.catalogs).toEqual({ testing: { "no-deps": "1.0.0" } }); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + describe("literals other than a bare name", () => { + test("alias@npm:pkg is written verbatim, like plain add", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + + expectOk(await runAdd(dir.pkg1Dir, "foo@npm:no-deps", "--catalog")); + + expect((await dir.pkg1()).dependencies).toEqual({ foo: "catalog:" }); + expect((await dir.root()).workspaces.catalog).toEqual({ foo: "npm:no-deps" }); + expect((await dir.lock()).catalog).toEqual({ foo: "npm:no-deps" }); + expect(await dir.installed("foo")).toMatchObject({ name: "no-deps", version: "2.0.0" }); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("alias@npm:pkg@dist-tag gets the resolved range", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + + expectOk(await runAdd(dir.pkg1Dir, "foo@npm:no-deps@latest", "--catalog")); + + expect((await dir.pkg1()).dependencies).toEqual({ foo: "catalog:" }); + expect((await dir.root()).workspaces.catalog).toEqual({ foo: "npm:no-deps@^2.0.0" }); + expect((await dir.lock()).catalog).toEqual({ foo: "npm:no-deps@^2.0.0" }); + expect(await dir.installed("foo")).toMatchObject({ name: "no-deps", version: "2.0.0" }); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("name@tarball-url is written verbatim, dist-tags become a range", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const tarball = `${registry.registryUrl()}no-deps/-/no-deps-1.0.0.tgz`; + + expectOk(await runAdd(dir.pkg1Dir, `no-deps@${tarball}`, "dep-with-tags@pre-1", "--catalog")); + + expect((await dir.pkg1()).dependencies).toEqual({ "dep-with-tags": "catalog:", "no-deps": "catalog:" }); + const catalog = { "dep-with-tags": "^1.0.1", "no-deps": tarball }; + expect((await dir.root()).workspaces.catalog).toEqual(catalog); + expect((await dir.lock()).catalog).toEqual(catalog); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + expect((await dir.installed("dep-with-tags")).version).toBe("1.0.1"); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("scoped package into a named catalog", async () => { + const dir = await createDir(workspacesObject()); + + expectOk(await runAdd(dir.pkg1Dir, "@types/no-deps", "--catalog=types")); + + expect((await dir.pkg1()).dependencies).toEqual({ "@types/no-deps": "catalog:types" }); + expect((await dir.root()).workspaces.catalogs).toEqual({ types: { "@types/no-deps": "^2.0.0" } }); + expect((await dir.lock()).catalogs).toEqual({ types: { "@types/no-deps": "^2.0.0" } }); + expect((await dir.installed("@types/no-deps")).version).toBe("2.0.0"); + }); + }); + + describe("workspace sibling", () => { + test("bare name fails without writing anything", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = await runAdd(dir.pkg1Dir, "pkg2", "--catalog"); + + expect(stderr).toContain("error:"); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).not.toBe(0); + }); + + for (const from of ["member", "root"] as const) { + test(`name@workspace:* is rejected (from ${from})`, async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const cwd = from === "member" ? dir.pkg1Dir : dir.packageDir; + const { stderr, exitCode } = await runAdd(cwd, "pkg2@workspace:*", "--catalog"); + + expect(stderr).toContain('error: --catalog cannot add a workspace package, but got "pkg2@workspace:*"'); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + } + + test("name@workspace:* with --filter is rejected", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = await runAdd(dir.packageDir, "pkg2@workspace:*", "--catalog", "--filter", "pkg1"); + + expect(stderr).toContain('error: --catalog cannot add a workspace package, but got "pkg2@workspace:*"'); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + }); + + describe("--filter", () => { + test("edits only the filtered member and the root catalog", async () => { + const pkg1 = JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir(workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), pkg1, withPkg2()); + await runBunInstall(bunEnv, dir.packageDir); + + expectOk(await runAdd(dir.packageDir, "a-dep", "--catalog", "--filter", "pkg2")); + + expect((await dir.pkg2()).dependencies).toEqual({ "a-dep": "catalog:" }); + expect(await dir.pkg1Text()).toBe(pkg1); + const root = await dir.root(); + expect(root.dependencies).toBeUndefined(); + expect(root.workspaces.catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); + expect(Object.keys(root.workspaces.catalog)).toEqual(["a-dep", "no-deps"]); + expect((await dir.lock()).catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); + expect((await dir.installed("a-dep")).version).toBe("1.0.10"); + + expectOk(await runAdd(dir.packageDir, "a-dep", "--catalog", "--filter", "*")); + + expect((await dir.root()).dependencies).toEqual({ "a-dep": "catalog:" }); + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); + expect((await dir.pkg2()).dependencies).toEqual({ "a-dep": "catalog:" }); + expect((await dir.root()).workspaces.catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("--only-missing skips members that already have the package", async () => { + const pkg1 = { name: "pkg1", dependencies: { "no-deps": "^1.0.0" } }; + const dir = await createDir(workspacesObject({ catalog: {} }), pkg1, withPkg2()); + + expectOk( + await runAdd(dir.packageDir, "no-deps", "--catalog", "--only-missing", "--filter", "pkg1", "--filter", "pkg2"), + ); + + expect(await dir.pkg1()).toEqual(pkg1); + expect((await dir.pkg2()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + }); + + test("--only-missing with every member already having the package leaves the catalog empty", async () => { + const pkg1 = { name: "pkg1", dependencies: { "no-deps": "^1.0.0" } }; + const dir = await createDir(workspacesObject({ catalog: {} }), pkg1); + + const { stderr, exitCode } = await runAdd( + dir.packageDir, + "no-deps", + "--catalog", + "--only-missing", + "--filter", + "pkg1", + ); + + expect(stderr).not.toContain("panic:"); + expect(await dir.pkg1()).toEqual(pkg1); + expect(await dir.root()).toEqual(workspacesObject({ catalog: {} })); + expect(exitCode).toBe(0); + }); + }); + + test("--only-missing when the target already has the package leaves the catalog empty", async () => { + const pkg1 = { name: "pkg1", dependencies: { "no-deps": "^1.0.0" } }; + const dir = await createDir(workspacesObject({ catalog: {} }), pkg1); + const rootBefore = await dir.rootText(); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog", "--only-missing")); + + expect(await dir.pkg1()).toEqual(pkg1); + expect(await dir.rootText()).toBe(rootBefore); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + }); + + // pnpm #9647: an existing entry is moved to the version given on the command line. + test("explicit version replaces an entry other members reference", async () => { + const member = (name: string) => JSON.stringify({ name, dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir( + workspacesObject({ catalog: { "no-deps": "^2.0.0" } }), + member("pkg1"), + withPkg2(member("pkg2")), + ); + await runBunInstall(bunEnv, dir.packageDir); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps@1.0.0", "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "1.0.0" }); + expect(await dir.pkg2Text()).toBe(member("pkg2")); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + const lockText = await dir.lockText(); + expect(lockText).not.toContain("no-deps@2.0.0"); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "1.0.0" }); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("other dependencies of the target are left alone", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), { + name: "pkg1", + dependencies: { "a-dep": "1.0.1" }, + }); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "1.0.1", "no-deps": "catalog:" }); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect((await dir.installed("a-dep")).version).toBe("1.0.1"); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + }); + + test("existing direct range is converted in place", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), { + name: "pkg1", + dependencies: { "no-deps": "^1.0.0" }, + }); + await runBunInstall(bunEnv, dir.packageDir); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect(await dir.pkg1Text()).toBe( + JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:" } }, null, 2), + ); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + }); + + test("re-running the same add is idempotent", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + const [rootAfter, pkg1After, lockAfter] = await Promise.all([dir.rootText(), dir.pkg1Text(), dir.lockText()]); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect(await dir.rootText()).toBe(rootAfter); + expect(await dir.pkg1Text()).toBe(pkg1After); + expect(await dir.lockText()).toBe(lockAfter); + }); + + test("named catalog with other entries and other named catalogs present", async () => { + const dir = await createDir( + workspacesObject({ catalogs: { other: { "a-dep": "1.0.1" }, testing: { "no-deps": "1.0.0" } } }), + ); + + expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=testing")); + + const catalogs = { other: { "a-dep": "1.0.1" }, testing: { "a-dep": "^1.0.10", "no-deps": "1.0.0" } }; + const root = await dir.root(); + expect(root.workspaces.catalogs).toEqual(catalogs); + expect(Object.keys(root.workspaces.catalogs.testing)).toEqual(["a-dep", "no-deps"]); + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:testing" }); + expect((await dir.lock()).catalogs).toEqual(catalogs); + expect((await dir.installed("a-dep")).version).toBe("1.0.10"); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("one package failing to resolve writes nothing", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = await runAdd(dir.pkg1Dir, "no-deps", "this-package-does-not-exist-xyz", "--catalog"); + + expect(stderr).toContain("error:"); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).not.toBe(0); + }); + + for (const from of ["member", "root"] as const) { + test(`--no-save installs but writes neither file (from ${from})`, async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const cwd = from === "member" ? dir.pkg1Dir : dir.packageDir; + expectOk(await runAdd(cwd, "no-deps", "--catalog", "--no-save")); + + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + }); + } + + test("root package.json formatting survives the member-path rewrite", async () => { + const rootBefore = + JSON.stringify( + { private: true, workspaces: ["packages/*"], catalog: { "a-dep": "1.0.1" }, name: "root" }, + null, + 4, + ) + "\n"; + const dir = await createDir(rootBefore); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + const rootText = await dir.rootText(); + expect(rootText).toBe( + JSON.stringify( + { private: true, workspaces: ["packages/*"], catalog: { "a-dep": "1.0.1", "no-deps": "^2.0.0" }, name: "root" }, + null, + 4, + ) + "\n", + ); + }); + + // pnpm #7072: `catalog:` and `catalog:default` are one catalog, whether it is spelled `catalog` or `catalogs.default`. + describe("default catalog alias", () => { + const member = (name: string, ref = "catalog:") => JSON.stringify({ name, dependencies: { "no-deps": ref } }); + + test("--catalog refreshes an existing catalogs.default instead of adding a second catalog", async () => { + const dir = await createDir( + workspacesObject({ catalogs: { default: { "no-deps": "1.0.0" } } }), + member("pkg1"), + withPkg2(member("pkg2", "catalog:default")), + ); + await runBunInstall(bunEnv, dir.packageDir); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect((await dir.root()).workspaces).toEqual({ + packages: ["packages/*"], + catalogs: { default: { "no-deps": "^2.0.0" } }, + }); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect(await dir.pkg2Text()).toBe(member("pkg2", "catalog:default")); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + + const lock = await dir.lock(); + expect(lock.catalog).toBeUndefined(); + expect(lock.catalogs).toEqual({ default: { "no-deps": "^2.0.0" } }); + expect(lock.packages["no-deps"][0]).toBe("no-deps@2.0.0"); + + const { stderr, exitCode } = await run(dir.packageDir, ["install", "--frozen-lockfile"]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + + test("--catalog=default refreshes the singular catalog every catalog: reference resolves through", async () => { + const dir = await createDir( + workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), + PKG1, + withPkg2(member("pkg2")), + ); + await runBunInstall(bunEnv, dir.packageDir); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog=default")); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:default" }); + expect((await dir.root()).workspaces).toEqual({ + packages: ["packages/*"], + catalog: { "no-deps": "^2.0.0" }, + }); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + + const lock = await dir.lock(); + expect(lock.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect(lock.catalogs).toBeUndefined(); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:default" }); + expect(await dir.lockText()).not.toContain("no-deps@1.1.0"); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("--catalog=default with no catalog defined creates the singular catalog", async () => { + const dir = await createDir(workspacesObject()); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog=default")); + + expect(await dir.root()).toEqual({ + name: "root", + workspaces: { packages: ["packages/*"], catalog: { "no-deps": "^2.0.0" } }, + }); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:default" }); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "^2.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + }); + + test("bun update --latest refreshes a catalogs.default entry referenced as catalog:", async () => { + const dir = await createDir(workspacesObject({ catalogs: { default: { "no-deps": "^1.0.0" } } }), member("pkg1")); + await runBunInstall(bunEnv, dir.packageDir); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + expectOk(await run(dir.packageDir, ["update", "--latest"])); + + expect((await dir.root()).workspaces.catalogs).toEqual({ default: { "no-deps": "^2.0.0" } }); + expect(await dir.pkg1Text()).toBe(member("pkg1")); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + expect((await dir.lock()).catalogs).toEqual({ default: { "no-deps": "^2.0.0" } }); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + test("bun pm pack substitutes a catalogs.default entry referenced as catalog:", async () => { + const dir = await createDir(workspacesObject({ catalogs: { default: { "no-deps": ">=1.0.0" } } }), { + name: "pkg1", + version: "1.0.0", + peerDependencies: { "no-deps": "catalog:" }, + }); + await runBunInstall(bunEnv, dir.packageDir); + + const { stderr, exitCode } = await run(dir.pkg1Dir, ["pm", "pack"]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const tarball = readTarball(join(dir.pkg1Dir, "pkg1-1.0.0.tgz")); + expect(JSON.parse(tarball.entries[0].contents)).toEqual({ + name: "pkg1", + version: "1.0.0", + peerDependencies: { "no-deps": ">=1.0.0" }, + }); + }); + }); + + // pnpm #8996: `bun pm pack` substitutes catalog references in every dependency group. + test("bun pm pack substitutes the catalog literal, including peerDependencies", async () => { + const dir = await createDir(workspacesObject({ catalog: { "no-deps": ">=1.0.0" } }), { + name: "pkg1", + version: "1.0.0", + peerDependencies: { "no-deps": "catalog:" }, + }); + await runBunInstall(bunEnv, dir.packageDir); + + expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=x")); + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:x" }); + + const { stderr, exitCode } = await run(dir.pkg1Dir, ["pm", "pack"]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const tarball = readTarball(join(dir.pkg1Dir, "pkg1-1.0.0.tgz")); + expect(JSON.parse(tarball.entries[0].contents)).toEqual({ + name: "pkg1", + version: "1.0.0", + peerDependencies: { "no-deps": ">=1.0.0" }, + dependencies: { "a-dep": "^1.0.10" }, + }); + }); + + // pnpm #10456: removing the last reference to an entry does not drop the catalog from bun.lock. + test("bun remove from members keeps the catalog definitions", async () => { + const member = (name: string) => JSON.stringify({ name, dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir( + workspacesObject({ catalog: { "no-deps": "1.0.0" } }), + member("pkg1"), + withPkg2(member("pkg2")), + ); + + expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=libs")); + expectOk(await run(dir.pkg1Dir, ["remove", "no-deps"])); + expectOk(await run(dir.pkg2Dir, ["remove", "no-deps"])); + + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:libs" }); + expect((await dir.pkg2()).dependencies).toBeUndefined(); + expect((await dir.root()).workspaces).toEqual({ + packages: ["packages/*"], + catalog: { "no-deps": "1.0.0" }, + catalogs: { libs: { "a-dep": "^1.0.10" } }, + }); + const lock = await dir.lock(); + expect(lock.catalog).toEqual({ "no-deps": "1.0.0" }); + expect(lock.catalogs).toEqual({ libs: { "a-dep": "^1.0.10" } }); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + // pnpm #8795: --frozen-lockfile notices a changed catalog entry. + for (const from of ["member", "root"] as const) { + test(`--frozen-lockfile passes after the add and fails once the entry is edited (from ${from})`, async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + + expectOk(await runAdd(from === "member" ? dir.pkg1Dir : dir.packageDir, "no-deps", "--catalog")); + + const frozen = await run(dir.packageDir, ["install", "--frozen-lockfile"]); + expect(frozen.stderr).not.toContain("error:"); + expect(frozen.exitCode).toBe(0); + + const root = await dir.root(); + root.workspaces.catalog["no-deps"] = "1.0.0"; + await write(dir.rootPath, JSON.stringify(root)); + + const { stderr, exitCode } = await run(dir.packageDir, ["install", "--frozen-lockfile"]); + expect(stderr).toContain("error:"); + expect(stderr).toContain("frozen-lockfile"); + expect(exitCode).toBe(1); + }); + } + + // pnpm #12115 / #11591: update never replaces a `catalog:` reference, even with an override or an explicit spec. + test("catalog: references survive bun update with an override", async () => { + const pkg1 = { name: "pkg1", dependencies: { "no-deps": "catalog:" } }; + const dir = await createDir( + { ...workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), overrides: { "no-deps": "1.0.0" } }, + pkg1, + ); + await runBunInstall(bunEnv, dir.packageDir); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + + for (const args of [["update"], ["update", "--recursive"], ["update", "no-deps"], ["update", "no-deps@1.1.0"]]) { + expectOk(await run(dir.pkg1Dir, args)); + expect(await dir.pkg1()).toEqual(pkg1); + } + expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog")); + + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); + const lock = await dir.lock(); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); + expect(lock.catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + + const { stderr, exitCode } = await run(dir.packageDir, ["install", "--frozen-lockfile"]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + + // pnpm #9660: overrides win at resolution, so the catalog records the overridden version. + test("an override decides what --catalog records", async () => { + const dir = await createDir({ ...workspacesObject({ catalog: {} }), overrides: { "no-deps": "1.0.0" } }); + + expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "^1.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + + const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }); + + describe("errors", () => { + test("root package.json without workspaces", async () => { + const { packageDir } = await registry.createTestDir({ + bunfigOpts: { linker: "hoisted", saveTextLockfile: true }, + files: { "package.json": JSON.stringify({ name: "solo" }) }, + }); + const before = await file(join(packageDir, "package.json")).text(); + + const { stderr, exitCode } = await runAdd(packageDir, "no-deps", "--catalog"); + + expect(stderr).toContain('error: --catalog requires a "workspaces" field in the root package.json'); + expect(await file(join(packageDir, "package.json")).text()).toBe(before); + expect(await file(join(packageDir, "bun.lock")).exists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + + test("positional without a name", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + const tarball = `${registry.registryUrl()}no-deps/-/no-deps-1.0.0.tgz`; + + const { stderr, exitCode } = await runAdd(dir.packageDir, "--catalog", tarball); + + expect(stderr).toContain(`error: --catalog can only add packages by name, but got "${tarball}"`); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(exitCode).toBe(1); + }); + + test("bun install --catalog without packages", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = await run(dir.packageDir, ["install", "--catalog"]); + + expect(stderr).toContain("error: --catalog requires at least one package to add"); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await file(join(dir.packageDir, "bun.lock")).exists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + + test("--catalog with --global", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + const globalDir = join(dir.packageDir, ".bun-global"); + + const { stderr, exitCode } = await run(dir.pkg1Dir, ["add", "no-deps", "--catalog", "-g"], { + ...bunEnv, + BUN_INSTALL: globalDir, + BUN_INSTALL_GLOBAL_DIR: join(globalDir, "install", "global"), + BUN_INSTALL_BIN: join(globalDir, "bin"), + }); + + expect(stderr).toContain("error: --catalog cannot be used with --global"); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await file(join(globalDir, "install", "global", "package.json")).exists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + }); +}); diff --git a/test/cli/install/bun-add-filter.test.ts b/test/cli/install/bun-add-filter.test.ts new file mode 100644 index 000000000000..abc36ec57244 --- /dev/null +++ b/test/cli/install/bun-add-filter.test.ts @@ -0,0 +1,1014 @@ +import { file, write } from "bun"; +import { afterAll, beforeAll, expect, test } from "bun:test"; +import { exists, mkdir } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe } from "harness"; +import { join } from "path"; + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +const ROOT = { name: "root", workspaces: ["packages/*"] }; +const API = { name: "api" }; +const WEB = { name: "web", dependencies: { "a-dep": "1.0.1" } }; +const PKG_A = { name: "pkg-a" }; +const PKG_B = { name: "pkg-b" }; + +type Workspace = "root" | "api" | "web" | "pkg-a" | "pkg-b"; + +type Linker = "hoisted" | "isolated"; + +// A string value is written verbatim, so a test can detect a rewrite that would otherwise be byte-identical. +async function makeMonorepo(extra: Partial> = {}, linker: Linker = "hoisted") { + const text = (value: object | string) => (typeof value === "string" ? value : JSON.stringify(value, null, 2)); + const { packageDir } = await registry.createTestDir({ + files: { + "package.json": text(extra.root ?? ROOT), + "packages/api/package.json": text(extra.api ?? API), + "packages/web/package.json": text(extra.web ?? WEB), + "packages/pkg-a/package.json": text(extra["pkg-a"] ?? PKG_A), + "packages/pkg-b/package.json": text(extra["pkg-b"] ?? PKG_B), + }, + bunfigOpts: { linker }, + }); + return packageDir; +} + +// `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. +async function run(args: string[], cwd: string, linker?: Linker) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...args, ...(linker ? ["--linker", linker] : [])], + cwd, + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +function pkgPath(dir: string, workspace: Workspace) { + return workspace === "root" ? join(dir, "package.json") : join(dir, "packages", workspace, "package.json"); +} + +function pkg(dir: string, workspace: Workspace) { + return file(pkgPath(dir, workspace)).json(); +} + +function pkgText(dir: string, workspace: Workspace) { + return file(pkgPath(dir, workspace)).text(); +} + +const WORKSPACES: Workspace[] = ["root", "api", "web", "pkg-a", "pkg-b"]; + +function allPackageJsons(dir: string) { + return Promise.all(WORKSPACES.map(w => pkg(dir, w))); +} + +function allPackageJsonTexts(dir: string) { + return Promise.all(WORKSPACES.map(w => pkgText(dir, w))); +} + +async function installOk(dir: string, linker?: Linker) { + const { stderr, exitCode } = await run(["install"], dir, linker); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); +} + +test.concurrent( + 'add --filter targets one workspace by name (and does not install an npm package called "api")', + async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await allPackageJsons(dir)).toEqual([ + ROOT, + { name: "api", dependencies: { "no-deps": "^2.0.0" } }, + WEB, + PKG_A, + PKG_B, + ]); + + expect(await file(join(dir, "node_modules", "no-deps", "package.json")).json()).toEqual({ + name: "no-deps", + version: "2.0.0", + }); + // node_modules/api is the linked workspace, not a registry package + expect(await file(join(dir, "node_modules", "api", "package.json")).json()).toEqual({ + name: "api", + dependencies: { "no-deps": "^2.0.0" }, + }); + const lockfile = await file(join(dir, "bun.lock")).text(); + expect(lockfile).toContain('"no-deps@2.0.0"'); + expect(lockfile).toContain('"api@workspace:packages/api"'); + expect(lockfile).not.toMatch(/"api@\d/); + }, +); + +test.concurrent("-F alias with --dev and --exact", async () => { + const dir = await makeMonorepo(); + + { + const { stderr, exitCode } = await run(["add", "a-dep", "-F", "api", "-d", "-E"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", devDependencies: { "a-dep": "1.0.10" } }); + expect(await pkg(dir, "root")).toEqual(ROOT); + expect(await pkg(dir, "web")).toEqual(WEB); + } + + // Same as unfiltered `bun add -d`: an entry that already exists in another list is updated in place. + { + const { stderr, exitCode } = await run(["add", "a-dep", "-F", "web", "-d", "-E"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "web")).toEqual({ name: "web", dependencies: { "a-dep": "1.0.10" } }); + expect(await pkg(dir, "api")).toEqual({ name: "api", devDependencies: { "a-dep": "1.0.10" } }); + expect(await pkg(dir, "root")).toEqual(ROOT); + } +}); + +test.concurrent("bun install --filter carries the filter through", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["install", "no-deps", "--filter", "api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "root")).toEqual(ROOT); + expect(await pkg(dir, "web")).toEqual(WEB); +}); + +test.concurrent("glob filter edits every match", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "pkg-*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await allPackageJsons(dir)).toEqual([ + ROOT, + API, + WEB, + { name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }, + { name: "pkg-b", dependencies: { "no-deps": "^2.0.0" } }, + ]); +}); + +test.concurrent("'*' edits every workspace including the root; '!' excludes", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "*", "--filter", "!api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await allPackageJsons(dir)).toEqual([ + { ...ROOT, dependencies: { "no-deps": "^2.0.0" } }, + API, + { name: "web", dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }, + { name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }, + { name: "pkg-b", dependencies: { "no-deps": "^2.0.0" } }, + ]); +}); + +test.concurrent("path filter, run from inside another workspace", async () => { + const dir = await makeMonorepo(); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "./packages/api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "root")).toEqual(ROOT); + expect(await pkg(dir, "web")).toEqual(WEB); + } + + { + const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "../api"], join(dir, "packages", "web")); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ + name: "api", + dependencies: { "a-dep": "^1.0.10", "no-deps": "^2.0.0" }, + }); + expect(await pkg(dir, "root")).toEqual(ROOT); + expect(await pkg(dir, "web")).toEqual(WEB); + } +}); + +test.concurrent("no match is an error and nothing is written", async () => { + const dir = await makeMonorepo(); + const before = await Promise.all(WORKSPACES.map(w => pkgText(dir, w))); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "does-not-exist"], dir); + expect(stderr).toContain("error: No workspace packages matched the filter"); + expect(exitCode).toBe(1); + + expect(await Promise.all(WORKSPACES.map(w => pkgText(dir, w)))).toEqual(before); + expect(await exists(join(dir, "bun.lock"))).toBeFalse(); + expect(await exists(join(dir, "node_modules"))).toBeFalse(); +}); + +test.concurrent("remove --filter removes from the matched workspace only", async () => { + const dir = await makeMonorepo({ + api: { name: "api", dependencies: { "no-deps": "^2.0.0" } }, + web: { name: "web", dependencies: { "no-deps": "^2.0.0" } }, + }); + + { + const { stderr, exitCode } = await run(["install"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await exists(join(dir, "bun.lock"))).toBeTrue(); + } + + { + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api" }); + expect(await pkg(dir, "web")).toEqual({ name: "web", dependencies: { "no-deps": "^2.0.0" } }); + expect(await exists(join(dir, "node_modules", "no-deps", "package.json"))).toBeTrue(); + } + + { + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "web"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "web")).toEqual({ name: "web" }); + expect(await exists(join(dir, "node_modules", "no-deps"))).toBeFalse(); + } +}); + +test.concurrent("remove --filter '*' with multiple packages", async () => { + const deps = { "no-deps": "^2.0.0", "a-dep": "^1.0.10" }; + const dir = await makeMonorepo({ + root: { ...ROOT, dependencies: deps }, + api: { name: "api", dependencies: deps }, + web: { name: "web", dependencies: deps }, + }); + + { + const { stderr, exitCode } = await run(["install"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + } + + const { stderr, exitCode } = await run(["remove", "no-deps", "a-dep", "--filter", "*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await allPackageJsons(dir)).toEqual([ROOT, API, { name: "web" }, PKG_A, PKG_B]); +}); + +test.concurrent("add --filter with an existing lockfile re-resolves only the added dep", async () => { + const dir = await makeMonorepo(); + + { + const { stderr, exitCode } = await run(["install"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + } + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "web"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "web")).toEqual({ + name: "web", + dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" }, + }); + expect(await file(join(dir, "node_modules", "a-dep", "package.json")).json()).toEqual({ + name: "a-dep", + version: "1.0.1", + }); + expect(await file(join(dir, "node_modules", "no-deps", "package.json")).json()).toEqual({ + name: "no-deps", + version: "2.0.0", + }); +}); + +test.concurrent("--dry-run writes nothing", async () => { + const dir = await makeMonorepo(); + const before = await pkgText(dir, "api"); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api", "--dry-run"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkgText(dir, "api")).toBe(before); + expect(await pkg(dir, "root")).toEqual(ROOT); +}); + +// Ported from pnpm's filtered add/remove suites (installing/commands/test/miscRecursive.ts, addRecursive.ts, +// remove/workspace.ts, pnpm/test/recursive/filter.ts) and pacquet's install_filters.rs. + +test.concurrent( + "add --only-missing --filter leaves an existing entry untouched in the target that has it", + async () => { + const dir = await makeMonorepo({ api: { name: "api", dependencies: { "no-deps": "1.0.0" } } }); + const apiBefore = await pkgText(dir, "api"); + + const { stderr, exitCode } = await run(["add", "no-deps", "--only-missing", "--filter", "*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkgText(dir, "api")).toBe(apiBefore); + expect(await allPackageJsons(dir)).toEqual([ + { ...ROOT, dependencies: { "no-deps": "^2.0.0" } }, + { name: "api", dependencies: { "no-deps": "1.0.0" } }, + { name: "web", dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }, + { name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }, + { name: "pkg-b", dependencies: { "no-deps": "^2.0.0" } }, + ]); + }, +); + +test.concurrent("add --only-missing --filter resolves from a target that received the request", async () => { + // The root (first target) already has the dep; web must still get the freshly resolved version. + const dir = await makeMonorepo({ root: { ...ROOT, dependencies: { "no-deps": "1.0.0" } } }); + const rootBefore = await pkgText(dir, "root"); + + const { stderr, exitCode } = await run( + ["add", "no-deps", "--only-missing", "--filter", "root", "--filter", "web"], + dir, + ); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkgText(dir, "root")).toBe(rootBefore); + expect(await pkg(dir, "web")).toEqual({ name: "web", dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }); +}); + +test.concurrent("add --only-missing --filter where every target already has it writes nothing", async () => { + const dir = await makeMonorepo({ + api: '{"name":"api","dependencies":{"no-deps":"1.0.0"}}', + web: '{"name":"web","dependencies":{"no-deps":"1.0.1"}}', + }); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run( + ["add", "no-deps", "--only-missing", "--filter", "api", "--filter", "web"], + dir, + ); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await allPackageJsonTexts(dir)).toEqual(before); +}); + +test.concurrent("remove --filter leaves targets that did not contain the dependency byte-identical", async () => { + const dir = await makeMonorepo({ + api: { name: "api", dependencies: { "no-deps": "^2.0.0" } }, + web: '{"name":"web","dependencies":{"a-dep":"1.0.1"}}', + "pkg-a": '{ "name": "pkg-a" }', + }); + await installOk(dir); + const [rootBefore, , webBefore, pkgABefore] = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api", "--filter", "web"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api" }); + expect(await pkgText(dir, "web")).toBe(webBefore); + expect(await pkgText(dir, "root")).toBe(rootBefore); + expect(await pkgText(dir, "pkg-a")).toBe(pkgABefore); + expect(await file(join(dir, "bun.lock")).text()).not.toContain("no-deps"); + expect(await exists(join(dir, "node_modules", "no-deps"))).toBeFalse(); +}); + +test.concurrent("remove --filter where no target contains the dependency writes nothing", async () => { + const dir = await makeMonorepo({ web: '{"name":"web"}' }); + await installOk(dir); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await allPackageJsonTexts(dir)).toEqual(before); +}); + +test.concurrent("add/remove --filter with the isolated linker links only the target workspace", async () => { + const dir = await makeMonorepo({}, "isolated"); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api"], dir, "isolated"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); + expect(await file(join(dir, "packages", "api", "node_modules", "no-deps", "package.json")).json()).toEqual({ + name: "no-deps", + version: "2.0.0", + }); + expect(await exists(join(dir, "packages", "web", "node_modules", "no-deps"))).toBeFalse(); + expect(await exists(join(dir, "node_modules", "no-deps"))).toBeFalse(); + } + + { + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api"], dir, "isolated"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api" }); + expect(await pkg(dir, "web")).toEqual(WEB); + // The isolated installer does not yet prune packages/api/node_modules/no-deps (same as unfiltered `bun remove`). + expect(await file(join(dir, "bun.lock")).text()).not.toContain("no-deps"); + } +}); + +test.concurrent("two name filters are unioned and unselected workspaces are byte-identical", async () => { + const dir = await makeMonorepo({ + root: '{ "name": "root", "workspaces": ["packages/*"] }', + "pkg-a": '{"name":"pkg-a"}', + }); + const [rootBefore, , , pkgABefore, pkgBBefore] = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api", "--filter", "web"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "web")).toEqual({ name: "web", dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }); + expect(await pkgText(dir, "root")).toBe(rootBefore); + expect(await pkgText(dir, "pkg-a")).toBe(pkgABefore); + expect(await pkgText(dir, "pkg-b")).toBe(pkgBBefore); +}); + +test.concurrent("remove --filter with no match names the pattern and touches nothing", async () => { + const dir = await makeMonorepo({ api: { name: "api", dependencies: { "no-deps": "^2.0.0" } } }); + await installOk(dir); + const before = await allPackageJsonTexts(dir); + const lockBefore = await file(join(dir, "bun.lock")).text(); + + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "nope"], dir); + expect(stderr).toContain('error: No workspace packages matched the filter "nope"'); + expect(exitCode).toBe(1); + + expect(await allPackageJsonTexts(dir)).toEqual(before); + expect(await file(join(dir, "bun.lock")).text()).toBe(lockBefore); + expect(await exists(join(dir, "node_modules", "no-deps", "package.json"))).toBeTrue(); +}); + +test.concurrent("a filter that matches nothing warns when another filter matched", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api", "--filter", "typo"], dir); + expect(stderr).toContain('warn: No workspace packages matched the filter "typo"'); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); +}); + +test.concurrent("negated filters that match nothing do not warn", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api", "--filter", "!nothing"], dir); + expect(stderr).not.toContain("warn:"); + expect(exitCode).toBe(0); +}); + +test.concurrent("an explicit version is written verbatim to every target", async () => { + const dir = await makeMonorepo(); + + { + const { stderr, exitCode } = await run(["add", "a-dep@1.0.1", "--filter", "pkg-*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "pkg-a")).toStrictEqual({ name: "pkg-a", dependencies: { "a-dep": "1.0.1" } }); + expect(await pkg(dir, "pkg-b")).toStrictEqual({ name: "pkg-b", dependencies: { "a-dep": "1.0.1" } }); + } + + { + const { stderr, exitCode } = await run(["add", "no-deps@^1.0.0", "--filter", "web"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "web")).toStrictEqual({ + name: "web", + dependencies: { "a-dep": "1.0.1", "no-deps": "^1.0.0" }, + }); + expect(await file(join(dir, "node_modules", "no-deps", "package.json")).json()).toEqual({ + name: "no-deps", + version: "1.1.0", + }); + } +}); + +test.concurrent("--peer and --optional target the right list in every selected workspace", async () => { + const dir = await makeMonorepo(); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--peer", "--filter", "pkg-*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "pkg-a")).toStrictEqual({ name: "pkg-a", peerDependencies: { "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "pkg-b")).toStrictEqual({ name: "pkg-b", peerDependencies: { "no-deps": "^2.0.0" } }); + } + + { + const { stderr, exitCode } = await run(["add", "a-dep", "--optional", "--filter", "pkg-a"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "pkg-a")).toStrictEqual({ + name: "pkg-a", + optionalDependencies: { "a-dep": "^1.0.10" }, + peerDependencies: { "no-deps": "^2.0.0" }, + }); + expect(await pkg(dir, "pkg-b")).toStrictEqual({ name: "pkg-b", peerDependencies: { "no-deps": "^2.0.0" } }); + } +}); + +test.concurrent("the root can be selected by name", async () => { + const dir = await makeMonorepo(); + const [, ...membersBefore] = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "root"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "root")).toEqual({ ...ROOT, dependencies: { "no-deps": "^2.0.0" } }); + const [, ...membersAfter] = await allPackageJsonTexts(dir); + expect(membersAfter).toEqual(membersBefore); +}); + +test.concurrent("'.' selects the workspace of the invoking directory", async () => { + const dir = await makeMonorepo(); + + { + const [, ...membersBefore] = await allPackageJsonTexts(dir); + const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "."], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "root")).toEqual({ ...ROOT, dependencies: { "a-dep": "^1.0.10" } }); + const [, ...membersAfter] = await allPackageJsonTexts(dir); + expect(membersAfter).toEqual(membersBefore); + } + + { + const rootBefore = await pkgText(dir, "root"); + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "."], join(dir, "packages", "web")); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "web")).toEqual({ name: "web", dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }); + expect(await pkgText(dir, "root")).toBe(rootBefore); + expect(await pkg(dir, "api")).toEqual(API); + } +}); + +test.concurrent("a directory excluded by a '!' entry in workspaces is never a target", async () => { + const dir = await makeMonorepo({ root: { name: "root", workspaces: ["packages/*", "!packages/web"] } }); + const webBefore = await pkgText(dir, "web"); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await allPackageJsons(dir)).toEqual([ + { name: "root", workspaces: ["packages/*", "!packages/web"], dependencies: { "no-deps": "^2.0.0" } }, + { name: "api", dependencies: { "no-deps": "^2.0.0" } }, + WEB, + { name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }, + { name: "pkg-b", dependencies: { "no-deps": "^2.0.0" } }, + ]); + expect(await pkgText(dir, "web")).toBe(webBefore); + } + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "web"], dir); + expect(stderr).toContain("error: No workspace packages matched the filter"); + expect(exitCode).toBe(1); + expect(await pkgText(dir, "web")).toBe(webBefore); + } +}); + +test.concurrent("workspaces: { packages: [...] } object form is honoured", async () => { + const dir = await makeMonorepo({ root: { name: "root", workspaces: { packages: ["packages/*"] } } }); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "root")).toEqual({ name: "root", workspaces: { packages: ["packages/*"] } }); +}); + +test.concurrent("scoped workspace names match a scope glob", async () => { + const dir = await makeMonorepo({ "pkg-a": { name: "@acme/a" }, "pkg-b": { name: "@acme/b" } }); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "@acme/*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await allPackageJsons(dir)).toEqual([ + ROOT, + API, + WEB, + { name: "@acme/a", dependencies: { "no-deps": "^2.0.0" } }, + { name: "@acme/b", dependencies: { "no-deps": "^2.0.0" } }, + ]); + } + + { + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "@acme/a"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "pkg-a")).toEqual({ name: "@acme/a" }); + expect(await pkg(dir, "pkg-b")).toEqual({ name: "@acme/b", dependencies: { "no-deps": "^2.0.0" } }); + } +}); + +test.concurrent("add --filter --lockfile-only edits package.json and bun.lock but installs nothing", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api", "--lockfile-only"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); + expect(await file(join(dir, "bun.lock")).text()).toContain('"no-deps@2.0.0"'); + expect(await exists(join(dir, "node_modules"))).toBeFalse(); +}); + +test.concurrent("running from a non-package subdirectory selects by filter and scaffolds nothing", async () => { + const dir = await makeMonorepo(); + await Promise.all([ + mkdir(join(dir, "packages", "api", "src")), + mkdir(join(dir, "tools", "scratch"), { recursive: true }), + ]); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "web"], join(dir, "packages", "api", "src")); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await exists(join(dir, "packages", "api", "src", "package.json"))).toBeFalse(); + } + + { + const { stderr, exitCode } = await run(["install", "a-dep", "--filter", "web"], join(dir, "tools", "scratch")); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await exists(join(dir, "tools", "scratch", "package.json"))).toBeFalse(); + } + + expect(await pkg(dir, "web")).toEqual({ name: "web", dependencies: { "a-dep": "^1.0.10", "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "api")).toEqual(API); + expect(await pkg(dir, "root")).toEqual(ROOT); +}); + +test.concurrent.each(["add", "install"])( + "%s --filter outside any project fails without creating package.json", + async cmd => { + const { packageDir } = await registry.createTestDir(); + + const { stderr, exitCode } = await run([cmd, "no-deps", "--filter", "web"], packageDir); + expect(stderr).toContain("error:"); + expect(exitCode).toBe(1); + + expect(await exists(join(packageDir, "package.json"))).toBeFalse(); + }, +); + +test.concurrent("filtered remove keeps the unselected workspace's lockfile entries", async () => { + const dir = await makeMonorepo({ + api: { name: "api", dependencies: { "no-deps": "^2.0.0" } }, + web: { name: "web", dependencies: { api: "workspace:*", "no-deps": "^2.0.0" } }, + }); + await installOk(dir); + + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const lockfile = JSON.parse( + await file(join(dir, "bun.lock")) + .text() + .then(t => t.replace(/,(\s*[}\]])/g, "$1")), + ); + expect(lockfile.workspaces["packages/api"]).toEqual({ name: "api" }); + expect(lockfile.workspaces["packages/web"]).toEqual({ + name: "web", + dependencies: { api: "workspace:*", "no-deps": "^2.0.0" }, + }); + expect(lockfile.packages["no-deps"]).toBeDefined(); + expect(await exists(join(dir, "node_modules", "no-deps", "package.json"))).toBeTrue(); +}); + +test.concurrent("bun install -F targets the workspace", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["install", "no-deps", "-F", "api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "root")).toEqual(ROOT); +}); + +test.concurrent("--filter outside a workspace: the lone package is the only candidate", async () => { + const { packageDir } = await registry.createTestDir({ files: { "package.json": JSON.stringify({ name: "solo" }) } }); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "*"], packageDir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await file(join(packageDir, "package.json")).json()).toEqual({ + name: "solo", + dependencies: { "no-deps": "^2.0.0" }, + }); + } + + { + const before = await file(join(packageDir, "package.json")).text(); + const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "other"], packageDir); + expect(stderr).toContain("error: No workspace packages matched the filter"); + expect(exitCode).toBe(1); + expect(await file(join(packageDir, "package.json")).text()).toBe(before); + } +}); + +test.concurrent("--filter with --global is rejected", async () => { + const dir = await makeMonorepo(); + const globalDir = join(dir, ".global"); + + await using proc = Bun.spawn({ + cmd: [bunExe(), "add", "no-deps", "-g", "--filter", "api"], + cwd: dir, + env: { ...bunEnv, BUN_INSTALL: globalDir, BUN_INSTALL_GLOBAL_DIR: join(globalDir, "install", "global") }, + stdout: "pipe", + stderr: "pipe", + }); + const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); + expect(stderr).toContain("error: --filter cannot be used with --global"); + expect(exitCode).toBe(1); + + expect(await pkg(dir, "api")).toEqual(API); + expect(await exists(join(globalDir, "install", "global", "node_modules", "no-deps"))).toBeFalse(); +}); + +// Round 3: bugs and non-bugs mined from pnpm's open issue tracker. + +function lockfileJson(dir: string) { + return file(join(dir, "bun.lock")) + .text() + .then(t => JSON.parse(t.replace(/,(\s*[}\]])/g, "$1"))); +} + +const VENDOR_FOO = { name: "foo", version: "1.0.0" }; + +async function addVendorFoo(dir: string) { + await write(join(dir, "vendor", "foo", "package.json"), JSON.stringify(VENDOR_FOO)); +} + +test.concurrent("package.json is written even when a root postinstall fails (pnpm#8627)", async () => { + const dir = await makeMonorepo({ root: { ...ROOT, scripts: { postinstall: "exit 1" } } }); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api"], dir); + expect(stderr).toContain('postinstall script from "root" exited with 1'); + expect(exitCode).toBe(1); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); + expect((await lockfileJson(dir)).workspaces["packages/api"]).toEqual({ + name: "api", + dependencies: { "no-deps": "^2.0.0" }, + }); +}); + +test.concurrent("a failed resolution leaves every target untouched", async () => { + const dir = await makeMonorepo(); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "does-not-exist-anywhere", "--filter", "pkg-*"], dir); + expect(stderr).toContain("error:"); + expect(exitCode).toBe(1); + + expect(await allPackageJsonTexts(dir)).toEqual(before); +}); + +test.concurrent("bun.lock records the resolved range for every target, not just the first", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "pkg-*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const { workspaces } = await lockfileJson(dir); + expect(workspaces["packages/pkg-a"]).toEqual({ name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }); + expect(workspaces["packages/pkg-b"]).toEqual({ name: "pkg-b", dependencies: { "no-deps": "^2.0.0" } }); + + const second = await run(["install"], dir); + expect(second.stderr).not.toContain("Saved lockfile"); + expect(second.exitCode).toBe(0); +}); + +test.concurrent("--only-missing resolves each request from a target that received it", async () => { + const dir = await makeMonorepo({ + root: { ...ROOT, dependencies: { "a-dep": "1.0.1" } }, + web: { name: "web", dependencies: { "no-deps": "1.0.0" } }, + }); + + const { stderr, exitCode } = await run( + ["add", "no-deps", "a-dep", "--only-missing", "--filter", "root", "--filter", "web"], + dir, + ); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "root")).toEqual({ ...ROOT, dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "web")).toEqual({ name: "web", dependencies: { "a-dep": "^1.0.10", "no-deps": "1.0.0" } }); +}); + +test.concurrent("a local path is relative to the cwd and re-spelled for each target (pnpm#9368)", async () => { + const dir = await makeMonorepo(); + await addVendorFoo(dir); + + const { stderr, exitCode } = await run(["add", "./vendor/foo", "--filter", "root", "--filter", "api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "root")).toEqual({ ...ROOT, dependencies: { foo: "./vendor/foo" } }); + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { foo: "../../vendor/foo" } }); + expect(await pkg(dir, "web")).toEqual(WEB); + const { workspaces } = await lockfileJson(dir); + expect(workspaces[""].dependencies).toEqual({ foo: "./vendor/foo" }); + expect(workspaces["packages/api"].dependencies).toEqual({ foo: "../../vendor/foo" }); + expect(await file(join(dir, "node_modules", "foo", "package.json")).json()).toEqual(VENDOR_FOO); + + const frozen = await run(["install", "--frozen-lockfile"], dir); + expect(frozen.stderr).not.toContain("error:"); + expect(frozen.exitCode).toBe(0); +}); + +test.concurrent("a file: path keeps its prefix and resolves from a nested cwd", async () => { + const dir = await makeMonorepo(); + await addVendorFoo(dir); + + const { stderr, exitCode } = await run( + ["add", "file:../../vendor/foo", "--filter", "api"], + join(dir, "packages", "web"), + ); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { foo: "file:../../vendor/foo" } }); + expect(await pkg(dir, "web")).toEqual(WEB); +}); + +test.concurrent("a local path that does not exist relative to the cwd fails and writes nothing", async () => { + const dir = await makeMonorepo(); + await addVendorFoo(dir); + const before = await allPackageJsonTexts(dir); + + // Relative to packages/api this would exist; paths are relative to the cwd (the root) instead. + const { stderr, exitCode } = await run(["add", "../../vendor/foo", "--filter", "api"], dir); + expect(stderr).toContain("error:"); + expect(exitCode).toBe(1); + + expect(await allPackageJsonTexts(dir)).toEqual(before); +}); + +test.concurrent("--filter writes the same entry as running bun add inside the workspace (pnpm#7194)", async () => { + const dir = await makeMonorepo(); + + for (const flags of [[], ["-E"]]) { + const dep = flags.length ? "no-deps" : "a-dep"; + const filtered = await run(["add", dep, ...flags, "--filter", "pkg-a"], dir); + expect(filtered.stderr).not.toContain("error:"); + expect(filtered.exitCode).toBe(0); + + const inside = await run(["add", dep, ...flags], join(dir, "packages", "api")); + expect(inside.stderr).not.toContain("error:"); + expect(inside.exitCode).toBe(0); + } + + const [pkgA, api] = await Promise.all([pkg(dir, "pkg-a"), pkg(dir, "api")]); + expect(pkgA.dependencies).toEqual({ "a-dep": "^1.0.10", "no-deps": "2.0.0" }); + expect(api.dependencies).toEqual(pkgA.dependencies); +}); + +test.concurrent("a directory name is not a package name for a scoped workspace (pnpm#5601)", async () => { + const dir = await makeMonorepo({ "pkg-a": { name: "@org/pkg-a" } }); + const before = await allPackageJsonTexts(dir); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "pkg-a"], dir); + expect(stderr).toContain('error: No workspace packages matched the filter "pkg-a"'); + expect(exitCode).toBe(1); + expect(await allPackageJsonTexts(dir)).toEqual(before); + } + + for (const pattern of ["*/pkg-a", "./packages/pkg-a"]) { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", pattern], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + } + expect(await pkg(dir, "pkg-a")).toEqual({ name: "@org/pkg-a", dependencies: { "no-deps": "^2.0.0" } }); +}); + +test.concurrent("a name glob does not cross the scope separator (pnpm#3452)", async () => { + const dir = await makeMonorepo({ "pkg-a": { name: "@org/date-utils" }, "pkg-b": { name: "string-utils" } }); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "*-utils"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await pkg(dir, "pkg-a")).toEqual({ name: "@org/date-utils" }); + expect(await pkg(dir, "pkg-b")).toEqual({ name: "string-utils", dependencies: { "no-deps": "^2.0.0" } }); + } + + { + const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "*/*-utils"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await pkg(dir, "pkg-a")).toEqual({ name: "@org/date-utils", dependencies: { "a-dep": "^1.0.10" } }); + expect(await pkg(dir, "pkg-b")).toEqual({ name: "string-utils", dependencies: { "no-deps": "^2.0.0" } }); + } +}); + +test.concurrent("a negated pattern wins regardless of flag order (pnpm#9354)", async () => { + const dir = await makeMonorepo(); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "!./packages/*", "--filter", "api"], dir); + expect(stderr).toContain("error: No workspace packages matched the filter"); + expect(exitCode).toBe(1); + + expect(await allPackageJsonTexts(dir)).toEqual(before); +}); + +test.concurrent("a path pattern naming the parent directory selects nothing (pnpm#5508)", async () => { + const dir = await makeMonorepo(); + const before = await allPackageJsonTexts(dir); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "./packages"], dir); + expect(stderr).toContain("error: No workspace packages matched the filter"); + expect(exitCode).toBe(1); + expect(await allPackageJsonTexts(dir)).toEqual(before); + } + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "./packages/*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await allPackageJsons(dir)).toEqual([ + ROOT, + { name: "api", dependencies: { "no-deps": "^2.0.0" } }, + { name: "web", dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }, + { name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }, + { name: "pkg-b", dependencies: { "no-deps": "^2.0.0" } }, + ]); + } +}); + +test.concurrent("an empty --filter value is a no-match error, not a crash (pnpm#5051)", async () => { + const dir = await makeMonorepo(); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", ""], dir); + expect(stderr).toContain('error: No workspace packages matched the filter ""'); + expect(exitCode).toBe(1); + + expect(await allPackageJsonTexts(dir)).toEqual(before); + expect(await exists(join(dir, "node_modules"))).toBeFalse(); +}); + +test.concurrent("path filters resolve against --cwd (pnpm#5270)", async () => { + const dir = await makeMonorepo(); + const { packageDir: elsewhere } = await registry.createTestDir(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--cwd", dir, "--filter", "./packages/api"], elsewhere); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "root")).toEqual(ROOT); + expect(await exists(join(elsewhere, "package.json"))).toBeFalse(); +}); diff --git a/test/cli/install/bun-audit-fix.test.ts b/test/cli/install/bun-audit-fix.test.ts new file mode 100644 index 000000000000..fec7838ac31b --- /dev/null +++ b/test/cli/install/bun-audit-fix.test.ts @@ -0,0 +1,1020 @@ +import { file, write } from "bun"; +import { afterAll, beforeAll, expect, test } from "bun:test"; +import { exists, readlink } from "fs/promises"; +import { + VerdaccioRegistry, + bunEnv, + bunExe, + gunzipJsonRequest, + normalizeBunSnapshot, + runBunInstall, + tempDir, +} from "harness"; +import { join } from "path"; + +const verdaccio = new VerdaccioRegistry(); + +beforeAll(async () => { + await verdaccio.start(); +}); + +afterAll(() => { + verdaccio.stop(); +}); + +type Advisory = { id: number; title: string; severity: string; url: string; vulnerable_versions: string }; + +function adv(range: string, id = 1): Advisory { + return { + id, + title: "test advisory", + severity: "high", + url: "https://example.invalid/advisory/" + id, + vulnerable_versions: range, + }; +} + +type RegistryOptions = { + // Serve the bulk response verbatim instead of filtering advisories by the submitted versions. + bulkResponse?: unknown; + bulkStatus?: number; + // Package names whose manifest requests answer 404; mutable so a test can break the registry after installing. + denyManifests?: Set; +}; + +// Answers the bulk-advisory endpoint itself and proxies everything else to verdaccio. +function startRegistry(advisories: Record, options: RegistryOptions = {}) { + return Bun.serve({ + port: 0, + async fetch(req) { + const url = new URL(req.url); + if (req.method === "POST" && url.pathname === "/-/npm/v1/security/advisories/bulk") { + if (options.bulkStatus) return new Response("registry exploded", { status: options.bulkStatus }); + if (options.bulkResponse !== undefined) return Response.json(options.bulkResponse); + const body: Record = await gunzipJsonRequest(req); + const out: Record = {}; + for (const [name, versions] of Object.entries(body)) { + const matching = (advisories[name] ?? []).filter(a => + versions.some(v => Bun.semver.satisfies(v, a.vulnerable_versions)), + ); + if (matching.length > 0) out[name] = matching; + } + return Response.json(out); + } + + if (options.denyManifests?.has(decodeURIComponent(url.pathname.slice(1)))) { + return new Response("not found", { status: 404 }); + } + + const up = await fetch(new URL(url.pathname + url.search, verdaccio.registryUrl()), { + method: req.method, + headers: { accept: req.headers.get("accept") ?? "*/*" }, + }); + return new Response(up.body, { + status: up.status, + headers: { "content-type": up.headers.get("content-type") ?? "application/octet-stream" }, + }); + }, + }); +} + +type Registry = ReturnType; + +function writeBunfig(dir: string, server: Registry) { + return write( + join(dir, "bunfig.toml"), + Bun.TOML.stringify({ + install: { cache: join(dir, ".bun-cache"), registry: server.url.href, saveTextLockfile: true }, + }), + ); +} + +async function setup(server: Registry, pkgJson: object, extraFiles: Record = {}) { + const dir = tempDir("audit-fix-", { "package.json": JSON.stringify(pkgJson), ...extraFiles }); + await writeBunfig(dir, server); + await runBunInstall(bunEnv, dir); + return dir; +} + +async function reinstall(dir: string, pkgJson: object) { + await write(join(dir, "package.json"), JSON.stringify(pkgJson)); + await runBunInstall(bunEnv, dir); +} + +async function run(dir: string, args: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...args], + env: bunEnv, + cwd: dir, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +function auditFix(dir: string, ...args: string[]) { + return run(dir, ["audit", "fix", ...args]); +} + +function audit(dir: string, ...args: string[]) { + return run(dir, ["audit", ...args]); +} + +function lock(dir: string) { + return file(join(dir, "bun.lock")).text(); +} + +async function installedVersion(dir: string, ...segments: string[]) { + return (await file(join(dir, "node_modules", ...segments, "package.json")).json()).version; +} + +// a-dep@1.0.2 stays installed after the range is widened because the still-satisfied edge is not re-resolved. +async function setupVulnerableADep(server: Registry) { + const dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2" } }); + await reinstall(dir, { name: "foo", dependencies: { "a-dep": "^1.0.2" } }); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + return dir; +} + +test.concurrent("fixes a direct dependency to the lowest safe version, not the newest", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setupVulnerableADep(server); + const pkgJsonBefore = await file(join(dir, "package.json")).text(); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(stdout).not.toContain("remaining"); + expect(stderr).toContain("Saved lockfile"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).not.toContain('"a-dep@1.0.2"'); + expect(lockfile).not.toContain('"a-dep@1.0.10"'); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + expect(await file(join(dir, "package.json")).text()).toBe(pkgJsonBefore); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + + await runBunInstall(bunEnv, dir, { frozenLockfile: true }); +}); + +test.concurrent("fixes a transitive dependency and leaves its dependent alone", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "one-range-dep": "1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.1.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"one-range-dep@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); + + await runBunInstall(bunEnv, dir, { frozenLockfile: true }); +}); + +test.concurrent("reports a fix that would violate a dependent's range and changes nothing", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-dep": "1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.1"'); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps@1.0.1 → 1.1.0 + one-dep@1.0.0 depends on no-deps@1.0.1 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("--dry-run prints the plan and writes nothing", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const { stdout, stderr, exitCode } = await auditFix(dir, "--dry-run"); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "fixing: + a-dep@1.0.2 → 1.0.4 + + Would fix 1 vulnerability in 1 package" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.2"); +}); + +test.concurrent("no fix available", async () => { + await using server = startRegistry({ "no-deps": [adv(">=2.0.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "^2.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@2.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "no fix available: + no-deps@2.0.0 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("no vulnerabilities", async () => { + await using server = startRegistry({}); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const lockBefore = await lock(dir); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).toBe("No vulnerabilities found\n"); + expect(stderr).toContain("bun audit fix v"); + expect(exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("peer dependency edges constrain the fix and are re-pointed", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0", "no-deps": "^1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"peer-deps-fixed@1.0.0"'); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stderr).not.toContain("incorrect peer dependency"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + + await runBunInstall(bunEnv, dir, { frozenLockfile: true }); +}); + +test.concurrent("instances of the same package are planned independently", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "^1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("fixing:"); + expect(stdout).toContain("no-deps@1.0.0 → 1.1.0"); + expect(stdout).toContain("requires a semver-major update:"); + expect(stdout).toContain("no-deps@1.0.1 → 1.1.0"); + expect(stdout).toContain("one-dep@1.0.0 depends on no-deps@1.0.1"); + // pnpm#10646: the advisory still applies to no-deps@1.0.1, so it is remaining, not fixed, and `bun audit` agrees. + expect(stdout).toContain("Fixed 0 vulnerabilities in 1 package"); + expect(stdout).toContain("1 vulnerability remaining"); + expect(exitCode).toBe(1); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.1.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(await installedVersion(dir, "no-deps")).toBe("1.1.0"); + + const recheck = await audit(dir); + expect(recheck.stdout).toContain("1 vulnerabilities (1 high)"); + expect(recheck.exitCode).toBe(1); + + await runBunInstall(bunEnv, dir, { frozenLockfile: true }); +}); + +// pnpm#10646: one advisory hitting two installed versions is one vulnerability, as `bun audit` counts it. +test.concurrent("one advisory across two fixable versions counts once", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); + const workspace = (name: string, range: string) => JSON.stringify({ name, dependencies: { "no-deps": range } }); + const root = { name: "root", workspaces: ["packages/*"] }; + using dir = await setup(server, root, { + "packages/a/package.json": workspace("a", "1.0.0"), + "packages/b/package.json": workspace("b", "1.0.1"), + }); + await write(join(dir, "packages", "a", "package.json"), workspace("a", "1.0.0 || >=1.1.0")); + await write(join(dir, "packages", "b", "package.json"), workspace("b", "^1.0.1")); + await runBunInstall(bunEnv, dir); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("fixing:\n no-deps@1.0.0 → 1.1.0\n no-deps@1.0.1 → 1.1.0\n"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(stdout).not.toContain("remaining"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.1.0"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.0.1"'); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); +}); + +// pnpm#13605: an optional peer that only a devDependency brought in is not a production dependency. +test.concurrent("bun audit --prod skips a dev-only optional peer of a production package", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "one-optional-peer-dep": "1.0.2" }, + devDependencies: { "no-deps": "1.0.0" }, + }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const all = await audit(dir); + expect(all.stdout).toContain("no-deps"); + expect(all.exitCode).toBe(1); + + const prod = await audit(dir, "--prod"); + expect(prod.stdout).toBe("No vulnerabilities found\n"); + expect(prod.exitCode).toBe(0); +}); + +// pnpm#13605: production status is per installed version, not per name. +test.concurrent( + "bun audit --prod skips a dev-only version of a name that is also a production dependency", + async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "one-dep": "1.0.0" }, + devDependencies: { "no-deps": "1.0.0" }, + }); + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + + const all = await audit(dir); + expect(all.stdout).toContain("no-deps"); + expect(all.exitCode).toBe(1); + + const prod = await audit(dir, "--prod"); + expect(prod.stdout).toBe("No vulnerabilities found\n"); + expect(prod.exitCode).toBe(0); + }, +); + +test.concurrent( + "bun audit --prod still reports the production version of a name that also has a dev version", + async () => { + await using server = startRegistry({ "no-deps": [adv("1.0.1")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "one-dep": "1.0.0" }, + devDependencies: { "no-deps": "1.0.0" }, + }); + + const prod = await audit(dir, "--prod"); + expect(prod.stdout).toContain("no-deps"); + expect(prod.stdout).toContain("1 vulnerabilities (1 high)"); + expect(prod.exitCode).toBe(1); + }, +); + +// pnpm#8943: a patch release on the current line wins over the next major that the range would also allow. +test.concurrent("prefers an in-line patch over a major that the range also allows", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "no-deps": ">=1.0.0" } }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@2.0.0"'); +}); + +// pnpm#12651 / #13824: an advisory with no released fix must not invent a version or leave bun.lock unusable. +test.concurrent("an advisory covering the newest release leaves a lockfile that still installs frozen", async () => { + await using server = startRegistry({ "a-dep": [adv("<=1.0.10")] }); + using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "^1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"a-dep@1.0.10"'); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "no fix available: + a-dep@1.0.10 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + + await runBunInstall(bunEnv, dir, { frozenLockfile: true }); +}); + +// pnpm#11101: a workspace package sharing a name with an advised npm package is not audited. +test.concurrent("a workspace package is never matched against an advisory for its name", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup( + server, + { name: "root", workspaces: ["packages/*"] }, + { "packages/no-deps/package.json": JSON.stringify({ name: "no-deps", version: "1.0.0" }) }, + ); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@workspace:packages/no-deps"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toBe("No vulnerabilities found\n"); + expect(exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); +}); + +// pnpm#10486 / #12487: a package kept alive only by a peer edge is still upgraded. +test.concurrent("fixes a package reachable only through a peer dependency edge", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.1.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + + await runBunInstall(bunEnv, dir, { frozenLockfile: true }); +}); + +test.concurrent("honours catalog ranges", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup( + server, + { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "1.0.0" } }, + { "packages/a/package.json": JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }) }, + ); + await reinstall(dir, { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "^1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); +}); + +test.concurrent("--ignore and --audit-level filter what gets fixed", async () => { + await using server = startRegistry({ "a-dep": [{ ...adv("<1.0.4", 7), severity: "low" }] }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const ignored = await auditFix(dir, "--ignore", "7"); + expect(ignored.stdout).toBe("No vulnerabilities found\n"); + expect(ignored.exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + + const belowLevel = await auditFix(dir, "--audit-level", "high"); + expect(belowLevel.stdout).toBe("No vulnerabilities found\n"); + expect(belowLevel.exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + + const fixed = await auditFix(dir); + expect(fixed.stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(fixed.exitCode).toBe(0); + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); +}); + +test.concurrent("rejects --json and extra arguments", async () => { + await using server = startRegistry({}); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const lockBefore = await lock(dir); + + const json = await auditFix(dir, "--json"); + expect(json.stderr).toContain("error: --json is not supported by bun audit fix"); + expect(json.exitCode).toBe(1); + + const extra = await auditFix(dir, "extra"); + expect(extra.stderr).toContain("error: bun audit fix does not take arguments"); + expect(extra.exitCode).toBe(1); + + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("refuses to run against a frozen lockfile", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + for (const flag of ["--frozen-lockfile", "--production"]) { + const { stderr, exitCode } = await auditFix(dir, flag); + expect(stderr).toContain("error: bun audit fix needs to write bun.lock, but the lockfile is frozen"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + } + + const dryRun = await auditFix(dir, "--frozen-lockfile", "--dry-run"); + expect(dryRun.stdout).toContain("Would fix 1 vulnerability in 1 package"); + expect(dryRun.exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("an optional peer edge does not keep the vulnerable version alive", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "optional-peer-deps": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "optional-peer-deps": "1.0.0", "no-deps": "^1.0.0" } }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + + await runBunInstall(bunEnv, dir, { frozenLockfile: true }); +}); + +test.concurrent("an advisory for an installed prerelease is matched and reported", async () => { + await using server = startRegistry({}, { bulkResponse: { "no-deps-backward-tags": [adv("<1.1.0")] } }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps-backward-tags": "1.0.0-rc.1" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps-backward-tags@1.0.0-rc.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps-backward-tags@1.0.0-rc.1 → 1.1.0 + foo depends on no-deps-backward-tags@1.0.0-rc.1 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("advisories that match no installed version are listed, not just counted", async () => { + await using server = startRegistry( + {}, + { bulkResponse: { "no-deps": [adv(">=5.0.0"), adv(">=5.0.0", 2), adv("not a range", 3)] } }, + ); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "not matched to an installed version: + no-deps@>=5.0.0 + no-deps@not a range + + No fixable vulnerabilities + 3 vulnerabilities remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("an unparsable advisory does not hide a real fix for the same package", async () => { + await using server = startRegistry({}, { bulkResponse: { "a-dep": [adv("<1.0.4"), adv("not a range", 2)] } }); + using dir = await setupVulnerableADep(server); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(stdout).toContain("a-dep@not a range"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(stdout).toContain("1 vulnerability remaining"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); +}); + +test.concurrent("a bundled dependency is never claimed as fixed", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "bundled-1": "1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.0"'); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps@1.0.0 → 1.0.1 + bundled-1@1.0.0 bundles no-deps@1.0.0 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + expect(await installedVersion(dir, "bundled-1", "node_modules", "no-deps")).toBe("1.0.0"); +}); + +test.concurrent("multiple advisories on one instance are cleared together", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4", 1), adv("<1.0.6", 2)] }); + using dir = await setupVulnerableADep(server); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.6"); + expect(stdout).toContain("Fixed 2 vulnerabilities in 1 package"); + expect(stdout).not.toContain("remaining"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.6"'); + expect(lockfile).not.toContain('"a-dep@1.0.4"'); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); +}); + +// pnpm fixtures/update-multiple: two advisories for one name with disjoint ranges. +test.concurrent("disjoint advisory ranges for one package are all avoided", async () => { + await using server = startRegistry({ "no-deps": [adv(">=1.0.0 <1.0.1", 1), adv(">=1.1.0 <2.0.0", 2)] }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "no-deps": ">=1.0.0" } }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + expect(await lock(dir)).toContain('"no-deps@1.0.1"'); +}); + +// pnpm fixtures/update-single-depth-2/responses/unfixable-vulnerability.json: npm-style advisory objects carry +// fields (findings, patched_versions, ...) that must be ignored, and `>=0.0.0` is unfixable. +test.concurrent("ignores unknown advisory fields and treats >=0.0.0 as unfixable", async () => { + await using server = startRegistry( + {}, + { + bulkResponse: { + "no-deps": [ + { + ...adv(">=0.0.0", 1234), + findings: [{ version: "1.0.0", paths: ["no-deps"] }], + patched_versions: "<0.0.0", + recommendation: "None", + cwe: ["CWE-1"], + cvss: { score: 0, vectorString: null }, + }, + ], + }, + }, + ); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "^1.0.0" } }); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "no fix available: + no-deps@1.1.0 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +// Ported verbatim from pnpm's test/audit/utils/responses/all-vulnerabilities-response.json (51 packages, 111 advisories). +test.concurrent("parses a real bulk response and only plans installed packages", async () => { + const bulkResponse = await file( + join(import.meta.dir, "registry/fixtures/audit/pnpm-all-vulnerabilities-response.json"), + ).json(); + await using server = startRegistry({}, { bulkResponse }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).not.toContain("fixing:"); + expect(stdout).not.toContain("no-deps"); + expect(stdout).toContain("not matched to an installed version:"); + expect(stdout).toContain(" axios@<0.21.2\n"); + expect(stdout).toContain(" semver@>=2.0.0-alpha <5.7.2\n"); + expect(stdout).toContain("111 vulnerabilities remaining"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("fixes two packages in one run and leaves the rest of the lockfile alone", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")], "no-deps": [adv("<1.0.1", 2)] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "a-dep": "1.0.2", "one-range-dep": "1.0.0", "no-deps": "1.0.0", "@types/is-number": "1.0.0" }, + }); + await reinstall(dir, { + name: "foo", + dependencies: { "a-dep": "^1.0.2", "one-range-dep": "1.0.0", "@types/is-number": "1.0.0" }, + }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"a-dep@1.0.2"'); + expect(lockBefore).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("fixing:\n a-dep@1.0.2 → 1.0.4\n no-deps@1.0.0 → 1.0.1\n"); + expect(stdout).toContain("Fixed 2 vulnerabilities in 2 packages"); + expect(exitCode).toBe(0); + + const lockAfter = await lock(dir); + const packageRows = lockBefore.split("\n").filter(line => /^ "[^"]+": \["/.test(line)); + const untouched = packageRows.filter(line => !line.includes('"a-dep@') && !line.includes('"no-deps@')); + expect(untouched.map(line => line.split('"')[1]).sort()).toEqual(["@types/is-number", "one-range-dep"]); + for (const line of untouched) expect(lockAfter).toContain(line); + expect(lockAfter).toContain('"a-dep@1.0.4"'); + expect(lockAfter).toContain('"no-deps@1.0.1"'); +}); + +test.concurrent("fixes a scoped package", async () => { + await using server = startRegistry({ "@types/is-number": [adv("<2.0.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "@types/is-number": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "@types/is-number": ">=1.0.0" } }); + expect(await lock(dir)).toContain('"@types/is-number@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("@types/is-number@1.0.0 → 2.0.0"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"@types/is-number@2.0.0"'); + expect(lockfile).not.toContain('"@types/is-number@1.0.0"'); + expect(await installedVersion(dir, "@types", "is-number")).toBe("2.0.0"); +}); + +test.concurrent("fixes an npm: alias pointing at a vulnerable package", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setup(server, { name: "foo", dependencies: { nd: "npm:a-dep@1.0.2" } }); + await reinstall(dir, { name: "foo", dependencies: { nd: "npm:a-dep@^1.0.2" } }); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).not.toContain('"a-dep@1.0.2"'); + expect(await installedVersion(dir, "nd")).toBe("1.0.4"); + + await runBunInstall(bunEnv, dir, { frozenLockfile: true }); +}); + +test.concurrent("an overrides pin is reported as the blocker", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "a-dep": "^1.0.2" }, + overrides: { "a-dep": "1.0.2" }, + }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + a-dep@1.0.2 → 1.0.4 + foo depends on a-dep@1.0.2 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +// pnpm fixtures/update-workspace-catalog-pinned: a pinned catalog entry blocks the fix. +test.concurrent("a pinned catalog entry is reported as the blocker", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup( + server, + { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "1.0.0" } }, + { "packages/a/package.json": JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }) }, + ); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps@1.0.0 → 1.0.1 + a depends on no-deps@1.0.0 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +// mismatched-peer-deps-lvl1 declares peer no-deps@<=1.0.1; its own dependency lvl2 declares peer no-deps@1.0.0, +// so the installs warn about an incorrect peer and cannot go through runBunInstall. +test.concurrent("a peer range is a blocker and is labelled with the peer dependent", async () => { + await using server = startRegistry({ "no-deps": [adv("<=1.0.1")] }); + const pkgJson = (noDeps: string) => + JSON.stringify({ name: "foo", dependencies: { "mismatched-peer-deps-lvl1": "1.0.0", "no-deps": noDeps } }); + using dir = tempDir("audit-fix-", { "package.json": pkgJson("1.0.1") }); + await writeBunfig(dir, server); + expect((await run(dir, ["install"])).exitCode).toBe(0); + await write(join(dir, "package.json"), pkgJson("^1.0.0")); + expect((await run(dir, ["install"])).exitCode).toBe(0); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.1 → 1.1.0"); + expect(stdout).toContain("mismatched-peer-deps-lvl1@1.0.0 depends on no-deps@<=1.0.1"); + expect(stdout).not.toContain("foo depends on"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("a depth-3 blocker names the immediate dependent", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-one-dep": "1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps@1.0.1 → 1.1.0 + one-dep@1.0.0 depends on no-deps@1.0.1 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +// Workspaces default to the isolated linker, so the linker is pinned to keep node_modules paths predictable. +test.concurrent("fixes a workspace member's dependency when run from the member directory", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + const member = (range: string) => JSON.stringify({ name: "a", dependencies: { "a-dep": range } }); + const rootPkgJson = JSON.stringify({ name: "root", workspaces: ["packages/*"] }); + using dir = tempDir("audit-fix-", { "package.json": rootPkgJson, "packages/a/package.json": member("1.0.2") }); + await writeBunfig(dir, server); + expect((await run(dir, ["install", "--linker", "hoisted"])).exitCode).toBe(0); + await write(join(dir, "packages", "a", "package.json"), member("^1.0.2")); + expect((await run(dir, ["install", "--linker", "hoisted"])).exitCode).toBe(0); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await run(join(dir, "packages", "a"), ["audit", "fix", "--linker", "hoisted"]); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).not.toContain('"a-dep@1.0.2"'); + expect(await exists(join(dir, "packages", "a", "bun.lock"))).toBeFalse(); + expect(await file(join(dir, "package.json")).text()).toBe(rootPkgJson); + expect(await file(join(dir, "packages", "a", "package.json")).text()).toBe(member("^1.0.2")); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + + const frozen = await run(dir, ["install", "--frozen-lockfile", "--linker", "hoisted"]); + expect(frozen.stderr).not.toContain("error:"); + expect(frozen.exitCode).toBe(0); +}); + +// `--linker isolated` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. +test.concurrent("isolated linker layout", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = tempDir("audit-fix-", { + "package.json": JSON.stringify({ name: "foo", dependencies: { "a-dep": "1.0.2" } }), + }); + await writeBunfig(dir, server); + expect((await run(dir, ["install", "--linker", "isolated"])).exitCode).toBe(0); + await write(join(dir, "package.json"), JSON.stringify({ name: "foo", dependencies: { "a-dep": "^1.0.2" } })); + expect((await run(dir, ["install", "--linker", "isolated"])).exitCode).toBe(0); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + expect(await readlink(join(dir, "node_modules", "a-dep"))).toContain("a-dep@1.0.2"); + + const { stdout, exitCode } = await auditFix(dir, "--linker", "isolated"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); + expect(await readlink(join(dir, "node_modules", "a-dep"))).toContain("a-dep@1.0.4"); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + + const frozen = await run(dir, ["install", "--frozen-lockfile", "--linker", "isolated"]); + expect(frozen.stderr).not.toContain("error:"); + expect(frozen.exitCode).toBe(0); +}); + +// Every a-dep release was published in 2023, so a 100-year minimum age gates all of them. +test.concurrent("a fix gated by --minimum-release-age is reported distinctly", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const gated = await auditFix(dir, "--minimum-release-age", "3153600000"); + expect(normalizeBunSnapshot(gated.stdout)).toMatchInlineSnapshot(` + "no fix available: + a-dep@1.0.2 (1.0.4 is newer than --minimum-release-age) + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(gated.exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + + const fixed = await auditFix(dir, "--minimum-release-age", "60"); + expect(fixed.stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(fixed.exitCode).toBe(0); + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); +}); + +test.concurrent("a failing bulk endpoint changes nothing", async () => { + await using server = startRegistry({}, { bulkStatus: 500 }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).not.toContain("fixing:"); + expect(stderr).toContain("audit request failed"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("a manifest that fails to download is reported, not fixed", async () => { + const denyManifests = new Set(); + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }, { denyManifests }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + denyManifests.add("a-dep"); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "no fix available: + a-dep@1.0.2 (failed to fetch the manifest) + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).toContain("a-dep"); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("refuses to run without a lockfile", async () => { + await using server = startRegistry({}); + using dir = tempDir("audit-fix-", { + "package.json": JSON.stringify({ name: "foo" }), + "bunfig.toml": Bun.TOML.stringify({ install: { registry: server.url.href } }), + }); + + const { stderr, exitCode } = await auditFix(dir); + expect(stderr).toContain("Lockfile not found"); + expect(exitCode).toBe(1); + expect(await exists(join(dir, "bun.lock"))).toBeFalse(); +}); + +test.concurrent("refuses --no-save before contacting the registry", async () => { + await using server = startRegistry({}, { bulkStatus: 500 }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const { stdout, stderr, exitCode } = await auditFix(dir, "--no-save"); + expect(stderr).toContain("error: bun audit fix needs to write bun.lock, but saving the lockfile is disabled"); + expect(stderr).not.toContain("audit request failed"); + expect(stdout).not.toContain("Fixed"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); +}); diff --git a/test/cli/install/bun-dedupe.test.ts b/test/cli/install/bun-dedupe.test.ts new file mode 100644 index 000000000000..9093cd2b8ea4 --- /dev/null +++ b/test/cli/install/bun-dedupe.test.ts @@ -0,0 +1,927 @@ +import { file, write } from "bun"; +import { afterAll, beforeAll, expect, test } from "bun:test"; +import { exists, realpath, rm } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe, normalizeBunSnapshot, readdirSorted, runBunInstall } from "harness"; +import { dirname, join } from "path"; + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +async function run(dir: string, ...cmd: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...cmd], + env: bunEnv, + cwd: dir, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +const dedupe = (dir: string, ...args: string[]) => run(dir, "dedupe", ...args); + +function lock(dir: string) { + return file(join(dir, "bun.lock")).text(); +} + +function firstLines(stdout: string, n: number) { + return normalizeBunSnapshot(stdout).split("\n").slice(0, n); +} + +function nodeModulesVersion(packageDir: string, ...segments: string[]) { + return file(join(packageDir, "node_modules", ...segments, "package.json")) + .json() + .then(pkg => pkg.version); +} + +// A still-satisfied range edge is never re-resolved by a later install, so adding an exact pin afterwards leaves a duplicate. +async function installTwice(packageDir: string, packageJson: string, first: object, second: object) { + await write(packageJson, JSON.stringify(first)); + await runBunInstall(bunEnv, packageDir); + await write(packageJson, JSON.stringify(second)); + await runBunInstall(bunEnv, packageDir); + return lock(packageDir); +} + +// one-range-dep@1.0.0 depends on no-deps@^1.0.0 (locked 1.1.0); root then pins no-deps@1.0.0. +async function setupRangeDuplicate(packageDir: string, packageJson: string, extra: Record = {}) { + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "foo", ...extra, dependencies: { "one-range-dep": "1.0.0" } }, + { name: "foo", ...extra, dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" } }, + ); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + expect(lockfile).toContain('"one-range-dep/no-deps"'); + return lockfile; +} + +test.concurrent("collapses a range onto the exact version that satisfies every edge", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + const pkgJsonBefore = await file(packageJson).text(); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(stderr).toContain("Saved lockfile"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const lockfile = await lock(packageDir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.1.0"'); + expect(lockfile).not.toContain('"one-range-dep/no-deps"'); + expect(await file(packageJson).text()).toBe(pkgJsonBefore); + + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +// pnpm/pnpm#6550: root's range moves up onto the version its dependents already use. +test.concurrent("prefers the highest version when several satisfy every edge", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + + await write( + packageJson, + JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "^1.0.0" } }), + ); + await runBunInstall(bunEnv, packageDir); + let lockfile = await lock(packageDir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.0.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + lockfile = await lock(packageDir); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + expect(await file(join(packageDir, "node_modules", "no-deps", "package.json")).json()).toEqual({ + name: "no-deps", + version: "1.1.0", + }); + + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +// pnpm/pnpm#13503: `--check` must not touch node_modules either. +test.concurrent("--check reports and exits 1 without writing", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockBefore = await setupRangeDuplicate(packageDir, packageJson); + const pkgJsonBefore = await file(packageJson).text(); + const nodeModulesBefore = await readdirSorted(join(packageDir, "node_modules")); + const nestedPkgJson = join(packageDir, "node_modules", "one-range-dep", "node_modules", "no-deps", "package.json"); + expect(await file(nestedPkgJson).json()).toEqual({ name: "no-deps", version: "1.1.0" }); + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + 1 duplicate version can be removed: no-deps@1.1.0" + `); + expect(check.stderr).toContain("note: run 'bun dedupe' to remove them"); + expect(check.stderr).not.toContain("Saved lockfile"); + expect(check.exitCode).toBe(1); + expect(await lock(packageDir)).toBe(lockBefore); + expect(await file(packageJson).text()).toBe(pkgJsonBefore); + expect(await readdirSorted(join(packageDir, "node_modules"))).toEqual(nodeModulesBefore); + expect(await file(nestedPkgJson).json()).toEqual({ name: "no-deps", version: "1.1.0" }); + + const dryRun = await dedupe(packageDir, "--dry-run"); + expect(normalizeBunSnapshot(dryRun.stdout)).toBe(normalizeBunSnapshot(check.stdout)); + expect(dryRun.stderr).not.toContain("Saved lockfile"); + expect(dryRun.exitCode).toBe(1); + expect(await lock(packageDir)).toBe(lockBefore); +}); + +test.concurrent("already deduplicated", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + + const first = await dedupe(packageDir); + expect(first.stderr).not.toContain("error:"); + expect(first.exitCode).toBe(0); + const lockAfterFirst = await lock(packageDir); + expect(lockAfterFirst).not.toContain('"no-deps@1.1.0"'); + + const second = await dedupe(packageDir); + expect(second.stdout).toContain("Already deduplicated."); + expect(second.stderr).not.toContain("Saved lockfile"); + expect(second.exitCode).toBe(0); + expect(await lock(packageDir)).toBe(lockAfterFirst); + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + Already deduplicated." + `); + expect(check.exitCode).toBe(0); + expect(await lock(packageDir)).toBe(lockAfterFirst); +}); + +test.concurrent("keeps versions that no single version can replace", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write( + packageJson, + JSON.stringify({ name: "foo", dependencies: { "one-dep": "1.0.0", "one-fixed-dep": "1.0.0" } }), + ); + await runBunInstall(bunEnv, packageDir); + const lockBefore = await lock(packageDir); + expect(lockBefore).toContain('"no-deps@1.0.0"'); + expect(lockBefore).toContain('"no-deps@1.0.1"'); + + const check = await dedupe(packageDir, "--check"); + expect(check.stdout).toContain("Already deduplicated."); + expect(check.exitCode).toBe(0); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(stdout).toContain("Already deduplicated."); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(0); + const lockAfter = await lock(packageDir); + expect(lockAfter).toContain('"no-deps@1.0.0"'); + expect(lockAfter).toContain('"no-deps@1.0.1"'); + expect(lockAfter).toBe(lockBefore); +}); + +test.concurrent("honours catalog ranges", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const root = (catalogRange: string, dependencies?: Record) => + JSON.stringify({ name: "root", workspaces: ["packages/*"], catalog: { "no-deps": catalogRange }, dependencies }); + + await Promise.all([ + write(packageJson, root("1.0.0")), + write( + join(packageDir, "packages", "a", "package.json"), + JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }), + ), + ]); + await runBunInstall(bunEnv, packageDir); + let lockfile = await lock(packageDir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + + await write(packageJson, root("^1.0.0")); + await runBunInstall(bunEnv, packageDir); + lockfile = await lock(packageDir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.1.0"'); + + await write(packageJson, root("^1.0.0", { "no-deps": "1.1.0" })); + await runBunInstall(bunEnv, packageDir); + lockfile = await lock(packageDir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.0.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + lockfile = await lock(packageDir); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +test.concurrent("override range wins over the edge's own range", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockBefore = await setupRangeDuplicate(packageDir, packageJson); + expect(lockBefore).toContain('\n "packages": {'); + const withOverride = lockBefore.replace( + '\n "packages": {', + '\n "overrides": {\n "no-deps": "1.1.0",\n },\n "packages": {', + ); + await write(join(packageDir, "bun.lock"), withOverride); + + const { stdout, stderr, exitCode } = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + 1 duplicate version can be removed: no-deps@1.0.0" + `); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(1); + expect(await lock(packageDir)).toBe(withOverride); +}); + +test.concurrent("errors without a lockfile", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); + + const apply = await dedupe(packageDir); + expect(apply.stderr).toContain("error: missing lockfile, nothing to dedupe"); + expect(apply.exitCode).toBe(1); + + const check = await dedupe(packageDir, "--check"); + expect(check.stderr).toContain("error: missing lockfile, nothing to dedupe"); + expect(check.exitCode).toBe(1); + + expect(await exists(join(packageDir, "bun.lock"))).toBeFalse(); +}); + +test.concurrent("rejects positional arguments", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); + await runBunInstall(bunEnv, packageDir); + const lockBefore = await lock(packageDir); + + const { stderr, exitCode } = await dedupe(packageDir, "no-deps"); + expect(stderr).toContain("error: bun dedupe does not take arguments"); + expect(exitCode).toBe(1); + expect(await lock(packageDir)).toBe(lockBefore); +}); + +test.concurrent("--help", async () => { + const { packageDir } = await registry.createTestDir(); + + const { stdout, exitCode } = await dedupe(packageDir, "--help"); + expect(stdout).toContain("bun dedupe"); + expect(stdout).toContain("--check"); + expect(exitCode).toBe(0); +}); + +test.concurrent("works with the isolated linker", async () => { + const { packageDir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); + // `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. + const install = async (...args: string[]) => { + const result = await run(packageDir, "install", ...args, "--linker", "isolated"); + expect(result.stderr).not.toContain("error:"); + expect(result.exitCode).toBe(0); + return result; + }; + await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0" } })); + await install(); + await write( + packageJson, + JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" } }), + ); + await install(); + let lockfile = await lock(packageDir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + // Store entry names may carry hash suffixes, so reach one-range-dep's store node_modules through the top-level link. + const nestedNoDeps = async () => { + const storeEntry = await realpath(join(packageDir, "node_modules", "one-range-dep")); + return file(join(dirname(storeEntry), "no-deps", "package.json")).json(); + }; + expect(await nestedNoDeps()).toEqual({ name: "no-deps", version: "1.1.0" }); + + const { stdout, stderr, exitCode } = await dedupe(packageDir, "--linker", "isolated"); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(stderr).toContain("Saved lockfile"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + lockfile = await lock(packageDir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.1.0"'); + await install("--frozen-lockfile"); + + expect(await nestedNoDeps()).toEqual({ name: "no-deps", version: "1.0.0" }); +}); + +// Loading re-hoists an optional peer onto a satisfying no-deps placed before it, so every other holder must sort after one-optional-peer-dep (hence the alias name). +test.concurrent("duplicate held only by an optional peer edge is deduplicated", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "foo", dependencies: { "one-optional-peer-dep": "1.0.2", "one-range-dep": "1.0.0" } }, + { + name: "foo", + dependencies: { + "one-optional-peer-dep": "1.0.2", + "one-range-dep": "1.0.0", + "z-fixed-dep": "npm:one-fixed-dep@1.0.0", + }, + }, + ); + expect(lockfile).toContain('"no-deps": ["no-deps@1.1.0"'); + expect(lockfile).toContain('"z-fixed-dep/no-deps": ["no-deps@1.0.0"'); + // Swap placements: root's no-deps becomes 1.0.0 (still satisfying one-range-dep and z-fixed-dep); only the peer edge resolves to 1.1.0. + const heldByPeer = lockfile + .replace('"no-deps": ["no-deps@1.1.0"', '"one-optional-peer-dep/no-deps": ["no-deps@1.1.0"') + .replace('"z-fixed-dep/no-deps": ["no-deps@1.0.0"', '"no-deps": ["no-deps@1.0.0"'); + await write(join(packageDir, "bun.lock"), heldByPeer); + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + 1 duplicate version can be removed: no-deps@1.1.0" + `); + expect(check.exitCode).toBe(1); + expect(await lock(packageDir)).toBe(heldByPeer); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).toContain('"no-deps@1.0.0"'); + expect(after).not.toContain('"no-deps@1.1.0"'); + expect((await dedupe(packageDir, "--check")).exitCode).toBe(0); + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +test.concurrent.each([ + ["--frozen-lockfile", "the lockfile is frozen"], + ["--production", "the lockfile is frozen"], + ["--no-save", "saving the lockfile is disabled"], +])("%s with duplicates errors instead of claiming removal", async (flag, reason) => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockBefore = await setupRangeDuplicate(packageDir, packageJson); + + const { stdout, stderr, exitCode } = await dedupe(packageDir, flag); + expect(stdout).not.toContain("Removed"); + expect(stderr).toContain(`error: 1 duplicate version can be removed, but ${reason}: no-deps@1.1.0`); + expect(stderr).toContain("note: run 'bun dedupe --check' to only report duplicates"); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(packageDir)).toBe(lockBefore); +}); + +test.concurrent("--frozen-lockfile succeeds when already deduplicated", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0" } })); + await runBunInstall(bunEnv, packageDir); + const lockBefore = await lock(packageDir); + + const { stdout, stderr, exitCode } = await dedupe(packageDir, "--frozen-lockfile"); + expect(stdout).toContain("Already deduplicated."); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await lock(packageDir)).toBe(lockBefore); +}); + +// Modelled on pnpm's workspace-with-lockfile-dupes fixture: root's ">=1.0.0" edge collapses onto the aliased 1.0.0, orphaning one-fixed-dep@2.0.0 -> no-deps@2.0.0. +test.concurrent("cascading removal lists every unreachable duplicate in name order", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "foo", dependencies: { "one-fixed-dep": ">=1.0.0" } }, + { name: "foo", dependencies: { "one-fixed-dep": ">=1.0.0", "ofd": "npm:one-fixed-dep@1.0.0" } }, + ); + for (const label of ['"one-fixed-dep@1.0.0"', '"one-fixed-dep@2.0.0"', '"no-deps@1.0.0"', '"no-deps@2.0.0"']) { + expect(lockfile).toContain(label); + } + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + 2 duplicate versions can be removed: no-deps@2.0.0, one-fixed-dep@2.0.0" + `); + expect(check.exitCode).toBe(1); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 2 duplicate versions: no-deps@2.0.0, one-fixed-dep@2.0.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).toContain('"one-fixed-dep@1.0.0"'); + expect(after).toContain('"no-deps@1.0.0"'); + expect(after).not.toContain('"one-fixed-dep@2.0.0"'); + expect(after).not.toContain('"no-deps@2.0.0"'); + expect(await nodeModulesVersion(packageDir, "one-fixed-dep")).toBe("1.0.0"); + expect(await nodeModulesVersion(packageDir, "ofd")).toBe("1.0.0"); + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +test.concurrent("multiple names removed in one run are sorted, scoped names first", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "foo", dependencies: { "two-range-deps": "1.0.0" } }, + { name: "foo", dependencies: { "two-range-deps": "1.0.0", "no-deps": "1.0.0", "@types/is-number": "1.0.0" } }, + ); + for (const label of ['"no-deps@1.0.0"', '"no-deps@1.1.0"', '"@types/is-number@1.0.0"', '"@types/is-number@2.0.0"']) { + expect(lockfile).toContain(label); + } + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + 2 duplicate versions can be removed: @types/is-number@2.0.0, no-deps@1.1.0" + `); + expect(check.exitCode).toBe(1); + + const { stdout, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 2 duplicate versions: @types/is-number@2.0.0, no-deps@1.1.0", + ]); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).not.toContain('"@types/is-number@2.0.0"'); + expect(after).not.toContain('"no-deps@1.1.0"'); + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +test.concurrent("workspace package edge is re-pointed when run from the workspace directory", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const workspaceDir = join(packageDir, "packages", "a"); + await write( + join(workspaceDir, "package.json"), + JSON.stringify({ name: "a", dependencies: { "one-range-dep": "1.0.0" } }), + ); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "root", workspaces: ["packages/*"] }, + { name: "root", workspaces: ["packages/*"], dependencies: { "no-deps": "1.0.0" } }, + ); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + + const { stdout, stderr, exitCode } = await dedupe(workspaceDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).not.toContain('"no-deps@1.1.0"'); + expect(await exists(join(workspaceDir, "bun.lock"))).toBeFalse(); + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +test.concurrent("corrupt bun.lock fails without rewriting it", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); + await runBunInstall(bunEnv, packageDir); + const corrupt = "{ not valid"; + await write(join(packageDir, "bun.lock"), corrupt); + + for (const args of [[], ["--check"]]) { + const { stderr, exitCode } = await dedupe(packageDir, ...args); + expect(stderr).toContain("failed to parse lockfile"); + expect(stderr).not.toContain("Ignoring lockfile"); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).not.toBe(0); + expect(await lock(packageDir)).toBe(corrupt); + } +}); + +test.concurrent("--check never creates node_modules", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockBefore = await setupRangeDuplicate(packageDir, packageJson); + await rm(join(packageDir, "node_modules"), { recursive: true }); + + const check = await dedupe(packageDir, "--check"); + expect(check.stdout).toContain("1 duplicate version can be removed: no-deps@1.1.0"); + expect(check.exitCode).toBe(1); + expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); + expect(await lock(packageDir)).toBe(lockBefore); + + await runBunInstall(bunEnv, packageDir, { savesLockfile: false }); + await dedupe(packageDir); + const lockDeduped = await lock(packageDir); + await rm(join(packageDir, "node_modules"), { recursive: true }); + + const clean = await dedupe(packageDir, "--check"); + expect(clean.stdout).toContain("Already deduplicated."); + expect(clean.exitCode).toBe(0); + expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); + expect(await lock(packageDir)).toBe(lockDeduped); +}); + +test.concurrent("never upgrades past the locked version", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); + await runBunInstall(bunEnv, packageDir); + const pinned = await lock(packageDir); + expect(pinned).toContain('"no-deps": "1.0.0"'); + const widened = pinned.replace('"no-deps": "1.0.0"', '"no-deps": "^1.0.0"'); + await Promise.all([ + write(join(packageDir, "bun.lock"), widened), + write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "^1.0.0" } })), + ]); + + const check = await dedupe(packageDir, "--check"); + expect(check.stdout).toContain("Already deduplicated."); + expect(check.exitCode).toBe(0); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(stdout).toContain("Already deduplicated."); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + const after = await lock(packageDir); + expect(after).toContain('"no-deps@1.0.0"'); + expect(after).not.toContain('"no-deps@1.1.0"'); + expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("1.0.0"); +}); + +test.concurrent.each([ + ["--ignore-scripts", false], + [undefined, true], +])("root lifecycle scripts with %s", async (flag, runs) => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson, { scripts: { postinstall: "echo ran > postinstall.txt" } }); + const marker = join(packageDir, "postinstall.txt"); + await rm(marker, { force: true }); + + const { stdout, stderr, exitCode } = await dedupe(packageDir, ...(flag ? [flag] : [])); + expect(stdout).toContain("Removed 1 duplicate version: no-deps@1.1.0"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await lock(packageDir)).not.toContain('"no-deps@1.1.0"'); + expect(await exists(marker)).toBe(runs); +}); + +// "Fewest versions" policy (pnpm/pnpm#4753, #6762): a direct dependency may be moved to an older version that still satisfies its range. +test.concurrent("root range collapses onto a transitive exact pin", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write( + packageJson, + JSON.stringify({ name: "foo", dependencies: { "a-dep": "^1.0.0", "uses-a-dep-3": "1.0.0" } }), + ); + await runBunInstall(bunEnv, packageDir); + const pkgJsonBefore = await file(packageJson).text(); + const lockfile = await lock(packageDir); + expect(lockfile).toContain('"a-dep@1.0.10"'); + expect(lockfile).toContain('"a-dep@1.0.3"'); + expect(await nodeModulesVersion(packageDir, "a-dep")).toBe("1.0.10"); + + const { stdout, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: a-dep@1.0.10", + ]); + expect(exitCode).toBe(0); + expect(await file(packageJson).text()).toBe(pkgJsonBefore); + + const after = await lock(packageDir); + expect(after).toContain('"a-dep@1.0.3"'); + expect(after).not.toContain('"a-dep@1.0.10"'); + expect(await nodeModulesVersion(packageDir, "a-dep")).toBe("1.0.3"); + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +test.concurrent("npm: alias edges are deduplicated by the aliased range", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "foo", dependencies: { "one-range-dep": "1.0.0" } }, + { name: "foo", dependencies: { "one-range-dep": "1.0.0", "nd": "npm:no-deps@1.0.0" } }, + ); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + + const { stdout, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(exitCode).toBe(0); + + expect(await lock(packageDir)).not.toContain('"no-deps@1.1.0"'); + expect(await nodeModulesVersion(packageDir, "nd")).toBe("1.0.0"); + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +test.concurrent("dist-tag edges keep the version the tag resolved to", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write( + packageJson, + JSON.stringify({ name: "foo", dependencies: { "dep-with-tags": "pre-1", "dwt": "npm:dep-with-tags@1.0.0" } }), + ); + await runBunInstall(bunEnv, packageDir); + const lockBefore = await lock(packageDir); + expect(lockBefore).toContain('"dep-with-tags@1.0.1"'); + expect(lockBefore).toContain('"dep-with-tags@1.0.0"'); + + const check = await dedupe(packageDir, "--check"); + expect(check.stdout).toContain("Already deduplicated."); + expect(check.exitCode).toBe(0); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(stdout).toContain("Already deduplicated."); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(0); + expect(await lock(packageDir)).toBe(lockBefore); + expect(await nodeModulesVersion(packageDir, "dep-with-tags")).toBe("1.0.1"); +}); + +test.concurrent("edges pointing at a patched version are never moved", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockBefore = await setupRangeDuplicate(packageDir, packageJson); + const patched = lockBefore.replace( + '\n "packages": {', + '\n "patchedDependencies": {\n "no-deps@1.1.0": "patches/no-deps@1.1.0.patch",\n },\n "packages": {', + ); + expect(patched).not.toBe(lockBefore); + await write(join(packageDir, "bun.lock"), patched); + + const { stdout, stderr, exitCode } = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + Already deduplicated." + `); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await lock(packageDir)).toBe(patched); +}); + +test.concurrent("--lockfile-only rewrites bun.lock without installing", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + const nested = () => nodeModulesVersion(packageDir, "one-range-dep", "node_modules", "no-deps"); + expect(await nested()).toBe("1.1.0"); + + const { stdout, stderr, exitCode } = await dedupe(packageDir, "--lockfile-only"); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(stderr).toContain("Saved lockfile"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await lock(packageDir)).not.toContain('"no-deps@1.1.0"'); + expect(await nested()).toBe("1.1.0"); + + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +test.concurrent("--silent keeps the exit codes", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockBefore = await setupRangeDuplicate(packageDir, packageJson); + + const check = await dedupe(packageDir, "--check", "--silent"); + expect(check.stdout).toBe(""); + expect(check.stderr).toBe(""); + expect(check.exitCode).toBe(1); + expect(await lock(packageDir)).toBe(lockBefore); + + const apply = await dedupe(packageDir, "--silent"); + expect(apply.stdout).toBe(""); + expect(apply.stderr).toBe(""); + expect(apply.exitCode).toBe(0); + expect(await lock(packageDir)).not.toContain('"no-deps@1.1.0"'); +}); + +// dedupe runs against the ranges recorded in bun.lock; a stale package.json is applied by the install that follows. +test.concurrent("stale package.json is resolved after the dedupe", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + await write( + packageJson, + JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "^2.0.0" } }), + ); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(stdout).toContain("Removed 1 duplicate version: no-deps@1.1.0"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).toContain('"no-deps@2.0.0"'); + expect(after).not.toContain('"no-deps@1.1.0"'); + expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("2.0.0"); + expect((await dedupe(packageDir, "--check")).exitCode).toBe(0); + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +// pnpm/pnpm#9213 (and #6619): one-fixed-dep@1.0.0 dies in this run, so its exact no-deps@1.0.0 edge must not drag the live "^1.0.0" edges down. +test.concurrent("a version removed by the run does not vote for its own dependencies", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write( + packageJson, + JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "^1.0.0" } }), + ); + await runBunInstall(bunEnv, packageDir); + expect(await lock(packageDir)).toContain('"no-deps@1.1.0"'); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "^1.0.0", "one-fixed-dep": "1.0.0" } }, + { + name: "foo", + dependencies: { + "one-range-dep": "1.0.0", + "no-deps": "^1.0.0", + "one-fixed-dep": ">=1.0.0", + "ofd2": "npm:one-fixed-dep@2.0.0", + }, + }, + ); + for (const label of [ + '"one-fixed-dep@1.0.0"', + '"one-fixed-dep@2.0.0"', + '"no-deps@1.0.0"', + '"no-deps@1.1.0"', + '"no-deps@2.0.0"', + ]) { + expect(lockfile).toContain(label); + } + expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("1.1.0"); + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + 2 duplicate versions can be removed: no-deps@1.0.0, one-fixed-dep@1.0.0" + `); + expect(check.exitCode).toBe(1); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 2 duplicate versions: no-deps@1.0.0, one-fixed-dep@1.0.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).toContain('"no-deps@1.1.0"'); + expect(after).toContain('"no-deps@2.0.0"'); + expect(after).toContain('"one-fixed-dep@2.0.0"'); + expect(after).not.toContain('"no-deps@1.0.0"'); + expect(after).not.toContain('"one-fixed-dep@1.0.0"'); + expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("1.1.0"); + expect(await nodeModulesVersion(packageDir, "one-fixed-dep")).toBe("2.0.0"); + + const recheck = await dedupe(packageDir, "--check"); + expect(recheck.stdout).toContain("Already deduplicated."); + expect(recheck.exitCode).toBe(0); + expect(await lock(packageDir)).toBe(after); + await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); +}); + +// A bundled edge is satisfied by the tarball's own copy, so it stays put and root's range collapses onto it instead. +test.concurrent("bundled edges are never re-pointed", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "foo", dependencies: { "no-deps": "^1.0.0" } }, + { name: "foo", dependencies: { "no-deps": "^1.0.0", "bundled-1": "1.0.0" } }, + ); + expect(lockfile).toContain('"bundled-1/no-deps": ["no-deps@1.0.0"'); + expect(lockfile).toContain('"bundled": true'); + expect(lockfile).toContain('{ "dependencies": { "no-deps": "1.0.0" } }'); + const widened = lockfile.replace( + '{ "dependencies": { "no-deps": "1.0.0" } }', + '{ "dependencies": { "no-deps": "^1.0.0" } }', + ); + await write(join(packageDir, "bun.lock"), widened); + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + 1 duplicate version can be removed: no-deps@1.1.0" + `); + expect(check.exitCode).toBe(1); + expect(await lock(packageDir)).toBe(widened); + + const { stdout, stderr, exitCode } = await dedupe(packageDir, "--lockfile-only"); + expect(firstLines(stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + const after = await lock(packageDir); + expect(after).toContain('"bundled-1/no-deps": ["no-deps@1.0.0"'); + expect(after).toContain('"bundled": true'); + expect(after).not.toContain('"no-deps@1.1.0"'); +}); + +// pnpm/pnpm#11238, #10329, #8446: dedupe never re-resolves, so it works with the registry down and ignores minimumReleaseAge. +test.concurrent("does not contact the registry", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + const closed = Bun.listen({ hostname: "localhost", port: 0, socket: { data() {} } }); + const port = closed.port; + closed.stop(true); + await write( + join(packageDir, "bunfig.toml"), + Bun.TOML.stringify({ + install: { + cache: join(packageDir, ".bun-cache"), + registry: `http://localhost:${port}/`, + minimumReleaseAge: 60 * 60 * 24 * 365 * 100, + }, + }), + ); + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + + 1 duplicate version can be removed: no-deps@1.1.0" + `); + expect(check.stderr).not.toContain("error:"); + expect(check.exitCode).toBe(1); + + const apply = await dedupe(packageDir, "--lockfile-only"); + expect(firstLines(apply.stdout, 3)).toEqual([ + "bun dedupe ()", + "", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(apply.stderr).toContain("Saved lockfile"); + expect(apply.stderr).not.toContain("error:"); + expect(apply.exitCode).toBe(0); + expect(await lock(packageDir)).not.toContain('"no-deps@1.1.0"'); + + const recheck = await dedupe(packageDir, "--check"); + expect(recheck.stdout).toContain("Already deduplicated."); + expect(recheck.stderr).not.toContain("error:"); + expect(recheck.exitCode).toBe(0); +}); diff --git a/test/cli/install/bun-pm-licenses.test.ts b/test/cli/install/bun-pm-licenses.test.ts new file mode 100644 index 000000000000..8146a909ce22 --- /dev/null +++ b/test/cli/install/bun-pm-licenses.test.ts @@ -0,0 +1,662 @@ +import { spawn } from "bun"; +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { existsSync, readFileSync, readdirSync, rmSync, writeFileSync } from "fs"; +import { VerdaccioRegistry, bunEnv, bunExe, normalizeBunSnapshot, tempDir } from "harness"; +import { join } from "path"; +import { pathToFileURL } from "url"; + +type Linker = "hoisted" | "isolated"; +type Files = Record; + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +const gitEnv = { + ...bunEnv, + GIT_CONFIG_NOSYSTEM: "1", + GIT_AUTHOR_NAME: "Test", + GIT_AUTHOR_EMAIL: "test@example.com", + GIT_COMMITTER_NAME: "Test", + GIT_COMMITTER_EMAIL: "test@example.com", +}; + +const fixturePackageJson = JSON.stringify({ + name: "licenses-fixture", + version: "1.0.0", + dependencies: { + "resolve": "1.9.0", + "no-deps": "1.0.0", + "one-dep": "1.0.0", + }, + devDependencies: { + "a-dep": "1.0.1", + }, +}); + +function pkg(fields: Record) { + return JSON.stringify({ name: "licenses-fixture", version: "1.0.0", ...fields }); +} + +// Shape of pnpm's commands/test/licenses/fixtures/workspace-licenses: dependency-less private root, foo with deps+devDeps, bar with deps. +const monorepoFiles: Files = { + "package.json": JSON.stringify({ name: "mono", private: true, workspaces: ["packages/*"] }), + "packages/foo/package.json": JSON.stringify({ + name: "foo", + version: "1.0.0", + dependencies: { "no-deps": "1.0.0" }, + devDependencies: { "a-dep": "1.0.1" }, + }), + "packages/bar/package.json": JSON.stringify({ + name: "bar", + version: "1.0.0", + dependencies: { resolve: "1.9.0" }, + }), +}; + +// `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. +async function install(dir: string, linker: Linker, ...args: string[]) { + await using proc = spawn({ + cmd: [bunExe(), "install", "--linker", linker, ...args], + env: bunEnv, + cwd: dir, + stdin: "ignore", + stdout: "pipe", + stderr: "pipe", + }); + const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); +} + +async function setup(linker: Linker = "hoisted", files: Files = { "package.json": fixturePackageJson }) { + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker }, files }); + await install(packageDir, linker); + return packageDir; +} + +// `bun install --production` never writes a lockfile, so the lockfile (with its dev-inclusive tree) is created first. +async function setupProductionInstall(files: Files) { + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" }, files }); + await install(packageDir, "hoisted", "--lockfile-only"); + await install(packageDir, "hoisted", "--production"); + return packageDir; +} + +async function licenses(dir: string, ...args: string[]): Promise<[string, string, number]> { + await using proc = spawn({ + cmd: [bunExe(), "pm", "licenses", ...args], + env: bunEnv, + cwd: dir, + stdout: "pipe", + stderr: "pipe", + }); + return await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); +} + +async function licensesJson(dir: string, ...args: string[]) { + const [stdout, stderr, exitCode] = await licenses(dir, ...args, "--json"); + expect(stderr).toBe(""); + const parsed = JSON.parse(stdout); + expect(exitCode).toBe(0); + return parsed as Record; +} + +function names(parsed: Record) { + return Object.values(parsed) + .flat() + .map(entry => entry.name) + .sort(); +} + +function patchInstalledManifest(dir: string, pkg: string, fields: Record) { + const path = join(dir, "node_modules", pkg, "package.json"); + const manifest = { ...JSON.parse(readFileSync(path, "utf8")), ...fields }; + for (const [key, value] of Object.entries(fields)) if (value === undefined) delete manifest[key]; + writeFileSync(path, JSON.stringify(manifest)); +} + +const fullJson = { + MIT: [ + { + name: "path-parse", + versions: ["1.0.6"], + homepage: "https://github.com/jbgutierrez/path-parse#readme", + author: "Javier Blanco ", + }, + { + name: "resolve", + versions: ["1.9.0"], + author: "James Halliday (http://substack.net)", + }, + ], + Unknown: [ + { name: "a-dep", versions: ["1.0.1"] }, + { name: "no-deps", versions: ["1.0.0", "1.0.1"] }, + { name: "one-dep", versions: ["1.0.0"] }, + ], +}; + +const prodJson = { + MIT: fullJson.MIT, + Unknown: [ + { name: "no-deps", versions: ["1.0.0", "1.0.1"] }, + { name: "one-dep", versions: ["1.0.0"] }, + ], +}; + +describe("bun pm licenses", () => { + let hoistedDir: string; + + beforeAll(async () => { + hoistedDir = await setup(); + }); + + test.concurrent("text output groups packages by license, Unknown last", async () => { + const [stdout, stderr, exitCode] = await licenses(hoistedDir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "MIT (2) + ├── path-parse@1.0.6 + └── resolve@1.9.0 + + Unknown (4) + ├── a-dep@1.0.1 + ├── no-deps@1.0.0 + ├── no-deps@1.0.1 + └── one-dep@1.0.0" + `); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + + test.concurrent("--json shape", async () => { + const [stdout, stderr, exitCode] = await licenses(hoistedDir, "--json"); + const parsed = JSON.parse(stdout); + expect(parsed).toEqual(fullJson); + expect(Object.keys(parsed)).toEqual(["MIT", "Unknown"]); + expect(parsed.MIT[1]).not.toHaveProperty("homepage"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + + test.concurrent("legacy license shapes", async () => { + const dir = await setup(); + patchInstalledManifest(dir, "a-dep", { license: { type: "BSD-3-Clause", url: "x" } }); + patchInstalledManifest(dir, "one-dep", { licenses: [{ type: "MIT" }, { type: "Apache-2.0" }] }); + patchInstalledManifest(dir, "no-deps", { licenses: { type: "ISC" } }); + + const parsed = await licensesJson(dir); + expect(Object.keys(parsed)).toEqual(["(MIT OR Apache-2.0)", "BSD-3-Clause", "ISC", "MIT", "Unknown"]); + expect(parsed["ISC"]).toEqual([{ name: "no-deps", versions: ["1.0.0"] }]); + expect(parsed["Unknown"]).toEqual([{ name: "no-deps", versions: ["1.0.1"] }]); + expect(parsed["(MIT OR Apache-2.0)"]).toEqual([{ name: "one-dep", versions: ["1.0.0"] }]); + expect(parsed["BSD-3-Clause"]).toEqual([{ name: "a-dep", versions: ["1.0.1"] }]); + expect(parsed["MIT"].map(entry => entry.name)).toEqual(["path-parse", "resolve"]); + }); + + // pnpm license-resolver/test/parseLicenseFromManifest.test.ts, replayed against installed manifests. + test.concurrent("`license` array and legacy `name` key", async () => { + const dir = await setup(); + patchInstalledManifest(dir, "a-dep", { license: [{ type: "MIT" }, { type: "Apache-2.0" }] }); + patchInstalledManifest(dir, "one-dep", { licenses: [{ name: "ISC" }] }); + patchInstalledManifest(dir, "no-deps", { license: ["BSD-2-Clause"] }); + patchInstalledManifest(dir, "one-dep/node_modules/no-deps", { license: { name: "0BSD" } }); + + const parsed = await licensesJson(dir); + expect(parsed["(MIT OR Apache-2.0)"]).toEqual([{ name: "a-dep", versions: ["1.0.1"] }]); + expect(parsed["ISC"]).toEqual([{ name: "one-dep", versions: ["1.0.0"] }]); + expect(parsed["BSD-2-Clause"]).toEqual([{ name: "no-deps", versions: ["1.0.0"] }]); + expect(parsed["0BSD"]).toEqual([{ name: "no-deps", versions: ["1.0.1"] }]); + expect(parsed).not.toHaveProperty("Unknown"); + }); + + test.concurrent("empty `license` falls through to `licenses`; `license` wins when both are present", async () => { + const dir = await setup(); + patchInstalledManifest(dir, "a-dep", { license: "", licenses: [{ type: "MIT" }] }); + patchInstalledManifest(dir, "one-dep", { license: "Apache-2.0", licenses: [{ type: "MIT" }] }); + + const parsed = await licensesJson(dir); + expect(parsed["MIT"].map(entry => entry.name)).toEqual(["a-dep", "path-parse", "resolve"]); + expect(parsed["Apache-2.0"]).toEqual([{ name: "one-dep", versions: ["1.0.0"] }]); + }); + + test.concurrent("non-string license shapes are Unknown; entries with non-string type are skipped", async () => { + const dir = await setup(); + patchInstalledManifest(dir, "a-dep", { license: 42 }); + patchInstalledManifest(dir, "one-dep", { licenses: [] }); + patchInstalledManifest(dir, "no-deps", { licenses: [{ url: "x" }] }); + patchInstalledManifest(dir, "resolve", { license: undefined, licenses: [{ type: 42 }, { type: "MIT" }] }); + + const parsed = await licensesJson(dir); + expect(parsed).toEqual({ + MIT: [fullJson.MIT[0], { name: "resolve", versions: ["1.9.0"], author: fullJson.MIT[1].author }], + Unknown: fullJson.Unknown, + }); + }); + + test.concurrent("repeated legacy entries are not deduplicated", async () => { + const dir = await setup(); + patchInstalledManifest(dir, "one-dep", { licenses: [{ type: "MIT" }, { type: "MIT" }, { type: "Apache-2.0" }] }); + + const parsed = await licensesJson(dir); + expect(parsed["(MIT OR MIT OR Apache-2.0)"]).toEqual([{ name: "one-dep", versions: ["1.0.0"] }]); + }); + + test.concurrent("one package with two versions: per-license grouping, metadata from the newest version", async () => { + const dir = await setup(); + patchInstalledManifest(dir, "no-deps", { license: "MIT", homepage: "https://example.com/old" }); + patchInstalledManifest(dir, "one-dep/node_modules/no-deps", { + license: "MIT", + homepage: "https://example.com/new", + }); + + const parsed = await licensesJson(dir); + expect(parsed["MIT"]).toEqual([ + { name: "no-deps", versions: ["1.0.0", "1.0.1"], homepage: "https://example.com/new" }, + ...fullJson.MIT, + ]); + expect(parsed["Unknown"]).toEqual([ + { name: "a-dep", versions: ["1.0.1"] }, + { name: "one-dep", versions: ["1.0.0"] }, + ]); + }); + + test.concurrent("versions are ordered by semver, names bytewise", async () => { + const dir = await setup("hoisted", { + "package.json": pkg({ dependencies: { "uses-a-dep-9": "1.0.0", "uses-a-dep-10": "1.0.0" } }), + }); + + const [stdout, stderr, exitCode] = await licenses(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "Unknown (4) + ├── a-dep@1.0.9 + ├── a-dep@1.0.10 + ├── uses-a-dep-10@1.0.0 + └── uses-a-dep-9@1.0.0" + `); + expect(stderr).toBe(""); + expect(exitCode).toBe(0); + + const parsed = await licensesJson(dir); + expect(parsed["Unknown"][0]).toEqual({ name: "a-dep", versions: ["1.0.9", "1.0.10"] }); + }); + + test.concurrent.each(["--prod", "--production", "-p", "-P"])("%s omits devDependencies (--json)", async flag => { + const parsed = await licensesJson(hoistedDir, flag); + expect(JSON.stringify(parsed)).not.toContain("a-dep"); + expect(parsed).toEqual(prodJson); + }); + + test.concurrent("--prod omits devDependencies (text)", async () => { + const [stdout, stderr, exitCode] = await licenses(hoistedDir, "--prod"); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "MIT (2) + ├── path-parse@1.0.6 + └── resolve@1.9.0 + + Unknown (3) + ├── no-deps@1.0.0 + ├── no-deps@1.0.1 + └── one-dep@1.0.0" + `); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + + test.concurrent("--prod keeps optionalDependencies", async () => { + const dir = await setup("hoisted", { + "package.json": pkg({ + dependencies: { "no-deps": "1.0.0" }, + optionalDependencies: { "one-dep": "1.0.0" }, + devDependencies: { "a-dep": "1.0.1" }, + }), + }); + + expect(await licensesJson(dir, "--prod")).toEqual({ + Unknown: [ + { name: "no-deps", versions: ["1.0.0", "1.0.1"] }, + { name: "one-dep", versions: ["1.0.0"] }, + ], + }); + expect(names(await licensesJson(dir))).toEqual(["a-dep", "no-deps", "one-dep"]); + }); + + test.concurrent("os/cpu-skipped optional dependencies are omitted, their parent is listed", async () => { + const dir = await setup("hoisted", { "package.json": pkg({ dependencies: { "optional-native": "1.0.0" } }) }); + const installedNatives = [ + "native-bar-x64", + "native-foo-x64", + "native-foo-x86", + "native-libc-glibc", + "native-libc-musl", + ] + .filter(name => existsSync(join(dir, "node_modules", name, "package.json"))) + .map(name => ({ name, versions: ["1.0.0"] })); + expect(installedNatives.map(entry => entry.name)).not.toContain("native-foo-x64"); + + const [stdout, stderr, exitCode] = await licenses(dir, "--json"); + expect(JSON.parse(stdout)).toEqual({ + Unknown: [...installedNatives, { name: "optional-native", versions: ["1.0.0"] }], + }); + expect(stderr).toBe(""); + expect(exitCode).toBe(0); + }); + + test.concurrent("isolated linker", async () => { + const dir = await setup("isolated"); + const [expected] = await licenses(hoistedDir); + const [stdout, stderr, exitCode] = await licenses(dir); + expect(stdout).toBe(expected); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "MIT (2) + ├── path-parse@1.0.6 + └── resolve@1.9.0 + + Unknown (4) + ├── a-dep@1.0.1 + ├── no-deps@1.0.0 + ├── no-deps@1.0.1 + └── one-dep@1.0.0" + `); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + + test.concurrent("isolated linker: scoped transitive dependency is found through the store", async () => { + const dir = await setup("isolated", { "package.json": pkg({ dependencies: { "two-range-deps": "1.0.0" } }) }); + expect(existsSync(join(dir, "node_modules", "@types", "is-number"))).toBeFalse(); + expect(existsSync(join(dir, "node_modules", ".bun", "@types+is-number@2.0.0"))).toBeTrue(); + + expect(await licensesJson(dir)).toEqual({ + Unknown: [ + { name: "@types/is-number", versions: ["2.0.0"] }, + { name: "no-deps", versions: ["1.1.0"] }, + { name: "two-range-deps", versions: ["1.0.0"] }, + ], + }); + }); + + test.concurrent("isolated linker: store entries with a peer hash suffix are matched", async () => { + const dir = await setup("isolated", { + "package.json": pkg({ dependencies: { "peer-deps-lvl0": "1.0.0" } }), + }); + const store = readdirSync(join(dir, "node_modules", ".bun")); + expect(store.some(name => /^peer-deps-lvl[12]@1\.0\.0\+[0-9a-f]{16}$/.test(name))).toBeTrue(); + + expect(await licensesJson(dir)).toEqual({ + Unknown: [ + { name: "no-deps", versions: ["1.0.0"] }, + { name: "peer-deps-lvl0", versions: ["1.0.0"] }, + { name: "peer-deps-lvl1", versions: ["1.0.0"] }, + { name: "peer-deps-lvl2", versions: ["1.0.0"] }, + ], + }); + }); + + // pnpm 'should work with file protocol dependency' (fixtures/with-file-protocol): a license-less folder dep is listed as Unknown. + test.concurrent.each(["hoisted", "isolated"] as Linker[])("file: dependency is listed (%s)", async linker => { + const dir = await setup(linker, { + "package.json": pkg({ dependencies: { "no-deps": "1.0.0", "sub-dep": "file:./sub-dep" } }), + "sub-dep/package.json": JSON.stringify({ name: "sub-dep", version: "2.5.0" }), + }); + + expect(await licensesJson(dir)).toEqual({ + Unknown: [ + { name: "no-deps", versions: ["1.0.0"] }, + { name: "sub-dep", versions: ["sub-dep"] }, + ], + }); + }); + + test.concurrent.each(["hoisted", "isolated"] as Linker[])("link: dependency is not listed (%s)", async linker => { + const { packageDir } = await registry.createTestDir({ + bunfigOpts: { linker }, + files: { + "package.json": pkg({ dependencies: { "no-deps": "1.0.0", "linked": "link:linked" } }), + "linked/package.json": JSON.stringify({ name: "linked", version: "1.0.0", license: "MIT" }), + }, + }); + const env = { ...bunEnv, BUN_INSTALL_GLOBAL_DIR: join(packageDir, ".global") }; + for (const [cmd, cwd] of [ + [[bunExe(), "link"], join(packageDir, "linked")], + [[bunExe(), "install", "--linker", linker], packageDir], + ] as const) { + await using proc = spawn({ cmd: [...cmd], env, cwd, stdin: "ignore", stdout: "pipe", stderr: "pipe" }); + const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + } + expect(existsSync(join(packageDir, "node_modules", "linked", "package.json"))).toBeTrue(); + + await using proc = spawn({ + cmd: [bunExe(), "pm", "licenses", "--json"], + env, + cwd: packageDir, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, exitCode] = await Promise.all([proc.stdout.text(), proc.exited]); + expect(JSON.parse(stdout)).toEqual({ Unknown: [{ name: "no-deps", versions: ["1.0.0"] }] }); + expect(exitCode).toBe(0); + }); + + // pnpm license-scanner 'lists versions installed under different aliases'. + test.concurrent.each(["hoisted", "isolated"] as Linker[])( + "npm: alias is listed under the real name (%s)", + async linker => { + const dir = await setup(linker, { + "package.json": pkg({ dependencies: { "no-deps": "1.0.0", "nd2": "npm:no-deps@1.0.1" } }), + }); + + expect(await licensesJson(dir)).toEqual({ Unknown: [{ name: "no-deps", versions: ["1.0.0", "1.0.1"] }] }); + }, + ); + + // pnpm 'path should be correct for workspaces' / 'filter outputs'; pnpm#5689 (same output from every directory of a monorepo). + test.concurrent("workspace root lists every member's dependencies; a member lists only its own closure", async () => { + const dir = await setup("hoisted", monorepoFiles); + + const fromRoot = await licensesJson(dir); + expect(names(fromRoot)).toEqual(["a-dep", "no-deps", "path-parse", "resolve"]); + expect(await licensesJson(join(dir, "packages", "bar"))).toEqual({ MIT: fullJson.MIT }); + expect(await licensesJson(join(dir, "packages", "foo"))).toEqual({ + Unknown: [ + { name: "a-dep", versions: ["1.0.1"] }, + { name: "no-deps", versions: ["1.0.0"] }, + ], + }); + }); + + test.concurrent("--prod inside a workspace drops members' devDependencies", async () => { + const dir = await setup("hoisted", monorepoFiles); + + expect(names(await licensesJson(dir, "--prod"))).toEqual(["no-deps", "path-parse", "resolve"]); + expect(await licensesJson(join(dir, "packages", "foo"), "--prod")).toEqual({ + Unknown: [{ name: "no-deps", versions: ["1.0.0"] }], + }); + }); + + test.concurrent("nothing to list prints nothing / {}", async () => { + const dir = await setup("hoisted", { "package.json": pkg({ devDependencies: { "no-deps": "1.0.0" } }) }); + + const [stdout, stderr, exitCode] = await licenses(dir, "--prod"); + expect(stdout).toBe(""); + expect(stderr).toBe(""); + expect(exitCode).toBe(0); + expect(await licensesJson(dir, "--prod")).toEqual({}); + expect(await licensesJson(dir)).toEqual({ Unknown: [{ name: "no-deps", versions: ["1.0.0"] }] }); + }); + + test.concurrent("licenses list / ls aliases", async () => { + const [[plain, , plainExit], [list, , listExit], [ls, , lsExit]] = await Promise.all([ + licenses(hoistedDir), + licenses(hoistedDir, "list"), + licenses(hoistedDir, "ls"), + ]); + expect(plain).toContain("MIT (2)"); + expect(list).toBe(plain); + expect(ls).toBe(plain); + expect([plainExit, listExit, lsExit]).toEqual([0, 0, 0]); + + const [stdout, stderr, exitCode] = await licenses(hoistedDir, "bogus"); + expect(stdout).toBe(""); + expect(stderr).toContain("Unknown subcommand: bogus"); + expect(exitCode).toBe(1); + }); + + test.concurrent("missing lockfile", async () => { + const { packageDir } = await registry.createTestDir({ files: { "package.json": fixturePackageJson } }); + const [stdout, stderr, exitCode] = await licenses(packageDir); + expect(stdout).toBe(""); + expect(stderr).toContain("Lockfile not found"); + expect(exitCode).toBe(1); + }); + + test.concurrent("missing node_modules", async () => { + const { packageDir } = await registry.createTestDir({ files: { "package.json": fixturePackageJson } }); + await using install = spawn({ + cmd: [bunExe(), "install", "--lockfile-only"], + env: bunEnv, + cwd: packageDir, + stdout: "pipe", + stderr: "pipe", + }); + expect(await install.exited).toBe(0); + + const [stdout, stderr, exitCode] = await licenses(packageDir); + expect(stdout).toBe(""); + expect(stderr).toContain("node_modules not found"); + expect(exitCode).toBe(1); + }); + + // pnpm#5702: a package that cannot be read must not fail the whole listing; the omission is reported instead of silent. + test.concurrent( + "unparsable package.json is reported as Unknown, missing ones are omitted with a warning", + async () => { + const dir = await setup(); + writeFileSync(join(dir, "node_modules", "resolve", "package.json"), "{ not json"); + rmSync(join(dir, "node_modules", "one-dep"), { recursive: true, force: true }); + + const [stdout, stderr, exitCode] = await licenses(dir, "--json"); + expect(JSON.parse(stdout)).toEqual({ + MIT: [fullJson.MIT[0]], + Unknown: [ + { name: "a-dep", versions: ["1.0.1"] }, + { name: "no-deps", versions: ["1.0.0"] }, + { name: "resolve", versions: ["1.9.0"] }, + ], + }); + expect(normalizeBunSnapshot(stderr)).toMatchInlineSnapshot( + `"warn: omitted 2 packages from the lockfile not found in node_modules"`, + ); + expect(exitCode).toBe(0); + }, + ); + + // pnpm#8589: the lockfile's resolvable tree hoists the dev subtree's a-dep@1.0.9, but `--production` installs a-dep@1.0.10 there. + const outHoistedFixture = { + "package.json": pkg({ + dependencies: { "uses-a-dep-10": "1.0.0" }, + devDependencies: { "uses-a-dep-9": "1.0.0" }, + }), + }; + + test.concurrent("pnpm#8589: --prod after `bun install --production` finds packages hoisted differently", async () => { + const dir = await setupProductionInstall(outHoistedFixture); + expect(JSON.parse(readFileSync(join(dir, "node_modules", "a-dep", "package.json"), "utf8")).version).toBe("1.0.10"); + expect(existsSync(join(dir, "node_modules", "uses-a-dep-10", "node_modules"))).toBeFalse(); + + const [stdout, stderr, exitCode] = await licenses(dir, "--prod", "--json"); + expect(JSON.parse(stdout)).toEqual({ + Unknown: [ + { name: "a-dep", versions: ["1.0.10"] }, + { name: "uses-a-dep-10", versions: ["1.0.0"] }, + ], + }); + expect(stderr).toBe(""); + expect(exitCode).toBe(0); + }); + + test.concurrent("pnpm#8589: a different version at the tree path is not misattributed", async () => { + const dir = await setupProductionInstall(outHoistedFixture); + patchInstalledManifest(dir, "a-dep", { license: "MIT" }); + + const [stdout, stderr, exitCode] = await licenses(dir, "--json"); + expect(JSON.parse(stdout)).toEqual({ + MIT: [{ name: "a-dep", versions: ["1.0.10"] }], + Unknown: [{ name: "uses-a-dep-10", versions: ["1.0.0"] }], + }); + expect(normalizeBunSnapshot(stderr)).toMatchInlineSnapshot( + `"warn: omitted 2 packages from the lockfile not found in node_modules"`, + ); + expect(exitCode).toBe(0); + }); + + // The lockfile nests bundled deps under their parent, which is also where the tarball unpacks them. + test.concurrent.each(["hoisted", "isolated"] as Linker[])( + "bundled dependencies are listed from inside their parent (%s)", + async linker => { + const dir = await setup(linker, { "package.json": pkg({ dependencies: { "bundled-1": "1.0.0" } }) }); + expect(existsSync(join(dir, "node_modules", "bundled-1", "node_modules", "no-deps", "package.json"))).toBeTrue(); + + const [stdout, stderr, exitCode] = await licenses(dir, "--json"); + expect(JSON.parse(stdout)).toEqual({ + Unknown: [ + { name: "bundled-1", versions: ["1.0.0"] }, + { name: "no-deps", versions: ["1.0.0"] }, + ], + }); + expect(stderr).toBe(""); + expect(exitCode).toBe(0); + }, + ); + + // pnpm#8739: git dependencies are looked up the same way the installer wrote them. + test.concurrent.each(["hoisted", "isolated"] as Linker[])("git dependency is listed (%s)", async linker => { + using repoDir = tempDir("licenses-git-repo", { + "package.json": JSON.stringify({ name: "git-pkg", version: "3.0.0", license: "ISC" }), + }); + const repo = String(repoDir); + for (const args of [ + ["init", "-q"], + ["add", "package.json"], + ["commit", "-q", "-m", "init", "--no-gpg-sign"], + ]) { + await using proc = spawn({ cmd: ["git", ...args], cwd: repo, env: gitEnv, stdout: "ignore", stderr: "pipe" }); + const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); + expect(stderr).not.toContain("fatal:"); + expect(exitCode).toBe(0); + } + + const dir = await setup(linker, { + "package.json": pkg({ dependencies: { "no-deps": "1.0.0", "git-pkg": `git+${pathToFileURL(repo)}` } }), + }); + + const [stdout, stderr, exitCode] = await licenses(dir, "--json"); + const parsed = JSON.parse(stdout); + expect(Object.keys(parsed)).toEqual(["ISC", "Unknown"]); + expect(parsed.ISC).toEqual([{ name: "git-pkg", versions: [expect.stringContaining("git+file://")] }]); + expect(parsed.Unknown).toEqual([{ name: "no-deps", versions: ["1.0.0"] }]); + expect(stderr).toBe(""); + expect(exitCode).toBe(0); + }); + + test.concurrent("bun pm help lists licenses", async () => { + await using proc = spawn({ + cmd: [bunExe(), "pm"], + env: bunEnv, + cwd: hoistedDir, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, exitCode] = await Promise.all([proc.stdout.text(), proc.exited]); + expect(stdout).toContain("bun pm licenses"); + expect(exitCode).toBe(0); + }); +}); diff --git a/test/cli/install/bun-prune.test.ts b/test/cli/install/bun-prune.test.ts new file mode 100644 index 000000000000..0315de0d5c73 --- /dev/null +++ b/test/cli/install/bun-prune.test.ts @@ -0,0 +1,1202 @@ +import { file, write } from "bun"; +import { afterAll, beforeAll, expect, test } from "bun:test"; +import { VerdaccioRegistry, bunEnv, bunExe, isWindows, normalizeBunSnapshot, runBunInstall } from "harness"; +import { + chmodSync, + existsSync, + lstatSync, + mkdirSync, + readdirSync, + renameSync, + rmSync, + symlinkSync, + writeFileSync, +} from "node:fs"; +import { basename, join } from "node:path"; + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +async function prune(dir: string, ...args: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), "prune", ...args], + env: bunEnv, + cwd: dir, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +function out(stdout: string) { + return normalizeBunSnapshot(stdout).replaceAll("\\", "/"); +} + +function plant(dir: string, rel: string) { + const abs = join(dir, rel); + mkdirSync(abs, { recursive: true }); + writeFileSync(join(abs, "package.json"), JSON.stringify({ name: basename(rel) })); + return abs; +} + +function lock(dir: string) { + return file(join(dir, "bun.lock")).text(); +} + +function isSymlink(path: string) { + return lstatSync(path).isSymbolicLink(); +} + +// On Windows a bin is a `.exe` + `.bunx` shim pair instead of a symlink. +function binFiles(nm: string, name: string) { + const bin = join(nm, ".bin", name); + return isWindows ? [`${bin}.exe`, `${bin}.bunx`] : [bin]; +} + +function expectBinInstalled(nm: string, name: string) { + for (const path of binFiles(nm, name)) { + expect(existsSync(path)).toBeTrue(); + } +} + +function expectBinRemoved(nm: string, name: string) { + for (const path of binFiles(nm, name)) { + expect(() => lstatSync(path)).toThrow(); + } +} + +type BunfigOpts = NonNullable[0]>["bunfigOpts"]; + +async function setup(pkgJson: Record, bunfigOpts?: BunfigOpts) { + const { packageDir, packageJson } = await registry.createTestDir({ bunfigOpts }); + await write(packageJson, JSON.stringify(pkgJson)); + await runBunInstall(bunEnv, packageDir); + return packageDir; +} + +async function install(dir: string, ...args: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), "install", ...args], + env: bunEnv, + cwd: dir, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + return stdout; +} + +type Linker = "hoisted" | "isolated"; + +// `--linker` is passed on the command line as well: an `install-strategy` in ~/.npmrc overrides the bunfig linker. +async function setupWithLinker(linker: Linker, pkgJson: Record, bunfigOpts?: BunfigOpts) { + const { packageDir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker, ...bunfigOpts } }); + await write(packageJson, JSON.stringify(pkgJson)); + await install(packageDir, "--linker", linker); + return packageDir; +} + +function linkOutside(dir: string, rel: string, contents: Record = {}) { + const outside = join(dir, "outside", basename(rel)); + mkdirSync(outside, { recursive: true }); + for (const [name, text] of Object.entries(contents)) { + writeFileSync(join(outside, name), text); + } + const link = join(dir, rel); + mkdirSync(join(link, ".."), { recursive: true }); + symlinkSync(outside, link, "junction"); + expect(isSymlink(link)).toBeTrue(); + return outside; +} + +test.concurrent("removes extraneous packages, keeps everything the lockfile installs", async () => { + const dir = await setup({ + name: "foo", + dependencies: { "no-deps": "1.0.0", "@scoped/has-bin-entry": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + const planted = [ + plant(dir, "node_modules/junk"), + plant(dir, "node_modules/@scoped/junk"), + plant(dir, "node_modules/@other/thing"), + ]; + writeFileSync(join(nm, "README.txt"), ""); + plant(dir, "node_modules/.cache/x"); + const lockBefore = await lock(dir); + + const first = await prune(dir); + expect(out(first.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/@other/thing + - node_modules/@scoped/junk + - node_modules/junk + Removed 3 packages" + `); + expect(first.exitCode).toBe(0); + + for (const path of planted) { + expect(existsSync(path)).toBeFalse(); + } + expect(existsSync(join(nm, "@other"))).toBeFalse(); + expect(existsSync(join(nm, "no-deps"))).toBeTrue(); + expect(existsSync(join(nm, "@scoped", "has-bin-entry"))).toBeTrue(); + expect(existsSync(join(nm, "README.txt"))).toBeTrue(); + expect(existsSync(join(nm, ".cache", "x"))).toBeTrue(); + expectBinInstalled(nm, "has-bin-entry"); + expect(await lock(dir)).toBe(lockBefore); + + const second = await prune(dir); + expect(out(second.stdout)).toEndWith("Nothing to prune."); + expect(second.exitCode).toBe(0); +}); + +test.concurrent("prunes nested node_modules folders the tree installs into", async () => { + const dir = await setup({ name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "2.0.0" } }); + const nested = join(dir, "node_modules", "one-dep", "node_modules", "no-deps"); + expect(await file(join(nested, "package.json")).json()).toMatchObject({ version: "1.0.1" }); + const junk = plant(dir, "node_modules/one-dep/node_modules/junk"); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/one-dep/node_modules/junk + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + expect(existsSync(join(nested, "package.json"))).toBeTrue(); +}); + +test.concurrent.each([["--production"], ["--prod"], ["--omit=dev"]])( + "%s removes packages only reachable through devDependencies", + async (...flags: string[]) => { + const dir = await setup({ + name: "foo", + dependencies: { "no-deps": "1.0.0", "@scoped/has-bin-entry": "1.0.0" }, + devDependencies: { "one-fixed-dep-bins": "1.0.0", "what-bin": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + expect(existsSync(join(nm, "no-deps-bins"))).toBeTrue(); + expectBinInstalled(nm, "what-bin"); + expectBinInstalled(nm, "has-bin-entry"); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await prune(dir, ...flags); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/no-deps-bins + - node_modules/one-fixed-dep-bins + - node_modules/what-bin + Removed 3 packages" + `); + expect(exitCode).toBe(0); + + expect(existsSync(join(nm, "no-deps"))).toBeTrue(); + expect(existsSync(join(nm, "@scoped", "has-bin-entry"))).toBeTrue(); + expect(existsSync(join(nm, "no-deps-bins"))).toBeFalse(); + expect(existsSync(join(nm, "one-fixed-dep-bins"))).toBeFalse(); + expect(existsSync(join(nm, "what-bin"))).toBeFalse(); + // pnpm#2326: bins of removed packages are cleaned up, on Windows too (shim files instead of links). + expectBinRemoved(nm, "what-bin"); + expectBinInstalled(nm, "has-bin-entry"); + + const { out: installOut } = await runBunInstall(bunEnv, dir, { production: true }); + expect(installOut).toContain("no changes"); + expect(await lock(dir)).toBe(lockBefore); + }, +); + +test.concurrent("--production keeps a package that prod and dev both need", async () => { + const pkg = { + name: "foo", + dependencies: { "no-deps": "1.0.0" }, + devDependencies: { "one-fixed-dep": "1.0.0" }, + }; + const [dir, plainDir] = await Promise.all([setup(pkg), setup(pkg)]); + + const production = await prune(dir, "--production"); + expect(out(production.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/one-fixed-dep + Removed 1 package" + `); + expect(production.exitCode).toBe(0); + expect(existsSync(join(dir, "node_modules", "no-deps"))).toBeTrue(); + expect(existsSync(join(dir, "node_modules", "one-fixed-dep"))).toBeFalse(); + + const plain = await prune(plainDir); + expect(out(plain.stdout)).toMatchInlineSnapshot(` + "bun prune () + Nothing to prune." + `); + expect(plain.exitCode).toBe(0); + expect(existsSync(join(plainDir, "node_modules", "one-fixed-dep"))).toBeTrue(); +}); + +test.concurrent("--dry-run prints without deleting; --silent deletes without printing", async () => { + const dir = await setup({ name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const junk = plant(dir, "node_modules/junk"); + + const dryRun = await prune(dir, "--dry-run"); + expect(out(dryRun.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk + Would remove 1 package" + `); + expect(dryRun.exitCode).toBe(0); + expect(existsSync(junk)).toBeTrue(); + + const silent = await prune(dir, "--silent"); + expect(silent.stdout).toBe(""); + expect(silent.exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + + const clean = await prune(dir, "--dry-run"); + expect(out(clean.stdout)).toEndWith("Nothing to prune."); + expect(clean.exitCode).toBe(0); +}); + +test.concurrent("nothing to prune when node_modules is missing or clean", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); + { + await using proc = Bun.spawn({ + cmd: [bunExe(), "install", "--lockfile-only"], + env: bunEnv, + cwd: packageDir, + stdout: "pipe", + stderr: "pipe", + }); + const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + } + const nm = join(packageDir, "node_modules"); + rmSync(nm, { recursive: true, force: true }); + expect(existsSync(join(packageDir, "bun.lock"))).toBeTrue(); + + const missing = await prune(packageDir); + expect(out(missing.stdout)).toMatchInlineSnapshot(` + "bun prune () + Nothing to prune." + `); + expect(missing.exitCode).toBe(0); + expect(existsSync(nm)).toBeFalse(); + + const cleanDir = await setup({ name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const clean = await prune(cleanDir, "--production"); + expect(out(clean.stdout)).toMatchInlineSnapshot(` + "bun prune () + Nothing to prune." + `); + expect(clean.exitCode).toBe(0); + expect(existsSync(join(cleanDir, "node_modules", "no-deps"))).toBeTrue(); +}); + +test.concurrent("never follows symlinks out of node_modules", async () => { + const dir = await setup({ name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const nm = join(dir, "node_modules"); + const outside = join(dir, "outside"); + mkdirSync(outside); + writeFileSync(join(outside, "keep.txt"), "keep"); + const link = join(nm, "linked-junk"); + symlinkSync(outside, link, "junction"); + expect(isSymlink(link)).toBeTrue(); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/linked-junk + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(() => lstatSync(link)).toThrow(); + expect(existsSync(join(outside, "keep.txt"))).toBeTrue(); + expect(existsSync(join(nm, "no-deps"))).toBeTrue(); +}); + +test.concurrent("refuses to run without a lockfile", async () => { + const [{ packageDir: noLockDir, packageJson }, installedDir] = await Promise.all([ + registry.createTestDir(), + setup({ name: "foo", dependencies: { "no-deps": "1.0.0" } }), + ]); + await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); + const junk = plant(noLockDir, "node_modules/junk"); + + const noLock = await prune(noLockDir); + expect(noLock.stderr).toContain("missing lockfile, nothing to prune"); + expect(noLock.exitCode).toBe(1); + expect(existsSync(junk)).toBeTrue(); + + const positional = await prune(installedDir, "foo"); + expect(positional.stderr).toContain("bun prune does not take arguments"); + expect(positional.exitCode).toBe(1); + expect(existsSync(join(installedDir, "node_modules", "no-deps"))).toBeTrue(); + + const help = await prune(noLockDir, "--help"); + expect(help.stdout).toContain("bun prune"); + expect(help.stdout).toContain("--production"); + expect(help.stdout).toContain("--linker"); + expect(help.exitCode).toBe(0); + expect(existsSync(junk)).toBeTrue(); +}); + +// pnpm#9796: --production never removes workspace links. +test.concurrent("workspaces: prunes workspace folders, keeps workspace links, runs from the root", async () => { + const { packageDir: dir, packageJson } = await registry.createTestDir(); + await Promise.all([ + write( + packageJson, + JSON.stringify({ name: "root", workspaces: ["packages/*"], dependencies: { "no-deps": "2.0.0" } }), + ), + write( + join(dir, "packages", "a", "package.json"), + JSON.stringify({ + name: "a", + version: "1.0.0", + dependencies: { "no-deps": "1.0.0" }, + devDependencies: { "a-dep": "1.0.1" }, + }), + ), + ]); + await runBunInstall(bunEnv, dir); + const nm = join(dir, "node_modules"); + const workspaceNoDeps = join(dir, "packages", "a", "node_modules", "no-deps"); + expect(isSymlink(join(nm, "a"))).toBeTrue(); + expect(existsSync(workspaceNoDeps)).toBeTrue(); + expect(existsSync(join(nm, "a-dep"))).toBeTrue(); + const junk = plant(dir, "packages/a/node_modules/junk"); + + const { stdout, exitCode } = await prune(join(dir, "packages", "a"), "--production"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/a-dep + - node_modules/a/node_modules/junk + Removed 2 packages" + `); + expect(exitCode).toBe(0); + + expect(isSymlink(join(nm, "a"))).toBeTrue(); + expect(existsSync(workspaceNoDeps)).toBeTrue(); + expect(existsSync(junk)).toBeFalse(); + expect(existsSync(join(nm, "a-dep"))).toBeFalse(); +}); + +test.concurrent("keeps dependencies bundled inside a package", async () => { + const dir = await setup({ name: "foo", dependencies: { "bundled-transitive": "1.0.0" } }); + const bundled = join(dir, "node_modules", "bundled-transitive", "node_modules", "no-deps", "package.json"); + expect(existsSync(bundled)).toBeTrue(); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + Nothing to prune." + `); + expect(exitCode).toBe(0); + expect(existsSync(bundled)).toBeTrue(); +}); + +// pnpm#881: --production also removes dev-only entries from the store. +test.concurrent("isolated linker: removes unused store entries and their links", async () => { + const dir = await setup( + { name: "foo", dependencies: { "no-deps": "1.0.0" }, devDependencies: { "one-dep": "1.0.0" } }, + { linker: "isolated" }, + ); + const nm = join(dir, "node_modules"); + const store = join(nm, ".bun"); + expect(existsSync(join(store, "one-dep@1.0.0"))).toBeTrue(); + expect(existsSync(join(store, "no-deps@1.0.1"))).toBeTrue(); + expect(existsSync(join(store, "no-deps@1.0.0"))).toBeTrue(); + expect(isSymlink(join(nm, "one-dep"))).toBeTrue(); + + plant(dir, "node_modules/.bun/junk@1.0.0/node_modules/junk"); + const peerVariant = plant(dir, "node_modules/.bun/no-deps@1.0.0+0123456789abcdef/node_modules/no-deps"); + const junkReal = plant(dir, "node_modules/junk-real"); + const hiddenHoist = join(store, "node_modules"); + mkdirSync(hiddenHoist, { recursive: true }); + symlinkSync("../zzz@1.0.0/node_modules/zzz", join(hiddenHoist, "zzz")); + expect(isSymlink(join(hiddenHoist, "zzz"))).toBeTrue(); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await prune(dir, "--production"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/junk@1.0.0 + - node_modules/.bun/no-deps@1.0.1 + - node_modules/.bun/one-dep@1.0.0 + - node_modules/junk-real + Removed 4 packages" + `); + expect(exitCode).toBe(0); + + expect(existsSync(join(store, "junk@1.0.0"))).toBeFalse(); + expect(existsSync(join(store, "no-deps@1.0.1"))).toBeFalse(); + expect(existsSync(join(store, "one-dep@1.0.0"))).toBeFalse(); + expect(existsSync(junkReal)).toBeFalse(); + expect(existsSync(join(store, "no-deps@1.0.0"))).toBeTrue(); + expect(existsSync(peerVariant)).toBeTrue(); + expect(() => lstatSync(join(nm, "one-dep"))).toThrow(); + expect(existsSync(join(nm, "no-deps", "package.json"))).toBeTrue(); + expect(existsSync(hiddenHoist)).toBeTrue(); + expect(() => lstatSync(join(hiddenHoist, "zzz"))).toThrow(); + expect(await lock(dir)).toBe(lockBefore); + + await runBunInstall(bunEnv, dir, { production: true }); +}); + +test.concurrent("isolated linker + global store: unlinks the store link, never deletes the shared entry", async () => { + const dir = await setup( + { name: "foo", devDependencies: { "one-dep": "1.0.0" } }, + { linker: "isolated", globalStore: true }, + ); + const storeEntry = join(dir, "node_modules", ".bun", "one-dep@1.0.0"); + expect(isSymlink(storeEntry)).toBeTrue(); + const linksDir = join(dir, ".bun-cache", "links"); + const globalEntry = readdirSync(linksDir).find(name => name.startsWith("one-dep@1.0.0-")); + expect(globalEntry).toBeDefined(); + const globalPkgJson = join(linksDir, globalEntry!, "node_modules", "one-dep", "package.json"); + expect(existsSync(globalPkgJson)).toBeTrue(); + + const { stdout, exitCode } = await prune(dir, "--production"); + expect(out(stdout)).toContain("- node_modules/.bun/one-dep@1.0.0"); + expect(out(stdout)).toContain("Removed 2 packages"); + expect(exitCode).toBe(0); + + expect(() => lstatSync(storeEntry)).toThrow(); + expect(lstatSync(join(linksDir, globalEntry!)).isDirectory()).toBeTrue(); + expect(await file(globalPkgJson).json()).toMatchObject({ name: "one-dep", version: "1.0.0" }); +}); + +test.concurrent("isolated linker: bins of removed packages are removed, live ones kept", async () => { + const dir = await setupWithLinker("isolated", { + name: "foo", + dependencies: { "@scoped/has-bin-entry": "1.0.0" }, + devDependencies: { "what-bin": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + expectBinInstalled(nm, "what-bin"); + expectBinInstalled(nm, "has-bin-entry"); + + const { stdout, exitCode } = await prune(dir, "--production", "--linker", "isolated"); + expect(out(stdout)).toContain("- node_modules/.bun/what-bin@1.0.0"); + expect(exitCode).toBe(0); + + expectBinRemoved(nm, "what-bin"); + expectBinInstalled(nm, "has-bin-entry"); +}); + +test.concurrent("hoisted: dot entries and files are never touched even when the lockfile is empty", async () => { + const { packageDir: dir, packageJson } = await registry.createTestDir(); + await Promise.all([ + write(packageJson, JSON.stringify({ name: "empty" })), + write( + join(dir, "bun.lock"), + `{ + "lockfileVersion": 1, + "workspaces": { + "": { + "name": "empty", + }, + }, + "packages": {} +} +`, + ), + ]); + const nm = join(dir, "node_modules"); + mkdirSync(nm); + const junk = plant(dir, "node_modules/junk"); + const leftoverStore = plant(dir, "node_modules/.bun/whatever@1.0.0"); + const integrity = join(nm, ".yarn-integrity"); + writeFileSync(integrity, ""); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + expect(existsSync(leftoverStore)).toBeTrue(); + expect(existsSync(integrity)).toBeTrue(); +}); + +test.concurrent( + "hoisted: a nested tree owned by a package that was replaced with a symlink is not walked", + async () => { + const dir = await setup({ name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "2.0.0" } }); + const nm = join(dir, "node_modules"); + expect(existsSync(join(nm, "one-dep", "node_modules", "no-deps"))).toBeTrue(); + rmSync(join(nm, "one-dep"), { recursive: true }); + const outside = linkOutside(dir, "node_modules/one-dep", { + "package.json": JSON.stringify({ name: "one-dep", version: "1.0.0" }), + }); + plant(outside, "node_modules/no-deps"); + const keepMe = plant(outside, "node_modules/keep-me"); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + Nothing to prune." + `); + expect(exitCode).toBe(0); + expect(existsSync(keepMe)).toBeTrue(); + expect(isSymlink(join(nm, "one-dep"))).toBeTrue(); + }, +); + +test.concurrent("hoisted: a symlinked scope dir is unlinked, not followed", async () => { + const dir = await setup({ name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const nm = join(dir, "node_modules"); + const outside = linkOutside(dir, "node_modules/@fake"); + const inner = plant(outside, "thing"); + plant(dir, "node_modules/@real/junk"); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/@fake + - node_modules/@real/junk + Removed 2 packages" + `); + expect(exitCode).toBe(0); + expect(() => lstatSync(join(nm, "@fake"))).toThrow(); + expect(existsSync(inner)).toBeTrue(); + expect(existsSync(join(nm, "@real"))).toBeFalse(); +}); + +test.concurrent.skipIf(isWindows)("a symlinked .bin directory is never cleaned through", async () => { + const dir = await setup({ name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const nm = join(dir, "node_modules"); + expect(existsSync(join(nm, ".bin"))).toBeFalse(); + const outsideBins = linkOutside(dir, "node_modules/.bin"); + const dangling = join(outsideBins, "dangling"); + symlinkSync("./does-not-exist", dangling); + const junk = plant(dir, "node_modules/junk"); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + expect(isSymlink(dangling)).toBeTrue(); + expect(isSymlink(join(nm, ".bin"))).toBeTrue(); +}); + +test.concurrent("isolated: extraneous symlinks are removed even when the store is clean", async () => { + const dir = await setupWithLinker("isolated", { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const nm = join(dir, "node_modules"); + const outside = linkOutside(dir, "node_modules/ext", { "keep.txt": "keep" }); + const scopedOutside = linkOutside(dir, "node_modules/@ext/thing", { "keep.txt": "keep" }); + + const first = await prune(dir, "--linker", "isolated"); + expect(out(first.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/@ext/thing + - node_modules/ext + Removed 2 packages" + `); + expect(first.exitCode).toBe(0); + expect(() => lstatSync(join(nm, "ext"))).toThrow(); + expect(existsSync(join(nm, "@ext"))).toBeFalse(); + expect(existsSync(join(outside, "keep.txt"))).toBeTrue(); + expect(existsSync(join(scopedOutside, "keep.txt"))).toBeTrue(); + expect(existsSync(join(nm, "no-deps", "package.json"))).toBeTrue(); + + const second = await prune(dir, "--linker", "isolated"); + expect(out(second.stdout)).toEndWith("Nothing to prune."); + expect(second.exitCode).toBe(0); +}); + +test.concurrent( + "hoisted: lockfile shrinks -> removed packages, their nested deps, scope dir and bin links go away", + async () => { + const dir = await setup({ + name: "foo", + dependencies: { "one-dep": "1.0.0", "no-deps": "2.0.0", "@scoped/has-bin-entry": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + expect(existsSync(join(nm, "one-dep", "node_modules", "no-deps"))).toBeTrue(); + await write(join(dir, "package.json"), JSON.stringify({ name: "foo", dependencies: { "no-deps": "2.0.0" } })); + await install(dir, "--lockfile-only"); + expect(existsSync(join(nm, "one-dep"))).toBeTrue(); + expect(existsSync(join(nm, "@scoped", "has-bin-entry"))).toBeTrue(); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/@scoped/has-bin-entry + - node_modules/one-dep + Removed 2 packages" + `); + expect(exitCode).toBe(0); + expect(existsSync(join(nm, "one-dep"))).toBeFalse(); + expect(existsSync(join(nm, "@scoped"))).toBeFalse(); + expect(await file(join(nm, "no-deps", "package.json")).json()).toMatchObject({ version: "2.0.0" }); + expectBinRemoved(nm, "has-bin-entry"); + const { out: installOut } = await runBunInstall(bunEnv, dir, { savesLockfile: false }); + expect(installOut).toContain("no changes"); + }, +); + +test.concurrent("hoisted: removing only a scoped package also removes its bin link", async () => { + const dir = await setup({ + name: "foo", + dependencies: { "no-deps": "1.0.0" }, + devDependencies: { "@scoped/has-bin-entry": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + expectBinInstalled(nm, "has-bin-entry"); + + const { stdout, exitCode } = await prune(dir, "--production"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/@scoped/has-bin-entry + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(join(nm, "@scoped"))).toBeFalse(); + expectBinRemoved(nm, "has-bin-entry"); +}); + +test.concurrent( + "isolated: lockfile shrinks -> store entries, alias links and the emptied scope dir go away", + async () => { + const dir = await setupWithLinker("isolated", { + name: "foo", + dependencies: { "one-dep": "1.0.0", "no-deps": "2.0.0", "@scoped/has-bin-entry": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + const store = join(nm, ".bun"); + await write(join(dir, "package.json"), JSON.stringify({ name: "foo", dependencies: { "no-deps": "2.0.0" } })); + await install(dir, "--lockfile-only", "--linker", "isolated"); + expect(isSymlink(join(nm, "one-dep"))).toBeTrue(); + expect(isSymlink(join(nm, "@scoped", "has-bin-entry"))).toBeTrue(); + + const { stdout, exitCode } = await prune(dir, "--linker", "isolated"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/@scoped+has-bin-entry@1.0.0 + - node_modules/.bun/no-deps@1.0.1 + - node_modules/.bun/one-dep@1.0.0 + Removed 3 packages" + `); + expect(exitCode).toBe(0); + expect(() => lstatSync(join(nm, "one-dep"))).toThrow(); + expect(existsSync(join(nm, "@scoped"))).toBeFalse(); + expect(existsSync(join(store, "no-deps@2.0.0"))).toBeTrue(); + expect(await file(join(nm, "no-deps", "package.json")).json()).toMatchObject({ version: "2.0.0" }); + expectBinRemoved(nm, "has-bin-entry"); + }, +); + +test.concurrent("hoisted: --production empties a workspace folder that only held nested devDependencies", async () => { + const { packageDir: dir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); + await Promise.all([ + write( + packageJson, + JSON.stringify({ name: "root", workspaces: ["packages/*"], dependencies: { "no-deps": "2.0.0" } }), + ), + write( + join(dir, "packages", "a", "package.json"), + JSON.stringify({ name: "a", version: "1.0.0", devDependencies: { "no-deps": "1.0.0" } }), + ), + ]); + await install(dir, "--linker", "hoisted"); + const nm = join(dir, "node_modules"); + const nested = join(dir, "packages", "a", "node_modules", "no-deps"); + expect(await file(join(nested, "package.json")).json()).toMatchObject({ version: "1.0.0" }); + + const { stdout, exitCode } = await prune(dir, "--production", "--linker", "hoisted"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - packages/a/node_modules/no-deps + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(nested)).toBeFalse(); + expect(isSymlink(join(nm, "a"))).toBeTrue(); + expect(await file(join(nm, "no-deps", "package.json")).json()).toMatchObject({ version: "2.0.0" }); + await runBunInstall(bunEnv, dir, { production: true }); +}); + +test.concurrent( + "isolated + workspaces: --production prunes a workspace's registry devDependency, keeps workspace links", + async () => { + const { packageDir: dir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); + await Promise.all([ + write(packageJson, JSON.stringify({ name: "root", workspaces: ["packages/*"] })), + write( + join(dir, "packages", "app", "package.json"), + JSON.stringify({ + name: "app", + version: "1.0.0", + dependencies: { lib: "workspace:*", "no-deps": "1.0.0" }, + devDependencies: { tool: "workspace:*", "a-dep": "1.0.1" }, + }), + ), + write(join(dir, "packages", "lib", "package.json"), JSON.stringify({ name: "lib", version: "1.0.0" })), + write(join(dir, "packages", "tool", "package.json"), JSON.stringify({ name: "tool", version: "1.0.0" })), + ]); + await install(dir, "--linker", "isolated"); + const appNm = join(dir, "packages", "app", "node_modules"); + expect(existsSync(join(dir, "node_modules", ".bun"))).toBeTrue(); + expect(isSymlink(join(appNm, "a-dep"))).toBeTrue(); + expect(isSymlink(join(appNm, "tool"))).toBeTrue(); + + const { stdout, exitCode } = await prune(join(dir, "packages", "app"), "--production", "--linker", "isolated"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/a-dep@1.0.1 + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(() => lstatSync(join(appNm, "a-dep"))).toThrow(); + expect(existsSync(join(appNm, "no-deps", "package.json"))).toBeTrue(); + expect(existsSync(join(appNm, "lib", "package.json"))).toBeTrue(); + // Diverges from pnpm on purpose: an alias that names a workspace is never removed. + expect(existsSync(join(appNm, "tool", "package.json"))).toBeTrue(); + expect(existsSync(join(dir, "packages", "tool", "package.json"))).toBeTrue(); + await runBunInstall(bunEnv, dir, { production: true }); + }, +); + +test.concurrent.each(["hoisted", "isolated"] as Linker[])( + "%s: optionalDependencies survive --production and go away with --omit=optional", + async linker => { + const pkg = { + name: "foo", + dependencies: { "no-deps": "1.0.0" }, + optionalDependencies: { "a-dep": "1.0.1" }, + devDependencies: { "one-fixed-dep": "1.0.0" }, + }; + const [prodDir, omitDir] = await Promise.all([setupWithLinker(linker, pkg), setupWithLinker(linker, pkg)]); + const removed = (name: string) => (linker === "hoisted" ? `- node_modules/${name}` : `- node_modules/.bun/${name}`); + + const production = await prune(prodDir, "--production", "--linker", linker); + expect(out(production.stdout)).toBe( + `bun prune ()\n${removed(linker === "hoisted" ? "one-fixed-dep" : "one-fixed-dep@1.0.0")}\nRemoved 1 package`, + ); + expect(production.exitCode).toBe(0); + expect(existsSync(join(prodDir, "node_modules", "a-dep", "package.json"))).toBeTrue(); + expect(existsSync(join(prodDir, "node_modules", "no-deps", "package.json"))).toBeTrue(); + + const omit = await prune(omitDir, "--omit=optional", "--linker", linker); + expect(out(omit.stdout)).toBe( + `bun prune ()\n${removed(linker === "hoisted" ? "a-dep" : "a-dep@1.0.1")}\nRemoved 1 package`, + ); + expect(omit.exitCode).toBe(0); + expect(() => lstatSync(join(omitDir, "node_modules", "a-dep"))).toThrow(); + expect(existsSync(join(omitDir, "node_modules", "no-deps", "package.json"))).toBeTrue(); + expect(existsSync(join(omitDir, "node_modules", "one-fixed-dep", "package.json"))).toBeTrue(); + }, +); + +test.concurrent.each([["--os=aix"], ["--cpu=s390x"]])( + "%s removes packages that are disabled for that platform, plain prune keeps them", + async (flag: string) => { + const dir = await setup({ + name: "foo", + dependencies: { "no-deps": "1.0.0", "test-postinstall-skip-native": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + const native = join(nm, "test-postinstall-skip-native"); + expect(existsSync(native)).toBeTrue(); + + const host = await prune(dir); + expect(out(host.stdout)).toEndWith("Nothing to prune."); + expect(host.exitCode).toBe(0); + expect(existsSync(native)).toBeTrue(); + + const other = await prune(dir, flag); + expect(out(other.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/test-postinstall-skip-native + Removed 1 package" + `); + expect(other.exitCode).toBe(0); + expect(existsSync(native)).toBeFalse(); + expect(existsSync(join(nm, "no-deps"))).toBeTrue(); + expect(await install(dir, flag)).toContain("no changes"); + }, +); + +test.concurrent.each(["hoisted", "isolated"] as Linker[])( + "%s: an npm: alias is kept under its alias name", + async linker => { + const dir = await setupWithLinker(linker, { + name: "foo", + dependencies: { "my-alias": "npm:no-deps@1.0.0", "one-dep": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + expect(await file(join(nm, "my-alias", "package.json")).json()).toMatchObject({ + name: "no-deps", + version: "1.0.0", + }); + const junk = plant(dir, "node_modules/junk"); + + const { stdout, exitCode } = await prune(dir, "--linker", linker); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + expect(await file(join(nm, "my-alias", "package.json")).json()).toMatchObject({ + name: "no-deps", + version: "1.0.0", + }); + if (linker === "isolated") { + expect(existsSync(join(nm, ".bun", "no-deps@1.0.0"))).toBeTrue(); + expect(existsSync(join(nm, ".bun", "no-deps@1.0.1"))).toBeTrue(); + } + }, +); + +test.concurrent("isolated + publicHoistPattern: hoisted links follow their store entries", async () => { + const dir = await setupWithLinker( + "isolated", + { name: "foo", dependencies: { "no-deps": "1.0.0" }, devDependencies: { "one-dep": "1.0.0" } }, + { publicHoistPattern: ["no-deps"], hoistPattern: ["one-dep"] }, + ); + const nm = join(dir, "node_modules"); + const store = join(nm, ".bun"); + expect(existsSync(join(nm, "no-deps", "package.json"))).toBeTrue(); + expect(isSymlink(join(store, "node_modules", "one-dep"))).toBeTrue(); + + const clean = await prune(dir, "--linker", "isolated"); + expect(out(clean.stdout)).toEndWith("Nothing to prune."); + expect(clean.exitCode).toBe(0); + + const production = await prune(dir, "--production", "--linker", "isolated"); + expect(out(production.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/no-deps@1.0.1 + - node_modules/.bun/one-dep@1.0.0 + Removed 2 packages" + `); + expect(production.exitCode).toBe(0); + expect(() => lstatSync(join(store, "node_modules", "one-dep"))).toThrow(); + expect(() => lstatSync(join(nm, "one-dep"))).toThrow(); + expect(await file(join(nm, "no-deps", "package.json")).json()).toMatchObject({ version: "1.0.0" }); +}); + +test.concurrent.skipIf(isWindows || process.getuid?.() === 0)( + "a failed deletion is reported, the rest is removed, exit code 1", + async () => { + const dir = await setup({ name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const nm = join(dir, "node_modules"); + const junkA = plant(dir, "node_modules/junk-a"); + const inner = plant(dir, "node_modules/junk-b/inner"); + const junkB = join(nm, "junk-b"); + chmodSync(junkB, 0o555); + try { + const { stdout, stderr, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk-a + Removed 1 package" + `); + expect(stderr).toContain("failed to remove"); + expect(stderr).toContain("junk-b"); + expect(exitCode).toBe(1); + expect(existsSync(junkA)).toBeFalse(); + expect(existsSync(inner)).toBeTrue(); + } finally { + chmodSync(junkB, 0o755); + } + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk-b + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(junkB)).toBeFalse(); + }, +); + +// pnpm#4770 / #5092 / #10275: `prepare` must not run after --production removed the tools it needs. +test.concurrent("never runs the project's lifecycle scripts", async () => { + const dir = await setup({ + name: "foo", + dependencies: { "no-deps": "1.0.0" }, + devDependencies: { "a-dep": "1.0.1" }, + scripts: { + preinstall: "echo PRE >> ran.txt", + postinstall: "echo POST >> ran.txt", + prepare: "echo PREPARE >> ran.txt", + }, + }); + const ran = join(dir, "ran.txt"); + expect(existsSync(ran)).toBeTrue(); + rmSync(ran); + const junk = plant(dir, "node_modules/junk"); + + const plain = await prune(dir); + expect(out(plain.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk + Removed 1 package" + `); + expect(plain.exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + expect(existsSync(ran)).toBeFalse(); + + const production = await prune(dir, "--production"); + expect(out(production.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/a-dep + Removed 1 package" + `); + expect(production.exitCode).toBe(0); + expect(existsSync(ran)).toBeFalse(); +}); + +test.concurrent.each(["hoisted", "isolated"] as Linker[])( + "%s: --omit=peer removes what bun install --omit=peer would not install", + async linker => { + const pkg = { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0" } }; + const [dir, plainDir] = await Promise.all([setupWithLinker(linker, pkg), setupWithLinker(linker, pkg)]); + const nm = join(dir, "node_modules"); + const installedNoDeps = () => + linker === "hoisted" + ? existsSync(join(nm, "no-deps")) + : readdirSync(join(nm, ".bun")).some(name => name.startsWith("no-deps@")); + expect(existsSync(join(nm, "peer-deps-fixed", "package.json"))).toBeTrue(); + expect(installedNoDeps()).toBeTrue(); + + const omit = await prune(dir, "--omit=peer", "--linker", linker); + expect(out(omit.stdout)).toContain( + linker === "hoisted" ? "- node_modules/no-deps" : "- node_modules/.bun/no-deps@", + ); + expect(out(omit.stdout)).toEndWith("Removed 1 package"); + expect(omit.exitCode).toBe(0); + expect(installedNoDeps()).toBeFalse(); + expect(() => lstatSync(join(nm, "no-deps"))).toThrow(); + expect(existsSync(join(nm, "peer-deps-fixed", "package.json"))).toBeTrue(); + if (linker === "hoisted") { + expect(await install(dir, "--omit=peer")).toContain("no changes"); + } + + const plain = await prune(plainDir, "--linker", linker); + expect(out(plain.stdout)).toEndWith("Nothing to prune."); + expect(plain.exitCode).toBe(0); + }, +); + +test.concurrent("keeps dependencies bundled inside a file: dependency", async () => { + const { packageDir: dir, packageJson } = await registry.createTestDir(); + await Promise.all([ + write(packageJson, JSON.stringify({ name: "foo", dependencies: { local: "file:./local" } })), + write( + join(dir, "local", "package.json"), + JSON.stringify({ name: "local", version: "1.0.0", bundleDependencies: ["inner"] }), + ), + write( + join(dir, "local", "node_modules", "inner", "package.json"), + JSON.stringify({ name: "inner", version: "1.0.0" }), + ), + ]); + await runBunInstall(bunEnv, dir); + const inner = join(dir, "node_modules", "local", "node_modules", "inner", "package.json"); + expect(existsSync(inner)).toBeTrue(); + const junk = plant(dir, "node_modules/junk"); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + expect(existsSync(inner)).toBeTrue(); +}); + +// pnpm#13676 +test.concurrent("hoisted: a nested copy left behind after its dependency started hoisting is removed", async () => { + const dir = await setup({ name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "2.0.0" } }); + const nm = join(dir, "node_modules"); + const nested = join(nm, "one-dep", "node_modules", "no-deps"); + expect(await file(join(nested, "package.json")).json()).toMatchObject({ version: "1.0.1" }); + + await write( + join(dir, "package.json"), + JSON.stringify({ name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "1.0.1" } }), + ); + await install(dir, "--lockfile-only"); + expect(existsSync(nested)).toBeTrue(); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/one-dep/node_modules/no-deps + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(nested)).toBeFalse(); + expect(existsSync(join(nm, "one-dep", "package.json"))).toBeTrue(); + expect(existsSync(join(nm, "no-deps", "package.json"))).toBeTrue(); +}); + +// pnpm#13676 +test.concurrent("hoisted: nested node_modules of packages without a tree node are pruned", async () => { + const dir = await setup({ + name: "foo", + dependencies: { "no-deps": "1.0.0", "@scoped/has-bin-entry": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + const junk = plant(dir, "node_modules/no-deps/node_modules/junk"); + const scopedJunk = plant(dir, "node_modules/@scoped/has-bin-entry/node_modules/@other/thing"); + writeFileSync(join(nm, "no-deps", "node_modules", "keep.txt"), ""); + + const { stdout, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/@scoped/has-bin-entry/node_modules/@other/thing + - node_modules/no-deps/node_modules/junk + Removed 2 packages" + `); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + expect(existsSync(scopedJunk)).toBeFalse(); + expect(existsSync(join(nm, "@scoped", "has-bin-entry", "node_modules", "@other"))).toBeFalse(); + expect(existsSync(join(nm, "no-deps", "node_modules", "keep.txt"))).toBeTrue(); + expect(existsSync(join(nm, "no-deps", "package.json"))).toBeTrue(); + expect(existsSync(join(nm, "@scoped", "has-bin-entry", "package.json"))).toBeTrue(); +}); + +// pnpm#8307 +test.concurrent("refuses to prune a hoisted install with the isolated linker", async () => { + const dir = await setupWithLinker("hoisted", { name: "foo", dependencies: { "one-dep": "1.0.0" } }); + const nm = join(dir, "node_modules"); + const hoisted = join(nm, "no-deps", "package.json"); + expect(existsSync(hoisted)).toBeTrue(); + expect(existsSync(join(nm, ".bun"))).toBeFalse(); + + for (const flags of [ + ["--linker", "isolated"], + ["--linker", "isolated", "--dry-run"], + ]) { + const { stdout, stderr, exitCode } = await prune(dir, ...flags); + expect(out(stdout)).toMatchInlineSnapshot(`"bun prune ()"`); + expect(stderr).toContain("node_modules was installed with the hoisted linker"); + expect(stderr).toContain("bun prune --linker hoisted"); + expect(exitCode).toBe(1); + expect(existsSync(hoisted)).toBeTrue(); + } + + const same = await prune(dir, "--linker", "hoisted"); + expect(out(same.stdout)).toEndWith("Nothing to prune."); + expect(same.exitCode).toBe(0); +}); + +// pnpm#8307 +test.concurrent("refuses to prune an isolated install with the hoisted linker", async () => { + const dir = await setupWithLinker("isolated", { name: "foo", devDependencies: { "one-dep": "1.0.0" } }); + const nm = join(dir, "node_modules"); + expect(isSymlink(join(nm, "one-dep"))).toBeTrue(); + + const mismatch = await prune(dir, "--production", "--linker", "hoisted"); + expect(out(mismatch.stdout)).toMatchInlineSnapshot(`"bun prune ()"`); + expect(mismatch.stderr).toContain("node_modules was installed with the isolated linker"); + expect(mismatch.stderr).toContain("bun prune --linker isolated"); + expect(mismatch.exitCode).toBe(1); + expect(isSymlink(join(nm, "one-dep"))).toBeTrue(); + expect(existsSync(join(nm, "one-dep", "package.json"))).toBeTrue(); + + const same = await prune(dir, "--production", "--linker", "isolated"); + expect(out(same.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/no-deps@1.0.1 + - node_modules/.bun/one-dep@1.0.0 + Removed 2 packages" + `); + expect(same.exitCode).toBe(0); + expect(() => lstatSync(join(nm, "one-dep"))).toThrow(); +}); + +// pnpm#5960 +test.concurrent.each(["hoisted", "isolated"] as Linker[])( + "%s: --production keeps an npm: alias whose real name is also a dev-only dependency", + async linker => { + const dir = await setupWithLinker(linker, { + name: "foo", + dependencies: { aliased: "npm:no-deps@1.0.0" }, + devDependencies: { "no-deps": "2.0.0" }, + }); + const nm = join(dir, "node_modules"); + expect(await file(join(nm, "aliased", "package.json")).json()).toMatchObject({ version: "1.0.0" }); + expect(await file(join(nm, "no-deps", "package.json")).json()).toMatchObject({ version: "2.0.0" }); + + const { stdout, exitCode } = await prune(dir, "--production", "--linker", linker); + expect(out(stdout)).toBe( + `bun prune ()\n- ${linker === "hoisted" ? "node_modules/no-deps" : "node_modules/.bun/no-deps@2.0.0"}\nRemoved 1 package`, + ); + expect(exitCode).toBe(0); + expect(() => lstatSync(join(nm, "no-deps"))).toThrow(); + expect(await file(join(nm, "aliased", "package.json")).json()).toMatchObject({ version: "1.0.0" }); + if (linker === "isolated") { + expect(existsSync(join(nm, ".bun", "no-deps@1.0.0"))).toBeTrue(); + } + await runBunInstall(bunEnv, dir, { production: true }); + }, +); + +// pnpm#10081 +test.concurrent.each(["hoisted", "isolated"] as Linker[])( + "%s: the dangling link of a renamed workspace is removed, the renamed workspace is not", + async linker => { + const { packageDir: dir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker } }); + const appJson = (lib: string) => + JSON.stringify({ name: "app", version: "1.0.0", dependencies: { [lib]: "workspace:*" } }); + const libJson = (name: string) => JSON.stringify({ name, version: "1.0.0", dependencies: { "no-deps": "1.0.0" } }); + await Promise.all([ + write(packageJson, JSON.stringify({ name: "root", workspaces: ["packages/*"] })), + write(join(dir, "packages", "app", "package.json"), appJson("a")), + write(join(dir, "packages", "a", "package.json"), libJson("a")), + ]); + await install(dir, "--linker", linker); + // The hoisted linker links every workspace into the root folder; the isolated linker links it where it is depended on. + const linkFolder = linker === "hoisted" ? "node_modules" : "packages/app/node_modules"; + const staleLink = join(dir, linkFolder, "a"); + expect(isSymlink(staleLink)).toBeTrue(); + + renameSync(join(dir, "packages", "a"), join(dir, "packages", "b")); + await Promise.all([ + write(join(dir, "packages", "b", "package.json"), libJson("b")), + write(join(dir, "packages", "app", "package.json"), appJson("b")), + ]); + await install(dir, "--lockfile-only", "--linker", linker); + expect(isSymlink(staleLink)).toBeTrue(); + expect(existsSync(staleLink)).toBeFalse(); + + const { stdout, exitCode } = await prune(dir, "--linker", linker); + expect(out(stdout)).toBe(`bun prune ()\n- ${linkFolder}/a\nRemoved 1 package`); + expect(exitCode).toBe(0); + expect(() => lstatSync(staleLink)).toThrow(); + expect(existsSync(join(dir, "packages", "b", "package.json"))).toBeTrue(); + const noDeps = + linker === "hoisted" + ? join(dir, "node_modules", "no-deps") + : join(dir, "packages", "b", "node_modules", "no-deps"); + expect(existsSync(join(noDeps, "package.json"))).toBeTrue(); + }, +); diff --git a/test/cli/install/catalog-peer-hoist.test.ts b/test/cli/install/catalog-peer-hoist.test.ts new file mode 100644 index 000000000000..c28aa9ea814f --- /dev/null +++ b/test/cli/install/catalog-peer-hoist.test.ts @@ -0,0 +1,466 @@ +import { readTarball } from "bun:internal-for-testing"; +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { existsSync, lstatSync, readlinkSync } from "fs"; +import { readdir, rm } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe, pack } from "harness"; +import { join } from "path"; + +var registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +type Linker = "hoisted" | "isolated"; + +// `--linker` in addition to bunfig: a user-level ~/.npmrc `install-strategy` would otherwise override bunfig's linker. +async function spawnInstall(dir: string, linker: Linker, ...args: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), "install", "--linker", linker, ...args], + cwd: dir, + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + }); + const [out, err, code] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { out, err, code }; +} + +async function install(dir: string, linker: Linker, ...args: string[]) { + const result = await spawnInstall(dir, linker, ...args); + expect(result.err).not.toContain("error:"); + expect(result.code).toBe(0); + return result; +} + +async function packageKeys(dir: string): Promise { + const lockfile = Bun.JSONC.parse(await Bun.file(join(dir, "bun.lock")).text()) as { + packages: Record; + }; + return Object.keys(lockfile.packages).sort(); +} + +async function layout(dir: string, peerName = "no-deps"): Promise { + const path = join(dir, "packages", "lib", "node_modules", peerName); + if (!existsSync(path)) return ""; + if (lstatSync(path).isSymbolicLink()) return readlinkSync(path); + const { version } = await Bun.file(join(path, "package.json")).json(); + return `nested:${version}`; +} + +function rootPackageJson(opts: { + catalog?: Record; + catalogs?: Record>; + rootDependencies?: Record; + overrides?: Record; +}) { + return { + name: "root", + workspaces: { + packages: ["packages/*"], + ...(opts.catalog ? { catalog: opts.catalog } : {}), + ...(opts.catalogs ? { catalogs: opts.catalogs } : {}), + }, + dependencies: opts.rootDependencies ?? { "one-fixed-dep": "2.0.0" }, + ...(opts.overrides ? { overrides: opts.overrides } : {}), + }; +} + +type RepoOpts = { + peerSpec: string; + peerName?: string; + catalog?: Record; + catalogs?: Record>; + rootDependencies?: Record; + overrides?: Record; + optionalPeer?: boolean; + appDependencies?: Record; + extraWorkspaces?: Record; + libVersion?: string; + linker: Linker; + saveTextLockfile?: boolean; +}; + +async function makeRepo(opts: RepoOpts): Promise { + const peerName = opts.peerName ?? "no-deps"; + const extraFiles: Record = {}; + for (const [name, pkg] of Object.entries(opts.extraWorkspaces ?? {})) { + extraFiles[`packages/${name}/package.json`] = JSON.stringify({ name, ...pkg }); + } + const { packageDir } = await registry.createTestDir({ + bunfigOpts: { linker: opts.linker, saveTextLockfile: opts.saveTextLockfile }, + files: { + "package.json": JSON.stringify(rootPackageJson(opts)), + "packages/app/package.json": JSON.stringify({ + name: "app", + dependencies: opts.appDependencies ?? { + "no-deps": "1.0.0", + lib: "workspace:*", + }, + }), + "packages/lib/package.json": JSON.stringify({ + name: "lib", + ...(opts.libVersion ? { version: opts.libVersion } : {}), + peerDependencies: { + [peerName]: opts.peerSpec, + }, + ...(opts.optionalPeer ? { peerDependenciesMeta: { [peerName]: { optional: true } } } : {}), + }), + ...extraFiles, + }, + }); + return packageDir; +} + +async function rewriteRootPackageJson(dir: string, opts: Parameters[0]) { + await Bun.write(join(dir, "package.json"), JSON.stringify(rootPackageJson(opts))); +} + +async function rmNodeModules(dir: string) { + const workspaces = await readdir(join(dir, "packages")); + await Promise.all( + [join(dir, "node_modules"), ...workspaces.map(ws => join(dir, "packages", ws, "node_modules"))].map(path => + rm(path, { recursive: true, force: true }), + ), + ); +} + +const dedupedKeys = ["app", "lib", "no-deps", "one-fixed-dep", "one-fixed-dep/no-deps"]; +const nestedKeys = ["app", "lib", "lib/no-deps", "no-deps", "one-fixed-dep", "one-fixed-dep/no-deps"]; +const isolatedNoDeps = (version: string) => + join("..", "..", "..", "node_modules", ".bun", `no-deps@${version}`, "node_modules", "no-deps"); +const isolatedNoDeps2 = isolatedNoDeps("2.0.0"); +const linkers = ["hoisted", "isolated"] as const; + +test.concurrent("default catalog peer dedupes onto the satisfying ancestor", async () => { + const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker: "hoisted" }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); + expect(existsSync(join(dir, "packages", "lib", "node_modules", "no-deps"))).toBeFalse(); +}); + +test.concurrent("named catalog peer (catalog:peers) dedupes the same way", async () => { + const dir = await makeRepo({ + catalogs: { peers: { "no-deps": ">=1.0.0" } }, + peerSpec: "catalog:peers", + linker: "hoisted", + }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); + expect(existsSync(join(dir, "packages", "lib", "node_modules", "no-deps"))).toBeFalse(); +}); + +test.concurrent("optional catalog peer dedupes too", async () => { + const dir = await makeRepo({ + catalog: { "no-deps": ">=1.0.0" }, + peerSpec: "catalog:", + optionalPeer: true, + linker: "hoisted", + }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); + expect(existsSync(join(dir, "packages", "lib", "node_modules", "no-deps"))).toBeFalse(); +}); + +test.concurrent("scoped package name as a catalog peer", async () => { + const dir = await makeRepo({ + rootDependencies: {}, + catalog: { "@scoped/has-bin-entry": ">=1.0.0" }, + peerName: "@scoped/has-bin-entry", + peerSpec: "catalog:", + appDependencies: { "@scoped/has-bin-entry": "1.0.0", lib: "workspace:*" }, + linker: "hoisted", + }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(["@scoped/has-bin-entry", "app", "lib"]); + expect(existsSync(join(dir, "packages", "lib", "node_modules", "@scoped"))).toBeFalse(); +}); + +async function record(dir: string, linker: Linker, peerName?: string) { + await install(dir, linker); + const keys = await packageKeys(dir); + const fresh = await layout(dir, peerName); + await rmNodeModules(dir); + const { err } = await install(dir, linker); + const keysAfterReload = await packageKeys(dir); + const reload = await layout(dir, peerName); + return { keys, fresh, keysAfterReload, reload, reloadSavedLockfile: err.includes("Saved lockfile") }; +} + +describe.each([ + [">=1.0.0", "dedupes"], + ["^2.0.0", "stays nested"], +] as const)("peer range %s (%s)", (range, outcome) => { + describe.each(linkers)("linker=%s", linker => { + test.concurrent("catalog: peer produces the same lockfile and layout as the inline range", async () => { + const [catalogDir, inlineDir] = await Promise.all([ + makeRepo({ catalog: { "no-deps": range }, peerSpec: "catalog:", linker }), + makeRepo({ peerSpec: range, linker }), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + + const expectedKeys = outcome === "dedupes" ? dedupedKeys : nestedKeys; + expect(fromInline.keys).toEqual(expectedKeys); + expect(fromInline.keysAfterReload).toEqual(expectedKeys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + if (linker === "isolated") { + expect(fromInline.fresh).toEndWith(isolatedNoDeps2); + expect(fromInline.reload).toEndWith(isolatedNoDeps2); + } else { + const expectedLayout = outcome === "dedupes" ? "" : "nested:2.0.0"; + expect(fromInline.fresh).toBe(expectedLayout); + expect(fromInline.reload).toBe(expectedLayout); + } + + expect(fromCatalog).toEqual(fromInline); + }); + }); +}); + +describe.each(linkers)("linker=%s", linker => { + test.concurrent("catalog `*` peer behaves exactly like an inline `*` peer", async () => { + const [catalogDir, inlineDir] = await Promise.all([ + makeRepo({ catalog: { "no-deps": "*" }, peerSpec: "catalog:", linker }), + makeRepo({ peerSpec: "*", linker }), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + expect(fromInline.keys).toEqual(dedupedKeys); + expect(fromInline.keysAfterReload).toEqual(dedupedKeys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + expect(fromCatalog).toEqual(fromInline); + }); + + test.concurrent("aliased catalog entry peer matches the inline alias", async () => { + const [catalogDir, inlineDir] = await Promise.all([ + makeRepo({ catalog: { "no-deps": "npm:no-deps@>=1.0.0" }, peerSpec: "catalog:", linker }), + makeRepo({ peerSpec: "npm:no-deps@>=1.0.0", linker }), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + expect(fromInline.keysAfterReload).toEqual(fromInline.keys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + expect(fromCatalog).toEqual(fromInline); + }); + + test.concurrent("optional catalog peer matches the inline optional peer on reload", async () => { + const [catalogDir, inlineDir] = await Promise.all([ + makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", optionalPeer: true, linker }), + makeRepo({ peerSpec: ">=1.0.0", optionalPeer: true, linker }), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + expect(fromInline.keys).toEqual(dedupedKeys); + expect(fromInline.keysAfterReload).toEqual(dedupedKeys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + expect(fromCatalog).toEqual(fromInline); + }); + + // pnpm: deps-installer/test/catalogs.ts "importer with different peers uses correct peer" + test.concurrent( + "two consumers providing different peer versions: catalog peer still equals inline peer", + async () => { + const twoConsumers = (peerSpec: string, catalog?: Record) => + makeRepo({ + peerSpec, + catalog, + rootDependencies: {}, + appDependencies: { "no-deps": "1.0.0", lib: "workspace:*" }, + extraWorkspaces: { app2: { dependencies: { "no-deps": "2.0.0", lib: "workspace:*" } } }, + linker, + }); + const [catalogDir, inlineDir] = await Promise.all([ + twoConsumers("catalog:", { "no-deps": ">=1.0.0" }), + twoConsumers(">=1.0.0"), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + expect(fromInline.keys).toEqual(["app", "app2", "app2/no-deps", "lib", "no-deps"]); + expect(fromInline.keysAfterReload).toEqual(fromInline.keys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + expect(fromCatalog).toEqual(fromInline); + }, + ); + + // pnpm: deps-installer/test/catalogs.ts "catalog resolutions should be consistent with peer dependencies" + test.concurrent("warm install leaves bun.lock byte-identical and --frozen-lockfile passes", async () => { + const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker }); + await install(dir, linker); + const lockfile = await Bun.file(join(dir, "bun.lock")).text(); + await rmNodeModules(dir); + const warm = await install(dir, linker); + expect(warm.err).not.toContain("Saved lockfile"); + expect(await Bun.file(join(dir, "bun.lock")).text()).toBe(lockfile); + await rmNodeModules(dir); + const frozen = await install(dir, linker, "--frozen-lockfile"); + expect(frozen.err).not.toContain("lockfile had changes"); + if (linker === "hoisted") expect(await layout(dir)).toBe(""); + else expect(await layout(dir)).toEndWith(isolatedNoDeps2); + }); +}); + +// pnpm: deps-installer/test/catalogs.ts "lockfile is updated if catalog config changes" +test.concurrent("changing the catalog range of a peer re-hoists on the next install (both directions)", async () => { + const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker: "hoisted" }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); + + await rewriteRootPackageJson(dir, { catalog: { "no-deps": "^2.0.0" } }); + let { err } = await install(dir, "hoisted"); + expect(err).toContain("Saved lockfile"); + expect(await packageKeys(dir)).toEqual(nestedKeys); + expect(await layout(dir)).toBe("nested:2.0.0"); + + await rewriteRootPackageJson(dir, { catalog: { "no-deps": ">=1.0.0" } }); + ({ err } = await install(dir, "hoisted")); + expect(err).toContain("Saved lockfile"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); +}); + +// pnpm: deps-installer/test/catalogs.ts "frozen lockfile error is thrown if catalog config changes" +test.concurrent("--frozen-lockfile fails when only a peer's catalog range changed", async () => { + const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker: "hoisted" }); + await install(dir, "hoisted"); + const lockfile = await Bun.file(join(dir, "bun.lock")).text(); + + await rewriteRootPackageJson(dir, { catalog: { "no-deps": "^2.0.0" } }); + const { err, code } = await spawnInstall(dir, "hoisted", "--frozen-lockfile"); + expect(err).toContain("error: lockfile had changes, but lockfile is frozen"); + expect(code).not.toBe(0); + expect(await Bun.file(join(dir, "bun.lock")).text()).toBe(lockfile); +}); + +test.concurrent("catalog peer with bun.lockb dedupes and reloads identically", async () => { + const dir = await makeRepo({ + catalog: { "no-deps": ">=1.0.0" }, + peerSpec: "catalog:", + linker: "hoisted", + saveTextLockfile: false, + }); + await install(dir, "hoisted"); + expect(existsSync(join(dir, "bun.lock"))).toBeFalse(); + const lockb = await Bun.file(join(dir, "bun.lockb")).bytes(); + expect(await layout(dir)).toBe(""); + await rmNodeModules(dir); + await install(dir, "hoisted"); + expect(await layout(dir)).toBe(""); + expect(await Bun.file(join(dir, "bun.lockb")).bytes()).toEqual(lockb); +}); + +// pnpm applies overrides before catalogs, keyed by the peer's own name, even when the catalog entry aliases another package. +describe.each([ + ["plain", ">=2.0.0"], + ["aliased", "npm:a-dep@1.0.1"], +] as const)("override beats the %s catalog entry of a peer", (_, entry) => { + test.concurrent("fresh == reload", async () => { + const dir = await makeRepo({ + overrides: { "no-deps": "1.0.0" }, + rootDependencies: { "one-fixed-dep": "2.0.0", "a-dep": "1.0.1" }, + catalog: { "no-deps": entry }, + peerSpec: "catalog:", + linker: "isolated", + }); + const result = await record(dir, "isolated"); + expect(result.fresh).toEndWith(isolatedNoDeps("1.0.0")); + const keys = ["a-dep", "app", "lib", "no-deps", "one-fixed-dep"]; + expect(result).toEqual({ + keys, + fresh: result.fresh, + keysAfterReload: keys, + reload: result.fresh, + reloadSavedLockfile: false, + }); + }); +}); + +// Only the entry the spec names is `^2.0.0`; a `>=1.0.0` decoy or an unresolved peer would give dedupedKeys instead (pnpm: resolveFromCatalog.test.ts). +describe.each([ + ["catalog:peers", { catalog: { "no-deps": ">=1.0.0" }, catalogs: { peers: { "no-deps": "^2.0.0" } } }], + ["catalog:", { catalog: { "no-deps": "^2.0.0" }, catalogs: { peers: { "no-deps": ">=1.0.0" } } }], + ["catalog:default", { catalog: { "no-deps": "^2.0.0" } }], + ["catalog:", { catalogs: { default: { "no-deps": "^2.0.0" } } }], + ["catalog:default", { catalogs: { default: { "no-deps": "^2.0.0" } } }], + ["catalog:default", { catalog: { "no-deps": ">=1.0.0" }, catalogs: { default: { "no-deps": "^2.0.0" } } }], + ["catalog:", { catalog: { "no-deps": "^2.0.0" }, catalogs: { default: { "no-deps": ">=1.0.0" } } }], +] as const)("peer %s resolves through the entry named by its spec (%o)", (peerSpec, catalogFields) => { + test.concurrent("fresh and reload", async () => { + const dir = await makeRepo({ ...catalogFields, peerSpec, linker: "hoisted" }); + expect(await record(dir, "hoisted")).toEqual({ + keys: nestedKeys, + fresh: "nested:2.0.0", + keysAfterReload: nestedKeys, + reload: "nested:2.0.0", + reloadSavedLockfile: false, + }); + }); +}); + +// pnpm errors here; Bun never fails an install over an unresolved peer, so the peer must simply not get a nested copy. +describe.each([ + ["catalog:", { catalog: {} }], + ["catalog:peers", { catalogs: { other: { "no-deps": ">=1.0.0" } } }], + ["catalog:default", { catalogs: { other: { "no-deps": ">=1.0.0" } } }], + ["catalog:", { catalog: { "no-deps": "catalog:other" }, catalogs: { other: { "no-deps": ">=1.0.0" } } }], +] as const)("peer %s with no usable catalog entry (%o)", (peerSpec, catalogFields) => { + test.concurrent("installs without a nested copy and is stable on reload", async () => { + const dir = await makeRepo({ ...catalogFields, peerSpec, linker: "hoisted" }); + expect(await record(dir, "hoisted")).toEqual({ + keys: dedupedKeys, + fresh: "", + keysAfterReload: dedupedKeys, + reload: "", + reloadSavedLockfile: false, + }); + }); +}); + +// pnpm #12159 shape: an override whose value is a catalog reference wins over the peer's own range, fresh and on reload. +describe.each(linkers)("linker=%s", linker => { + describe.each([ + [">=1.0.0", "catalog:", { catalog: { "no-deps": "1.0.0" } }], + ["^2.0.0", "catalog:", { catalog: { "no-deps": "1.0.0" } }], + ["catalog:", "catalog:", { catalog: { "no-deps": "1.0.0" } }], + ["catalog:", "catalog:pins", { catalog: { "no-deps": ">=1.0.0" }, catalogs: { pins: { "no-deps": "1.0.0" } } }], + ] as const)("peer %s overridden to %s", (peerSpec, override, catalogFields) => { + test.concurrent("binds to the overriding catalog entry, fresh == reload", async () => { + const dir = await makeRepo({ ...catalogFields, overrides: { "no-deps": override }, peerSpec, linker }); + const result = await record(dir, linker); + const keys = ["app", "lib", "no-deps", "one-fixed-dep"]; + expect(result).toEqual({ + keys, + fresh: linker === "isolated" ? expect.stringContaining(isolatedNoDeps("1.0.0")) : "", + keysAfterReload: keys, + reload: result.fresh, + reloadSavedLockfile: false, + }); + const { packages } = Bun.JSONC.parse(await Bun.file(join(dir, "bun.lock")).text()) as { + packages: Record; + }; + expect(packages["no-deps"][0]).toBe("no-deps@1.0.0"); + }); + }); +}); + +// pnpm #8996 (`catalog:` peers survive `pack` unsubstituted) and #7072 (`catalog:` / `catalog:default` are one catalog). +describe.each([ + ["catalog:", { catalog: { "no-deps": ">=1.0.0" } }], + ["catalog:peers", { catalogs: { peers: { "no-deps": ">=1.0.0" } } }], + ["catalog:", { catalogs: { default: { "no-deps": ">=1.0.0" } } }], + ["catalog:default", { catalog: { "no-deps": ">=1.0.0" } }], +] as const)("bun pm pack substitutes the %s peer (%o)", (peerSpec, catalogFields) => { + test.concurrent("with the catalog's range", async () => { + const dir = await makeRepo({ ...catalogFields, peerSpec, libVersion: "1.2.3", linker: "hoisted" }); + await install(dir, "hoisted"); + const libDir = join(dir, "packages", "lib"); + await pack(libDir, bunEnv); + const tarball = readTarball(join(libDir, "lib-1.2.3.tgz")); + const packageJson = tarball.entries.find( + (entry: { pathname: string }) => entry.pathname === "package/package.json", + ); + expect(JSON.parse(packageJson.contents)).toEqual({ + name: "lib", + version: "1.2.3", + peerDependencies: { "no-deps": ">=1.0.0" }, + }); + }); +}); diff --git a/test/cli/install/catalogs.test.ts b/test/cli/install/catalogs.test.ts index 08cc2b4044db..9dc234e2ad61 100644 --- a/test/cli/install/catalogs.test.ts +++ b/test/cli/install/catalogs.test.ts @@ -165,6 +165,38 @@ describe("basic", () => { }); }); } + + test("switching a dependency to a different catalog is detected", async () => { + const { packageDir } = await registry.createTestDir({ bunfigOpts: { saveTextLockfile: true, linker: "hoisted" } }); + const pkg1Path = join(packageDir, "packages", "pkg1", "package.json"); + await Promise.all([ + write( + join(packageDir, "package.json"), + JSON.stringify({ + name: "catalog-switch", + workspaces: { + packages: ["packages/*"], + catalogs: { a: { "no-deps": "1.0.0" }, b: { "no-deps": "2.0.0" } }, + }, + }), + ), + write(pkg1Path, JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:a" } })), + ]); + + await runBunInstall(bunEnv, packageDir); + expect((await file(join(packageDir, "node_modules", "no-deps", "package.json")).json()).version).toBe("1.0.0"); + + await write(pkg1Path, JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:b" } })); + await runBunInstall(bunEnv, packageDir); + + expect((await file(join(packageDir, "node_modules", "no-deps", "package.json")).json()).version).toBe("2.0.0"); + const lock = Bun.JSONC.parse(await file(join(packageDir, "bun.lock")).text()) as any; + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:b" }); + expect(Object.keys(lock.packages)).toEqual(["no-deps", "pkg1"]); + expect(lock.packages["no-deps"][0]).toBe("no-deps@2.0.0"); + + await runBunInstall(bunEnv, packageDir, { savesLockfile: false }); + }); }); describe("update", () => { diff --git a/test/cli/install/config-precedence.test.ts b/test/cli/install/config-precedence.test.ts new file mode 100644 index 000000000000..fa083177bf0d --- /dev/null +++ b/test/cli/install/config-precedence.test.ts @@ -0,0 +1,210 @@ +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { existsSync, readFileSync } from "fs"; +import { VerdaccioRegistry, bunEnv, bunExe, tempDir } from "harness"; +import { join } from "path"; + +// Own config dir: other install files' VerdaccioRegistry start()/stop() delete the shared htpasswd, invalidating our token. +const sharedRegistryDir = join(import.meta.dir, "registry"); +const registryDir = tempDir("config-precedence-registry", { + "verdaccio.yaml": readFileSync(join(sharedRegistryDir, "verdaccio.yaml"), "utf8").replace( + "storage: ./packages", + `storage: ${JSON.stringify(join(sharedRegistryDir, "packages"))}`, + ), +}); +const registry = new VerdaccioRegistry({ configPath: join(String(registryDir), "verdaccio.yaml") }); +let authToken: string; + +beforeAll(async () => { + await registry.start(); + authToken = await registry.generateUser("config-precedence", "verysecure"); +}); + +afterAll(() => { + registry.stop(); + registryDir[Symbol.dispose](); +}); + +const authLine = () => `//localhost:${registry.port}/:_authToken=${authToken}\n`; +const bunfig = (install: Record) => Bun.TOML.stringify({ install }); +const packageJson = (dependencies: Record) => + JSON.stringify({ name: "config-precedence", version: "1.0.0", dependencies }); + +/** A registry that must never be contacted. */ +function deadRegistry() { + let hits = 0; + const server = Bun.serve({ + port: 0, + fetch() { + hits++; + return new Response("wrong registry", { status: 500 }); + }, + }); + return { + server, + url: `http://localhost:${server.port}/`, + get hits() { + return hits; + }, + [Symbol.dispose]() { + server.stop(true); + }, + }; +} + +async function install(root: string, args: string[] = []) { + const home = join(root, "home"); + await using proc = Bun.spawn({ + cmd: [bunExe(), "install", ...args], + cwd: join(root, "project"), + env: { + ...bunEnv, + HOME: home, + USERPROFILE: home, + XDG_CONFIG_HOME: home, + BUN_INSTALL_CACHE_DIR: join(root, "cache"), + }, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +const isIsolated = (root: string) => existsSync(join(root, "project", "node_modules", ".bun")); + +describe.concurrent("bun install config precedence", () => { + test("project bunfig linker beats ~/.npmrc install-strategy", async () => { + using dir = tempDir("config-precedence", { + "home/.npmrc": "install-strategy=hoisted\n", + "project/bunfig.toml": bunfig({ registry: registry.registryUrl(), linker: "isolated" }), + "project/package.json": packageJson({ "no-deps": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(isIsolated(String(dir))).toBe(true); + }); + + test("project bunfig registry beats project .npmrc registry", async () => { + using dead = deadRegistry(); + using dir = tempDir("config-precedence", { + "project/.npmrc": `registry=${dead.url}\n`, + "project/bunfig.toml": bunfig({ registry: registry.registryUrl() }), + "project/package.json": packageJson({ "no-deps": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(dead.hits).toBe(0); + expect(existsSync(join(String(dir), "project", "node_modules", "no-deps", "package.json"))).toBe(true); + }); + + test("project bunfig registry beats ~/.npmrc registry", async () => { + using dead = deadRegistry(); + using dir = tempDir("config-precedence", { + "home/.npmrc": `registry=${dead.url}\n`, + "project/bunfig.toml": bunfig({ registry: registry.registryUrl() }), + "project/package.json": packageJson({ "no-deps": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(dead.hits).toBe(0); + }); + + test("project .npmrc registry beats ~/.npmrc registry", async () => { + using dead = deadRegistry(); + using dir = tempDir("config-precedence", { + "home/.npmrc": `registry=${dead.url}\n`, + "project/.npmrc": `registry=${registry.registryUrl()}\n`, + "project/package.json": packageJson({ "no-deps": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(dead.hits).toBe(0); + expect(existsSync(join(String(dir), "project", "node_modules", "no-deps", "package.json"))).toBe(true); + }); + + test("~/.npmrc _authToken applies to the registry set in project bunfig", async () => { + using dir = tempDir("config-precedence", { + "home/.npmrc": authLine(), + "project/bunfig.toml": bunfig({ registry: registry.registryUrl() }), + "project/package.json": packageJson({ "@needs-auth/test-pkg": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(existsSync(join(String(dir), "project", "node_modules", "@needs-auth", "test-pkg", "package.json"))).toBe( + true, + ); + }); + + test("project .npmrc scoped registry and token apply alongside a bunfig registry", async () => { + using dead = deadRegistry(); + using dir = tempDir("config-precedence", { + "project/.npmrc": `@needs-auth:registry=${registry.registryUrl()}\n${authLine()}`, + "project/bunfig.toml": bunfig({ registry: dead.url }), + "project/package.json": packageJson({ "@needs-auth/test-pkg": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(dead.hits).toBe(0); + expect(existsSync(join(String(dir), "project", "node_modules", "@needs-auth", "test-pkg", "package.json"))).toBe( + true, + ); + }); + + test("bunfig scoped registry keeps credentials from ~/.npmrc", async () => { + using dead = deadRegistry(); + using dir = tempDir("config-precedence", { + "home/.npmrc": authLine(), + "project/bunfig.toml": bunfig({ registry: dead.url, scopes: { "needs-auth": registry.registryUrl() } }), + "project/package.json": packageJson({ "@needs-auth/test-pkg": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(dead.hits).toBe(0); + }); + + test("bunfig scoped registry beats the same scope in .npmrc", async () => { + using dead = deadRegistry(); + using dir = tempDir("config-precedence", { + "project/.npmrc": `@types:registry=${dead.url}\n`, + "project/bunfig.toml": bunfig({ registry: dead.url, scopes: { types: registry.registryUrl() } }), + "project/package.json": packageJson({ "@types/no-deps": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(dead.hits).toBe(0); + expect(existsSync(join(String(dir), "project", "node_modules", "@types", "no-deps", "package.json"))).toBe(true); + }); + + test("--linker beats ~/.npmrc, project .npmrc and bunfig", async () => { + using dir = tempDir("config-precedence", { + "home/.npmrc": "install-strategy=hoisted\n", + "project/.npmrc": "install-strategy=hoisted\n", + "project/bunfig.toml": bunfig({ registry: registry.registryUrl(), linker: "hoisted" }), + "project/package.json": packageJson({ "no-deps": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir), ["--linker", "isolated"]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(isIsolated(String(dir))).toBe(true); + }); + + test("~/.npmrc install-strategy applies when bunfig does not set a linker", async () => { + using dir = tempDir("config-precedence", { + "home/.npmrc": "install-strategy=linked\n", + "project/bunfig.toml": bunfig({ registry: registry.registryUrl() }), + "project/package.json": packageJson({ "no-deps": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(isIsolated(String(dir))).toBe(true); + }); +}); diff --git a/test/cli/install/frozen-lockfile-pruned.test.ts b/test/cli/install/frozen-lockfile-pruned.test.ts new file mode 100644 index 000000000000..94c3a30a78c7 --- /dev/null +++ b/test/cli/install/frozen-lockfile-pruned.test.ts @@ -0,0 +1,823 @@ +import { file, write } from "bun"; +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { exists, lstat, readlink } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe } from "harness"; +import { join } from "path"; + +type Linker = "hoisted" | "isolated"; +type PackageJson = Record; +type Tree = { root: PackageJson; packages: Record; files?: Record }; + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +const rootPackageJson: PackageJson = { name: "mono", workspaces: ["packages/*"] }; +const appPackageJson: PackageJson = { + name: "app", + version: "1.0.0", + dependencies: { shared: "workspace:*", "a-dep": "1.0.1" }, +}; +const sharedPackageJson: PackageJson = { + name: "shared", + version: "1.0.0", + peerDependencies: { "no-deps": "*" }, + peerDependenciesMeta: { "no-deps": { optional: true } }, +}; +const otherPackageJson: PackageJson = { + name: "other", + version: "1.0.0", + dependencies: { shared: "workspace:*", "no-deps": "1.0.0", "left-pad": "1.0.0" }, +}; + +async function writeTree(dir: string, tree: Tree, workspaces: string[] = Object.keys(tree.packages)) { + await Promise.all([ + write(join(dir, "package.json"), JSON.stringify(tree.root)), + ...workspaces.map(path => write(join(dir, path, "package.json"), JSON.stringify(tree.packages[path]))), + ...Object.entries(tree.files ?? {}) + .filter(([path]) => workspaces.some(ws => path.startsWith(ws + "/"))) + .map(([path, contents]) => write(join(dir, path), contents)), + ]); +} + +const monorepo: Tree = { + root: rootPackageJson, + packages: { + "packages/app": appPackageJson, + "packages/shared": sharedPackageJson, + "packages/other": otherPackageJson, + }, +}; + +const survivors = ["packages/app", "packages/shared"]; + +async function writeMonorepo(dir: string, { withOther }: { withOther: boolean }) { + await writeTree(dir, monorepo, withOther ? undefined : survivors); +} + +const explicitMonorepo: Tree = { + ...monorepo, + root: { name: "mono", workspaces: ["packages/app", "packages/shared", "packages/other"] }, +}; + +// `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. +async function raw(dir: string, linker: Linker, args: string[], cwd = dir) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...args, "--linker", linker], + cwd, + env: bunEnv, + stdin: "ignore", + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + expect(stderr).not.toContain("panic:"); + return { stdout, stderr, exitCode }; +} + +const prunedNote = "note: skipped 1 workspace listed in bun.lock but not on disk"; + +async function run(dir: string, linker: Linker, args: string[], expectedExitCode: number, cwd = dir) { + const { stdout, stderr, exitCode } = await raw(dir, linker, args, cwd); + if (expectedExitCode === 0) { + expect(stderr).not.toContain("error:"); + } else { + expect(stderr).toContain("lockfile had changes, but lockfile is frozen"); + } + expect(exitCode).toBe(expectedExitCode); + return { stdout, stderr, exitCode }; +} + +const install = (dir: string, linker: Linker) => run(dir, linker, ["install"], 0); +const frozen = (dir: string, linker: Linker, expectedExitCode: number, cmd = ["install", "--frozen-lockfile"]) => + run(dir, linker, cmd, expectedExitCode); + +async function fullInstall(linker: Linker, tree: Tree) { + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker } }); + await writeTree(packageDir, tree); + const { stderr } = await install(packageDir, linker); + expect(stderr).toContain("Saved lockfile"); + return { fullDir: packageDir, full: await file(join(packageDir, "bun.lock")).text() }; +} + +// A full install of `tree`, then a second checkout with only `keep` on disk and the full bun.lock copied verbatim. +async function verbatimScenario(linker: Linker, tree: Tree, keep: string[]) { + const { fullDir, full } = await fullInstall(linker, tree); + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker } }); + await writeTree(packageDir, tree, keep); + await write(join(packageDir, "bun.lock"), full); + return { packageDir, fullDir, full }; +} + +const fullLockfiles = new Map>(); + +function fullLockfile(linker: Linker): Promise { + let lock = fullLockfiles.get(linker); + if (!lock) { + lock = fullInstall(linker, monorepo).then(({ full }) => { + expect(full).toContain('"no-deps"'); + expect(full).toContain('"left-pad"'); + expect(full).toContain('"packages/other"'); + return full; + }); + fullLockfiles.set(linker, lock); + } + return lock; +} + +// Same shape `turbo prune` emits: the workspace and its exclusive packages go, the peer-reachable no-deps entry stays. +function turboPrune(lock: string): string { + const pruned = lock + .replace(/ "packages\/other": \{\n(?: .*\n)* \},\n/, "") + .replace(/\n "(?:other|left-pad)": \[[^\n]*\],\n\n?/g, "\n"); + expect(pruned).toContain('"no-deps": ["no-deps@1.0.0"'); + expect(pruned).not.toContain('"other"'); + expect(pruned).not.toContain('"left-pad"'); + return pruned; +} + +async function prunedTree(linker: Linker) { + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker } }); + await writeMonorepo(packageDir, { withOther: false }); + const pruned = turboPrune(await fullLockfile(linker)); + await write(join(packageDir, "bun.lock"), pruned); + return { packageDir, pruned }; +} + +async function verbatimTree(linker: Linker) { + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker } }); + await writeMonorepo(packageDir, { withOther: false }); + const full = await fullLockfile(linker); + await write(join(packageDir, "bun.lock"), full); + return { packageDir, full }; +} + +const lockText = (dir: string) => file(join(dir, "bun.lock")).text(); + +async function editApp(dir: string, edit: (app: any) => void) { + const appJson = join(dir, "packages", "app", "package.json"); + const app = await file(appJson).json(); + edit(app); + await write(appJson, JSON.stringify(app)); +} + +function installedPath(dir: string, linker: Linker, name: string, version: string) { + return linker === "hoisted" + ? join(dir, "node_modules", name, "package.json") + : join(dir, "node_modules", ".bun", `${name}@${version}`, "node_modules", name, "package.json"); +} + +describe.each(["hoisted", "isolated"] as Linker[])("linker: %s", linker => { + // Also pins that the optional-peer-bound no-deps is installed even though only the pruned workspace needed it (pnpm#6264). + test.concurrent("turbo-pruned lockfile: package held only by an optional peer passes --frozen-lockfile", async () => { + const { packageDir, pruned } = await prunedTree(linker); + + await frozen(packageDir, linker, 0); + + expect(await lockText(packageDir)).toBe(pruned); + const noDeps = + linker === "hoisted" + ? join(packageDir, "node_modules", "no-deps", "package.json") + : join(packageDir, "packages", "shared", "node_modules", "no-deps", "package.json"); + expect(await file(noDeps).json()).toEqual({ name: "no-deps", version: "1.0.0" }); + }); + + test.concurrent("turbo-pruned lockfile: plain bun install does not rewrite it", async () => { + const { packageDir, pruned } = await prunedTree(linker); + + const { stderr } = await install(packageDir, linker); + + expect(stderr).not.toContain("Saved lockfile"); + expect(await lockText(packageDir)).toBe(pruned); + }); + + test.concurrent("verbatim full lockfile with a workspace folder missing passes --frozen-lockfile", async () => { + const { packageDir, full } = await verbatimTree(linker); + + await frozen(packageDir, linker, 0); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + if (linker === "hoisted") { + expect((await lstat(join(packageDir, "node_modules", "app"))).isSymbolicLink()).toBeTrue(); + expect(await exists(join(packageDir, "node_modules", "left-pad"))).toBeFalse(); + expect(await file(join(packageDir, "node_modules", "a-dep", "package.json")).json()).toMatchObject({ + name: "a-dep", + version: "1.0.1", + }); + } else { + expect(await exists(join(packageDir, "node_modules", ".bun", "left-pad@1.0.0"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", ".bun", "a-dep@1.0.1"))).toBeTrue(); + expect( + await file(join(packageDir, "packages", "app", "node_modules", "a-dep", "package.json")).json(), + ).toMatchObject({ + name: "a-dep", + version: "1.0.1", + }); + } + }); + + test.concurrent("second --frozen-lockfile install on a pruned tree is a no-op", async () => { + const { packageDir, pruned } = await prunedTree(linker); + await frozen(packageDir, linker, 0); + + const { stderr } = await frozen(packageDir, linker, 0); + + expect(stderr).not.toContain("Saved lockfile"); + expect(await lockText(packageDir)).toBe(pruned); + const noDeps = + linker === "hoisted" + ? join(packageDir, "node_modules", "no-deps", "package.json") + : join(packageDir, "packages", "shared", "node_modules", "no-deps", "package.json"); + expect(await file(noDeps).json()).toEqual({ name: "no-deps", version: "1.0.0" }); + }); + + test.concurrent("a real package.json change in a pruned tree still fails --frozen-lockfile", async () => { + const { packageDir, pruned } = await prunedTree(linker); + await editApp(packageDir, app => (app.dependencies["no-deps"] = "2.0.0")); + + await frozen(packageDir, linker, 1); + + expect(await lockText(packageDir)).toBe(pruned); + }); + + test.concurrent("a missing workspace is still removed from the lockfile by a non-frozen install", async () => { + const { packageDir } = await verbatimTree(linker); + + const { stderr } = await install(packageDir, linker); + + expect(stderr).toContain("Saved lockfile"); + const lock = await lockText(packageDir); + expect(lock).not.toContain('"packages/other"'); + expect(lock).not.toContain('"left-pad"'); + expect(await exists(join(packageDir, "packages", "other"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "left-pad", "1.0.0"))).toBeFalse(); + }); + + test.concurrent("scoped workspace whose folder name differs from its package name is pruned", async () => { + const tree: Tree = { + root: rootPackageJson, + packages: { + "packages/app": appPackageJson, + "packages/shared": sharedPackageJson, + "packages/other-dir": { name: "@mono/other", version: "1.0.0", dependencies: { "left-pad": "1.0.0" } }, + }, + }; + const { packageDir, full } = await verbatimScenario(linker, tree, survivors); + expect(full).toContain('"packages/other-dir"'); + + await frozen(packageDir, linker, 0); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "@mono"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "left-pad", "1.0.0"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "a-dep", "1.0.1"))).toBeTrue(); + }); + + test.concurrent("several workspaces pruned at once", async () => { + const tree: Tree = { + root: rootPackageJson, + packages: { + ...monorepo.packages, + "packages/other2": { name: "other2", version: "1.0.0", dependencies: { "is-number": "1.0.0" } }, + }, + }; + const { packageDir, full } = await verbatimScenario(linker, tree, survivors); + expect(full).toContain('"packages/other2"'); + + await frozen(packageDir, linker, 0); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", "other2"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "left-pad", "1.0.0"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "is-number", "1.0.0"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "a-dep", "1.0.1"))).toBeTrue(); + + await editApp(packageDir, app => (app.dependencies["is-number"] = "2.0.0")); + + await frozen(packageDir, linker, 1); + + expect(await lockText(packageDir)).toBe(full); + }); + + test.concurrent("bun ci behaves the same on the verbatim full lockfile", async () => { + const { packageDir, full } = await verbatimTree(linker); + + await frozen(packageDir, linker, 0, ["ci"]); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "left-pad", "1.0.0"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "a-dep", "1.0.1"))).toBeTrue(); + }); + + // pnpm#11364: the root lists workspaces by path instead of a glob (what `turbo prune` copies verbatim). + test.concurrent("explicitly listed workspace whose folder is missing passes --frozen-lockfile", async () => { + const { packageDir, full } = await verbatimScenario(linker, explicitMonorepo, survivors); + + const { stderr } = await frozen(packageDir, linker, 0); + + expect(stderr).not.toContain("Workspace not found"); + expect(stderr).toContain(prunedNote); + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "left-pad", "1.0.0"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "a-dep", "1.0.1"))).toBeTrue(); + }); + + test.concurrent("bun ci with an explicitly listed workspace whose folder is missing", async () => { + const { packageDir, full } = await verbatimScenario(linker, explicitMonorepo, survivors); + + const { stderr } = await frozen(packageDir, linker, 0, ["ci"]); + + expect(stderr).not.toContain("Workspace not found"); + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "a-dep", "1.0.1"))).toBeTrue(); + }); + + // pnpm#7823: the relaxation is not silent, so a stale bun.lock is greppable in CI logs. + test.concurrent("--frozen-lockfile names how many workspaces it skipped", async () => { + const { packageDir } = await verbatimTree(linker); + + const { stderr } = await frozen(packageDir, linker, 0); + + expect(stderr).toContain(prunedNote); + expect(stderr).not.toContain('skipping workspace "other"'); + }); + + // pnpm#6094: `--lockfile-only` must not write under `--frozen-lockfile`; the extra newline is a byte-level canary. + test.concurrent("--frozen-lockfile --lockfile-only leaves the pruned bun.lock byte-identical", async () => { + const { packageDir, pruned } = await prunedTree(linker); + const canary = pruned + "\n"; + await write(join(packageDir, "bun.lock"), canary); + + const { stdout, stderr } = await frozen(packageDir, linker, 0, ["install", "--frozen-lockfile", "--lockfile-only"]); + + expect(stdout + stderr).not.toContain("Saved"); + expect(await lockText(packageDir)).toBe(canary); + expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); + }); +}); + +describe("hoisted", () => { + test.concurrent("bun ci behaves the same on the pruned turbo lockfile", async () => { + const { packageDir, pruned } = await prunedTree("hoisted"); + + await frozen(packageDir, "hoisted", 0, ["ci"]); + + expect(await lockText(packageDir)).toBe(pruned); + expect(await file(join(packageDir, "node_modules", "no-deps", "package.json")).json()).toEqual({ + name: "no-deps", + version: "1.0.0", + }); + }); + + // pnpm#11364, turbo shape: bun.lock no longer lists the workspace but the copied root package.json still does. + test.concurrent("explicitly listed workspace missing from disk and from a turbo-pruned bun.lock", async () => { + const { full } = await fullInstall("hoisted", explicitMonorepo); + const pruned = turboPrune(full); + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); + await writeTree(packageDir, explicitMonorepo, survivors); + await write(join(packageDir, "bun.lock"), pruned); + + const { stderr } = await frozen(packageDir, "hoisted", 0); + + expect(stderr).not.toContain("Workspace not found"); + expect(stderr).not.toContain("not on disk"); + expect(await lockText(packageDir)).toBe(pruned); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await file(join(packageDir, "node_modules", "no-deps", "package.json")).json()).toMatchObject({ + version: "1.0.0", + }); + }); + + test.concurrent( + "explicitly listed workspace missing: --frozen-lockfile from inside a surviving workspace", + async () => { + const { packageDir, full } = await verbatimScenario("hoisted", explicitMonorepo, survivors); + + await run(packageDir, "hoisted", ["install", "--frozen-lockfile"], 0, join(packageDir, "packages", "app")); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "packages", "app", "bun.lock"))).toBeFalse(); + expect((await lstat(join(packageDir, "node_modules", "app"))).isSymbolicLink()).toBeTrue(); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + }, + ); + + test.concurrent("explicitly listed workspace missing from disk still errors on a non-frozen install", async () => { + const { packageDir, full } = await verbatimScenario("hoisted", explicitMonorepo, survivors); + + const { stderr, exitCode } = await raw(packageDir, "hoisted", ["install"]); + + expect(stderr).toContain('Workspace not found "packages/other"'); + expect(exitCode).toBe(1); + expect(await lockText(packageDir)).toBe(full); + }); + + test.concurrent("--verbose names each skipped workspace", async () => { + const { packageDir } = await verbatimTree("hoisted"); + + const { stderr } = await frozen(packageDir, "hoisted", 0, ["install", "--frozen-lockfile", "--verbose"]); + + expect(stderr).toContain('note: skipping workspace "other": listed in bun.lock but not on disk'); + expect(stderr).toContain(prunedNote); + }); + + test.concurrent("--silent suppresses the skipped-workspace note", async () => { + const { packageDir } = await verbatimTree("hoisted"); + + const { stdout, stderr } = await frozen(packageDir, "hoisted", 0, ["install", "--frozen-lockfile", "--silent"]); + + expect(stdout + stderr).toBe(""); + expect(await exists(join(packageDir, "node_modules", "app"))).toBeTrue(); + }); + + test.concurrent("no skipped-workspace note when nothing was skipped", async () => { + const { packageDir } = await prunedTree("hoisted"); + + const { stderr } = await frozen(packageDir, "hoisted", 0); + + expect(stderr).not.toContain("not on disk"); + }); + + test.concurrent("skipped-workspace note is pluralized", async () => { + const tree: Tree = { + root: rootPackageJson, + packages: { + ...monorepo.packages, + "packages/other2": { name: "other2", version: "1.0.0" }, + }, + }; + const { packageDir } = await verbatimScenario("hoisted", tree, survivors); + + const { stderr } = await frozen(packageDir, "hoisted", 0); + + expect(stderr).toContain("note: skipped 2 workspaces listed in bun.lock but not on disk"); + }); + + // The one sanctioned frozen write: the bun.lockb -> bun.lock migration recipe from the docs. + test.concurrent( + "--save-text-lockfile --frozen-lockfile --lockfile-only still migrates a pruned bun.lockb", + async () => { + const tree: Tree = { + root: rootPackageJson, + packages: { + "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "1.0.1" } }, + "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "1.0.0" } }, + }, + }; + const bunfigOpts = { linker: "hoisted", saveTextLockfile: false } as const; + const { packageDir: fullDir } = await registry.createTestDir({ bunfigOpts }); + await writeTree(fullDir, tree); + await install(fullDir, "hoisted"); + const lockb = await file(join(fullDir, "bun.lockb")).bytes(); + const { packageDir } = await registry.createTestDir({ bunfigOpts }); + await writeTree(packageDir, tree, ["packages/app"]); + await write(join(packageDir, "bun.lockb"), lockb); + + await frozen(packageDir, "hoisted", 0, [ + "install", + "--save-text-lockfile", + "--frozen-lockfile", + "--lockfile-only", + ]); + + expect(await exists(join(packageDir, "bun.lockb"))).toBeFalse(); + const lock = await lockText(packageDir); + expect(lock).toContain('"packages/other"'); + expect(lock).toContain('"left-pad"'); + expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); + }, + ); + + // pnpm#8795: a catalog change is caught by the frozen check before `--lockfile-only` gets a chance to write. + test.concurrent("--frozen-lockfile --lockfile-only still fails on a catalog change in a pruned tree", async () => { + const tree: Tree = { + root: { name: "mono", workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.1" } } }, + packages: { + "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "catalog:" } }, + "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "1.0.0" } }, + }, + }; + const { packageDir, full } = await verbatimScenario("hoisted", tree, ["packages/app"]); + await write( + join(packageDir, "package.json"), + JSON.stringify({ name: "mono", workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.2" } } }), + ); + + await frozen(packageDir, "hoisted", 1, ["install", "--frozen-lockfile", "--lockfile-only"]); + + expect(await lockText(packageDir)).toBe(full); + }); + + // pnpm#4861: `--frozen-lockfile --dry-run` is the install-free lockfile check, and it understands pruned trees. + test.concurrent("--frozen-lockfile --dry-run checks a pruned tree without installing", async () => { + const { packageDir, full } = await verbatimTree("hoisted"); + + await frozen(packageDir, "hoisted", 0, ["install", "--frozen-lockfile", "--dry-run"]); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); + + await editApp(packageDir, app => (app.dependencies["is-number"] = "1.0.0")); + + await frozen(packageDir, "hoisted", 1, ["install", "--frozen-lockfile", "--dry-run"]); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); + }); + + // pnpm#6312 / pnpm#9741: bun.lock's shape does not depend on the linker or on --production/--omit. + test.concurrent("pruned bun.lock written with the hoisted linker passes frozen under other settings", async () => { + const pruned = turboPrune(await fullLockfile("hoisted")); + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); + await writeMonorepo(packageDir, { withOther: false }); + await write(join(packageDir, "bun.lock"), pruned); + + await frozen(packageDir, "isolated", 0); + expect(await lockText(packageDir)).toBe(pruned); + expect(await exists(installedPath(packageDir, "isolated", "a-dep", "1.0.1"))).toBeTrue(); + + await frozen(packageDir, "hoisted", 0, ["install", "--frozen-lockfile", "--production", "--omit=optional"]); + expect(await lockText(packageDir)).toBe(pruned); + }); + + // pnpm#5794: a pruned bun.lock a plain install would rewrite also fails frozen (the two checks are the same comparison). + test.concurrent( + "a hand-edited specifier in the pruned bun.lock fails frozen and is rewritten by a plain install", + async () => { + const { packageDir, pruned } = await prunedTree("hoisted"); + const edited = pruned.replace('"a-dep": "1.0.1"', '"a-dep": "1.0.0"'); + expect(edited).not.toBe(pruned); + await write(join(packageDir, "bun.lock"), edited); + + await frozen(packageDir, "hoisted", 1); + expect(await lockText(packageDir)).toBe(edited); + + const { stderr } = await install(packageDir, "hoisted"); + + expect(stderr).toContain("Saved lockfile"); + expect(await lockText(packageDir)).toContain('"a-dep": "1.0.1"'); + }, + ); + + test.concurrent("--frozen-lockfile run from inside a surviving workspace", async () => { + const { packageDir, full } = await verbatimTree("hoisted"); + + await run(packageDir, "hoisted", ["install", "--frozen-lockfile"], 0, join(packageDir, "packages", "app")); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "packages", "app", "bun.lock"))).toBeFalse(); + expect((await lstat(join(packageDir, "node_modules", "app"))).isSymbolicLink()).toBeTrue(); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", "left-pad"))).toBeFalse(); + }); + + test.concurrent("workspace folder without a package.json is treated as pruned", async () => { + const { packageDir, full } = await verbatimTree("hoisted"); + await write(join(packageDir, "packages", "other", "dist", "index.js"), ""); + + await frozen(packageDir, "hoisted", 0); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", "left-pad"))).toBeFalse(); + expect(await exists(join(packageDir, "packages", "other", "node_modules"))).toBeFalse(); + }); + + test.concurrent("pruned workspace's lifecycle scripts and bins are neither run nor linked", async () => { + const tree: Tree = { + root: rootPackageJson, + packages: { + "packages/app": { ...appPackageJson, bin: { "app-cli": "cli.js" } }, + "packages/shared": sharedPackageJson, + "packages/other": { + ...otherPackageJson, + bin: { "other-cli": "cli.js" }, + scripts: { postinstall: "echo other-postinstall > ../../other-postinstall.txt" }, + }, + }, + files: { "packages/app/cli.js": "", "packages/other/cli.js": "" }, + }; + const { packageDir, fullDir } = await verbatimScenario("hoisted", tree, survivors); + expect(await exists(join(fullDir, "other-postinstall.txt"))).toBeTrue(); + expect(await exists(join(fullDir, "node_modules", ".bin", "other-cli"))).toBeTrue(); + + const { stdout, stderr } = await frozen(packageDir, "hoisted", 0); + + expect(stdout + stderr).not.toContain("other-postinstall"); + expect(stdout + stderr).not.toContain("ENOENT"); + expect(await exists(join(packageDir, "other-postinstall.txt"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", ".bin", "other-cli"))).toBeFalse(); + expect(await readlink(join(packageDir, "node_modules", ".bin", "app-cli"))).toContain("app"); + }); + + test.concurrent("--production composes with the pruned-workspace filter", async () => { + const tree: Tree = { + root: rootPackageJson, + packages: { + "packages/app": { + name: "app", + version: "1.0.0", + dependencies: { shared: "workspace:*" }, + devDependencies: { "a-dep": "1.0.1" }, + }, + "packages/shared": sharedPackageJson, + "packages/other": { name: "other", version: "1.0.0", devDependencies: { "left-pad": "1.0.0" } }, + }, + }; + const { packageDir, full } = await verbatimScenario("hoisted", tree, survivors); + + await frozen(packageDir, "hoisted", 0, ["install", "--frozen-lockfile", "--production"]); + + expect(await lockText(packageDir)).toBe(full); + expect((await lstat(join(packageDir, "node_modules", "shared"))).isSymbolicLink()).toBeTrue(); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", "a-dep"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", "left-pad"))).toBeFalse(); + }); + + test.concurrent("non-frozen prune writes the same bun.lock a fresh install of the survivors would", async () => { + const { packageDir } = await verbatimTree("hoisted"); + const { full: expected } = await fullInstall("hoisted", { + root: rootPackageJson, + packages: { "packages/app": appPackageJson, "packages/shared": sharedPackageJson }, + }); + + const { stderr } = await install(packageDir, "hoisted"); + + expect(stderr).toContain("Saved lockfile"); + expect(await lockText(packageDir)).toBe(expected); + }); + + // No `workspace:` edges here: bun.lockb round-trips those as a diff on its own, independent of pruning. + test.concurrent("verbatim bun.lockb with a workspace folder missing passes --frozen-lockfile", async () => { + const tree: Tree = { + root: rootPackageJson, + packages: { + "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "1.0.1" } }, + "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "1.0.0" } }, + }, + }; + const bunfigOpts = { linker: "hoisted", saveTextLockfile: false } as const; + const { packageDir: fullDir } = await registry.createTestDir({ bunfigOpts }); + await writeTree(fullDir, tree); + await install(fullDir, "hoisted"); + const lockb = await file(join(fullDir, "bun.lockb")).bytes(); + const { packageDir } = await registry.createTestDir({ bunfigOpts }); + await writeTree(packageDir, tree, ["packages/app"]); + await write(join(packageDir, "bun.lockb"), lockb); + + await frozen(packageDir, "hoisted", 0); + + expect(await file(join(packageDir, "bun.lockb")).bytes()).toEqual(lockb); + expect(await exists(join(packageDir, "bun.lock"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(join(packageDir, "node_modules", "left-pad"))).toBeFalse(); + expect(await file(join(packageDir, "node_modules", "a-dep", "package.json")).json()).toMatchObject({ + version: "1.0.1", + }); + }); + + test.concurrent("pruned tree using catalogs passes --frozen-lockfile when the catalog is left intact", async () => { + const tree: Tree = { + root: { + name: "mono", + workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.1", "left-pad": "1.0.0" } }, + }, + packages: { + "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "catalog:" } }, + "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "catalog:" } }, + }, + }; + const { packageDir, full } = await verbatimScenario("hoisted", tree, ["packages/app"]); + expect(full).toContain('"left-pad": "1.0.0"'); + + await frozen(packageDir, "hoisted", 0); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "left-pad"))).toBeFalse(); + expect(await file(join(packageDir, "node_modules", "a-dep", "package.json")).json()).toMatchObject({ + version: "1.0.1", + }); + }); + + // Pins the boundary: unlike pnpm, a catalog entry trimmed from bun.lock is a real package.json/lockfile disagreement. + test.concurrent("pruned tree whose bun.lock catalog was trimmed still fails --frozen-lockfile", async () => { + const tree: Tree = { + root: { + name: "mono", + workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.1", "left-pad": "1.0.0" } }, + }, + packages: { + "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "catalog:" } }, + "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "catalog:" } }, + }, + }; + const { packageDir, full } = await verbatimScenario("hoisted", tree, ["packages/app"]); + const trimmed = full.replace(/\n +"left-pad": "1\.0\.0",?\n/, "\n"); + expect(trimmed).not.toBe(full); + await write(join(packageDir, "bun.lock"), trimmed); + + await frozen(packageDir, "hoisted", 1); + + expect(await lockText(packageDir)).toBe(trimmed); + }); + + // The guards below pass on main too; they pin the boundaries of the frozen-lockfile relaxation. + test.concurrent("a pruned workspace does not mask a workspace added on disk", async () => { + const { packageDir, full } = await verbatimTree("hoisted"); + await write(join(packageDir, "packages", "extra", "package.json"), JSON.stringify({ name: "extra" })); + + await frozen(packageDir, "hoisted", 1); + + expect(await lockText(packageDir)).toBe(full); + }); + + test.concurrent("a pruned workspace does not mask a dependency added to the root package.json", async () => { + const { packageDir, full } = await verbatimTree("hoisted"); + await write( + join(packageDir, "package.json"), + JSON.stringify({ ...rootPackageJson, dependencies: { "is-number": "1.0.0" } }), + ); + + await frozen(packageDir, "hoisted", 1); + + expect(await lockText(packageDir)).toBe(full); + }); + + test.concurrent( + "a dependency-free workspace on disk but missing from bun.lock still fails --frozen-lockfile", + async () => { + const { packageDir, pruned } = await prunedTree("hoisted"); + await write( + join(packageDir, "packages", "newpkg", "package.json"), + JSON.stringify({ name: "newpkg", version: "1.0.0" }), + ); + + await frozen(packageDir, "hoisted", 1); + + expect(await lockText(packageDir)).toBe(pruned); + }, + ); + + test.concurrent("a package.json change still drops the optional-peer-only entry on a normal install", async () => { + const { packageDir } = await prunedTree("hoisted"); + await editApp(packageDir, app => delete app.dependencies["a-dep"]); + + const { stderr } = await install(packageDir, "hoisted"); + + expect(stderr).toContain("Saved lockfile"); + const lock = await lockText(packageDir); + expect(lock).not.toContain('"no-deps": ["no-deps@'); + expect(lock).not.toContain('"a-dep"'); + }); + + test.concurrent("a workspace that is on disk but no longer globbed is not treated as pruned", async () => { + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); + await writeMonorepo(packageDir, { withOther: true }); + const full = await fullLockfile("hoisted"); + await write(join(packageDir, "bun.lock"), full); + await write( + join(packageDir, "package.json"), + JSON.stringify({ name: "mono", workspaces: ["packages/app", "packages/shared"] }), + ); + + await frozen(packageDir, "hoisted", 1); + + expect(await lockText(packageDir)).toBe(full); + }); + + // An invalid prune (turbo always keeps transitive workspace deps): the survivor's `workspace:` edge is reported. + test.concurrent("a survivor depending on a pruned workspace fails naming the missing workspace", async () => { + const tree: Tree = { + root: rootPackageJson, + packages: { + ...monorepo.packages, + "packages/app": { + ...appPackageJson, + dependencies: { ...(appPackageJson.dependencies as object), other: "workspace:*" }, + }, + }, + }; + const { packageDir, full } = await verbatimScenario("hoisted", tree, survivors); + + const { stderr, exitCode } = await raw(packageDir, "hoisted", ["install", "--frozen-lockfile"]); + + expect(stderr).toContain('Workspace dependency "other" not found'); + expect(stderr).toContain("other@workspace:* failed to resolve"); + expect(exitCode).toBe(1); + expect(await lockText(packageDir)).toBe(full); + expect(await lstat(join(packageDir, "node_modules", "other")).catch(e => e.code)).toBe("ENOENT"); + }); +}); diff --git a/test/cli/install/migration/__snapshots__/pnpm-lock-v9.test.ts.snap b/test/cli/install/migration/__snapshots__/pnpm-lock-v9.test.ts.snap new file mode 100644 index 000000000000..7c9edc41e8a4 --- /dev/null +++ b/test/cli/install/migration/__snapshots__/pnpm-lock-v9.test.ts.snap @@ -0,0 +1,51 @@ +// Bun Snapshot v1, https://bun.sh/docs/test/snapshots + +exports[`pnpm-lock.yaml v9 v9 git and userinfo-tarball references migrate: bun.lock 1`] = ` +"{ + "lockfileVersion": 1, + "configVersion": 1, + "workspaces": { + "": { + "name": "v9-git-references", + "dependencies": { + "hue": "github:org/hue#ec3d1d1", + "pkg-a": "git+ssh://git@example.com/org/pkg-a.git#v1", + "priv": "https://token@tarballs.example.com/priv-1.0.0.tgz", + }, + }, + }, + "packages": { + "hue": ["hue@git+ssh://git@example.com:org/hue.git#ec3d1d18f73ab023b1fa3e31e1f4316f476566a5", { "dependencies": { "priv": "https://token@tarballs.example.com/priv-1.0.0.tgz" } }, ""], + + "pkg-a": ["pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567", { "dependencies": { "pkg-b": "git+ssh://git@example.com/org/pkg-b.git#89abcdef0123456789abcdef0123456789abcdef" } }, ""], + + "pkg-b": ["pkg-b@git+ssh://git@example.com/org/pkg-b.git#89abcdef0123456789abcdef0123456789abcdef", {}, ""], + + "priv": ["priv@https://token@tarballs.example.com/priv-1.0.0.tgz", {}], + } +} +" +`; + +exports[`pnpm-lock.yaml v9 snapshot alias whose dep-path version is a file: directory or tarball: bun.lock 1`] = ` +"{ + "lockfileVersion": 2, + "configVersion": 1, + "workspaces": { + "": { + "name": "v9-alias-non-registry-dep-path", + "dependencies": { + "outer": "file:outer", + }, + }, + }, + "packages": { + "fork": ["bar@github:o/bar#aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b", {}, ""], + + "outer": ["outer@file:outer", { "dependencies": { "config": "file:shared/config", "fork": "https://codeload.github.com/o/bar/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b" } }], + + "outer/config": ["hi2@file:shared/config", {}], + } +} +" +`; diff --git a/test/cli/install/migration/pnpm-lock-v9.test.ts b/test/cli/install/migration/pnpm-lock-v9.test.ts new file mode 100644 index 000000000000..182c399b4bcc --- /dev/null +++ b/test/cli/install/migration/pnpm-lock-v9.test.ts @@ -0,0 +1,819 @@ +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { existsSync, rmSync } from "fs"; +import { bunEnv, bunExe, nodeModulesPackages, tempDir, VerdaccioRegistry } from "harness"; +import { join } from "path"; + +const verdaccio = new VerdaccioRegistry(); + +beforeAll(async () => { + await verdaccio.start(); +}); + +afterAll(() => { + verdaccio.stop(); +}); + +async function migrate(cwd: string) { + await using proc = Bun.spawn({ + cmd: [bunExe(), "pm", "migrate"], + cwd, + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + }); + + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + + return { stdout, stderr, exitCode }; +} + +function fixture(name: string) { + return tempDir(`pnpm-${name}`, join(import.meta.dir, "pnpm", name)); +} + +async function bunLockOf(dir: string) { + return await Bun.file(join(dir, "bun.lock")).text(); +} + +const PKG_A_GIT = "pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567"; +const NO_DEPS_1_0_1_INTEGRITY = + "sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A=="; + +function registryLockfileWithTarball(tarball: string) { + return `lockfileVersion: '9.0' + +importers: + + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.1 + +packages: + + no-deps@1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}, tarball: ${tarball}} + +snapshots: + + no-deps@1.0.1: {} +`; +} + +describe("pnpm-lock.yaml v9", () => { + // Cases using toMatchSnapshot are sequential: snapshot matchers are unsupported inside a concurrent group. + test("v9 git and userinfo-tarball references migrate", async () => { + using dir = fixture("v9-git-references"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + + expect(bunLock).toContain(`"pkg-a": "git+ssh://git@example.com/org/pkg-a.git#v1"`); + expect(bunLock).toContain(`"hue": "github:org/hue#ec3d1d1"`); + expect(bunLock).toContain(`"${PKG_A_GIT}"`); + expect(bunLock).toContain( + `"pkg-b": "git+ssh://git@example.com/org/pkg-b.git#89abcdef0123456789abcdef0123456789abcdef"`, + ); + expect(bunLock).toContain( + `"pkg-b@git+ssh://git@example.com/org/pkg-b.git#89abcdef0123456789abcdef0123456789abcdef"`, + ); + expect(bunLock).toContain(`"hue@git+ssh://git@example.com:org/hue.git#ec3d1d18f73ab023b1fa3e31e1f4316f476566a5"`); + expect(bunLock).toContain(`"priv@https://token@tarballs.example.com/priv-1.0.0.tgz"`); + expect(bunLock).toContain(`"priv": "https://token@tarballs.example.com/priv-1.0.0.tgz"`); + expect(bunLock).not.toContain("npm:"); + + expect(bunLock).toMatchSnapshot("bun.lock"); + }); + + test("v9 alias in snapshot optionalDependencies gets the npm: prefix", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: join(import.meta.dir, "pnpm/v9-alias-in-optional-dependencies"), + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + + expect(bunLock).toContain(`"aliased-no-deps": "npm:no-deps@2.0.0"`); + expect(bunLock).toContain(`"aliased-no-deps": ["no-deps@2.0.0"`); + + await using install = Bun.spawn({ + cmd: [bunExe(), "install", "--frozen-lockfile"], + cwd: packageDir, + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + }); + + const [installStdout, installStderr, installExitCode] = await Promise.all([ + install.stdout.text(), + install.stderr.text(), + install.exited, + ]); + + expect(installStderr).not.toContain("error:"); + expect(installStdout).toContain("packages installed"); + expect(installExitCode).toBe(0); + + expect(nodeModulesPackages(packageDir)).toMatchInlineSnapshot(` + "node_modules/aliased-no-deps/no-deps@2.0.0 + node_modules/no-deps/no-deps@1.0.1 + node_modules/one-dep/one-dep@1.0.0" + `); + }); + + test.concurrent("reports the missing packages entry", async () => { + using dir = fixture("v9-missing-package-entry"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain( + "pnpm-lock.yaml has no package entry 'no-deps@1.0.0' for dependency 'no-deps' of importer '.'", + ); + expect(stderr).toContain("Error loading lockfile"); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + + test.concurrent("reports the missing packages entry of a workspace importer", async () => { + using dir = fixture("v9-missing-package-entry-workspace"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain( + "pnpm-lock.yaml has no package entry 'no-deps@1.0.0' for dependency 'no-deps' of importer 'packages/a'", + ); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + + test.concurrent("reports the missing packages entry of a transitive dependency", async () => { + using dir = fixture("v9-missing-package-entry-transitive"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain( + "pnpm-lock.yaml has no package entry 'no-deps@9.9.9' for dependency 'no-deps' of package 'pkg-a'", + ); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + + test.concurrent("reports an importer whose package.json is missing", async () => { + using dir = fixture("v9-missing-importer-package-json"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain( + "pnpm-lock.yaml lists importer 'packages/gone' but 'packages/gone/package.json' does not exist", + ); + expect(stderr).toContain("Error loading lockfile"); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + + test("registry-qualified dep path resolves from the configured registry with a warning", async () => { + // shape from pnpm11/deps/path/test/index.ts parse() `foo@work:1.0.0` + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ name: "registry-qualified", dependencies: { "no-deps": "^1.0.0" } }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +importers: + + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: work:1.0.1 + +packages: + + no-deps@work:1.0.1: + resolution: {integrity: sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A==} + +snapshots: + + no-deps@work:1.0.1: {} +`, + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain( + "pnpm-lock.yaml package 'no-deps@work:1.0.1' is from pnpm registry 'work', resolving it from the configured registry instead", + ); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"no-deps@1.0.1"`); + expect(bunLock).not.toContain("work:"); + expect(bunLock).not.toContain("git+ssh"); + }); + + test.concurrent("reports a package whose resolution cannot be parsed", async () => { + using dir = tempDir("pnpm-v9-unsupported-resolution", { + "package.json": JSON.stringify({ name: "unsupported-resolution", dependencies: { foo: "^1.0.0" } }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +importers: + + .: + dependencies: + foo: + specifier: ^1.0.0 + version: '1.0' + +packages: + + foo@1.0: + resolution: {integrity: sha512-foo==} + +snapshots: + + foo@1.0: {} +`, + }); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("pnpm-lock.yaml package 'foo@1.0' has an unsupported resolution"); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + + test.concurrent("warns about a lockfileVersion newer than 9", async () => { + using dir = tempDir("pnpm-v9-newer-version", { + "package.json": JSON.stringify({ name: "newer-version" }), + "pnpm-lock.yaml": `lockfileVersion: '10.0' + +importers: + + .: {} +`, + }); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("lockfileVersion 10 is newer than the supported 9.0"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + }); + + describe("multi-document lockfile", () => { + // pnpm 11 writes `------` (pnpm11/lockfile/fs/src/envLockfile.ts) + test.concurrent("migrates the last document", async () => { + using dir = fixture("v9-multi-document"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"${PKG_A_GIT}"`); + expect(bunLock).not.toContain("plugin-better-defaults"); + }); + + test.concurrent("rejects a file whose main document is empty", async () => { + using dir = tempDir("pnpm-v9-empty-main-document", { + "package.json": JSON.stringify({ name: "empty-main-document" }), + "pnpm-lock.yaml": `--- +lockfileVersion: '9.0' + +importers: + + .: + configDependencies: {} + +--- +`, + }); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("pnpm-lock.yaml root must be an object"); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + }); + + test.concurrent("runtime: entries are skipped with a warning", async () => { + using dir = fixture("v9-runtime-entries"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("pnpm-lock.yaml runtime dependency 'node@runtime:22.0.0' is not migrated"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"${PKG_A_GIT}"`); + expect(bunLock).not.toContain(`"node"`); + expect(bunLock).not.toContain("runtime:"); + }); + + describe("patchedDependencies", () => { + // fixture ported from pnpm11/deps/compliance/commands/test/licenses/fixtures/with-git-protocol-patched-deps + const IS_POSITIVE = "is-positive@github:kevva/is-positive#97edff6f525f192a3f83cea1944765f769ae2678"; + + test.concurrent.each(["legacy", "bare"])("%s hash form on a git-hosted package", async form => { + using dir = fixture(`v9-patched-git-hosted-${form}`); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"${IS_POSITIVE}"`); + expect(bunLock).toContain(`"${IS_POSITIVE}": "patches/is-positive@3.1.0.patch"`); + expect(bunLock).not.toContain("is-positive@3.1.0@"); + + const packageJson = await Bun.file(join(String(dir), "package.json")).json(); + expect(packageJson.patchedDependencies).toEqual({ "is-positive@3.1.0": "patches/is-positive@3.1.0.patch" }); + }); + + test.concurrent("bare hash whose patch is not in the config is skipped with a warning", async () => { + using dir = fixture("v9-patched-git-hosted-bare"); + rmSync(join(String(dir), "pnpm-workspace.yaml")); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("pnpm-lock.yaml patch for 'is-positive@3.1.0' is not in patchedDependencies"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"${IS_POSITIVE}"`); + expect(bunLock).not.toContain("patchedDependencies"); + }); + + test("bare hash on a registry package with a bare `name` key in pnpm-workspace.yaml", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: join(import.meta.dir, "pnpm/v9-patch-bare-hash-registry"), + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"no-deps@1.0.1": "patches/no-deps.patch"`); + + const packageJson = await Bun.file(join(packageDir, "package.json")).json(); + expect(packageJson.patchedDependencies).toEqual({ "no-deps@1.0.1": "patches/no-deps.patch" }); + }); + }); + + test("catalog:default is the default catalog", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: join(import.meta.dir, "pnpm/v9-catalog-default"), + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).not.toContain("missing entry"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"no-deps@1.0.1"`); + }); + + test("reference shapes: scoped + peer suffix, short alias, scoped alias, file: tarball", async () => { + // reference vectors from pnpm11/deps/path/test/index.ts refToRelative() + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: join(import.meta.dir, "pnpm/v9-reference-shapes"), + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).not.toContain("no package entry"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"@types/no-deps@1.0.0"`); + expect(bunLock).toContain(`"@types/no-deps@2.0.0"`); + expect(bunLock).toContain(`"@types/is-number@1.0.0"`); + expect(bunLock).toContain(`"no-deps@1.0.1"`); + expect(bunLock).toContain(`"tb@tb-1.0.0.tgz"`); + expect(bunLock).toContain(`"nd": "npm:no-deps@1.0.1"`); + expect(bunLock).toContain(`"tnd": "npm:@types/no-deps@2.0.0"`); + expect(bunLock).toContain(`"tb": "file:tb-1.0.0.tgz"`); + }); + + test("snapshot alias whose dep-path version is a file: directory or tarball", async () => { + using dir = fixture("v9-alias-non-registry-dep-path"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"config": "file:shared/config"`); + expect(bunLock).toContain( + `"fork": "https://codeload.github.com/o/bar/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b"`, + ); + expect(bunLock).toContain(`"hi2@file:shared/config"`); + expect(bunLock).toContain(`"bar@github:o/bar#aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b"`); + expect(bunLock).not.toContain("npm:"); + + expect(bunLock).toMatchSnapshot("bun.lock"); + }); + + test.concurrent("local file: tarballs with tarball+integrity and integrity-only .tar.gz resolutions", async () => { + // tar-pkg entry ported from pnpm11/installing/deps-restorer/test/fixtures/has-local-dep/pkg/pnpm-lock.yaml + using dir = fixture("v9-local-tarballs"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain( + `["tar-pkg@../tar-pkg-1.0.0.tgz", {}, "sha512-HP/5Rgt3pVFLzjmN9qJJ6vZMgCwoCIl/m2bPndYT283CUqnmFiMx0GeeIJ7SyK6TYoJM78SEvFEOQie++caHqw=="]`, + ); + expect(bunLock).toContain(`["tar-gz-pkg@../tar-gz-pkg-1.0.0.tar.gz", {}, "sha512-`); + expect(bunLock).not.toContain("tar-gz-pkg@file:"); + }); + + test.concurrent("file: directory with type: directory and a nested file: dependency", async () => { + // ported from pnpm11/deps/compliance/commands/test/licenses/fixtures/with-file-protocol + using dir = fixture("v9-file-directory"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"sub-dep": "file:./sub-dep"`); + expect(bunLock).toContain(`["sub-dep@file:sub-dep", { "dependencies": { "nested-child": "file:sub-dep/child" } }]`); + expect(bunLock).toContain(`["nested-child@file:sub-dep/child", {}]`); + }); + + test.concurrent("codeload tarballs with and without gitHosted: true", async () => { + // ported from pnpm11/__fixtures__/with-git-protocol-dep and with-non-package-dep + using dir = fixture("v9-codeload-tarballs"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"is-negative@github:kevva/is-negative#1d7e288222b53a0cab90a331f1865220ec29560c"`); + expect(bunLock).toContain(`"camelcase@github:denolib/camelcase#aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b"`); + expect(bunLock).not.toContain("codeload.github.com"); + }); + + test.concurrent("git resolutions keep the ssh port and userinfo; orphan packages entries are ignored", async () => { + using dir = fixture("v9-git-urls-and-orphan"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"a@git+ssh://git@example.com:2222/org/a.git#aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"`); + expect(bunLock).toContain( + `"b@git+https://TOKEN:x-oauth-basic@github.com/foo/bar.git#bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"`, + ); + expect(bunLock).not.toContain("orphan"); + }); + + describe("injected workspace packages", () => { + test.concurrent("resolve to the workspace package instead of a folder package", async () => { + using dir = fixture("v9-injected-workspace"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"foo": ["foo@workspace:packages/foo"]`); + expect(bunLock).not.toContain("foo@file:"); + + await using install = Bun.spawn({ + cmd: [bunExe(), "install", "--frozen-lockfile", "--linker", "hoisted"], + cwd: String(dir), + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + }); + const [installStderr, installExitCode] = await Promise.all([install.stderr.text(), install.exited]); + expect(installStderr).not.toContain("error:"); + expect(installExitCode).toBe(0); + }); + + test.concurrent( + "pruned lockfile with a peer-suffixed packages key and a directory-typed registry key", + async () => { + // ported from pnpm11/installing/deps-restorer/test/fixtures/peer-variant-missing-resolution + using dir = fixture("v9-peer-variant-missing-resolution"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"pkg-a": ["pkg-a@workspace:packages/pkg-a"]`); + expect(bunLock).toContain(`"peer": ["peer@workspace:packages/peer"]`); + expect(bunLock).not.toContain("pkg-a@file:"); + expect(bunLock).not.toContain("peer@1.0.0"); + }, + ); + }); + + describe("registry tarball: urls", () => { + // pnpm/pnpm#13534: GitHub Packages / npm Enterprise tarballs are not on the canonical `/-/` path + test("recorded under the configured registry is kept", async () => { + const registry = verdaccio.registryUrl(); + const tarball = `${registry}download/no-deps/1.0.1/0123456789abcdef`; + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ name: "kept-tarball", dependencies: { "no-deps": "^1.0.0" } }), + "pnpm-lock.yaml": registryLockfileWithTarball(tarball), + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`["no-deps@1.0.1", "${tarball}", {}, "${NO_DEPS_1_0_1_INTEGRITY}"]`); + }); + + // pnpm/pnpm#5920 / #4361: a stale or injected off-registry tarball is rebuilt from the configured registry + test("recorded on a foreign host is rebuilt from the configured registry", async () => { + const registry = verdaccio.registryUrl(); + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ name: "foreign-tarball", dependencies: { "no-deps": "^1.0.0" } }), + "pnpm-lock.yaml": registryLockfileWithTarball("https://evil.example.com/no-deps/-/no-deps-1.0.1.tgz"), + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain( + `["no-deps@1.0.1", "${registry}no-deps/-/no-deps-1.0.1.tgz", {}, "${NO_DEPS_1_0_1_INTEGRITY}"]`, + ); + expect(bunLock).not.toContain("evil.example.com"); + }); + + test("under a registry whose hostname is a prefix of the recorded url's is rebuilt", async () => { + const registry = verdaccio.registryUrl(); + const lookalike = `${registry.slice(0, -1)}.evil.example.com/no-deps/-/no-deps-1.0.1.tgz`; + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ name: "lookalike-tarball", dependencies: { "no-deps": "^1.0.0" } }), + "pnpm-lock.yaml": registryLockfileWithTarball(lookalike), + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"${registry}no-deps/-/no-deps-1.0.1.tgz"`); + expect(bunLock).not.toContain("evil.example.com"); + }); + }); + + describe("git sub-directory dependencies", () => { + // pnpm/pnpm#8243: `repo#commit&path:sub/dir` has no bun equivalent; refuse instead of installing the repo root + test.concurrent("type: git resolution with path: is rejected naming the package", async () => { + using dir = fixture("v9-git-subdirectory"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain( + "pnpm-lock.yaml package 'pkg@git+ssh://git@example.com/org/monorepo.git#cba04669e621b85fbdb33371604de1a2898e68e9&path:packages/pkg' is a git sub-directory dependency", + ); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + + test.concurrent("git-hosted tarball resolution with path: is rejected naming the package", async () => { + // shape from pnpm11/resolving/git-resolver/test/index.ts "with both sub folder and branch" + const id = + "https://codeload.github.com/o/mono/tar.gz/777e8a3e78cc89bbf41fb3fd9f6cf922d5463313#path:/packages/pkg"; + using dir = tempDir("pnpm-v9-git-hosted-subdirectory", { + "package.json": JSON.stringify({ + name: "git-hosted-subdirectory", + dependencies: { pkg: "github:o/mono#beta&path:/packages/pkg" }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +importers: + + .: + dependencies: + pkg: + specifier: github:o/mono#beta&path:/packages/pkg + version: ${id} + +packages: + + pkg@${id}: + resolution: {gitHosted: true, path: /packages/pkg, tarball: https://codeload.github.com/o/mono/tar.gz/777e8a3e78cc89bbf41fb3fd9f6cf922d5463313} + version: 1.0.0 + +snapshots: + + pkg@${id}: {} +`, + }); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain(`pnpm-lock.yaml package 'pkg@${id}' is a git sub-directory dependency`); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + }); + + describe("overrides", () => { + function overridesLockfile(overrides: string) { + return `lockfileVersion: '9.0' + +overrides: +${overrides} + +importers: + + .: {} +`; + } + + // pnpm/pnpm#5928 (`-` removes the dependency) and pnpm/pnpm#6774 (`name@range` / `parent>child` keys) + test.concurrent("unsupported removal and selector keys warn but migrate", async () => { + using dir = tempDir("pnpm-v9-overrides-unsupported", { + "package.json": JSON.stringify({ name: "overrides-unsupported" }), + "pnpm-lock.yaml": overridesLockfile(` left-pad: '-' + semver@<7.5.2: 7.5.2 + foo>bar: 2.0.0 + '@scope/pkg@^1': 1.9.0 + '@scope/plain': 3.0.0 + plain: 1.0.0`), + }); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("pnpm-lock.yaml override 'left-pad' removes the dependency ('-')"); + expect(stderr).toContain( + "pnpm-lock.yaml override 'semver@<7.5.2' is scoped to a version range or parent package", + ); + expect(stderr).toContain("pnpm-lock.yaml override 'foo>bar' is scoped to a version range or parent package"); + expect(stderr).toContain( + "pnpm-lock.yaml override '@scope/pkg@^1' is scoped to a version range or parent package", + ); + expect(stderr).not.toContain("override '@scope/plain'"); + expect(stderr).not.toContain("override 'plain'"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"plain": "1.0.0"`); + expect(bunLock).toContain(`"@scope/plain": "3.0.0"`); + }); + + test.concurrent("an unparsable value names the override", async () => { + using dir = tempDir("pnpm-v9-overrides-invalid", { + "package.json": JSON.stringify({ name: "overrides-invalid" }), + "pnpm-lock.yaml": overridesLockfile(" foo: ftp://example.com/foo.tgz"), + }); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("pnpm-lock.yaml override 'foo' has an invalid value 'ftp://example.com/foo.tgz'"); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + }); + + test.concurrent("link: version with a semver specifier resolves to the workspace (pnpm/pnpm#7712)", async () => { + // save-workspace-protocol=false / link-workspace-packages shape + using dir = fixture("v9-link-semver-specifier"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"common": ["common@workspace:shared/common"]`); + expect(bunLock).not.toContain("link:"); + + await using install = Bun.spawn({ + cmd: [bunExe(), "install", "--frozen-lockfile", "--linker", "hoisted"], + cwd: String(dir), + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + }); + const [installStderr, installExitCode] = await Promise.all([install.stderr.text(), install.exited]); + expect(installStderr).not.toContain("error:"); + expect(installExitCode).toBe(0); + }); + + test.concurrent("prettier-style multi-line resolution mappings migrate identically (pnpm/pnpm#4084)", async () => { + using plain = fixture("v9-git-references"); + using formatted = fixture("v9-git-references"); + + const lockfilePath = join(String(formatted), "pnpm-lock.yaml"); + const original = await Bun.file(lockfilePath).text(); + const oneLine = " resolution: {tarball: https://token@tarballs.example.com/priv-1.0.0.tgz}\n"; + expect(original).toContain(oneLine); + await Bun.write( + lockfilePath, + original.replace( + oneLine, + ` resolution: + { + tarball: https://token@tarballs.example.com/priv-1.0.0.tgz, + } +`, + ), + ); + + const [plainResult, formattedResult] = await Promise.all([migrate(String(plain)), migrate(String(formatted))]); + + expect(formattedResult.stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(formattedResult.exitCode).toBe(0); + expect(plainResult.exitCode).toBe(0); + expect(await bunLockOf(String(formatted))).toBe(await bunLockOf(String(plain))); + }); + + describe("catalogs", () => { + // pnpm/pnpm#10551: pruned Docker contexts ship the lockfile without pnpm-workspace.yaml + test("lockfile catalogs: section is enough without pnpm-workspace.yaml", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: join(import.meta.dir, "pnpm/v9-catalog-default"), + }); + rmSync(join(packageDir, "pnpm-workspace.yaml")); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"catalog": {\n "no-deps": "^1.0.0",\n }`); + expect(bunLock).toContain(`"no-deps@1.0.1"`); + }); + + // pnpm/pnpm#10456: `pnpm remove` can drop the catalogs: section while importers still say catalog: + test.concurrent("importer catalog: reference without a catalogs: section is reported", async () => { + using dir = fixture("v9-catalog-default"); + + const lockfilePath = join(String(dir), "pnpm-lock.yaml"); + const original = await Bun.file(lockfilePath).text(); + const catalogsBlock = original.slice(original.indexOf("catalogs:"), original.indexOf("importers:")); + expect(catalogsBlock).toContain("no-deps"); + await Bun.write(lockfilePath, original.replace(catalogsBlock, "")); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("pnpm-lock.yaml catalog 'default' missing entry for dependency 'no-deps'"); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + }); +}); diff --git a/test/cli/install/migration/pnpm/v9-alias-in-optional-dependencies/package.json b/test/cli/install/migration/pnpm/v9-alias-in-optional-dependencies/package.json new file mode 100644 index 000000000000..6def62bc4828 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-alias-in-optional-dependencies/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-alias-in-optional-dependencies", + "version": "1.0.0", + "dependencies": { + "one-dep": "^1.0.0" + } +} diff --git a/test/cli/install/migration/pnpm/v9-alias-in-optional-dependencies/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-alias-in-optional-dependencies/pnpm-lock.yaml new file mode 100644 index 000000000000..31f77573e4a6 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-alias-in-optional-dependencies/pnpm-lock.yaml @@ -0,0 +1,36 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + one-dep: + specifier: ^1.0.0 + version: 1.0.0 + +packages: + + no-deps@1.0.1: + resolution: {integrity: sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A==} + + no-deps@2.0.0: + resolution: {integrity: sha512-W3duJKZPcMIG5rA1io5cSK/bhW9rWFz+jFxZsKS/3suK4qHDkQNxUTEXee9/hTaAoDCeHWQqogukWYKzfr6X4g==} + + one-dep@1.0.0: + resolution: {integrity: sha512-qG6lZjwM1vFmRCHwP+XpOKu6FkrBmwr20+54+qaHGdjZlw/wz8aJrhFqX4dZksqmBLZtj2mzL77Yf04WKs1+Kg==} + +snapshots: + + no-deps@1.0.1: {} + + no-deps@2.0.0: {} + + one-dep@1.0.0: + dependencies: + no-deps: 1.0.1 + optionalDependencies: + aliased-no-deps: no-deps@2.0.0 diff --git a/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/outer/package.json b/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/outer/package.json new file mode 100644 index 000000000000..b7c86a6e639f --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/outer/package.json @@ -0,0 +1,8 @@ +{ + "name": "outer", + "version": "1.0.0", + "dependencies": { + "config": "file:../shared/config", + "fork": "github:o/bar#aeb6b15" + } +} diff --git a/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/package.json b/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/package.json new file mode 100644 index 000000000000..c95d5efd9741 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-alias-non-registry-dep-path", + "version": "1.0.0", + "dependencies": { + "outer": "file:outer" + } +} diff --git a/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/pnpm-lock.yaml new file mode 100644 index 000000000000..c2c369f8d6db --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/pnpm-lock.yaml @@ -0,0 +1,38 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + outer: + specifier: file:outer + version: file:outer + +packages: + + bar@https://codeload.github.com/o/bar/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b: + resolution: {tarball: https://codeload.github.com/o/bar/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b} + version: 2.0.0 + + hi2@file:shared/config: + resolution: {directory: shared/config, type: directory} + version: 1.0.0 + + outer@file:outer: + resolution: {directory: outer, type: directory} + version: 1.0.0 + +snapshots: + + bar@https://codeload.github.com/o/bar/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b: {} + + hi2@file:shared/config: {} + + outer@file:outer: + dependencies: + config: hi2@file:shared/config + fork: bar@https://codeload.github.com/o/bar/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b diff --git a/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/shared/config/package.json b/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/shared/config/package.json new file mode 100644 index 000000000000..b010b6b0ce94 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-alias-non-registry-dep-path/shared/config/package.json @@ -0,0 +1,4 @@ +{ + "name": "hi2", + "version": "1.0.0" +} diff --git a/test/cli/install/migration/pnpm/v9-catalog-default/package.json b/test/cli/install/migration/pnpm/v9-catalog-default/package.json new file mode 100644 index 000000000000..932f22ff4009 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-catalog-default/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-catalog-default", + "version": "1.0.0", + "dependencies": { + "no-deps": "catalog:default" + } +} diff --git a/test/cli/install/migration/pnpm/v9-catalog-default/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-catalog-default/pnpm-lock.yaml new file mode 100644 index 000000000000..693ecf5db9a7 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-catalog-default/pnpm-lock.yaml @@ -0,0 +1,28 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +catalogs: + default: + no-deps: + specifier: ^1.0.0 + version: 1.0.1 + +importers: + + .: + dependencies: + no-deps: + specifier: catalog:default + version: 1.0.1 + +packages: + + no-deps@1.0.1: + resolution: {integrity: sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A==} + +snapshots: + + no-deps@1.0.1: {} diff --git a/test/cli/install/migration/pnpm/v9-catalog-default/pnpm-workspace.yaml b/test/cli/install/migration/pnpm/v9-catalog-default/pnpm-workspace.yaml new file mode 100644 index 000000000000..561ac6fd9950 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-catalog-default/pnpm-workspace.yaml @@ -0,0 +1,2 @@ +catalog: + no-deps: ^1.0.0 diff --git a/test/cli/install/migration/pnpm/v9-codeload-tarballs/package.json b/test/cli/install/migration/pnpm/v9-codeload-tarballs/package.json new file mode 100644 index 000000000000..a8cd1473fde4 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-codeload-tarballs/package.json @@ -0,0 +1,8 @@ +{ + "name": "v9-codeload-tarballs", + "version": "1.0.0", + "dependencies": { + "camelcase": "denolib/camelcase#aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b", + "is-negative": "github:kevva/is-negative#master" + } +} diff --git a/test/cli/install/migration/pnpm/v9-codeload-tarballs/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-codeload-tarballs/pnpm-lock.yaml new file mode 100644 index 000000000000..bacc06ef77ea --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-codeload-tarballs/pnpm-lock.yaml @@ -0,0 +1,33 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + camelcase: + specifier: denolib/camelcase#aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b + version: https://codeload.github.com/denolib/camelcase/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b + is-negative: + specifier: github:kevva/is-negative#master + version: https://codeload.github.com/kevva/is-negative/tar.gz/1d7e288222b53a0cab90a331f1865220ec29560c + +packages: + + camelcase@https://codeload.github.com/denolib/camelcase/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b: + resolution: {tarball: https://codeload.github.com/denolib/camelcase/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b} + version: 0.0.0 + + is-negative@https://codeload.github.com/kevva/is-negative/tar.gz/1d7e288222b53a0cab90a331f1865220ec29560c: + resolution: {gitHosted: true, tarball: https://codeload.github.com/kevva/is-negative/tar.gz/1d7e288222b53a0cab90a331f1865220ec29560c} + version: 2.1.0 + engines: {node: '>=0.10.0'} + +snapshots: + + camelcase@https://codeload.github.com/denolib/camelcase/tar.gz/aeb6b15f9c9957c8fa56f9731e914c4d8a6d2f2b: {} + + is-negative@https://codeload.github.com/kevva/is-negative/tar.gz/1d7e288222b53a0cab90a331f1865220ec29560c: {} diff --git a/test/cli/install/migration/pnpm/v9-file-directory/package.json b/test/cli/install/migration/pnpm/v9-file-directory/package.json new file mode 100644 index 000000000000..9835cf0905e6 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-file-directory/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-file-directory", + "version": "1.0.0", + "dependencies": { + "sub-dep": "file:./sub-dep" + } +} diff --git a/test/cli/install/migration/pnpm/v9-file-directory/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-file-directory/pnpm-lock.yaml new file mode 100644 index 000000000000..ad355298a2da --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-file-directory/pnpm-lock.yaml @@ -0,0 +1,33 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + sub-dep: + specifier: file:./sub-dep + version: file:sub-dep + +packages: + + nested-child@file:sub-dep/child: + resolution: {directory: sub-dep/child, type: directory} + version: 1.0.0 + + sub-dep@file:sub-dep: + resolution: {directory: sub-dep, type: directory} + version: 1.0.0 + +snapshots: + + nested-child@file:sub-dep/child: + dev: false + + sub-dep@file:sub-dep: + dependencies: + nested-child: file:sub-dep/child + dev: false diff --git a/test/cli/install/migration/pnpm/v9-file-directory/sub-dep/child/package.json b/test/cli/install/migration/pnpm/v9-file-directory/sub-dep/child/package.json new file mode 100644 index 000000000000..359a7458f71d --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-file-directory/sub-dep/child/package.json @@ -0,0 +1,4 @@ +{ + "name": "nested-child", + "version": "1.0.0" +} diff --git a/test/cli/install/migration/pnpm/v9-file-directory/sub-dep/package.json b/test/cli/install/migration/pnpm/v9-file-directory/sub-dep/package.json new file mode 100644 index 000000000000..3b339b5b1338 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-file-directory/sub-dep/package.json @@ -0,0 +1,7 @@ +{ + "name": "sub-dep", + "version": "1.0.0", + "dependencies": { + "nested-child": "file:./child" + } +} diff --git a/test/cli/install/migration/pnpm/v9-git-references/package.json b/test/cli/install/migration/pnpm/v9-git-references/package.json new file mode 100644 index 000000000000..a51dd8f34d9e --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-git-references/package.json @@ -0,0 +1,9 @@ +{ + "name": "v9-git-references", + "version": "1.0.0", + "dependencies": { + "hue": "github:org/hue#ec3d1d1", + "pkg-a": "git+ssh://git@example.com/org/pkg-a.git#v1", + "priv": "https://token@tarballs.example.com/priv-1.0.0.tgz" + } +} diff --git a/test/cli/install/migration/pnpm/v9-git-references/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-git-references/pnpm-lock.yaml new file mode 100644 index 000000000000..9ab16126312e --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-git-references/pnpm-lock.yaml @@ -0,0 +1,51 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + hue: + specifier: github:org/hue#ec3d1d1 + version: git+https://git@example.com:org/hue.git#ec3d1d18f73ab023b1fa3e31e1f4316f476566a5 + pkg-a: + specifier: git+ssh://git@example.com/org/pkg-a.git#v1 + version: git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567 + priv: + specifier: https://token@tarballs.example.com/priv-1.0.0.tgz + version: https://token@tarballs.example.com/priv-1.0.0.tgz + +packages: + + hue@git+https://git@example.com:org/hue.git#ec3d1d18f73ab023b1fa3e31e1f4316f476566a5: + resolution: {commit: ec3d1d18f73ab023b1fa3e31e1f4316f476566a5, repo: git@example.com:org/hue.git, type: git} + version: 0.2.3 + + pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567: + resolution: {commit: 0123456789abcdef0123456789abcdef01234567, repo: ssh://git@example.com/org/pkg-a.git, type: git} + version: 1.0.0 + + pkg-b@git+ssh://git@example.com/org/pkg-b.git#89abcdef0123456789abcdef0123456789abcdef: + resolution: {commit: 89abcdef0123456789abcdef0123456789abcdef, repo: ssh://git@example.com/org/pkg-b.git, type: git} + version: 2.0.0 + + priv@https://token@tarballs.example.com/priv-1.0.0.tgz: + resolution: {tarball: https://token@tarballs.example.com/priv-1.0.0.tgz} + version: 1.0.0 + +snapshots: + + hue@git+https://git@example.com:org/hue.git#ec3d1d18f73ab023b1fa3e31e1f4316f476566a5: + dependencies: + priv: https://token@tarballs.example.com/priv-1.0.0.tgz + + pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567: + dependencies: + pkg-b: git+ssh://git@example.com/org/pkg-b.git#89abcdef0123456789abcdef0123456789abcdef + + pkg-b@git+ssh://git@example.com/org/pkg-b.git#89abcdef0123456789abcdef0123456789abcdef: {} + + priv@https://token@tarballs.example.com/priv-1.0.0.tgz: {} diff --git a/test/cli/install/migration/pnpm/v9-git-subdirectory/package.json b/test/cli/install/migration/pnpm/v9-git-subdirectory/package.json new file mode 100644 index 000000000000..08de06bd0f45 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-git-subdirectory/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-git-subdirectory", + "version": "1.0.0", + "dependencies": { + "pkg": "git+ssh://git@example.com/org/monorepo.git#main&path:packages/pkg" + } +} diff --git a/test/cli/install/migration/pnpm/v9-git-subdirectory/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-git-subdirectory/pnpm-lock.yaml new file mode 100644 index 000000000000..b1a75f2d380b --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-git-subdirectory/pnpm-lock.yaml @@ -0,0 +1,23 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + pkg: + specifier: git+ssh://git@example.com/org/monorepo.git#main&path:packages/pkg + version: git+ssh://git@example.com/org/monorepo.git#cba04669e621b85fbdb33371604de1a2898e68e9&path:packages/pkg + +packages: + + pkg@git+ssh://git@example.com/org/monorepo.git#cba04669e621b85fbdb33371604de1a2898e68e9&path:packages/pkg: + resolution: {commit: cba04669e621b85fbdb33371604de1a2898e68e9, path: packages/pkg, repo: git@example.com:org/monorepo.git, type: git} + version: 1.0.0 + +snapshots: + + pkg@git+ssh://git@example.com/org/monorepo.git#cba04669e621b85fbdb33371604de1a2898e68e9&path:packages/pkg: {} diff --git a/test/cli/install/migration/pnpm/v9-git-urls-and-orphan/package.json b/test/cli/install/migration/pnpm/v9-git-urls-and-orphan/package.json new file mode 100644 index 000000000000..b07f6d315c4a --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-git-urls-and-orphan/package.json @@ -0,0 +1,8 @@ +{ + "name": "v9-git-urls", + "version": "1.0.0", + "dependencies": { + "a": "git+ssh://git@example.com:2222/org/a.git", + "b": "git+https://TOKEN:x-oauth-basic@github.com/foo/bar.git" + } +} diff --git a/test/cli/install/migration/pnpm/v9-git-urls-and-orphan/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-git-urls-and-orphan/pnpm-lock.yaml new file mode 100644 index 000000000000..9a6e4100af8a --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-git-urls-and-orphan/pnpm-lock.yaml @@ -0,0 +1,35 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + a: + specifier: git+ssh://git@example.com:2222/org/a.git + version: git+ssh://git@example.com:2222/org/a.git#aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + b: + specifier: git+https://TOKEN:x-oauth-basic@github.com/foo/bar.git + version: git+https://TOKEN:x-oauth-basic@github.com/foo/bar.git#bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb + +packages: + + a@git+ssh://git@example.com:2222/org/a.git#aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa: + resolution: {commit: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa, repo: ssh://git@example.com:2222/org/a.git, type: git} + version: 1.0.0 + + b@git+https://TOKEN:x-oauth-basic@github.com/foo/bar.git#bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb: + resolution: {commit: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb, repo: https://TOKEN:x-oauth-basic@github.com/foo/bar.git, type: git} + version: 1.0.0 + + orphan@1.0.0: + resolution: {integrity: sha512-orphan==} + +snapshots: + + a@git+ssh://git@example.com:2222/org/a.git#aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa: {} + + b@git+https://TOKEN:x-oauth-basic@github.com/foo/bar.git#bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb: {} diff --git a/test/cli/install/migration/pnpm/v9-injected-workspace/package.json b/test/cli/install/migration/pnpm/v9-injected-workspace/package.json new file mode 100644 index 000000000000..f7154ecabc03 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-injected-workspace/package.json @@ -0,0 +1,11 @@ +{ + "name": "v9-injected-workspace", + "version": "1.0.0", + "private": true, + "workspaces": [ + "packages/*" + ], + "dependencies": { + "foo": "workspace:*" + } +} diff --git a/test/cli/install/migration/pnpm/v9-injected-workspace/packages/foo/package.json b/test/cli/install/migration/pnpm/v9-injected-workspace/packages/foo/package.json new file mode 100644 index 000000000000..da86787ad3ec --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-injected-workspace/packages/foo/package.json @@ -0,0 +1,4 @@ +{ + "name": "foo", + "version": "1.0.0" +} diff --git a/test/cli/install/migration/pnpm/v9-injected-workspace/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-injected-workspace/pnpm-lock.yaml new file mode 100644 index 000000000000..ff2fcaa0f954 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-injected-workspace/pnpm-lock.yaml @@ -0,0 +1,29 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + injectWorkspacePackages: true + +importers: + + .: + dependencies: + foo: + specifier: workspace:* + version: file:packages/foo + dependenciesMeta: + foo: + injected: true + + packages/foo: {} + +packages: + + foo@file:packages/foo: + resolution: {directory: packages/foo, type: directory} + version: 1.0.0 + +snapshots: + + foo@file:packages/foo: {} diff --git a/test/cli/install/migration/pnpm/v9-link-semver-specifier/apps/web/package.json b/test/cli/install/migration/pnpm/v9-link-semver-specifier/apps/web/package.json new file mode 100644 index 000000000000..805d5d9d22fe --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-link-semver-specifier/apps/web/package.json @@ -0,0 +1,7 @@ +{ + "name": "web", + "version": "1.0.0", + "dependencies": { + "common": "^1.0.0" + } +} diff --git a/test/cli/install/migration/pnpm/v9-link-semver-specifier/package.json b/test/cli/install/migration/pnpm/v9-link-semver-specifier/package.json new file mode 100644 index 000000000000..51a0e585ebb2 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-link-semver-specifier/package.json @@ -0,0 +1,9 @@ +{ + "name": "v9-link-semver-specifier", + "version": "1.0.0", + "private": true, + "workspaces": [ + "apps/*", + "shared/*" + ] +} diff --git a/test/cli/install/migration/pnpm/v9-link-semver-specifier/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-link-semver-specifier/pnpm-lock.yaml new file mode 100644 index 000000000000..7370b6dcf24e --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-link-semver-specifier/pnpm-lock.yaml @@ -0,0 +1,17 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: {} + + apps/web: + dependencies: + common: + specifier: ^1.0.0 + version: link:../../shared/common + + shared/common: {} diff --git a/test/cli/install/migration/pnpm/v9-link-semver-specifier/shared/common/package.json b/test/cli/install/migration/pnpm/v9-link-semver-specifier/shared/common/package.json new file mode 100644 index 000000000000..6d03cf1034bc --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-link-semver-specifier/shared/common/package.json @@ -0,0 +1,4 @@ +{ + "name": "common", + "version": "1.2.0" +} diff --git a/test/cli/install/migration/pnpm/v9-local-tarballs/package.json b/test/cli/install/migration/pnpm/v9-local-tarballs/package.json new file mode 100644 index 000000000000..494743727fa0 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-local-tarballs/package.json @@ -0,0 +1,8 @@ +{ + "name": "v9-local-tarballs", + "version": "1.0.0", + "dependencies": { + "tar-gz-pkg": "file:../tar-gz-pkg-1.0.0.tar.gz", + "tar-pkg": "file:../tar-pkg-1.0.0.tgz" + } +} diff --git a/test/cli/install/migration/pnpm/v9-local-tarballs/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-local-tarballs/pnpm-lock.yaml new file mode 100644 index 000000000000..ebc8fe861962 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-local-tarballs/pnpm-lock.yaml @@ -0,0 +1,32 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + tar-gz-pkg: + specifier: file:../tar-gz-pkg-1.0.0.tar.gz + version: file:../tar-gz-pkg-1.0.0.tar.gz + tar-pkg: + specifier: file:../tar-pkg-1.0.0.tgz + version: file:../tar-pkg-1.0.0.tgz + +packages: + + tar-gz-pkg@file:../tar-gz-pkg-1.0.0.tar.gz: + resolution: {integrity: sha512-HP/5Rgt3pVFLzjmN9qJJ6vZMgCwoCIl/m2bPndYT283CUqnmFiMx0GeeIJ7SyK6TYoJM78SEvFEOQie++caHqw==} + version: 1.0.0 + + tar-pkg@file:../tar-pkg-1.0.0.tgz: + resolution: {integrity: sha512-HP/5Rgt3pVFLzjmN9qJJ6vZMgCwoCIl/m2bPndYT283CUqnmFiMx0GeeIJ7SyK6TYoJM78SEvFEOQie++caHqw==, tarball: file:../tar-pkg-1.0.0.tgz} + version: 1.0.0 + +snapshots: + + tar-gz-pkg@file:../tar-gz-pkg-1.0.0.tar.gz: {} + + tar-pkg@file:../tar-pkg-1.0.0.tgz: {} diff --git a/test/cli/install/migration/pnpm/v9-missing-importer-package-json/package.json b/test/cli/install/migration/pnpm/v9-missing-importer-package-json/package.json new file mode 100644 index 000000000000..686c7db1a236 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-missing-importer-package-json/package.json @@ -0,0 +1,8 @@ +{ + "name": "v9-missing-importer-package-json", + "version": "1.0.0", + "private": true, + "workspaces": [ + "packages/*" + ] +} diff --git a/test/cli/install/migration/pnpm/v9-missing-importer-package-json/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-missing-importer-package-json/pnpm-lock.yaml new file mode 100644 index 000000000000..d7b5a79dbd1c --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-missing-importer-package-json/pnpm-lock.yaml @@ -0,0 +1,11 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: {} + + packages/gone: {} diff --git a/test/cli/install/migration/pnpm/v9-missing-package-entry-transitive/package.json b/test/cli/install/migration/pnpm/v9-missing-package-entry-transitive/package.json new file mode 100644 index 000000000000..555635a64dd4 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-missing-package-entry-transitive/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-missing-package-entry-transitive", + "version": "1.0.0", + "dependencies": { + "pkg-a": "git+ssh://git@example.com/org/pkg-a.git#v1" + } +} diff --git a/test/cli/install/migration/pnpm/v9-missing-package-entry-transitive/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-missing-package-entry-transitive/pnpm-lock.yaml new file mode 100644 index 000000000000..1056d565e005 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-missing-package-entry-transitive/pnpm-lock.yaml @@ -0,0 +1,25 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + pkg-a: + specifier: git+ssh://git@example.com/org/pkg-a.git#v1 + version: git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567 + +packages: + + pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567: + resolution: {commit: 0123456789abcdef0123456789abcdef01234567, repo: ssh://git@example.com/org/pkg-a.git, type: git} + version: 1.0.0 + +snapshots: + + pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567: + dependencies: + no-deps: 9.9.9 diff --git a/test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/package.json b/test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/package.json new file mode 100644 index 000000000000..ddc25ada08ac --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "v9-missing-package-entry-workspace", + "version": "1.0.0", + "private": true, + "workspaces": [ + "packages/*" + ] +} diff --git a/test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/packages/a/package.json b/test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/packages/a/package.json new file mode 100644 index 000000000000..3918473b22c6 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/packages/a/package.json @@ -0,0 +1,7 @@ +{ + "name": "a", + "version": "1.0.0", + "dependencies": { + "no-deps": "^1.0.0" + } +} diff --git a/test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/pnpm-lock.yaml new file mode 100644 index 000000000000..8b057beeab1e --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-missing-package-entry-workspace/pnpm-lock.yaml @@ -0,0 +1,15 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: {} + + packages/a: + dependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.0 diff --git a/test/cli/install/migration/pnpm/v9-missing-package-entry/package.json b/test/cli/install/migration/pnpm/v9-missing-package-entry/package.json new file mode 100644 index 000000000000..5c2d2c8b0011 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-missing-package-entry/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-missing-package-entry", + "version": "1.0.0", + "dependencies": { + "no-deps": "^1.0.0" + } +} diff --git a/test/cli/install/migration/pnpm/v9-missing-package-entry/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-missing-package-entry/pnpm-lock.yaml new file mode 100644 index 000000000000..0638a5cdf16e --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-missing-package-entry/pnpm-lock.yaml @@ -0,0 +1,13 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.0 diff --git a/test/cli/install/migration/pnpm/v9-multi-document/package.json b/test/cli/install/migration/pnpm/v9-multi-document/package.json new file mode 100644 index 000000000000..9b64cc89bb0e --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-multi-document/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-multi-document", + "version": "1.0.0", + "dependencies": { + "pkg-a": "git+ssh://git@example.com/org/pkg-a.git#v1" + } +} diff --git a/test/cli/install/migration/pnpm/v9-multi-document/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-multi-document/pnpm-lock.yaml new file mode 100644 index 000000000000..ef5b5d610660 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-multi-document/pnpm-lock.yaml @@ -0,0 +1,42 @@ +--- +lockfileVersion: '9.0' + +importers: + + .: + configDependencies: + '@pnpm/plugin-better-defaults': 0.3.0+sha512-aaaa + +packages: + + '@pnpm/plugin-better-defaults@0.3.0': + resolution: {integrity: sha512-aaaa} + +snapshots: + + '@pnpm/plugin-better-defaults@0.3.0': {} + +--- +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + pkg-a: + specifier: git+ssh://git@example.com/org/pkg-a.git#v1 + version: git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567 + +packages: + + pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567: + resolution: {commit: 0123456789abcdef0123456789abcdef01234567, repo: ssh://git@example.com/org/pkg-a.git, type: git} + version: 1.0.0 + +snapshots: + + pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567: {} diff --git a/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/package.json b/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/package.json new file mode 100644 index 000000000000..6f391a29e604 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-patch-bare-hash-registry", + "version": "1.0.0", + "dependencies": { + "no-deps": "^1.0.0" + } +} diff --git a/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/patches/no-deps.patch b/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/patches/no-deps.patch new file mode 100644 index 000000000000..ffa09982a910 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/patches/no-deps.patch @@ -0,0 +1,6 @@ +diff --git a/index.js b/index.js +--- a/index.js ++++ b/index.js +@@ -1 +1,2 @@ ++// patched + module.exports = 1; diff --git a/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/pnpm-lock.yaml new file mode 100644 index 000000000000..fd557794aa7a --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/pnpm-lock.yaml @@ -0,0 +1,25 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +patchedDependencies: + no-deps: 2mxqxzgazgkaqoljbgoadrshgq + +importers: + + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.1(patch_hash=2mxqxzgazgkaqoljbgoadrshgq) + +packages: + + no-deps@1.0.1: + resolution: {integrity: sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A==} + +snapshots: + + no-deps@1.0.1(patch_hash=2mxqxzgazgkaqoljbgoadrshgq): {} diff --git a/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/pnpm-workspace.yaml b/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/pnpm-workspace.yaml new file mode 100644 index 000000000000..ed6ce20cd587 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patch-bare-hash-registry/pnpm-workspace.yaml @@ -0,0 +1,2 @@ +patchedDependencies: + no-deps: patches/no-deps.patch diff --git a/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/package.json b/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/package.json new file mode 100644 index 000000000000..efb292860648 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-patched-git-hosted", + "version": "1.0.0", + "dependencies": { + "is-positive": "github:kevva/is-positive" + } +} diff --git a/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/patches/is-positive@3.1.0.patch b/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/patches/is-positive@3.1.0.patch new file mode 100644 index 000000000000..b2cd854982aa --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/patches/is-positive@3.1.0.patch @@ -0,0 +1,9 @@ +diff --git a/index.js b/index.js +--- a/index.js ++++ b/index.js +@@ -1,4 +1,5 @@ + 'use strict'; ++console.log('patched') + module.exports = function (n) { + return toString.call(n) === '[object Number]' && n > 0; + }; diff --git a/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/pnpm-lock.yaml new file mode 100644 index 000000000000..4260e850e36a --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/pnpm-lock.yaml @@ -0,0 +1,27 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +patchedDependencies: + is-positive@3.1.0: b27bbf2d83e68cac4491a38dd8b846aadd55d9c7bf8a4971139465c4de3566ce + +importers: + + .: + dependencies: + is-positive: + specifier: github:kevva/is-positive + version: https://codeload.github.com/kevva/is-positive/tar.gz/97edff6f525f192a3f83cea1944765f769ae2678(patch_hash=b27bbf2d83e68cac4491a38dd8b846aadd55d9c7bf8a4971139465c4de3566ce) + +packages: + + is-positive@https://codeload.github.com/kevva/is-positive/tar.gz/97edff6f525f192a3f83cea1944765f769ae2678: + resolution: {tarball: https://codeload.github.com/kevva/is-positive/tar.gz/97edff6f525f192a3f83cea1944765f769ae2678} + version: 3.1.0 + engines: {node: '>=0.10.0'} + +snapshots: + + is-positive@https://codeload.github.com/kevva/is-positive/tar.gz/97edff6f525f192a3f83cea1944765f769ae2678(patch_hash=b27bbf2d83e68cac4491a38dd8b846aadd55d9c7bf8a4971139465c4de3566ce): {} diff --git a/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/pnpm-workspace.yaml b/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/pnpm-workspace.yaml new file mode 100644 index 000000000000..cf9fbcc5985b --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patched-git-hosted-bare/pnpm-workspace.yaml @@ -0,0 +1,2 @@ +patchedDependencies: + is-positive@3.1.0: patches/is-positive@3.1.0.patch diff --git a/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/package.json b/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/package.json new file mode 100644 index 000000000000..efb292860648 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/package.json @@ -0,0 +1,7 @@ +{ + "name": "v9-patched-git-hosted", + "version": "1.0.0", + "dependencies": { + "is-positive": "github:kevva/is-positive" + } +} diff --git a/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/patches/is-positive@3.1.0.patch b/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/patches/is-positive@3.1.0.patch new file mode 100644 index 000000000000..b2cd854982aa --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/patches/is-positive@3.1.0.patch @@ -0,0 +1,9 @@ +diff --git a/index.js b/index.js +--- a/index.js ++++ b/index.js +@@ -1,4 +1,5 @@ + 'use strict'; ++console.log('patched') + module.exports = function (n) { + return toString.call(n) === '[object Number]' && n > 0; + }; diff --git a/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/pnpm-lock.yaml new file mode 100644 index 000000000000..e3111ae589c1 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/pnpm-lock.yaml @@ -0,0 +1,29 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +patchedDependencies: + is-positive@3.1.0: + hash: b27bbf2d83e68cac4491a38dd8b846aadd55d9c7bf8a4971139465c4de3566ce + path: patches/is-positive@3.1.0.patch + +importers: + + .: + dependencies: + is-positive: + specifier: github:kevva/is-positive + version: https://codeload.github.com/kevva/is-positive/tar.gz/97edff6f525f192a3f83cea1944765f769ae2678(patch_hash=b27bbf2d83e68cac4491a38dd8b846aadd55d9c7bf8a4971139465c4de3566ce) + +packages: + + is-positive@https://codeload.github.com/kevva/is-positive/tar.gz/97edff6f525f192a3f83cea1944765f769ae2678: + resolution: {tarball: https://codeload.github.com/kevva/is-positive/tar.gz/97edff6f525f192a3f83cea1944765f769ae2678} + version: 3.1.0 + engines: {node: '>=0.10.0'} + +snapshots: + + is-positive@https://codeload.github.com/kevva/is-positive/tar.gz/97edff6f525f192a3f83cea1944765f769ae2678(patch_hash=b27bbf2d83e68cac4491a38dd8b846aadd55d9c7bf8a4971139465c4de3566ce): {} diff --git a/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/pnpm-workspace.yaml b/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/pnpm-workspace.yaml new file mode 100644 index 000000000000..cf9fbcc5985b --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-patched-git-hosted-legacy/pnpm-workspace.yaml @@ -0,0 +1,2 @@ +patchedDependencies: + is-positive@3.1.0: patches/is-positive@3.1.0.patch diff --git a/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/package.json b/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/package.json new file mode 100644 index 000000000000..11fa61bac4be --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/package.json @@ -0,0 +1,11 @@ +{ + "name": "v9-peer-variant-missing-resolution", + "version": "1.0.0", + "private": true, + "workspaces": [ + "packages/*" + ], + "dependencies": { + "pkg-a": "workspace:*" + } +} diff --git a/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/packages/peer/package.json b/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/packages/peer/package.json new file mode 100644 index 000000000000..2b8b6606e669 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/packages/peer/package.json @@ -0,0 +1,5 @@ +{ + "name": "peer", + "version": "1.0.0", + "private": true +} diff --git a/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/packages/pkg-a/package.json b/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/packages/pkg-a/package.json new file mode 100644 index 000000000000..d0a16e5de2c7 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/packages/pkg-a/package.json @@ -0,0 +1,8 @@ +{ + "name": "pkg-a", + "version": "1.0.0", + "private": true, + "peerDependencies": { + "peer": "1.0.0" + } +} diff --git a/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/pnpm-lock.yaml new file mode 100644 index 000000000000..22bbce6e54e5 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-peer-variant-missing-resolution/pnpm-lock.yaml @@ -0,0 +1,41 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + injectWorkspacePackages: true + +importers: + + .: + dependencies: + pkg-a: + specifier: workspace:* + version: 'file:packages/pkg-a(peer@1.0.0)' + + packages/peer: {} + + packages/pkg-a: + peerDependencies: + peer: + specifier: 1.0.0 + version: 1.0.0 + +packages: + + 'pkg-a@file:packages/pkg-a(peer@1.0.0)': + dependencies: + peer: 1.0.0 + + 'peer@1.0.0': + resolution: { directory: packages/peer, type: directory } + +snapshots: + + 'pkg-a@file:packages/pkg-a': {} + + 'pkg-a@file:packages/pkg-a(peer@1.0.0)': + dependencies: + peer: 1.0.0 + + 'peer@1.0.0': {} diff --git a/test/cli/install/migration/pnpm/v9-reference-shapes/package.json b/test/cli/install/migration/pnpm/v9-reference-shapes/package.json new file mode 100644 index 000000000000..4c98ad591ff4 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-reference-shapes/package.json @@ -0,0 +1,11 @@ +{ + "name": "v9-reference-shapes", + "version": "1.0.0", + "dependencies": { + "@types/no-deps": "^1.0.0", + "nd": "npm:no-deps@1.0.1", + "one-dep": "^1.0.0", + "tb": "file:tb-1.0.0.tgz", + "tnd": "npm:@types/no-deps@2.0.0" + } +} diff --git a/test/cli/install/migration/pnpm/v9-reference-shapes/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-reference-shapes/pnpm-lock.yaml new file mode 100644 index 000000000000..c9089a5b5396 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-reference-shapes/pnpm-lock.yaml @@ -0,0 +1,69 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + '@types/no-deps': + specifier: ^1.0.0 + version: 1.0.0(@types/is-number@1.0.0) + nd: + specifier: npm:no-deps@1.0.1 + version: no-deps@1.0.1 + one-dep: + specifier: ^1.0.0 + version: 1.0.0 + tb: + specifier: file:tb-1.0.0.tgz + version: file:tb-1.0.0.tgz + tnd: + specifier: npm:@types/no-deps@2.0.0 + version: '@types/no-deps@2.0.0' + +packages: + + '@types/is-number@1.0.0': + resolution: {integrity: sha512-v7Teha9FjTcou+/dtF3KLYGcrEl3j5gbY7kIEF1LrwP4fjiiWUOh5qJbPc4tK2nB5pJ0O9cexMAZZ1ushh3GGQ==} + + '@types/no-deps@1.0.0': + resolution: {integrity: sha512-quthzD2O04AlTaZLJGf4a6/6aD7lf4Qa4HS7ViRWnTFdSbRbof20GFoq9YRCD3YQxd/HKI83YBAAiZ4ewoy+0Q==} + peerDependencies: + '@types/is-number': '*' + + '@types/no-deps@2.0.0': + resolution: {integrity: sha512-Zue3tPSS7wGh0k4QA1JyRHZSW9RJWTJxWllThag+zGm+3Ny3UNiOG6xt0OcX8yKaaRymhf9G1qGGOFLQdyEOpA==} + + no-deps@1.0.1: + resolution: {integrity: sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A==} + + one-dep@1.0.0: + resolution: {integrity: sha512-qG6lZjwM1vFmRCHwP+XpOKu6FkrBmwr20+54+qaHGdjZlw/wz8aJrhFqX4dZksqmBLZtj2mzL77Yf04WKs1+Kg==} + + tb@file:tb-1.0.0.tgz: + resolution: {tarball: file:tb-1.0.0.tgz} + version: 1.0.0 + +snapshots: + + '@types/is-number@1.0.0': {} + + '@types/no-deps@1.0.0(@types/is-number@1.0.0)': + dependencies: + '@types/is-number': 1.0.0 + + '@types/no-deps@2.0.0': {} + + no-deps@1.0.1: {} + + one-dep@1.0.0: + dependencies: + '@types/no-deps': 1.0.0(@types/is-number@1.0.0) + nd: no-deps@1.0.1 + tb: file:tb-1.0.0.tgz + tnd: '@types/no-deps@2.0.0' + + tb@file:tb-1.0.0.tgz: {} diff --git a/test/cli/install/migration/pnpm/v9-runtime-entries/package.json b/test/cli/install/migration/pnpm/v9-runtime-entries/package.json new file mode 100644 index 000000000000..4b8da23ead71 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-runtime-entries/package.json @@ -0,0 +1,14 @@ +{ + "name": "v9-runtime-entries", + "version": "1.0.0", + "dependencies": { + "pkg-a": "git+ssh://git@example.com/org/pkg-a.git#v1" + }, + "devEngines": { + "runtime": { + "name": "node", + "version": "22.0.0", + "onFail": "download" + } + } +} diff --git a/test/cli/install/migration/pnpm/v9-runtime-entries/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-runtime-entries/pnpm-lock.yaml new file mode 100644 index 000000000000..204eb6b55bb2 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-runtime-entries/pnpm-lock.yaml @@ -0,0 +1,43 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + pkg-a: + specifier: git+ssh://git@example.com/org/pkg-a.git#v1 + version: git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567 + devDependencies: + node: + specifier: runtime:22.0.0 + version: runtime:22.0.0 + +packages: + + node@runtime:22.0.0: + resolution: + type: variations + variants: + - targets: + - os: linux + cpu: x64 + resolution: + type: binary + archive: tarball + url: https://nodejs.org/download/release/v22.0.0/node-v22.0.0-linux-x64.tar.gz + integrity: sha512-aaaa + version: 22.0.0 + + pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567: + resolution: {commit: 0123456789abcdef0123456789abcdef01234567, repo: ssh://git@example.com/org/pkg-a.git, type: git} + version: 1.0.0 + +snapshots: + + node@runtime:22.0.0: {} + + pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567: {} diff --git a/test/cli/install/registry/fixtures/audit/pnpm-all-vulnerabilities-response.json b/test/cli/install/registry/fixtures/audit/pnpm-all-vulnerabilities-response.json new file mode 100644 index 000000000000..08eab9911821 --- /dev/null +++ b/test/cli/install/registry/fixtures/audit/pnpm-all-vulnerabilities-response.json @@ -0,0 +1,1690 @@ +{ + "axios": [ + { + "id": 1102326, + "url": "https://github.com/advisories/GHSA-cph5-m8f7-6c5x", + "title": "axios Inefficient Regular Expression Complexity vulnerability", + "severity": "high", + "vulnerable_versions": "<0.21.2", + "cwe": [ + "CWE-400", + "CWE-1333" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1111034, + "url": "https://github.com/advisories/GHSA-jr5f-v2jv-69x6", + "title": "axios Requests Vulnerable To Possible SSRF and Credential Leakage via Absolute URL", + "severity": "high", + "vulnerable_versions": "<0.30.0", + "cwe": [ + "CWE-918" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1113274, + "url": "https://github.com/advisories/GHSA-43fc-jf86-j433", + "title": "Axios is Vulnerable to Denial of Service via __proto__ Key in mergeConfig", + "severity": "high", + "vulnerable_versions": "<=0.30.2", + "cwe": [ + "CWE-754" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1116365, + "url": "https://github.com/advisories/GHSA-3p68-rc4w-qgx5", + "title": "Axios has a NO_PROXY Hostname Normalization Bypass Leads to SSRF", + "severity": "critical", + "vulnerable_versions": "<1.15.0", + "cwe": [ + "CWE-441", + "CWE-918" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1116605, + "url": "https://github.com/advisories/GHSA-fvcv-3m26-pcqx", + "title": "Axios has Unrestricted Cloud Metadata Exfiltration via Header Injection Chain", + "severity": "critical", + "vulnerable_versions": "<0.31.0", + "cwe": [ + "CWE-113", + "CWE-444", + "CWE-918" + ], + "cvss": { + "score": 10, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H" + } + }, + { + "id": 1090049, + "url": "https://github.com/advisories/GHSA-4w2v-q235-vp99", + "title": "Axios vulnerable to Server-Side Request Forgery", + "severity": "moderate", + "vulnerable_versions": "<0.21.1", + "cwe": [ + "CWE-918" + ], + "cvss": { + "score": 5.9, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:N" + } + }, + { + "id": 1097679, + "url": "https://github.com/advisories/GHSA-wf5p-g6vw-rhxx", + "title": "Axios Cross-Site Request Forgery Vulnerability", + "severity": "moderate", + "vulnerable_versions": ">=0.8.1 <0.28.0", + "cwe": [ + "CWE-352" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:N/A:N" + } + }, + { + "id": 1091722, + "url": "https://github.com/advisories/GHSA-42xw-2xvc-qx8m", + "title": "Denial of Service in axios", + "severity": "high", + "vulnerable_versions": "<=0.18.0", + "cwe": [ + "CWE-20", + "CWE-755" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "follow-redirects": [ + { + "id": 1102323, + "url": "https://github.com/advisories/GHSA-74fj-2j2h-c42q", + "title": "Exposure of sensitive information in follow-redirects", + "severity": "high", + "vulnerable_versions": "<1.14.7", + "cwe": [ + "CWE-359" + ], + "cvss": { + "score": 8, + "vectorString": "CVSS:3.0/AV:N/AC:L/PR:L/UI:R/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1109569, + "url": "https://github.com/advisories/GHSA-jchw-25xp-jwwc", + "title": "Follow Redirects improperly handles URLs in the url.parse() function", + "severity": "moderate", + "vulnerable_versions": "<1.15.4", + "cwe": [ + "CWE-20", + "CWE-601" + ], + "cvss": { + "score": 6.1, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:L/I:L/A:N" + } + }, + { + "id": 1096856, + "url": "https://github.com/advisories/GHSA-cxjh-pqwp-8mfp", + "title": "follow-redirects' Proxy-Authorization header kept across hosts", + "severity": "moderate", + "vulnerable_versions": "<=1.15.5", + "cwe": [ + "CWE-200" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N" + } + }, + { + "id": 1116560, + "url": "https://github.com/advisories/GHSA-r4q5-vmmm-2653", + "title": "follow-redirects leaks Custom Authentication Headers to Cross-Domain Redirect Targets", + "severity": "moderate", + "vulnerable_versions": "<=1.15.11", + "cwe": [ + "CWE-200" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1092623, + "url": "https://github.com/advisories/GHSA-pw2r-vq6v-hr8c", + "title": "Exposure of Sensitive Information to an Unauthorized Actor in follow-redirects", + "severity": "moderate", + "vulnerable_versions": "<1.14.8", + "cwe": [ + "CWE-200", + "CWE-212" + ], + "cvss": { + "score": 5.9, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:N" + } + } + ], + "debug": [ + { + "id": 1096792, + "url": "https://github.com/advisories/GHSA-gxpj-cx7g-858c", + "title": "Regular Expression Denial of Service in debug", + "severity": "low", + "vulnerable_versions": ">=4.0.0 <4.3.1", + "cwe": [ + "CWE-400" + ], + "cvss": { + "score": 3.7, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:L" + } + }, + { + "id": 1096793, + "url": "https://github.com/advisories/GHSA-gxpj-cx7g-858c", + "title": "Regular Expression Denial of Service in debug", + "severity": "low", + "vulnerable_versions": ">=3.2.0 <3.2.7", + "cwe": [ + "CWE-400" + ], + "cvss": { + "score": 3.7, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:L" + } + } + ], + "karma": [ + { + "id": 1090418, + "url": "https://github.com/advisories/GHSA-rc3x-jf5g-xvc5", + "title": "Open redirect in karma", + "severity": "moderate", + "vulnerable_versions": "<6.3.16", + "cwe": [ + "CWE-601" + ], + "cvss": { + "score": 5.4, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:L/I:L/A:N" + } + }, + { + "id": 1090439, + "url": "https://github.com/advisories/GHSA-7x7c-qm48-pq9c", + "title": "Cross-site Scripting in karma", + "severity": "moderate", + "vulnerable_versions": "<6.3.14", + "cwe": [ + "CWE-79" + ], + "cvss": { + "score": 6.1, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:L/I:L/A:N" + } + } + ], + "body-parser": [ + { + "id": 1099520, + "url": "https://github.com/advisories/GHSA-qwcr-r2fm-qrc7", + "title": "body-parser vulnerable to denial of service when url encoding is enabled", + "severity": "high", + "vulnerable_versions": "<1.20.3", + "cwe": [ + "CWE-405" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "qs": [ + { + "id": 1104115, + "url": "https://github.com/advisories/GHSA-hrpp-h998-j3pp", + "title": "qs vulnerable to Prototype Pollution", + "severity": "high", + "vulnerable_versions": "<6.2.4", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1104118, + "url": "https://github.com/advisories/GHSA-hrpp-h998-j3pp", + "title": "qs vulnerable to Prototype Pollution", + "severity": "high", + "vulnerable_versions": ">=6.5.0 <6.5.3", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1104120, + "url": "https://github.com/advisories/GHSA-hrpp-h998-j3pp", + "title": "qs vulnerable to Prototype Pollution", + "severity": "high", + "vulnerable_versions": ">=6.7.0 <6.7.3", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1113161, + "url": "https://github.com/advisories/GHSA-w7fw-mjwx-w883", + "title": "qs's arrayLimit bypass in comma parsing allows denial of service", + "severity": "low", + "vulnerable_versions": ">=6.7.0 <=6.14.1", + "cwe": [ + "CWE-20" + ], + "cvss": { + "score": 3.7, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:L" + } + }, + { + "id": 1113719, + "url": "https://github.com/advisories/GHSA-6rw7-vpxm-498p", + "title": "qs's arrayLimit bypass in its bracket notation allows DoS via memory exhaustion", + "severity": "moderate", + "vulnerable_versions": "<6.14.1", + "cwe": [ + "CWE-20" + ], + "cvss": { + "score": 3.7, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:L" + } + } + ], + "micromatch": [ + { + "id": 1098681, + "url": "https://github.com/advisories/GHSA-952p-6rrq-rcjv", + "title": "Regular Expression Denial of Service (ReDoS) in micromatch", + "severity": "moderate", + "vulnerable_versions": "<4.0.8", + "cwe": [ + "CWE-1333" + ], + "cvss": { + "score": 5.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L" + } + } + ], + "kind-of": [ + { + "id": 1095056, + "url": "https://github.com/advisories/GHSA-6c8f-qphg-qjgp", + "title": "Validation Bypass in kind-of", + "severity": "high", + "vulnerable_versions": ">=6.0.0 <6.0.3", + "cwe": [ + "CWE-668" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:H/A:N" + } + } + ], + "decode-uri-component": [ + { + "id": 1094087, + "url": "https://github.com/advisories/GHSA-w573-4hg7-7wgq", + "title": "decode-uri-component vulnerable to Denial of Service (DoS)", + "severity": "high", + "vulnerable_versions": "<0.2.1", + "cwe": [ + "CWE-20" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "braces": [ + { + "id": 1098094, + "url": "https://github.com/advisories/GHSA-grv7-fg5c-xmjg", + "title": "Uncontrolled resource consumption in braces", + "severity": "high", + "vulnerable_versions": "<3.0.3", + "cwe": [ + "CWE-400", + "CWE-1050" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "fsevents": [ + { + "id": 1094997, + "url": "https://github.com/advisories/GHSA-8r6j-v8pm-fqw3", + "title": "Code injection in fsevents", + "severity": "critical", + "vulnerable_versions": "<=1.2.10", + "cwe": [ + "CWE-94" + ], + "cvss": { + "score": 9.8, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1091853, + "url": "https://github.com/advisories/GHSA-xv2f-5jw4-v95m", + "title": "Malware in fsevents", + "severity": "critical", + "vulnerable_versions": ">=1.0.0 <1.2.11", + "cwe": [ + "CWE-506" + ], + "cvss": { + "score": 0, + "vectorString": null + } + } + ], + "minimist": [ + { + "id": 1097677, + "url": "https://github.com/advisories/GHSA-xvch-5gv4-984h", + "title": "Prototype Pollution in minimist", + "severity": "critical", + "vulnerable_versions": "<0.2.4", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 9.8, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1097678, + "url": "https://github.com/advisories/GHSA-xvch-5gv4-984h", + "title": "Prototype Pollution in minimist", + "severity": "critical", + "vulnerable_versions": ">=1.0.0 <1.2.6", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 9.8, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1096466, + "url": "https://github.com/advisories/GHSA-vh95-rmgr-6w4m", + "title": "Prototype Pollution in minimist", + "severity": "moderate", + "vulnerable_versions": "<0.2.1", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 5.6, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:L/I:L/A:L" + } + } + ], + "semver": [ + { + "id": 1112918, + "url": "https://github.com/advisories/GHSA-c2qf-rxjj-qqgw", + "title": "semver vulnerable to Regular Expression Denial of Service", + "severity": "high", + "vulnerable_versions": ">=2.0.0-alpha <5.7.2", + "cwe": [ + "CWE-1333" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "tar": [ + { + "id": 1112659, + "url": "https://github.com/advisories/GHSA-34x7-hfp2-rc4v", + "title": "node-tar Vulnerable to Arbitrary File Creation/Overwrite via Hardlink Path Traversal", + "severity": "high", + "vulnerable_versions": "<7.5.7", + "cwe": [ + "CWE-22", + "CWE-59" + ], + "cvss": { + "score": 8.2, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:L/A:N" + } + }, + { + "id": 1113300, + "url": "https://github.com/advisories/GHSA-8qq5-rm4j-mr97", + "title": "node-tar is Vulnerable to Arbitrary File Overwrite and Symlink Poisoning via Insufficient Path Sanitization", + "severity": "high", + "vulnerable_versions": "<=7.5.2", + "cwe": [ + "CWE-22" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1113375, + "url": "https://github.com/advisories/GHSA-83g3-92jg-28cx", + "title": "Arbitrary File Read/Write via Hardlink Target Escape Through Symlink Chain in node-tar Extraction", + "severity": "high", + "vulnerable_versions": "<7.5.8", + "cwe": [ + "CWE-22" + ], + "cvss": { + "score": 7.1, + "vectorString": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:N" + } + }, + { + "id": 1114200, + "url": "https://github.com/advisories/GHSA-qffp-2rhf-9h96", + "title": "tar has Hardlink Path Traversal via Drive-Relative Linkpath", + "severity": "high", + "vulnerable_versions": "<=7.5.9", + "cwe": [ + "CWE-22", + "CWE-59" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1114302, + "url": "https://github.com/advisories/GHSA-9ppj-qmqm-q256", + "title": "node-tar Symlink Path Traversal via Drive-Relative Linkpath", + "severity": "high", + "vulnerable_versions": "<=7.5.10", + "cwe": [ + "CWE-22" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1114680, + "url": "https://github.com/advisories/GHSA-r6q2-hw4h-h46w", + "title": "Race Condition in node-tar Path Reservations via Unicode Ligature Collisions on macOS APFS", + "severity": "high", + "vulnerable_versions": "<=7.5.3", + "cwe": [ + "CWE-176", + "CWE-367" + ], + "cvss": { + "score": 8.8, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:L/I:H/A:L" + } + }, + { + "id": 1095117, + "url": "https://github.com/advisories/GHSA-5955-9wpr-37jh", + "title": "Arbitrary File Creation/Overwrite on Windows via insufficient relative path sanitization", + "severity": "high", + "vulnerable_versions": "<4.4.18", + "cwe": [ + "CWE-22" + ], + "cvss": { + "score": 8.2, + "vectorString": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:N" + } + }, + { + "id": 1097493, + "url": "https://github.com/advisories/GHSA-f5x3-32g6-xq36", + "title": "Denial of service while parsing a tar file due to lack of folders count validation", + "severity": "moderate", + "vulnerable_versions": "<6.2.1", + "cwe": [ + "CWE-400" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1096376, + "url": "https://github.com/advisories/GHSA-9r2w-394v-53qc", + "title": "Arbitrary File Creation/Overwrite via insufficient symlink protection due to directory cache poisoning using symbolic links", + "severity": "high", + "vulnerable_versions": ">=3.0.0 <4.4.16", + "cwe": [ + "CWE-22", + "CWE-59" + ], + "cvss": { + "score": 8.2, + "vectorString": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:N" + } + }, + { + "id": 1096411, + "url": "https://github.com/advisories/GHSA-qq89-hq3f-393p", + "title": "Arbitrary File Creation/Overwrite via insufficient symlink protection due to directory cache poisoning using symbolic links", + "severity": "high", + "vulnerable_versions": ">=3.0.0 <4.4.18", + "cwe": [ + "CWE-22", + "CWE-59" + ], + "cvss": { + "score": 8.2, + "vectorString": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:N" + } + } + ], + "http-proxy": [ + { + "id": 1096334, + "url": "https://github.com/advisories/GHSA-6x33-pw7p-hmpq", + "title": "Denial of Service in http-proxy", + "severity": "high", + "vulnerable_versions": "<1.18.1", + "cwe": [ + "CWE-184", + "CWE-693" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "lodash": [ + { + "id": 1106913, + "url": "https://github.com/advisories/GHSA-35jh-r3h4-6jhm", + "title": "Command Injection in lodash", + "severity": "high", + "vulnerable_versions": "<4.17.21", + "cwe": [ + "CWE-77", + "CWE-94" + ], + "cvss": { + "score": 7.2, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1106920, + "url": "https://github.com/advisories/GHSA-p6mc-m468-83gw", + "title": "Prototype Pollution in lodash", + "severity": "high", + "vulnerable_versions": ">=3.7.0 <4.17.19", + "cwe": [ + "CWE-770", + "CWE-1321" + ], + "cvss": { + "score": 7.4, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:H/A:H" + } + }, + { + "id": 1108258, + "url": "https://github.com/advisories/GHSA-29mw-wpgm-hmr9", + "title": "Regular Expression Denial of Service (ReDoS) in lodash", + "severity": "moderate", + "vulnerable_versions": ">=4.0.0 <4.17.21", + "cwe": [ + "CWE-400", + "CWE-1333" + ], + "cvss": { + "score": 5.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L" + } + }, + { + "id": 1112455, + "url": "https://github.com/advisories/GHSA-xxjr-mmjv-4gpg", + "title": "Lodash has Prototype Pollution Vulnerability in `_.unset` and `_.omit` functions", + "severity": "moderate", + "vulnerable_versions": ">=4.0.0 <=4.17.22", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:L/A:L" + } + }, + { + "id": 1115806, + "url": "https://github.com/advisories/GHSA-r5fr-rjxr-66jc", + "title": "lodash vulnerable to Code Injection via `_.template` imports key names", + "severity": "high", + "vulnerable_versions": ">=4.0.0 <=4.17.23", + "cwe": [ + "CWE-94" + ], + "cvss": { + "score": 8.1, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1115810, + "url": "https://github.com/advisories/GHSA-f23m-r3pf-42rh", + "title": "lodash vulnerable to Prototype Pollution via array path bypass in `_.unset` and `_.omit`", + "severity": "moderate", + "vulnerable_versions": "<=4.17.23", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:L/A:L" + } + } + ], + "log4js": [ + { + "id": 1095531, + "url": "https://github.com/advisories/GHSA-82v2-mx6x-wq7q", + "title": "Incorrect Default Permissions in log4js", + "severity": "moderate", + "vulnerable_versions": "<6.4.0", + "cwe": [ + "CWE-276" + ], + "cvss": { + "score": 5.5, + "vectorString": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N" + } + } + ], + "url-parse": [ + { + "id": 1112827, + "url": "https://github.com/advisories/GHSA-hh27-ffr2-f2jc", + "title": "Open redirect in url-parse", + "severity": "moderate", + "vulnerable_versions": ">=0.1.0 <1.5.2", + "cwe": [ + "CWE-601" + ], + "cvss": { + "score": 6.1, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:L/I:L/A:N" + } + }, + { + "id": 1112828, + "url": "https://github.com/advisories/GHSA-9m6j-fcg5-2442", + "title": "Path traversal in url-parse", + "severity": "moderate", + "vulnerable_versions": ">=0.1.0 <1.5.0", + "cwe": [ + "CWE-23" + ], + "cvss": { + "score": 5.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:L/A:N" + } + }, + { + "id": 1112829, + "url": "https://github.com/advisories/GHSA-jf5r-8hm2-f872", + "title": "url-parse incorrectly parses hostname / protocol due to unstripped leading control characters.", + "severity": "moderate", + "vulnerable_versions": ">=0.1.0 <1.5.9", + "cwe": [ + "CWE-639" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:N" + } + }, + { + "id": 1113394, + "url": "https://github.com/advisories/GHSA-8v38-pw62-9cw2", + "title": "url-parse Incorrectly parses URLs that include an '@'", + "severity": "moderate", + "vulnerable_versions": ">=1.0.0 <1.5.7", + "cwe": [ + "CWE-639" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:N" + } + }, + { + "id": 1113395, + "url": "https://github.com/advisories/GHSA-rqff-837h-mm52", + "title": "Authorization bypass in url-parse", + "severity": "moderate", + "vulnerable_versions": ">=0.1.0 <1.5.6", + "cwe": [ + "CWE-639" + ], + "cvss": { + "score": 5.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:L/A:N" + } + }, + { + "id": 1095095, + "url": "https://github.com/advisories/GHSA-hgjh-723h-mx2j", + "title": "Authorization Bypass Through User-Controlled Key in url-parse", + "severity": "critical", + "vulnerable_versions": "<1.5.8", + "cwe": [ + "CWE-639" + ], + "cvss": { + "score": 9.1, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N" + } + } + ], + "request": [ + { + "id": 1096727, + "url": "https://github.com/advisories/GHSA-p8p7-x288-28g6", + "title": "Server-Side Request Forgery in Request", + "severity": "moderate", + "vulnerable_versions": "<=2.88.2", + "cwe": [ + "CWE-918" + ], + "cvss": { + "score": 6.1, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:L/I:L/A:N" + } + } + ], + "form-data": [ + { + "id": 1109540, + "url": "https://github.com/advisories/GHSA-fjxv-7rqg-78g4", + "title": "form-data uses unsafe random function in form-data for choosing boundary", + "severity": "critical", + "vulnerable_versions": "<2.5.4", + "cwe": [ + "CWE-330" + ], + "cvss": { + "score": 0, + "vectorString": null + } + } + ], + "ajv": [ + { + "id": 1113714, + "url": "https://github.com/advisories/GHSA-2g4f-4pwh-qvx6", + "title": "ajv has ReDoS when using `$data` option", + "severity": "moderate", + "vulnerable_versions": "<6.14.0", + "cwe": [ + "CWE-400", + "CWE-1333" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1097685, + "url": "https://github.com/advisories/GHSA-v88g-cgmw-v5xw", + "title": "Prototype Pollution in Ajv", + "severity": "moderate", + "vulnerable_versions": "<6.12.3", + "cwe": [ + "CWE-915", + "CWE-1321" + ], + "cvss": { + "score": 5.6, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:L/I:L/A:L" + } + } + ], + "json-schema": [ + { + "id": 1101855, + "url": "https://github.com/advisories/GHSA-896r-f27r-55mw", + "title": "json-schema is vulnerable to Prototype Pollution", + "severity": "critical", + "vulnerable_versions": "<0.4.0", + "cwe": [ + "CWE-915", + "CWE-1321" + ], + "cvss": { + "score": 9.8, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + } + ], + "tough-cookie": [ + { + "id": 1097682, + "url": "https://github.com/advisories/GHSA-72xf-g2v4-qvf3", + "title": "tough-cookie Prototype Pollution vulnerability", + "severity": "moderate", + "vulnerable_versions": "<4.1.3", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:N" + } + } + ], + "tunnel-agent": [ + { + "id": 1085744, + "url": "https://github.com/advisories/GHSA-xc7v-wxcw-j472", + "title": "Memory Exposure in tunnel-agent", + "severity": "moderate", + "vulnerable_versions": "<0.6.0", + "cwe": [ + "CWE-200" + ], + "cvss": { + "score": 0, + "vectorString": null + } + } + ], + "bl": [ + { + "id": 1090072, + "url": "https://github.com/advisories/GHSA-pp7h-53gx-mx7r", + "title": "Remote Memory Exposure in bl", + "severity": "moderate", + "vulnerable_versions": "<1.2.3", + "cwe": [ + "CWE-125", + "CWE-126" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:L" + } + } + ], + "jsonpointer": [ + { + "id": 1102906, + "url": "https://github.com/advisories/GHSA-282f-qqgm-c34q", + "title": "Prototype Pollution in node-jsonpointer", + "severity": "moderate", + "vulnerable_versions": "<5.0.0", + "cwe": [ + "CWE-843", + "CWE-1321" + ], + "cvss": { + "score": 5.6, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:L/I:L/A:L" + } + } + ], + "hawk": [ + { + "id": 1095062, + "url": "https://github.com/advisories/GHSA-44pw-h2cw-w3vq", + "title": "Uncontrolled Resource Consumption in Hawk", + "severity": "high", + "vulnerable_versions": "<9.0.1", + "cwe": [ + "CWE-400", + "CWE-1333" + ], + "cvss": { + "score": 7.4, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:N/I:N/A:H" + } + } + ], + "hoek": [ + { + "id": 1105092, + "url": "https://github.com/advisories/GHSA-c429-5p7v-vgjp", + "title": "hoek subject to prototype pollution via the clone function.", + "severity": "high", + "vulnerable_versions": "<=6.1.3", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 8.1, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1105121, + "url": "https://github.com/advisories/GHSA-jp4x-w63m-7wgm", + "title": "Prototype Pollution in hoek", + "severity": "high", + "vulnerable_versions": "<4.2.1", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 8.8, + "vectorString": "CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H" + } + } + ], + "timespan": [ + { + "id": 1093858, + "url": "https://github.com/advisories/GHSA-f523-2f5j-gfcg", + "title": "Regular Expression Denial of Service in timespan", + "severity": "high", + "vulnerable_versions": "<=2.3.0", + "cwe": [ + "CWE-400" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "async": [ + { + "id": 1097691, + "url": "https://github.com/advisories/GHSA-fwr7-v2mv-hh25", + "title": "Prototype Pollution in async", + "severity": "high", + "vulnerable_versions": ">=2.0.0 <2.6.4", + "cwe": [ + "CWE-1321" + ], + "cvss": { + "score": 7.8, + "vectorString": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H" + } + } + ], + "pac-resolver": [ + { + "id": 1090384, + "url": "https://github.com/advisories/GHSA-9j49-mfvp-vmhm", + "title": "Code Injection in pac-resolver", + "severity": "high", + "vulnerable_versions": "<5.0.0", + "cwe": [ + "CWE-94" + ], + "cvss": { + "score": 8.1, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + } + ], + "degenerator": [ + { + "id": 1090403, + "url": "https://github.com/advisories/GHSA-9j49-mfvp-vmhm", + "title": "Code Injection in pac-resolver", + "severity": "high", + "vulnerable_versions": "<3.0.1", + "cwe": [ + "CWE-94" + ], + "cvss": { + "score": 8.1, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + } + ], + "word-wrap": [ + { + "id": 1102444, + "url": "https://github.com/advisories/GHSA-j8xg-fqg3-53r7", + "title": "word-wrap vulnerable to Regular Expression Denial of Service", + "severity": "moderate", + "vulnerable_versions": "<1.2.4", + "cwe": [ + "CWE-1333" + ], + "cvss": { + "score": 5.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L" + } + } + ], + "ip": [ + { + "id": 1101851, + "url": "https://github.com/advisories/GHSA-2p57-rm9w-gvfp", + "title": "ip SSRF improper categorization in isPublic", + "severity": "high", + "vulnerable_versions": "<=2.0.1", + "cwe": [ + "CWE-918" + ], + "cvss": { + "score": 8.1, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1114831, + "url": "https://github.com/advisories/GHSA-78xj-cgh5-2h22", + "title": "NPM IP package incorrectly identifies some private IP addresses as public", + "severity": "low", + "vulnerable_versions": "<1.1.9", + "cwe": [ + "CWE-918" + ], + "cvss": { + "score": 0, + "vectorString": null + } + } + ], + "netmask": [ + { + "id": 1093560, + "url": "https://github.com/advisories/GHSA-pch5-whg9-qr2r", + "title": "netmask npm package mishandles octal input data", + "severity": "moderate", + "vulnerable_versions": "<2.0.1", + "cwe": [ + "CWE-20" + ], + "cvss": { + "score": 5.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:L/A:N" + } + }, + { + "id": 1089900, + "url": "https://github.com/advisories/GHSA-4c7m-wxvm-r7gc", + "title": "Improper parsing of octal bytes in netmask", + "severity": "critical", + "vulnerable_versions": "<1.1.0", + "cwe": [ + "CWE-20" + ], + "cvss": { + "score": 9.1, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N" + } + } + ], + "nodemailer": [ + { + "id": 1107232, + "url": "https://github.com/advisories/GHSA-9h6g-pr28-7cqp", + "title": "nodemailer ReDoS when trying to send a specially crafted email", + "severity": "moderate", + "vulnerable_versions": "<=6.9.8", + "cwe": [ + "CWE-1333" + ], + "cvss": { + "score": 5.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L" + } + }, + { + "id": 1109804, + "url": "https://github.com/advisories/GHSA-mm7p-fcc7-pg87", + "title": "Nodemailer: Email to an unintended domain can occur due to Interpretation Conflict", + "severity": "moderate", + "vulnerable_versions": "<7.0.7", + "cwe": [ + "CWE-20", + "CWE-436" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1113165, + "url": "https://github.com/advisories/GHSA-rcmh-qjqh-p98v", + "title": "Nodemailer’s addressparser is vulnerable to DoS caused by recursive calls", + "severity": "high", + "vulnerable_versions": "<=7.0.10", + "cwe": [ + "CWE-703" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1089709, + "url": "https://github.com/advisories/GHSA-hwqf-gcqm-7353", + "title": "Header injection in nodemailer", + "severity": "moderate", + "vulnerable_versions": "<6.6.1", + "cwe": [ + "CWE-74" + ], + "cvss": { + "score": 6.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:L/I:L/A:L" + } + }, + { + "id": 1115470, + "url": "https://github.com/advisories/GHSA-c7w3-x93f-qmm8", + "title": "Nodemailer has SMTP command injection due to unsanitized `envelope.size` parameter", + "severity": "low", + "vulnerable_versions": "<8.0.4", + "cwe": [ + "CWE-93" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1089880, + "url": "https://github.com/advisories/GHSA-48ww-j4fc-435p", + "title": "Command injection in nodemailer", + "severity": "critical", + "vulnerable_versions": "<6.4.16", + "cwe": [ + "CWE-88" + ], + "cvss": { + "score": 9.8, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1116270, + "url": "https://github.com/advisories/GHSA-vvjj-xcjg-gr5g", + "title": "Nodemailer Vulnerable to SMTP Command Injection via CRLF in Transport name Option (EHLO/HELO) ", + "severity": "moderate", + "vulnerable_versions": "<=8.0.4", + "cwe": [ + "CWE-93" + ], + "cvss": { + "score": 4.9, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:N/I:H/A:N" + } + } + ], + "underscore": [ + { + "id": 1109570, + "url": "https://github.com/advisories/GHSA-cf4h-3jhx-xvhq", + "title": "Arbitrary Code Execution in underscore", + "severity": "critical", + "vulnerable_versions": ">=1.3.2 <1.12.1", + "cwe": [ + "CWE-94" + ], + "cvss": { + "score": 9.8, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + }, + { + "id": 1113950, + "url": "https://github.com/advisories/GHSA-qpx9-hpmf-5gmw", + "title": "Underscore has unlimited recursion in _.flatten and _.isEqual, potential for DoS attack", + "severity": "high", + "vulnerable_versions": "<=1.13.7", + "cwe": [ + "CWE-674", + "CWE-770" + ], + "cvss": { + "score": 0, + "vectorString": null + } + } + ], + "redis": [ + { + "id": 1089196, + "url": "https://github.com/advisories/GHSA-35q2-47q7-3pc3", + "title": "Node-Redis potential exponential regex in monitor mode", + "severity": "high", + "vulnerable_versions": ">=2.6.0 <3.1.1", + "cwe": [ + "CWE-400" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "requestretry": [ + { + "id": 1090420, + "url": "https://github.com/advisories/GHSA-hjp8-2cm3-cc45", + "title": "Cookie exposure in requestretry", + "severity": "high", + "vulnerable_versions": "<7.0.0", + "cwe": [ + "CWE-200" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N" + } + } + ], + "minimatch": [ + { + "id": 1113459, + "url": "https://github.com/advisories/GHSA-3ppc-4f35-3m26", + "title": "minimatch has a ReDoS via repeated wildcards with non-matching literal in pattern", + "severity": "high", + "vulnerable_versions": "<3.1.3", + "cwe": [ + "CWE-1333" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1113538, + "url": "https://github.com/advisories/GHSA-7r86-cg39-jmmj", + "title": "minimatch has ReDoS: matchOne() combinatorial backtracking via multiple non-adjacent GLOBSTAR segments", + "severity": "high", + "vulnerable_versions": "<3.1.3", + "cwe": [ + "CWE-407" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1113546, + "url": "https://github.com/advisories/GHSA-23c5-xmqv-rm74", + "title": "minimatch ReDoS: nested *() extglobs generate catastrophically backtracking regular expressions", + "severity": "high", + "vulnerable_versions": "<3.1.4", + "cwe": [ + "CWE-1333" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1096485, + "url": "https://github.com/advisories/GHSA-f8q6-p94x-37v3", + "title": "minimatch ReDoS vulnerability", + "severity": "high", + "vulnerable_versions": "<3.0.5", + "cwe": [ + "CWE-400", + "CWE-1333" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "brace-expansion": [ + { + "id": 1105443, + "url": "https://github.com/advisories/GHSA-v6h2-p8h4-qcjw", + "title": "brace-expansion Regular Expression Denial of Service vulnerability", + "severity": "low", + "vulnerable_versions": ">=1.0.0 <=1.1.11", + "cwe": [ + "CWE-400" + ], + "cvss": { + "score": 3.1, + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:L/UI:N/S:U/C:N/I:N/A:L" + } + }, + { + "id": 1115540, + "url": "https://github.com/advisories/GHSA-f886-m6hf-6m8v", + "title": "brace-expansion: Zero-step sequence causes process hang and memory exhaustion", + "severity": "moderate", + "vulnerable_versions": "<1.1.13", + "cwe": [ + "CWE-400" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H" + } + } + ], + "socket.io": [ + { + "id": 1100551, + "url": "https://github.com/advisories/GHSA-25hc-qcg6-38wj", + "title": "socket.io has an unhandled 'error' event", + "severity": "moderate", + "vulnerable_versions": "<2.5.0", + "cwe": [ + "CWE-20", + "CWE-754" + ], + "cvss": { + "score": 7.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:L" + } + }, + { + "id": 1093718, + "url": "https://github.com/advisories/GHSA-fxwf-4rqh-v8g3", + "title": "CORS misconfiguration in socket.io", + "severity": "moderate", + "vulnerable_versions": "<2.4.0", + "cwe": [ + "CWE-346", + "CWE-453" + ], + "cvss": { + "score": 4.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:N" + } + } + ], + "engine.io": [ + { + "id": 1105123, + "url": "https://github.com/advisories/GHSA-j4f2-536g-r55m", + "title": "Resource exhaustion in engine.io", + "severity": "high", + "vulnerable_versions": "<3.6.0", + "cwe": [ + "CWE-400" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1089526, + "url": "https://github.com/advisories/GHSA-r7qp-cfhv-p84w", + "title": "Uncaught exception in engine.io", + "severity": "moderate", + "vulnerable_versions": "<3.6.1", + "cwe": [ + "CWE-248" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "cookie": [ + { + "id": 1103907, + "url": "https://github.com/advisories/GHSA-pxg6-pf52-xh8x", + "title": "cookie accepts cookie name, path, and domain with out of bounds characters", + "severity": "low", + "vulnerable_versions": "<0.7.0", + "cwe": [ + "CWE-74" + ], + "cvss": { + "score": 0, + "vectorString": null + } + } + ], + "ws": [ + { + "id": 1098395, + "url": "https://github.com/advisories/GHSA-3h5v-q93c-6h6q", + "title": "ws affected by a DoS when handling a request with many HTTP headers", + "severity": "high", + "vulnerable_versions": ">=2.1.0 <5.2.4", + "cwe": [ + "CWE-476" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "xmlhttprequest-ssl": [ + { + "id": 1095090, + "url": "https://github.com/advisories/GHSA-72mh-269x-7mh5", + "title": "Improper Certificate Validation in xmlhttprequest-ssl", + "severity": "critical", + "vulnerable_versions": "<1.6.1", + "cwe": [ + "CWE-295" + ], + "cvss": { + "score": 9.4, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:L" + } + }, + { + "id": 1095088, + "url": "https://github.com/advisories/GHSA-h4j5-c7cj-74xg", + "title": "xmlhttprequest and xmlhttprequest-ssl vulnerable to Arbitrary Code Injection", + "severity": "critical", + "vulnerable_versions": "<1.6.2", + "cwe": [ + "CWE-94" + ], + "cvss": { + "score": 9.8, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + } + ], + "parseuri": [ + { + "id": 1107224, + "url": "https://github.com/advisories/GHSA-6fx8-h7jm-663j", + "title": "parse-uri Regular expression Denial of Service (ReDoS)", + "severity": "moderate", + "vulnerable_versions": "<2.0.0", + "cwe": [ + "CWE-185", + "CWE-1333" + ], + "cvss": { + "score": 0, + "vectorString": null + } + } + ], + "socket.io-parser": [ + { + "id": 1100540, + "url": "https://github.com/advisories/GHSA-cqmj-92xf-r6r9", + "title": "Insufficient validation when decoding a Socket.IO packet", + "severity": "moderate", + "vulnerable_versions": "<3.3.4", + "cwe": [ + "CWE-20", + "CWE-754" + ], + "cvss": { + "score": 7.3, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:L" + } + }, + { + "id": 1115156, + "url": "https://github.com/advisories/GHSA-677m-j7p3-52f9", + "title": "socket.io allows an unbounded number of binary attachments", + "severity": "high", + "vulnerable_versions": "<3.3.5", + "cwe": [ + "CWE-754" + ], + "cvss": { + "score": 0, + "vectorString": null + } + }, + { + "id": 1089711, + "url": "https://github.com/advisories/GHSA-xfhh-g9f5-x4m4", + "title": "Resource exhaustion in socket.io-parser", + "severity": "high", + "vulnerable_versions": "<3.3.2", + "cwe": [ + "CWE-400" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + }, + { + "id": 1097134, + "url": "https://github.com/advisories/GHSA-qm95-pgcg-qqfq", + "title": "Insufficient validation when decoding a Socket.IO packet", + "severity": "critical", + "vulnerable_versions": "<3.3.3", + "cwe": [ + "CWE-20", + "CWE-89", + "CWE-1287" + ], + "cvss": { + "score": 9.8, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H" + } + } + ], + "tmp": [ + { + "id": 1109537, + "url": "https://github.com/advisories/GHSA-52f5-9888-hmc6", + "title": "tmp allows arbitrary temporary file / directory write via symbolic link `dir` parameter", + "severity": "low", + "vulnerable_versions": "<=0.2.3", + "cwe": [ + "CWE-59" + ], + "cvss": { + "score": 2.5, + "vectorString": "CVSS:3.1/AV:L/AC:H/PR:L/UI:N/S:U/C:N/I:L/A:N" + } + } + ], + "useragent": [ + { + "id": 1107230, + "url": "https://github.com/advisories/GHSA-mgfv-m47x-4wqp", + "title": "useragent Regular Expression Denial of Service vulnerability", + "severity": "moderate", + "vulnerable_versions": "<=2.3.0", + "cwe": [ + "CWE-1333" + ], + "cvss": { + "score": 7.5, + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H" + } + } + ], + "sync-exec": [ + { + "id": 1093475, + "url": "https://github.com/advisories/GHSA-38h8-x697-gh8q", + "title": "Tmp files readable by other users in sync-exec", + "severity": "moderate", + "vulnerable_versions": "<=0.6.2", + "cwe": [ + "CWE-377" + ], + "cvss": { + "score": 6.5, + "vectorString": "CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N" + } + } + ] +} \ No newline at end of file From e8358f33815f6c74cebc4ed709ae460f7f9ef542 Mon Sep 17 00:00:00 2001 From: Jarred Sumner Date: Thu, 13 Aug 2026 21:33:41 -0700 Subject: [PATCH 02/25] install: nested and version-scoped overrides, transitive updates, isolated relink Overrides: npm nested objects, yarn paths and pnpm parent>child selectors all lower to direct-parent rules; targets may carry a version selector ("lodash@<4.17.21": ...), matched against the dependent's declared range by intersection. Rules persist as objects inside the bun.lock overrides section and the file is stamped lockfileVersion 3 only when such rules exist, so existing lockfiles are byte-identical. Flat rules also fix $ref to workspace-member deps and catalog-valued values going stale. bun update always re-resolves transitive packages to the newest version each dependent's range allows (pnpm's default depth); bun update reaches any depth, matches npm: aliases by real name, updates in place and errors on unknown names; --latest never downgrades a locked version ahead of the tag; --interactive applies only the selection. The post-resolve package.json write-back now runs before bun.lock is saved and the lockfile's declared ranges, overrides and catalogs are re-derived from the final package.json, replacing the per-command literal rewriting. Isolated linker: an existing store entry whose dependencies re-resolved has its links refreshed (gated on a persisted store hash so unchanged installs do no extra work), and blocked entries resume through per-entry waiter lists instead of a scan of every entry after each completion. Also: one shared lockfile reachability walk for prune/dedupe/licenses/ audit; frozen installs only tolerate workspaces the lockfile knows about; optional-peer retention keyed on resolution-affecting diffs (turbo prune output); prune version-checks before deleting a shadowed copy and treats Windows reparse points as links; audit --json honors --audit-level and --ignore; --omit honored by audit and licenses; bare npm: aliases get a range on add and the install summary shows the alias target; per-test install caches in the concurrent suites; shell completions for the new commands. Co-authored-by: Kaj Kowalski No-Verification-Needed: user asked to skip --- completions/bun-cli.json | 446 +++++- completions/bun.bash | 47 +- completions/bun.fish | 45 +- completions/bun.zsh | 137 +- docs/pm/catalogs.mdx | 8 +- docs/pm/cli/add.mdx | 6 + docs/pm/cli/audit.mdx | 16 +- docs/pm/cli/dedupe.mdx | 3 +- docs/pm/cli/install.mdx | 10 +- docs/pm/cli/pm.mdx | 16 +- docs/pm/cli/prune.mdx | 12 +- docs/pm/cli/remove.mdx | 15 + docs/pm/cli/update.mdx | 37 +- docs/pm/filter.mdx | 4 +- docs/pm/isolated-installs.mdx | 1 + docs/pm/lockfile.mdx | 4 +- docs/pm/npmrc.mdx | 4 +- docs/pm/overrides.mdx | 94 +- docs/runtime/bunfig.mdx | 2 +- docs/snippets/cli/add.mdx | 4 + docs/snippets/cli/remove.mdx | 4 + docs/snippets/cli/update.mdx | 4 - src/install/PackageManager.rs | 38 +- .../PackageManager/CommandLineArguments.rs | 15 +- .../PackageManager/PackageJSONEditor.rs | 926 ++++-------- .../PackageManager/PackageManagerEnqueue.rs | 45 +- .../PackageManager/PackageManagerOptions.rs | 2 +- src/install/PackageManager/UpdateRequest.rs | 10 +- src/install/PackageManager/add_catalog.rs | 213 +-- .../PackageManager/add_remove_with_filter.rs | 116 +- .../PackageManager/install_with_manager.rs | 231 ++- .../PackageManager/package_json_write_back.rs | 430 ++++++ .../updatePackageJSONAndInstall.rs | 457 +----- src/install/audit_fix.rs | 69 +- src/install/dedupe.rs | 53 +- src/install/isolated_install.rs | 53 + src/install/isolated_install/Installer.rs | 310 ++-- src/install/isolated_install/Symlinker.rs | 23 +- src/install/lib.rs | 1 + src/install/lockfile.rs | 305 +--- src/install/lockfile/CatalogMap.rs | 37 +- src/install/lockfile/OverrideMap.rs | 1313 +++++++++++++---- src/install/lockfile/Package.rs | 75 +- src/install/lockfile/Package/WorkspaceMap.rs | 130 +- src/install/lockfile/bun.lock.rs | 368 ++++- src/install/lockfile/bun.lockb.rs | 71 + src/install/lockfile/override_selector.rs | 149 ++ src/install/lockfile/printer/tree_printer.rs | 68 +- src/install/lockfile/pruned_workspaces.rs | 9 + src/install/lockfile/reachable.rs | 99 ++ src/install/migration.rs | 4 +- src/install/pnpm.rs | 60 +- src/install/prune.rs | 379 +++-- src/install/update_transitive.rs | 453 ++++++ src/install/yarn.rs | 21 +- src/runtime/cli/audit_command.rs | 121 +- src/runtime/cli/dedupe_command.rs | 4 +- src/runtime/cli/pm_licenses_command.rs | 88 +- src/runtime/cli/prune_command.rs | 2 +- src/runtime/cli/update_interactive_command.rs | 34 +- src/semver/intersects.rs | 112 ++ src/semver/lib.rs | 1 + test/cli/install/bun-add-catalog.test.ts | 211 +-- test/cli/install/bun-add-filter.test.ts | 21 +- test/cli/install/bun-add.test.ts | 150 +- test/cli/install/bun-audit-fix.test.ts | 1020 ------------- test/cli/install/bun-audit.test.ts | 1063 ++++++++++++- test/cli/install/bun-dedupe.test.ts | 123 +- test/cli/install/bun-install-registry.test.ts | 13 +- test/cli/install/bun-pm-licenses.test.ts | 34 +- test/cli/install/bun-prune.test.ts | 256 +++- test/cli/install/bun-remove.test.ts | 83 +- .../bun-security-scanner-matrix-runner.ts | 3 + .../install/bun-update-lockfile-sync.test.ts | 559 +++++++ .../cli/install/bun-update-transitive.test.ts | 533 +++++++ test/cli/install/bun-update.test.ts | 304 +++- test/cli/install/catalog-peer-hoist.test.ts | 466 ------ test/cli/install/catalogs.test.ts | 458 +++++- test/cli/install/config-precedence.test.ts | 35 + .../frozen-lockfile-missing-workspace.test.ts | 112 ++ .../install/frozen-lockfile-pruned.test.ts | 200 ++- test/cli/install/isolated-relink.test.ts | 75 + test/cli/install/lockfile-only.test.ts | 137 +- .../yarn-lock-migration.test.ts.snap | 5 +- .../install/migration/pnpm-lock-v9.test.ts | 73 +- test/cli/install/nested-overrides.test.ts | 1016 +++++++++++++ .../packages/@scoped/pkg-1/package.json | 44 + .../packages/@scoped/pkg-1/pkg-1-1.1.1.tgz | Bin 0 -> 165 bytes 88 files changed, 10336 insertions(+), 4472 deletions(-) create mode 100644 src/install/PackageManager/package_json_write_back.rs create mode 100644 src/install/lockfile/override_selector.rs create mode 100644 src/install/lockfile/reachable.rs create mode 100644 src/install/update_transitive.rs create mode 100644 src/semver/intersects.rs delete mode 100644 test/cli/install/bun-audit-fix.test.ts create mode 100644 test/cli/install/bun-update-lockfile-sync.test.ts create mode 100644 test/cli/install/bun-update-transitive.test.ts delete mode 100644 test/cli/install/catalog-peer-hoist.test.ts create mode 100644 test/cli/install/frozen-lockfile-missing-workspace.test.ts create mode 100644 test/cli/install/isolated-relink.test.ts create mode 100644 test/cli/install/nested-overrides.test.ts create mode 100644 test/cli/install/registry/packages/@scoped/pkg-1/package.json create mode 100644 test/cli/install/registry/packages/@scoped/pkg-1/pkg-1-1.1.1.tgz diff --git a/completions/bun-cli.json b/completions/bun-cli.json index 6dfdac75a8f2..f4e4bb1bf4b5 100644 --- a/completions/bun-cli.json +++ b/completions/bun-cli.json @@ -503,6 +503,7 @@ }, { "name": "filter", + "shortName": "F", "description": "Install packages for the matching workspaces", "hasValue": true, "valueType": "val", @@ -523,6 +524,14 @@ "hasValue": false, "required": false, "multiple": false + }, + { + "name": "catalog", + "description": "Add the resolved version to the root package.json catalog and depend on it as \"catalog:\" (use --catalog=NAME for a named catalog)", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false } ], "positionalArgs": [ @@ -813,6 +822,15 @@ "required": false, "multiple": false }, + { + "name": "filter", + "shortName": "F", + "description": "Add the package(s) to the matching workspaces instead of the current package", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, { "name": "analyze", "shortName": "a", @@ -827,6 +845,14 @@ "hasValue": false, "required": false, "multiple": false + }, + { + "name": "catalog", + "description": "Add the resolved version to the root package.json catalog and depend on it as \"catalog:\" (use --catalog=NAME for a named catalog)", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false } ], "positionalArgs": [ @@ -1096,6 +1122,15 @@ "hasValue": false, "required": false, "multiple": false + }, + { + "name": "filter", + "shortName": "F", + "description": "Remove the package(s) from the matching workspaces instead of the current package", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false } ], "positionalArgs": [ @@ -1420,6 +1455,22 @@ "hasValue": false, "required": false, "multiple": false + }, + { + "name": "audit-level", + "description": "Only print advisories with severity greater than or equal to (low, moderate, high, critical)", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "ignore", + "description": "Ignore advisories by GHSA or numeric advisory ID (repeatable)", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false } ], "positionalArgs": [ @@ -1430,9 +1481,396 @@ "type": "string" } ], - "examples": ["bun audit", "bun audit --json"], + "examples": ["bun audit", "bun audit --json", "bun audit fix", "bun audit fix --dry-run"], "usage": "Usage: bun audit [flags]", "documentationUrl": "https://bun.com/docs/install/audit.", + "dynamicCompletions": {}, + "subcommands": { + "fix": { + "name": "fix", + "description": "Upgrade vulnerable packages to the lowest safe version that still satisfies every dependent's range", + "flags": [ + { + "name": "dry-run", + "description": "Show what bun audit fix would change without changing anything", + "hasValue": false, + "required": false, + "multiple": false + } + ] + } + } + }, + "dedupe": { + "name": "dedupe", + "description": "Remove duplicate versions from bun.lock by re-resolving dependency ranges onto versions that are already in the lockfile, then install.", + "flags": [ + { + "name": "config", + "shortName": "c", + "description": "Specify path to config file (bunfig.toml)", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "yarn", + "shortName": "y", + "description": "Write a yarn.lock file (yarn v1)", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "production", + "shortName": "p", + "description": "Don't install devDependencies", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "no-save", + "description": "Don't update package.json or save a lockfile", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "save", + "description": "Save to package.json (true by default)", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "ca", + "description": "Provide a Certificate Authority signing certificate", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "cafile", + "description": "The same as `--ca`, but is a file path to the certificate", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "dry-run", + "description": "Perform a dry run without making changes", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "frozen-lockfile", + "description": "Disallow changes to lockfile", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "force", + "shortName": "f", + "description": "Always request the latest versions from the registry & reinstall all dependencies", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "cache-dir", + "description": "Store & load cached data from a specific directory path", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "no-cache", + "description": "Ignore manifest cache entirely", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "silent", + "description": "Don't log anything", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "quiet", + "description": "Only show tarball name when packing", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "verbose", + "description": "Excessively verbose logging", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "no-progress", + "description": "Disable the progress bar", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "no-summary", + "description": "Don't print a summary", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "no-verify", + "description": "Skip verifying integrity of newly downloaded packages", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "ignore-scripts", + "description": "Skip lifecycle scripts in the project's package.json (dependency scripts are never run)", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "trust", + "description": "Add to trustedDependencies in the project's package.json and install the package(s)", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "global", + "shortName": "g", + "description": "Install globally", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "cwd", + "description": "Set a specific cwd", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "backend", + "description": "Platform-specific optimizations for installing dependencies. Possible values: \"clonefile\" (default), \"hardlink\", \"symlink\", \"copyfile\"", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "registry", + "description": "Use a specific registry by default, overriding .npmrc, bunfig.toml and environment variables", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "concurrent-scripts", + "description": "Maximum number of concurrent jobs for lifecycle scripts (default: 2x CPU cores)", + "hasValue": true, + "valueType": "val", + "defaultValue": "2x", + "required": false, + "multiple": false + }, + { + "name": "network-concurrency", + "description": "Maximum number of concurrent network requests (default 48)", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "save-text-lockfile", + "description": "Save a text-based lockfile", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "omit", + "description": "Exclude 'dev', 'optional', or 'peer' dependencies from install", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "lockfile-only", + "description": "Generate a lockfile without installing dependencies", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "linker", + "description": "Linker strategy (one of \"isolated\" or \"hoisted\")", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "minimum-release-age", + "description": "Only install packages published at least N seconds ago (security feature)", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "cpu", + "description": "Override CPU architecture for optional dependencies (e.g., x64, arm64, * for all)", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "os", + "description": "Override operating system for optional dependencies (e.g., linux, darwin, * for all)", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "help", + "shortName": "h", + "description": "Print this help menu", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "check", + "description": "Exit with code 1 if the lockfile has duplicate versions that can be removed, without changing anything", + "hasValue": false, + "required": false, + "multiple": false + } + ], + "positionalArgs": [ + { + "name": "flags", + "required": false, + "multiple": false, + "type": "string" + } + ], + "examples": ["bun dedupe", "bun dedupe --check", "bun dedupe --lockfile-only"], + "usage": "Usage: bun dedupe [flags]", + "documentationUrl": "https://bun.com/docs/pm/cli/dedupe.", + "dynamicCompletions": {} + }, + "prune": { + "name": "prune", + "description": "Remove packages from node_modules that are not in bun.lock. With --production, also remove packages that are only needed by devDependencies.", + "flags": [ + { + "name": "production", + "shortName": "p", + "description": "Also remove packages that are only needed by devDependencies (alias: --prod)", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "omit", + "description": "Also remove packages that are only needed by the given dependency types", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "dry-run", + "description": "Print what would be removed without deleting anything", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "os", + "description": "Prune for a different operating system than the current one", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "cpu", + "description": "Prune for a different CPU architecture than the current one", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "linker", + "description": "Prune a node_modules installed with the given linker (one of \"isolated\" or \"hoisted\")", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "silent", + "description": "Don't log anything", + "hasValue": false, + "required": false, + "multiple": false + }, + { + "name": "cwd", + "description": "Set a specific cwd", + "hasValue": true, + "valueType": "val", + "required": false, + "multiple": false + }, + { + "name": "help", + "shortName": "h", + "description": "Print this help menu", + "hasValue": false, + "required": false, + "multiple": false + } + ], + "positionalArgs": [ + { + "name": "flags", + "required": false, + "multiple": false, + "type": "string" + } + ], + "examples": ["bun prune", "bun prune --production", "bun prune --dry-run"], + "usage": "Usage: bun prune [flags]", + "documentationUrl": "https://bun.com/docs/pm/cli/prune.", "dynamicCompletions": {} }, "outdated": { @@ -2858,6 +3296,12 @@ "flags": [], "positionalArgs": [] }, + "licenses": { + "name": "licenses", + "description": "list installed packages grouped by license", + "flags": [], + "positionalArgs": [] + }, "whoami": { "name": "whoami", "description": "print the current npm username", diff --git a/completions/bun.bash b/completions/bun.bash index 7040f0ca51b2..3454292afc26 100644 --- a/completions/bun.bash +++ b/completions/bun.bash @@ -89,20 +89,25 @@ _bun_completions() { declare -A PACKAGE_OPTIONS; declare -A PM_OPTIONS; - local SUBCOMMANDS="dev bun create run install add remove upgrade completions discord help init pm x test repl update outdated link unlink build"; + local SUBCOMMANDS="dev bun create run install add remove upgrade completions discord help init pm x test repl update audit dedupe prune outdated link unlink build"; GLOBAL_OPTIONS[LONG_OPTIONS]="--use --cwd --bunfile --server-bunfile --config --disable-react-fast-refresh --disable-hmr --env-file --extension-order --jsx-factory --jsx-fragment --extension-order --jsx-factory --jsx-fragment --jsx-import-source --jsx-production --jsx-runtime --main-fields --no-summary --version --platform --public-dir --tsconfig-override --define --external --help --inject --loader --origin --port --dump-environment-variables --dump-limits --disable-bun-js"; GLOBAL_OPTIONS[SHORT_OPTIONS]="-c -v -d -e -h -i -l -u -p"; - PACKAGE_OPTIONS[ADD_OPTIONS_LONG]="--development --optional --peer"; - PACKAGE_OPTIONS[ADD_OPTIONS_SHORT]="-d"; - PACKAGE_OPTIONS[REMOVE_OPTIONS_LONG]=""; - PACKAGE_OPTIONS[REMOVE_OPTIONS_SHORT]=""; + PACKAGE_OPTIONS[ADD_OPTIONS_LONG]="--development --optional --peer --catalog --filter"; + PACKAGE_OPTIONS[ADD_OPTIONS_SHORT]="-d -F"; + PACKAGE_OPTIONS[REMOVE_OPTIONS_LONG]="--filter"; + PACKAGE_OPTIONS[REMOVE_OPTIONS_SHORT]="-F"; PACKAGE_OPTIONS[SHARED_OPTIONS_LONG]="--config --yarn --production --frozen-lockfile --no-save --dry-run --force --cache-dir --no-cache --silent --verbose --global --cwd --backend --link-native-bins --help"; PACKAGE_OPTIONS[SHARED_OPTIONS_SHORT]="-c -y -p -f -g"; - PM_OPTIONS[LONG_OPTIONS]="--config --yarn --production --frozen-lockfile --no-save --dry-run --force --cache-dir --no-cache --silent --verbose --no-progress --no-summary --no-verify --ignore-scripts --global --cwd --backend --link-native-bins --help" + PACKAGE_OPTIONS[DEDUPE_OPTIONS_LONG]="--check"; + PACKAGE_OPTIONS[PRUNE_OPTIONS_LONG]="--production --prod --omit --dry-run --os --cpu --linker --silent --cwd --help"; + PACKAGE_OPTIONS[PRUNE_OPTIONS_SHORT]="-p -P -h"; + PACKAGE_OPTIONS[AUDIT_OPTIONS_LONG]="--json --audit-level --ignore --prod --production --omit --dry-run --cwd --help"; + + PM_OPTIONS[LONG_OPTIONS]="--config --yarn --production --frozen-lockfile --no-save --dry-run --force --cache-dir --no-cache --silent --verbose --no-progress --no-summary --no-verify --ignore-scripts --global --cwd --backend --link-native-bins --json --help" PM_OPTIONS[SHORT_OPTIONS]="-c -y -p -f -g" local cur_word="${COMP_WORDS[${COMP_CWORD}]}"; @@ -115,11 +120,17 @@ _bun_completions() { --server-bunfile) _file_arguments "!*.server.bun" && return;; --backend) case "${COMP_WORDS[1]}" in - a|add|remove|rm|install|i) + a|add|remove|rm|install|i|dedupe) COMPREPLY=( $(compgen -W "clonefile copyfile hardlink clonefile_each_dir symlink" -- "${cur_word}") ); ;; esac return ;; + --omit) + COMPREPLY=( $(compgen -W "dev optional peer" -- "${cur_word}") ); + return;; + --linker) + COMPREPLY=( $(compgen -W "isolated hoisted" -- "${cur_word}") ); + return;; --cwd|--public-dir) COMPREPLY=( $(compgen -d -- "${cur_word}" )); return;; @@ -144,11 +155,29 @@ _bun_completions() { "${PACKAGE_OPTIONS[ADD_OPTIONS_LONG]} ${PACKAGE_OPTIONS[ADD_OPTIONS_SHORT]} ${PACKAGE_OPTIONS[SHARED_OPTIONS_LONG]} ${PACKAGE_OPTIONS[SHARED_OPTIONS_SHORT]}" \ "${PACKAGE_OPTIONS[ADD_OPTIONS_SHORT]} ${PACKAGE_OPTIONS[SHARED_OPTIONS_SHORT]}" return;; - remove|rm|i|install|link|unlink) + remove|rm|i|install) _long_short_completion \ "${PACKAGE_OPTIONS[REMOVE_OPTIONS_LONG]} ${PACKAGE_OPTIONS[REMOVE_OPTIONS_SHORT]} ${PACKAGE_OPTIONS[SHARED_OPTIONS_LONG]} ${PACKAGE_OPTIONS[SHARED_OPTIONS_SHORT]}" \ "${PACKAGE_OPTIONS[REMOVE_OPTIONS_SHORT]} ${PACKAGE_OPTIONS[SHARED_OPTIONS_SHORT]}"; return;; + link|unlink) + _long_short_completion \ + "${PACKAGE_OPTIONS[SHARED_OPTIONS_LONG]} ${PACKAGE_OPTIONS[SHARED_OPTIONS_SHORT]}" \ + "${PACKAGE_OPTIONS[SHARED_OPTIONS_SHORT]}"; + return;; + dedupe) + _long_short_completion \ + "${PACKAGE_OPTIONS[DEDUPE_OPTIONS_LONG]} ${PACKAGE_OPTIONS[SHARED_OPTIONS_LONG]} ${PACKAGE_OPTIONS[SHARED_OPTIONS_SHORT]}" \ + "${PACKAGE_OPTIONS[SHARED_OPTIONS_SHORT]}"; + return;; + prune) + _long_short_completion \ + "${PACKAGE_OPTIONS[PRUNE_OPTIONS_LONG]} ${PACKAGE_OPTIONS[PRUNE_OPTIONS_SHORT]}" \ + "${PACKAGE_OPTIONS[PRUNE_OPTIONS_SHORT]}"; + return;; + audit) + COMPREPLY=( $(compgen -W "fix ${PACKAGE_OPTIONS[AUDIT_OPTIONS_LONG]}" -- "${cur_word}") ); + return;; create|c) COMPREPLY=( $(compgen -W "--force --no-install --help --no-git --verbose --no-package-json --open next react" -- "${cur_word}") ); return;; @@ -166,7 +195,7 @@ _bun_completions() { pm) _long_short_completion \ "${PM_OPTIONS[LONG_OPTIONS]} ${PM_OPTIONS[SHORT_OPTIONS]}"; - COMPREPLY+=( $(compgen -W "bin ls cache hash hash-print hash-string" -- "${cur_word}") ); + COMPREPLY+=( $(compgen -W "bin ls licenses cache hash hash-print hash-string" -- "${cur_word}") ); return;; *) local replaced_script; diff --git a/completions/bun.fish b/completions/bun.fish index 6b2dc5bc9bcd..1b12cdcf794b 100644 --- a/completions/bun.fish +++ b/completions/bun.fish @@ -35,7 +35,7 @@ end set -l bun_install_boolean_flags yarn production optional development no-save dry-run force no-cache silent verbose global set -l bun_install_boolean_flags_descriptions "Write a yarn.lock file (yarn v1)" "Don't install devDependencies" "Add dependency to optionalDependencies" "Add dependency to devDependencies" "Don't update package.json or save a lockfile" "Don't install anything" "Always request the latest versions from the registry & reinstall all dependencies" "Ignore manifest cache entirely" "Don't output anything" "Excessively verbose logging" "Use global folder" -set -l bun_builtin_cmds_without_run dev create help bun upgrade discord install remove add update init pm x repl +set -l bun_builtin_cmds_without_run dev create help bun upgrade discord install remove add update audit dedupe prune init pm x repl set -l bun_builtin_cmds_accepting_flags create help bun upgrade discord run init link unlink pm x update function __bun_complete_bins_scripts --inherit-variable bun_builtin_cmds_without_run -d "Emit bun completions for bins and scripts" @@ -148,14 +148,23 @@ complete -c bun \ for i in (seq (count $bun_install_boolean_flags)) complete -c bun \ - -n "__fish_seen_subcommand_from install add remove update" -l "$bun_install_boolean_flags[$i]" -d "$bun_install_boolean_flags_descriptions[$i]" + -n "__fish_seen_subcommand_from install add remove update dedupe" -l "$bun_install_boolean_flags[$i]" -d "$bun_install_boolean_flags_descriptions[$i]" end complete -c bun \ - -n "__fish_seen_subcommand_from install add remove update" -l 'cwd' -d 'Change working directory' + -n "__fish_seen_subcommand_from install add remove update dedupe" -l 'cwd' -d 'Change working directory' complete -c bun \ - -n "__fish_seen_subcommand_from install add remove update" -l 'cache-dir' -d 'Choose a cache directory (default: $HOME/.bun/install/cache)' + -n "__fish_seen_subcommand_from install add remove update dedupe" -l 'cache-dir' -d 'Choose a cache directory (default: $HOME/.bun/install/cache)' + +complete -c bun \ + -n "__fish_seen_subcommand_from install add remove" -s 'F' -l 'filter' -r -d 'Apply to the matching workspaces instead of the current package' + +complete -c bun \ + -n "__fish_seen_subcommand_from install add" -l 'catalog' -d 'Add the resolved version to the root package.json catalog and depend on it as "catalog:" (--catalog=NAME for a named catalog)' + +complete -c bun \ + -n "__fish_seen_subcommand_from dedupe" -l 'check' -d 'Exit with code 1 if the lockfile has duplicate versions that can be removed, without changing anything' complete -c bun \ -n "__fish_seen_subcommand_from add" -d 'Popular' -a '(__fish__get_bun_packages)' @@ -164,11 +173,20 @@ complete -c bun \ -n "__fish_seen_subcommand_from add" -d 'History' -a '(__history_completions)' complete -c bun \ - -n "__fish_seen_subcommand_from pm; and not __fish_seen_subcommand_from (__fish__get_bun_bins) (__fish__get_bun_scripts) cache;" -a 'bin ls cache hash hash-print hash-string' -f + -n "__fish_seen_subcommand_from pm; and not __fish_seen_subcommand_from (__fish__get_bun_bins) (__fish__get_bun_scripts) cache;" -a 'bin ls licenses cache hash hash-print hash-string' -f complete -c bun \ -n "__fish_seen_subcommand_from pm; and __fish_seen_subcommand_from cache; and not __fish_seen_subcommand_from (__fish__get_bun_bins) (__fish__get_bun_scripts);" -a 'rm' -f +complete -c bun \ + -n "__fish_seen_subcommand_from pm; and __fish_seen_subcommand_from licenses" -l 'json' -d 'Output as JSON' -f + +complete -c bun \ + -n "__fish_seen_subcommand_from pm; and __fish_seen_subcommand_from licenses" -l 'prod' -d 'Omit devDependencies' -f + +complete -c bun \ + -n "__fish_seen_subcommand_from pm; and __fish_seen_subcommand_from licenses" -l 'production' -d 'Omit devDependencies' -f + # Add built-in subcommands with descriptions. complete -c bun -n "__fish_use_subcommand" -a "create" -f -d "Create a new project from a template" complete -c bun -n "__fish_use_subcommand" -a "build bun" --require-parameter -F -d "Transpile and bundle one or more files" @@ -183,6 +201,23 @@ complete -c bun -n "__fish_use_subcommand" -a "unlink" -d "Unregister a local np complete -c bun -n "__fish_use_subcommand" -a "pm" -d "Additional package management utilities" -f complete -c bun -n "__fish_use_subcommand" -a "x" -d "Execute a package binary, installing if needed" -f complete -c bun -n "__fish_use_subcommand" -a "outdated" -d "Display the latest versions of outdated dependencies" -f +complete -c bun -n "__fish_use_subcommand" -a "audit" -d "Check installed packages for vulnerabilities" -f +complete -c bun -n "__fish_use_subcommand" -a "dedupe" -d "Remove duplicate versions from the lockfile" -f +complete -c bun -n "__fish_use_subcommand" -a "prune" -d "Remove packages that are not in the lockfile from node_modules" -f +complete -c bun -n "__fish_seen_subcommand_from audit; and not __fish_seen_subcommand_from fix" -a "fix" -d "Upgrade vulnerable packages to the lowest safe version" -f +complete -c bun -n "__fish_seen_subcommand_from audit" -l "json" -d "Output in JSON format" -f +complete -c bun -n "__fish_seen_subcommand_from audit" -l "audit-level" -r -a "low moderate high critical" -d "Only print advisories at or above this severity" -f +complete -c bun -n "__fish_seen_subcommand_from audit" -l "ignore" -r -d "Ignore advisories by GHSA or numeric advisory ID" -f +complete -c bun -n "__fish_seen_subcommand_from audit" -l "prod" -d "Omit devDependencies" -f +complete -c bun -n "__fish_seen_subcommand_from audit prune" -l "omit" -r -a "dev optional peer" -d "Omit the given dependency type" -f +complete -c bun -n "__fish_seen_subcommand_from audit prune" -l "dry-run" -d "Print what would change without changing anything" -f +complete -c bun -n "__fish_seen_subcommand_from prune" -s "p" -l "production" -d "Also remove packages that are only needed by devDependencies" -f +complete -c bun -n "__fish_seen_subcommand_from prune" -s "P" -l "prod" -d "Also remove packages that are only needed by devDependencies" -f +complete -c bun -n "__fish_seen_subcommand_from prune" -l "os" -r -d "Prune for a different operating system than the current one" -f +complete -c bun -n "__fish_seen_subcommand_from prune" -l "cpu" -r -d "Prune for a different CPU architecture than the current one" -f +complete -c bun -n "__fish_seen_subcommand_from prune" -l "linker" -r -a "isolated hoisted" -d "Prune a node_modules installed with the given linker" -f +complete -c bun -n "__fish_seen_subcommand_from prune" -l "silent" -d "Don't log anything" -f +complete -c bun -n "__fish_seen_subcommand_from audit prune" -l "cwd" -r -d "Set a specific cwd" complete -c bun -n "__fish_use_subcommand" -a "update" -d "Update dependencies to their latest versions" -f complete -c bun -n "__fish_use_subcommand" -a "publish" -d "Publish your package from local to npm" -f complete -c bun -n "__fish_use_subcommand" -a "repl" -d "Start a REPL session with Bun" -f diff --git a/completions/bun.zsh b/completions/bun.zsh index 9ab15deedf66..26adc17a72b1 100644 --- a/completions/bun.zsh +++ b/completions/bun.zsh @@ -36,7 +36,10 @@ _bun_add_completion() { '--development[]' \ '--optional[Add dependency to "optionalDependencies]' \ '--peer[Add dependency to "peerDependencies]' \ - '--exact[Add the exact version instead of the ^range]' && + '--exact[Add the exact version instead of the ^range]' \ + '--catalog=-[Add the resolved version to the root package.json catalog and depend on it as "catalog:"]:catalog' \ + '*--filter[Add the package(s) to the matching workspaces instead of the current package]:workspace pattern' \ + '*-F[Add the package(s) to the matching workspaces instead of the current package]:workspace pattern' && ret=0 case $state in @@ -257,6 +260,7 @@ _bun_pm_completion() { sub_commands=( 'bin\:"print the path to bin folder" ' 'ls\:"list the dependency tree according to the current lockfile" ' + 'licenses\:"list installed packages grouped by license" ' 'hash\:"generate & print the hash of the current lockfile" ' 'hash-string\:"print the string used to hash the lockfile" ' 'hash-print\:"print the hash stored in the current lockfile" ' @@ -295,6 +299,20 @@ _bun_pm_completion() { "--trusted[list only trusted dependencies]" ) + _arguments -s -C \ + '1: :->cmd' \ + '2: :->cmd2' \ + $pmargs && + ret=0 + + ;; + licenses) + pmargs=( + "--json[output as JSON]" + "--prod[omit devDependencies]" + "--production[omit devDependencies]" + ) + _arguments -s -C \ '1: :->cmd' \ '2: :->cmd2' \ @@ -377,7 +395,10 @@ _bun_install_completion() { '-D[]' \ '--optional[Add dependency to "optionalDependencies]' \ '--peer[Add dependency to "peerDependencies]' \ - '--exact[Add the exact version instead of the ^range]' && + '--exact[Add the exact version instead of the ^range]' \ + '--catalog=-[Add the resolved version to the root package.json catalog and depend on it as "catalog:"]:catalog' \ + '*--filter[Install packages for the matching workspaces]:workspace pattern' \ + '*-F[Install packages for the matching workspaces]:workspace pattern' && ret=0 case $state in @@ -417,7 +438,9 @@ _bun_remove_completion() { '--cwd[Set a specific cwd]:cwd' \ '--backend[Platform-specific optimizations for installing dependencies]:backend:("copyfile" "hardlink" "symlink")' \ '--link-native-bins[Link "bin" from a matching platform-specific dependency instead. Default: esbuild, turbo]:link-native-bins' \ - '--help[Print this help menu]' && + '--help[Print this help menu]' \ + '*--filter[Remove the package(s) from the matching workspaces instead of the current package]:workspace pattern' \ + '*-F[Remove the package(s) from the matching workspaces instead of the current package]:workspace pattern' && ret=0 case $state in @@ -645,6 +668,87 @@ _bun_outdated_completion() { esac } +_bun_dedupe_completion() { + _arguments -s -C \ + '1: :->cmd1' \ + '--check[Exit with code 1 if the lockfile has duplicate versions that can be removed, without changing anything]' \ + '-c[Load config(bunfig.toml)]: :->config' \ + '--config[Load config(bunfig.toml)]: :->config' \ + '-y[Write a yarn.lock file (yarn v1)]' \ + '--yarn[Write a yarn.lock file (yarn v1)]' \ + '-p[Don'"'"'t install devDependencies]' \ + '--production[Don'"'"'t install devDependencies]' \ + '--no-save[Don'"'"'t save a lockfile]' \ + '--save[Save to package.json]' \ + '--dry-run[Don'"'"'t install anything]' \ + '--frozen-lockfile[Disallow changes to lockfile]' \ + '--lockfile-only[Generate a lockfile without installing dependencies]' \ + '-f[Always request the latest versions from the registry & reinstall all dependencies]' \ + '--force[Always request the latest versions from the registry & reinstall all dependencies]' \ + '--cache-dir[Store & load cached data from a specific directory path]:cache-dir' \ + '--no-cache[Ignore manifest cache entirely]' \ + '--silent[Don'"'"'t log anything]' \ + '--verbose[Excessively verbose logging]' \ + '--no-progress[Disable the progress bar]' \ + '--no-summary[Don'"'"'t print a summary]' \ + '--no-verify[Skip verifying integrity of newly downloaded packages]' \ + '--ignore-scripts[Skip lifecycle scripts in the package.json (dependency scripts are never run)]' \ + '--cwd[Set a specific cwd]:cwd' \ + '--backend[Platform-specific optimizations for installing dependencies]:backend:("copyfile" "hardlink" "symlink")' \ + '--linker[Linker strategy]:linker:(isolated hoisted)' \ + '--help[Print this help menu]' && + ret=0 + + case $state in + config) + _bun_list_bunfig_toml + + ;; + esac +} + +_bun_prune_completion() { + _arguments -s -C \ + '1: :->cmd1' \ + '-p[Also remove packages that are only needed by devDependencies]' \ + '--production[Also remove packages that are only needed by devDependencies]' \ + '-P[Also remove packages that are only needed by devDependencies]' \ + '--prod[Also remove packages that are only needed by devDependencies]' \ + '*--omit[Also remove packages that are only needed by the given dependency types]:type:(dev optional peer)' \ + '--dry-run[Print what would be removed without deleting anything]' \ + '*--os[Prune for a different operating system than the current one]:os' \ + '*--cpu[Prune for a different CPU architecture than the current one]:cpu' \ + '--linker[Prune a node_modules installed with the given linker]:linker:(isolated hoisted)' \ + '--silent[Don'"'"'t log anything]' \ + '--cwd[Set a specific cwd]:cwd' \ + '-h[Print this help menu]' \ + '--help[Print this help menu]' && + ret=0 +} + +_bun_audit_completion() { + _arguments -s -C \ + '1: :->cmd1' \ + '2: :->subcommand' \ + '--json[Output in JSON format]' \ + '--audit-level[Only print advisories with severity greater than or equal to the given level]:level:(low moderate high critical)' \ + '*--ignore[Ignore advisories by GHSA or numeric advisory ID]:advisory' \ + '--prod[Omit devDependencies]' \ + '--production[Omit devDependencies]' \ + '*--omit[Exclude dependency types from the audit]:type:(dev optional peer)' \ + '--dry-run[Show what bun audit fix would change without changing anything]' \ + '--cwd[Set a specific cwd]:cwd' \ + '--help[Print this help menu]' && + ret=0 + + case $state in + subcommand) + _alternative 'args:subcommand:((fix\:"Upgrade vulnerable packages to the lowest safe version"))' + + ;; + esac +} + _bun_test_completion() { _arguments -s -C \ '1: :->cmd1' \ @@ -753,6 +857,9 @@ _bun() { 'add\:"Add a dependency to package.json (bun a)" ' 'remove\:"Remove a dependency from package.json (bun rm)" ' 'update\:"Update outdated dependencies & save to package.json" ' + 'audit\:"Check installed packages for vulnerabilities" ' + 'dedupe\:"Remove duplicate versions from the lockfile" ' + 'prune\:"Remove packages that are not in the lockfile from node_modules" ' 'outdated\:"Display the latest versions of outdated dependencies" ' 'link\:"Link an npm package globally" ' 'unlink\:"Globally unlink an npm package" ' @@ -833,6 +940,18 @@ _bun() { outdated) _bun_outdated_completion + ;; + audit) + _bun_audit_completion + + ;; + dedupe) + _bun_dedupe_completion + + ;; + prune) + _bun_prune_completion + ;; 'test') _bun_test_completion @@ -920,6 +1039,18 @@ _bun() { outdated) _bun_outdated_completion + ;; + audit) + _bun_audit_completion + + ;; + dedupe) + _bun_dedupe_completion + + ;; + prune) + _bun_prune_completion + ;; 'test') _bun_test_completion diff --git a/docs/pm/catalogs.mdx b/docs/pm/catalogs.mdx index 8599d652b81c..bdeba48201f4 100644 --- a/docs/pm/catalogs.mdx +++ b/docs/pm/catalogs.mdx @@ -91,7 +91,7 @@ In your workspace packages, use the `catalog:` protocol to reference versions: } ``` -`catalog:` also works in `peerDependencies`. A catalog peer is resolved and hoisted exactly as if the catalog's range were written inline: if an ancestor already provides a version that satisfies the catalog range, the workspace package reuses that copy instead of getting its own. +`catalog:` and `catalog:` are accepted in `dependencies`, `devDependencies`, `optionalDependencies` and `peerDependencies`, and as the value of a root [`overrides` / `resolutions`](/pm/overrides) rule. A catalog peer is resolved and hoisted exactly as if the catalog's range were written inline: if an ancestor already provides a version that satisfies the catalog range, the workspace package reuses that copy instead of getting its own. ### 3. Run Bun Install @@ -238,13 +238,15 @@ To update versions across all packages, change the version in the root package.j Then run `bun install` to update all packages. -To add a new dependency to the catalog (or refresh an existing entry), run `bun add` with `--catalog` (or `--catalog=` for a named catalog) inside the workspace package: +To add a new dependency to the catalog (or refresh an existing entry), run `bun add` with `--catalog` (or `--catalog=` for a named catalog) from the workspace package or from the root: ```bash terminal icon="terminal" bun add react --catalog ``` -Bun writes the resolved version to the catalog in the root `package.json` and `"catalog:"` to the package's `package.json`. See [`bun add --catalog`](/pm/cli/add#--catalog). +Bun writes the resolved version to the catalog in the root `package.json` and `"catalog:"` to the `package.json` you ran the command in. See [`bun add --catalog`](/pm/cli/add#--catalog). + +The catalog is only consulted when `--catalog` is passed: `bun add react` writes a concrete range to the current `package.json` even if `react` is already in the catalog (Bun has no equivalent of pnpm's `catalogMode`). ## Lockfile Integration diff --git a/docs/pm/cli/add.mdx b/docs/pm/cli/add.mdx index 7b083903a6b6..f66afdfb45b2 100644 --- a/docs/pm/cli/add.mdx +++ b/docs/pm/cli/add.mdx @@ -19,6 +19,8 @@ bun add zod@^3.0.0 bun add zod@latest ``` +The package is written to `dependencies` unless `--dev`, `--optional` or `--peer` is given; if the current `package.json` already lists it in another group, that entry is updated in place instead. + ## `--dev` **Alias** — `--development`, `-d`, `-D` @@ -46,6 +48,8 @@ To add a package as a peer dependency (`"peerDependencies"`): bun add --peer @types/bun ``` +Only `peerDependencies` is written; because Bun installs peer dependencies by default, no `devDependencies` entry is needed for the package to be installed locally (pnpm's `--save-peer` adds one). + ## `--exact` **Alias** — `-E` @@ -113,6 +117,7 @@ A few things to know, some of which differ from pnpm's `--save-catalog`: - `--catalog` and `--catalog=default` are the same catalog: the entry is written to whichever of `catalog` or `catalogs.default` the root `package.json` already defines (`catalog` is created when there is neither), so a repository migrated from pnpm never ends up with the package in both. - The recorded version is the one that was installed, so an entry in [`overrides`](/pm/overrides) for the package ends up in the catalog too. - The name must be attached with `=` (`--catalog=testing`); `--catalog testing` adds a package called `testing`. +- pnpm's `--save-catalog` is spelled `--catalog` and `--save-catalog-name testing` is `--catalog=testing`; Bun silently skips flags it does not know, so the pnpm spellings add the package as an ordinary dependency (and `--save-catalog-name testing` also tries to add a package called `testing`) instead of failing. - Packages must be added by name (`react`, `react@^18`, `alias@npm:react`). Bare URLs, paths and `workspace:` versions are rejected. - The root `package.json` must have a `workspaces` field. @@ -137,6 +142,7 @@ A few things to know, some of which differ from pnpm: - Path patterns must match a workspace directory exactly, relative to the current directory: `--filter ./packages` selects nothing, `--filter ./packages/*` selects every package in it, and `--filter .` only works from a workspace's own directory. - Local paths (`./vendor/logger`, `file:../logger`, `./logger.tgz`) are relative to the current directory and are re-spelled relative to each selected package (`"logger": "../../vendor/logger"` in `packages/api/package.json`). - The flag chooses which `package.json` files are edited, not what is installed: a single install of the whole workspace runs afterwards. The `package.json` files are written as soon as the lockfile is saved, so they agree with `bun.lock` even if a root `postinstall` script then fails. +- The install summary is printed from the point of view of one selected workspace (the first one that received every requested package); the other selected `package.json` files get the same edits even though they are not listed in the output. - `--dry-run` skips writing; `--filter` cannot be combined with `--global`. ## `--global` diff --git a/docs/pm/cli/audit.mdx b/docs/pm/cli/audit.mdx index e8e4c031bd86..f9de0126f42d 100644 --- a/docs/pm/cli/audit.mdx +++ b/docs/pm/cli/audit.mdx @@ -11,6 +11,8 @@ bun audit Bun sends the list of installed packages and versions to npm, and prints a report of any vulnerabilities found. Packages installed from registries other than the default registry are skipped. +The package list comes from `bun.lock` alone, so `node_modules` does not need to exist, and `bun audit` writes nothing: `package.json`, `bun.lock` and `node_modules` are left as they are (only `bun audit fix`, below, changes `bun.lock` and `node_modules`). + If no vulnerabilities are found, the command prints: ``` @@ -29,13 +31,13 @@ To update all dependencies to the latest versions (including breaking changes): ### Filtering options -**`--audit-level=`** - Only show vulnerabilities at this severity level or higher: +**`--audit-level=`** - Only show vulnerabilities at this severity level or higher. Without the flag nothing is filtered out; an advisory whose severity is not one of these four values is counted and filtered as `moderate`: ```bash terminal icon="terminal" bun audit --audit-level=high ``` -**`--prod`** - Audit only production dependencies. A package is production when it is reachable from the root through `dependencies`, `optionalDependencies` or `peerDependencies` edges; packages that are only reached through `devDependencies` or an optional peer dependency are excluded, even when another version of the same package is a production dependency: +**`--prod`** (also `-P`, `-p` or `--production`) - Audit only production dependencies. A package is production when it is reachable from the root through `dependencies`, `optionalDependencies` or `peerDependencies` edges; packages that are only reached through `devDependencies` or an optional peer dependency are excluded, even when another version of the same package is a production dependency. There is no flag for auditing only `devDependencies`: ```bash terminal icon="terminal" bun audit --prod @@ -47,6 +49,8 @@ bun audit --prod bun audit --ignore GHSA-c2qf-rxjj-qqgw --ignore 1112918 ``` +These filters cannot be set in `bunfig.toml`; to ignore an advisory or raise the level for every run, put the flags in a `package.json` script. + ### `--json` Use the `--json` flag to print the raw JSON response from the registry instead of the formatted report: @@ -55,9 +59,11 @@ Use the `--json` flag to print the raw JSON response from the registry instead o bun audit --json ``` +The response is printed as received: `--prod` still limits which packages are sent, and `--audit-level` and `--ignore` do not remove anything from the printed JSON. They do decide the exit code, which is `1` only if an advisory is left after both filters are applied (and `0` otherwise), so `bun audit --json --audit-level=high` prints every advisory but only fails the run for high or critical ones. If the response cannot be parsed, it is still printed and the command exits with `1`. + ### `bun audit fix` -`bun audit fix` runs the audit and then upgrades each vulnerable package in `bun.lock` to the lowest version that is not affected by any advisory and still satisfies every range that depends on it (including `overrides` and catalog entries), then installs. `package.json` is never modified and unrelated packages are not touched. +`bun audit fix` runs the audit and then upgrades each vulnerable package in `bun.lock` to the lowest version that is not affected by any advisory and still satisfies every range that depends on it (including `overrides` and catalog entries), then installs. `package.json` is never modified (Bun does not add `overrides` on your behalf; see `requires a semver-major update:` below) and unrelated packages are not touched. `fix` is a subcommand, not a flag: `bun audit --fix` is rejected as an unknown flag. ```bash terminal icon="terminal" bun audit fix @@ -86,4 +92,6 @@ Fixed 1 vulnerability in 1 package ### Exit code -`bun audit` exits with code `0` if no vulnerabilities are found and `1` if the report lists any, including when `--json` is passed. `bun audit fix` exits with `0` when nothing vulnerable remains after the fix (or, with `--dry-run`, would remain) and `1` otherwise. +`bun audit` exits with code `0` if no vulnerabilities are found and `1` if any are left after `--audit-level` and `--ignore` are applied; `--json` uses the same rule even though it prints the unfiltered response. `bun audit fix` exits with `0` when nothing vulnerable remains after the fix (or, with `--dry-run`, would remain) and `1` otherwise. + +When the registry cannot be reached, or answers with a 4xx or 5xx status, both commands print `audit request failed` to stderr and also exit with `1`; there is no flag to turn a registry failure into a passing run, so a CI step that wants to distinguish an outage from a vulnerable tree has to look at stderr. diff --git a/docs/pm/cli/dedupe.mdx b/docs/pm/cli/dedupe.mdx index 732d7d53b9be..cbb7a88fd94c 100644 --- a/docs/pm/cli/dedupe.mdx +++ b/docs/pm/cli/dedupe.mdx @@ -23,7 +23,7 @@ Already deduplicated. ### `--check` -`bun dedupe --check` prints the versions that would be removed and exits with code `1` without writing anything. It exits with code `0` when the lockfile is already deduplicated, which makes it usable in CI. `--dry-run` is an alias. +`bun dedupe --check` prints the versions that would be removed and exits with code `1` without installing packages or modifying `bun.lock`. It exits with code `0` when the lockfile is already deduplicated, which makes it usable in CI. `--dry-run` is an alias. ```bash terminal icon="terminal" bun dedupe --check @@ -40,6 +40,7 @@ bun dedupe --check - Dependencies pointing at a version listed in `patchedDependencies` are never moved, bundled dependencies stay on the copy inside their tarball, and dependencies specified as a dist-tag (such as `latest`), a git URL, or a tarball keep the version they resolved to. - Only the ranges of packages that remain installed are counted; a version that the same run removes does not influence where its own dependencies end up, so running the command twice never changes anything the second time. - It works from the ranges recorded in `bun.lock`; changes made to `package.json` since the last install are applied by the install that follows. +- The command needs an existing `bun.lock` to work from: when there is none, `bun dedupe` (and `bun dedupe --check`) reports `missing lockfile, nothing to dedupe` and exits with code `1`, so run `bun install` first. - `bun.lock` and `node_modules` change; `package.json` never does. The result only lives in `bun.lock`: deleting the lockfile and reinstalling can bring the duplicates back. Add an [override](/pm/overrides) to pin a version permanently. - `--lockfile-only` rewrites `bun.lock` without installing. - `--frozen-lockfile`, `--production`, and `--no-save` cannot be combined with removing duplicates; the command exits with code `1` and lists the duplicates instead. Use `--check` in CI. diff --git a/docs/pm/cli/install.mdx b/docs/pm/cli/install.mdx index 5edba98d9e30..e201ff47ea66 100644 --- a/docs/pm/cli/install.mdx +++ b/docs/pm/cli/install.mdx @@ -164,15 +164,19 @@ To install in production mode (without `devDependencies`): bun install --production ``` +`--production` (`--prod`, `-p`, `-P`) also implies `--frozen-lockfile`: it never writes `bun.lock`, and it exits with an error if `package.json` and `bun.lock` disagree. It only controls which packages Bun links; anything already in `node_modules` that it would not install (for example `devDependencies` from an earlier `bun install`) is left there — run [`bun prune --production`](/pm/cli/prune) to delete it. + For reproducible installs, use `--frozen-lockfile`. Bun installs the exact versions specified in the lockfile and does not update it. If your `package.json` disagrees with `bun.lock`, Bun exits with an error. ```bash terminal icon="terminal" bun install --frozen-lockfile ``` -`--frozen-lockfile` also works on a pruned copy of a monorepo (for example the output of `turbo prune`, or a Docker context that copies `bun.lock` with only some workspace folders): workspaces listed in `bun.lock` whose `package.json` is not on disk are skipped rather than treated as a lockfile change, and packages the pruned lockfile still lists are installed as written. This works whether the root `workspaces` field uses globs or lists each folder explicitly, and Bun prints a `note:` with the number of skipped workspaces (`--verbose` names them), so a `bun.lock` that is stale because a workspace was deleted without re-running `bun install` is still visible in CI logs. A workspace is only skipped when its `package.json` is missing — removing it from the `workspaces` globs while the folder is still present, changing any `package.json` that is on disk, or trimming `overrides` or `catalog` entries from `bun.lock` still fails the install. The skipped workspaces' exclusive dependencies are not downloaded or installed, but they remain in `bun.lock`, so `bun pm ls` and `bun audit` still report them. A package that a surviving workspace lists as an optional peer dependency is installed if the pruned `bun.lock` still contains it, even when only a skipped workspace depended on it. +Bun does not enable `--frozen-lockfile` on its own when it detects a CI environment (`CI=1` only turns off the progress bar); pass the flag or run `bun ci` if a stale `bun.lock` should fail the build. When there is no `bun.lock` at all, `bun install --frozen-lockfile` (and `bun ci`) resolves and installs from `package.json` without writing a lockfile; the error is only raised when a lockfile exists and does not match `package.json`. + +`--frozen-lockfile` also works on a pruned copy of a monorepo (for example the output of `turbo prune`, or a Docker context that copies `bun.lock` with only some workspace folders): workspaces listed in `bun.lock` whose `package.json` is not on disk are skipped rather than treated as a lockfile change, and packages the pruned lockfile still lists are installed as written. This works whether the root `workspaces` field uses globs or lists each folder explicitly, and Bun prints a `note:` with the number of skipped workspaces (`--verbose` names them), so a `bun.lock` that is stale because a workspace was deleted without re-running `bun install` is still visible in CI logs. Only workspaces that `bun.lock` knows about are skipped: a `workspaces` entry whose folder is missing and which `bun.lock` does not list (for example a typo) still fails with `Workspace not found`, exactly as it does without `--frozen-lockfile`. A workspace is only skipped when its `package.json` is missing — removing it from the `workspaces` globs while the folder is still present, changing any `package.json` that is on disk, or trimming `overrides` or `catalog` entries from `bun.lock` still fails the install. The skipped workspaces' exclusive dependencies are not downloaded or installed, but they remain in `bun.lock`, so `bun pm ls` and `bun audit` still report them. A package that a surviving workspace lists as an optional peer dependency is installed if the pruned `bun.lock` still contains it, even when only a skipped workspace depended on it. This also holds when only `trustedDependencies` (which `turbo prune` leaves out of the pruned `bun.lock`) or `patchedDependencies` differ between `package.json` and `bun.lock`; a package is only dropped from the lockfile when a dependency, `overrides` or `catalog` entry actually changed, or when `bun audit fix` upgrades it. -`--frozen-lockfile` never writes `bun.lock`, including with `--lockfile-only`; the one exception is the `bun.lockb` to `bun.lock` migration requested with `--save-text-lockfile`. To check a lockfile without installing anything, use `bun install --frozen-lockfile --dry-run`. +`--frozen-lockfile` never writes `bun.lock`, including with `--lockfile-only`. The one exception is a lockfile format migration: converting `bun.lockb` with `--save-text-lockfile`, or migrating from `package-lock.json`, `yarn.lock` or `pnpm-lock.yaml`, still writes `bun.lock`. To check a lockfile without installing anything, use `bun install --frozen-lockfile --dry-run`. See [lockfile](/pm/lockfile) for more on `bun.lock`. @@ -300,7 +304,7 @@ On `bun install`, `bun remove`, and `bun add`, Bun looks for `bunfig.toml` in: 1. `$XDG_CONFIG_HOME/.bunfig.toml` or `$HOME/.bunfig.toml` 2. `./bunfig.toml` -If both are found, the results are merged together. +If both are found, both are loaded, and keys set in the project's `bunfig.toml` override the same keys in the global file. Configuring with `bunfig.toml` is optional. These are the default values: diff --git a/docs/pm/cli/pm.mdx b/docs/pm/cli/pm.mdx index 9a8388df645a..47a3af013ea2 100644 --- a/docs/pm/cli/pm.mdx +++ b/docs/pm/cli/pm.mdx @@ -169,6 +169,10 @@ To list every installed package grouped by its license (read from the `license` ```bash terminal icon="terminal" bun pm licenses +# or +bun pm licenses list +# or +bun pm licenses ls ``` ```txt @@ -183,7 +187,9 @@ Unknown (4) └── one-dep@1.0.0 ``` -Pass `--prod` to skip `devDependencies`, and `--json` to get a machine-readable object keyed by license, where each entry has `name`, `versions` (in semver order) and, when present, the `homepage` and `author` of the newest listed version: +License groups are printed in name order with `Unknown` always last, and packages within a group are sorted by name and then version; if nothing is installed the text output is empty and `--json` prints `{}`. + +Pass `--prod` (or `-P`, `-p`, `--production`) to skip `devDependencies`, and `--json` to get a machine-readable object keyed by license, where each entry has `name`, `versions` (in semver order) and, when present, the `homepage` and `author` of the newest listed version: ```bash terminal icon="terminal" bun pm licenses --json --prod @@ -202,7 +208,13 @@ bun pm licenses --json --prod } ``` -`--prod` only drops the `devDependencies` of the root package and of workspace packages; `optionalDependencies` and everything a production dependency pulls in are still listed, and from a workspace root every workspace's production dependencies are included. Run it inside a workspace package to list only that package's dependencies, and use [`bun why`](/pm/cli/why) to see which dependency pulls in an unexpected package. +There is no `--long` flag; the text output only shows names and versions, so use `--json` when you need `author` or `homepage`. + +`--prod` drops every `devDependencies` edge — the same packages `bun install --production` leaves out, including the `devDependencies` of `file:` dependencies; `optionalDependencies`, `peerDependencies` and everything a production dependency pulls in are still listed, and from a workspace root every workspace's production dependencies are included. Run it inside a workspace package to list only that package's dependencies, and use [`bun why`](/pm/cli/why) to see which dependency pulls in an unexpected package. + +`--omit=dev` is the same as `--prod` (as is [`install.production`](/runtime/bunfig#install-production) in `bunfig.toml`), and `--omit=optional` / `--omit=peer` skip `optionalDependencies` / `peerDependencies` as well, so the listing follows the same dependency types `bun install` would with those flags. + +`bun pm licenses` only reads `bun.lock` and `node_modules`; it never writes to either of them or to `package.json`, and it prints an error and exits with code `1` when the project has no lockfile or no `node_modules` directory. Packages that are in the lockfile but not in `node_modules` (for example after `bun install --production`) are omitted, and a `warn:` line with the number of omitted packages is printed to stderr. Packages that don't apply to the current platform (`os`/`cpu`) are omitted silently. diff --git a/docs/pm/cli/prune.mdx b/docs/pm/cli/prune.mdx index f3d2f6c26392..8e9fb734a7d1 100644 --- a/docs/pm/cli/prune.mdx +++ b/docs/pm/cli/prune.mdx @@ -3,7 +3,7 @@ title: "bun prune" description: "Remove packages that are not in bun.lock from node_modules" --- -`bun prune` deletes everything in `node_modules` that the current `bun.lock` does not install there — packages left behind after switching branches, editing `bun.lock`, or installing with another package manager. It only reads `bun.lock`; it never contacts the registry and never changes `bun.lock` or `package.json`. +`bun prune` deletes everything in `node_modules` that the current `bun.lock` does not install there — packages left behind after switching branches, editing `bun.lock`, or installing with another package manager. It only reads `bun.lock`; it never contacts the registry and never changes `bun.lock` or `package.json`. It takes no package names: passing one is an error, because it always removes every package `bun.lock` does not account for. ```bash terminal icon="terminal" bun prune @@ -23,7 +23,7 @@ Nothing to prune. ### `--production` -`bun prune --production` (alias `--prod`) additionally removes every package that is only needed by `devDependencies`, leaving exactly what a fresh `bun install --production` would install. `--omit=dev`, `--omit=optional` and `--omit=peer` work the same way as they do for `bun install`. +`bun prune --production` (alias `--prod`) additionally removes every package that is only needed by `devDependencies`, leaving exactly what a fresh `bun install --production` would install. `--omit=dev`, `--omit=optional` and `--omit=peer` work the same way as they do for `bun install`; there is no `--no-optional` flag, use `--omit=optional`. This makes it possible to build with `devDependencies` installed and ship without them: @@ -50,12 +50,12 @@ Would remove 1 package ### Notes -- Works with both linkers. With the hoisted linker it checks every `node_modules` folder `bun install` would install into — including workspace folders and the nested `node_modules` folders of installed packages, so copies left behind by an earlier install are removed too; with the isolated linker it removes unused entries in `node_modules/.bun` and the symlinks that pointed at them. +- Works with both linkers. With the hoisted linker it checks every `node_modules` folder `bun install` would install into — including workspace folders and the nested `node_modules` folders of installed packages, so copies left behind by an earlier install are removed too (subject to the version check below); with the isolated linker it removes unused entries in `node_modules/.bun` and the symlinks that pointed at them. With `--production`, the symlink of a dev-only dependency is removed and listed even when its `node_modules/.bun` entry stays because a production dependency also uses it. - The linker is chosen the same way `bun install` chooses it. If `node_modules` was installed with the other linker (for example `bun install --linker hoisted` in a project that defaults to isolated), `bun prune` refuses to run instead of removing packages the other layout needs; pass the same `--linker` you installed with, or run `bun install` to switch layouts. -- Always runs against the workspace root, even when invoked inside a workspace package. +- Always runs against the workspace root, even when invoked inside a workspace package, and prunes the `node_modules` folder of every workspace package in that one run; with `--production`, a package that only a workspace's `devDependencies` need is removed too. - Workspace symlinks (even `--production` keeps a workspace linked from another workspace's `devDependencies`), `.bin` entries that are still in use, dot-entries such as `.cache`, plain files, and dependencies bundled inside a package's tarball are never removed. `.bin` entries whose package was removed are cleaned up on every platform. - Nothing outside `node_modules` is ever deleted. A package folder that you replaced with a symlink is left alone, and the `node_modules` folder inside it is not pruned. With `install.globalStore`, only the project's links into the store are removed. - Packages disabled for the current `os`/`cpu` are removed, just as `bun install` would skip them. Pass `--os` / `--cpu` to prune for another platform. -- Only package names are compared, and only `bun.lock` is consulted: a dependency removed from `package.json` is pruned after the next `bun install` updates `bun.lock`. Run `bun install` if you also want the versions on disk re-verified. +- Only `bun.lock` is consulted, and packages are matched by name: a package that is installed under the right name at the wrong version is left for `bun install` to replace, and a dependency removed from `package.json` is pruned only after the next `bun install` updates `bun.lock`. A copy nested inside another package (`node_modules/a/node_modules/b`) is only removed once the copy that replaces it higher up is installed at the version `bun.lock` expects (checked from its `package.json`, or `.bun-tag` for git dependencies); until then it is kept and `bun prune` prints a `warn:` line naming both folders — typically after `bun prune --production` when a `devDependency` pinned a different version of `b` at the root. Run `bun install` (with the same flags), then `bun prune` again. - Lifecycle scripts are never run. -- Equivalent to `pnpm prune` and `npm prune`. +- Equivalent to `pnpm prune` and `npm prune`. It does not touch the global cache; the counterpart of `pnpm store prune` is [`bun pm cache rm`](/pm/cli/pm#cache). Not related to `turbo prune`, which copies a subset of a monorepo's workspaces, `package.json` files and `bun.lock` into an output directory: `bun prune` only deletes from `node_modules`, so a Dockerfile can use `turbo prune`, then `bun install`, then `bun prune --production` after building. diff --git a/docs/pm/cli/remove.mdx b/docs/pm/cli/remove.mdx index 9df4d760cd7d..d0c68bcce22b 100644 --- a/docs/pm/cli/remove.mdx +++ b/docs/pm/cli/remove.mdx @@ -7,10 +7,25 @@ import Remove from "/snippets/cli/remove.mdx"; ## Basic Usage +**Alias** — `bun rm`, `bun uninstall`, `bun r` + ```bash terminal icon="terminal" bun remove ts-node ``` +The package is deleted from every one of `dependencies`, `devDependencies`, `optionalDependencies` and `peerDependencies` it appears in (there is no flag to limit the removal to one of them), `bun.lock` is updated, and `node_modules/` is deleted once no package in the lockfile still depends on it. + +## `--filter` + +**Alias** — `-F` + +In a monorepo, `--filter` removes the package from the matching workspace package(s) instead of the package in the current directory, using the same patterns as [`bun add --filter`](/pm/cli/add#--filter). To remove a package from every workspace, use `--filter '*'`; there is no `--recursive`/`-r` flag. Selected workspaces that don't list the package are left untouched, and if none of them list it no `package.json` is edited and the command exits 0. + +```bash terminal icon="terminal" +bun remove zod --filter api +bun remove zod --filter '*' +``` + --- diff --git a/docs/pm/cli/update.mdx b/docs/pm/cli/update.mdx index 52afe28c03aa..fd5ae21936a9 100644 --- a/docs/pm/cli/update.mdx +++ b/docs/pm/cli/update.mdx @@ -1,24 +1,35 @@ --- title: "bun update" -description: "Update dependencies to latest versions" +description: "Update dependencies to the newest versions their ranges allow" --- import Update from "/snippets/cli/update.mdx"; To upgrade your Bun CLI version, see [`bun upgrade`](/installation#upgrading). -To update all dependencies to the latest version: +`bun update` updates dependencies to the newest versions allowed by the ranges in `package.json`; [`bun update --latest`](#--latest) ignores those ranges. ```sh terminal icon="terminal" bun update ``` -To update a specific dependency to the latest version: +To update specific packages, pass their names. A `@range` suffix applies to the entry in your `package.json`: `bun update jquery@3` moves it to the newest `3.x` even if its current range does not allow that. A suffix cannot be combined with `--latest`; `bun update jquery@3 --latest` is an error. Glob patterns such as `@types/*` are not accepted here; use [`bun outdated`](/pm/cli/outdated) to list candidates. ```sh terminal icon="terminal" bun update [package] +bun update zod jquery@3 ``` +A plain `bun update` updates the whole tree, like `pnpm update` and `npm update`: the dependencies declared in `package.json` move within their declared ranges and have their `package.json` entries rewritten, and every package that other packages in `bun.lock` depend on is re-resolved to the newest version its dependent's range allows. Each dependent is resolved against its own range, so two packages declaring different ranges on the same dependency can end up on different versions. This also applies when one of the dependents is your own `package.json`: an exact pin there stays put, and a package depending on a range of the same name moves to the newest version in that range, so `bun.lock` ends up with two copies. Ranges are never widened: a package depending on `^1.0.0` never picks up `2.x`; update the package that declares the range instead. Transitive updates only change `bun.lock` and `node_modules`, never `package.json`, and are listed before the install as `name@old → new`. + +`bun update ` updates `` everywhere it occurs, at any depth, and nothing else: dependencies you did not name keep their locked versions, and so do the dependencies of `` itself as long as its new version still allows them. `` is matched by the name of the package that gets installed, so a dependency declared as `"foo": "npm:@^1"` is updated as well. If `` is declared in your `package.json`, that entry is rewritten too; if it is only a dependency of your dependencies, only `bun.lock` changes, which is how to pick up a fix in a nested package (`bun update caniuse-lite`) without adding it to your own dependencies or using `overrides`. A name that is not in `bun.lock` at all is an error, not a no-op; use `bun add` to add a dependency. Package names cannot be combined with `--recursive` or `--filter`, and `--production` is rejected because it would freeze the lockfile `bun update` needs to write. + +## What an update writes + +`bun update` rewrites the version in `package.json` but keeps the operator you declared: `^1.1.0` becomes `^1.2.0`, `~` stays `~`, and an exact pin is left alone unless you pass `--latest`. With [`install.exact`](/runtime/bunfig#install-exact) (or `--exact`) the rewritten entry is an exact version even if it was declared with `^` or `~`. A `catalog:` reference is never rewritten; a plain `bun update` updates the catalog entry in the root `package.json` instead. A dist-tag such as `"foo": "latest"` or `"foo": "next"` stays as written, whether you run `bun update` or `bun update foo`, and only `bun.lock` moves; with `--latest` it is replaced by a `^` range on the version `latest` points at. `--dry-run` prints what would change without writing anything, and `--no-save` updates `node_modules` but leaves both `package.json` and `bun.lock` untouched, so it never changes what the next `bun install` produces. + +After any `bun update` or `bun add`, `bun.lock` records the same ranges that were written to `package.json`, so a following `bun install --frozen-lockfile` passes. + ## `--interactive` Use the `--interactive` flag to choose which packages to update: @@ -28,7 +39,7 @@ bun update --interactive bun update -i ``` -The flag opens a terminal interface that lists every outdated package with its current and target versions. +The flag opens a terminal interface that lists every outdated direct dependency with its current and target versions. Once you confirm, the selection is applied like `bun update ` with the selected names: those entries are rewritten in `package.json` and updated everywhere they occur in `bun.lock`, while the packages you left unselected, and every other transitive package, keep their locked versions. Confirming with nothing selected changes nothing. ### Interactive Interface @@ -99,22 +110,30 @@ Packages are organized in sections by dependency type: Within each section, individual packages may have a suffix (` dev`, ` peer`, ` optional`). -## `--recursive` +## `--recursive` and `--filter` -Use the `--recursive` flag with `--interactive` to update dependencies across all workspaces in a monorepo: +In a monorepo, a plain `bun update` rewrites only the `package.json` of the workspace you run it in (transitive packages are shared, so they update either way). `bun update --recursive` (`-r`) also updates the direct dependencies of every workspace and rewrites each workspace's `package.json`; `bun update --filter ` limits that to the matching workspaces (globs and `!` negation are accepted). Both need an existing `bun.lock`, and neither can be combined with package names. They also work with `--interactive`, which then adds a "Workspace" column showing which workspace each dependency belongs to: ```sh terminal icon="terminal" -bun update --interactive --recursive +bun update --recursive +bun update --filter 'packages/*' bun update -i -r ``` -With `--recursive`, the interface adds a "Workspace" column showing which workspace each dependency belongs to. +## `--global` + +`bun update -g` (`--global`) updates the packages installed with `bun add -g` instead of the current project, following the same rules; `bun update -g ` and `--latest` work there too: + +```sh terminal icon="terminal" +bun update -g +bun update -g typescript +``` ## `--latest` By default, `bun update` updates each dependency to the latest version that satisfies the version range in your `package.json`. -To update to the latest version regardless of whether it satisfies that range, use the `--latest` flag: +To update the dependencies declared in `package.json` to the latest version regardless of whether it satisfies that range, use the `--latest` flag. Transitive packages still move within the ranges their dependents declare, and a dependency that is already ahead of the `latest` tag (for example a prerelease) is left where it is rather than downgraded: ```sh terminal icon="terminal" bun update --latest diff --git a/docs/pm/filter.mdx b/docs/pm/filter.mdx index 802554702fd1..4e1448fe628b 100644 --- a/docs/pm/filter.mdx +++ b/docs/pm/filter.mdx @@ -5,7 +5,7 @@ description: "Select packages by pattern in a monorepo using the --filter flag" The `--filter` (or `-F`) flag selects packages in a monorepo by pattern. Patterns match package names or package paths, with full glob syntax. -`bun install` and `bun outdated` support `--filter`, and you can use it to run scripts in multiple packages at once. +`bun install`, `bun add`, `bun remove`, `bun update` and `bun outdated` support `--filter` (pass it after the subcommand, or as `--filter=` before it); for `bun add`/`bun remove` it selects which workspace `package.json` files are edited. You can also use it to run scripts in multiple packages at once. --- @@ -19,6 +19,8 @@ Name patterns select packages by the `name` field in `package.json`. For example Path patterns start with `./` and select all packages in directories matching the pattern. For example, to match all packages in subdirectories of `packages`, use `--filter './packages/**'`. To match the package in `packages/foo`, use `--filter ./packages/foo`. +Patterns are matched against the full package name -- `--filter core` does not select `@acme/core` (use `@acme/core` or `@acme/*`), and `*` does not cross `/`. Path patterns must match a workspace's directory itself: `--filter ./packages` selects nothing, `--filter './packages/*'` selects every workspace directly inside it, and `--filter '!./lib'` only excludes a workspace located at `./lib`. + --- ## `bun install` and `bun outdated` diff --git a/docs/pm/isolated-installs.mdx b/docs/pm/isolated-installs.mdx index 41a95f2799fb..8fe212449ff2 100644 --- a/docs/pm/isolated-installs.mdx +++ b/docs/pm/isolated-installs.mdx @@ -91,6 +91,7 @@ node_modules/ 2. **Symlinks** — Top-level `node_modules` contains symlinks pointing to the central store 3. **Peer resolution** — Complex peer dependencies create specialized directory names 4. **Deduplication** — Packages with identical package IDs and peer dependency sets are shared +5. **Re-linking** — On later installs, existing store entries are kept but their `node_modules/` symlinks are checked against the lockfile and re-pointed when a dependency was re-resolved (for example after [`bun dedupe`](/pm/cli/dedupe) or adding an override); store entries nothing references any more are left in place until you run [`bun prune`](/pm/cli/prune) ### Workspace handling diff --git a/docs/pm/lockfile.mdx b/docs/pm/lockfile.mdx index c0a7ce23fd82..5982c98f76e7 100644 --- a/docs/pm/lockfile.mdx +++ b/docs/pm/lockfile.mdx @@ -11,12 +11,14 @@ Yes #### Generate a lockfile without installing? -To generate a lockfile without installing to `node_modules`, use the `--lockfile-only` flag. The lockfile is always saved to disk, even if it is already up to date with your project's `package.json`(s). +To generate a lockfile without installing to `node_modules`, use the `--lockfile-only` flag. The lockfile is always saved to disk, even if it is already up to date with your project's `package.json`(s), unless `--frozen-lockfile` (or `--production`) is set. ```bash terminal icon="terminal" bun install --lockfile-only ``` +The flag is also accepted by `bun add`, `bun remove` and `bun update`, which still edit `package.json` and write `bun.lock` but leave `node_modules` untouched. + `--lockfile-only` still populates the global install cache with registry metadata and git/tarball dependencies. diff --git a/docs/pm/npmrc.mdx b/docs/pm/npmrc.mdx index 0c2ddd181662..37b5e8f8d524 100644 --- a/docs/pm/npmrc.mdx +++ b/docs/pm/npmrc.mdx @@ -5,7 +5,9 @@ description: Bun loads configuration options from [`.npmrc`](https://docs.npmjs.com/cli/v10/configuring-npm/npmrc) files, so you can reuse your existing registry and scope configuration. -Bun reads `~/.npmrc` (or `$XDG_CONFIG_HOME/.npmrc` if it exists), then the project's `./.npmrc`, then `bunfig.toml`. When the same option is set in more than one place, `./.npmrc` overrides `~/.npmrc`, `bunfig.toml` overrides both `.npmrc` files, and command-line flags override everything. Options that only appear in `.npmrc` (such as `///:_authToken`) still apply, including to registries configured in `bunfig.toml`. +Bun reads `~/.npmrc` (or `$XDG_CONFIG_HOME/.npmrc` if it exists), then the project's `./.npmrc`, then `bunfig.toml`; `NPM_CONFIG_USERCONFIG` is not consulted. When the same option is set in more than one place, `./.npmrc` overrides `~/.npmrc`, `bunfig.toml` overrides both `.npmrc` files (the project's `bunfig.toml` overriding the global `~/.bunfig.toml`), the `BUN_CONFIG_REGISTRY` / `NPM_CONFIG_REGISTRY` and `BUN_CONFIG_TOKEN` / `NPM_CONFIG_TOKEN` environment variables override the default registry and its token from any of those files, and command-line flags such as `--registry` override everything. Options that only appear in `.npmrc` (such as `///:_authToken`) still apply, including to registries configured in `bunfig.toml`. + +Values in both `~/.npmrc` and `./.npmrc` may reference environment variables: `${NAME}` is replaced with the variable's value and left as the literal text `${NAME}` when the variable is unset, while `${NAME?}` becomes an empty string when unset. We recommend migrating your `.npmrc` file to Bun's [`bunfig.toml`](/runtime/bunfig) format, which supports more diff --git a/docs/pm/overrides.mdx b/docs/pm/overrides.mdx index 8cb771da2dbd..2ce2a42fb197 100644 --- a/docs/pm/overrides.mdx +++ b/docs/pm/overrides.mdx @@ -45,11 +45,6 @@ If a security vulnerability is introduced in `bar@4.5.6`, you may want to pin `b Add `bar` to the `"overrides"` field in `package.json`. Bun defers to the specified version range when determining which version of `bar` to install, whether it's a dependency or a metadependency. - - Bun only supports top-level `"overrides"`, not [nested - overrides](https://docs.npmjs.com/cli/v9/configuring-npm/package-json#overrides). - - {/* prettier-ignore */} ```json package.json icon="file-json" { @@ -63,12 +58,14 @@ Add `bar` to the `"overrides"` field in `package.json`. Bun defers to the specif } ``` +Bun only reads `"overrides"` and `"resolutions"` from the root `package.json`; the field is ignored in workspace packages, and pnpm's `pnpm.overrides` field and `pnpm-workspace.yaml` are only consulted when migrating a `pnpm-lock.yaml`, which moves those rules into `"overrides"`. + +Rules also apply to `peerDependencies`. An overridden peer dependency is still a peer dependency; only its version range changes. + ## `"resolutions"` `"resolutions"` is Yarn's alternative to `"overrides"`, with similar syntax. Bun supports it to make migration from Yarn easier. -As with `"overrides"`, _nested resolutions_ are not supported. - {/* prettier-ignore */} ```json package.json icon="file-json" { @@ -81,3 +78,86 @@ As with `"overrides"`, _nested resolutions_ are not supported. } // [!code ++] } ``` + +## Values + +A value can be any dependency specifier, not only a version range. `npm:` swaps a package for a fork, and `catalog:` (or `catalog:`) keeps the overridden version in sync with a [workspace catalog](/pm/catalogs): + +```json package.json icon="file-json" +{ + "name": "my-app", + "overrides": { + "quux": "npm:@myorg/quux@^1.0.0", + "foo": "catalog:" + } +} +``` + +A value of `"$name"` copies the range you declared for `name` in your own `dependencies` (or `devDependencies`, `peerDependencies`, `optionalDependencies`); the referenced package doesn't have to be the one being overridden, so `"bar": "$foo"` pins `bar` to whatever range you declared for `foo`. If the root doesn't declare `name`, Bun looks in the workspace packages instead, which must all declare the same range for it. + +## Nested overrides + +A rule can be scoped to the dependencies of one package. Bun applies it to that package's direct dependency on the overridden package and nothing else, so the same package can resolve to different versions under different parents. + +The npm object form, the pnpm `>` form, and a parent with a version range are all accepted in `"overrides"`: + +```json package.json icon="file-json" +{ + "name": "my-app", + "overrides": { + "micromatch": { + ".": "^4.0.5", + "picomatch": "^2.3.2" + }, + "micromatch>picomatch": "^2.3.2", + "micromatch@^4>picomatch": "^2.3.2" + } +} +``` + +`"."` inside an object overrides `micromatch` itself, like a top-level `"micromatch"` rule. + +`"resolutions"` accepts Yarn's path form. `**` is accepted for compatibility, but only the parent's direct dependency is affected either way: + +```json package.json icon="file-json" +{ + "name": "my-app", + "resolutions": { + "micromatch/picomatch": "^2.3.2", + "**/micromatch/**/picomatch": "^2.3.2" + } +} +``` + +`$name` references and the other [value forms](#values) work in nested rules too. + +When several rules match one dependency, rules scoped to a parent with a version range win over rules scoped to a parent without one, which win over top-level rules. Within each of those groups, a rule with a matching [version selector](#version-scoped-overrides) wins over one without, and if several selectors match, the one whose range text sorts first is used. + +## Version-scoped overrides + +The overridden package's key can carry a version selector, so a rule only rewrites some of the edges pointing at that package. This is the shape `pnpm audit --fix` writes and the one npm documents, and it works in top-level and nested rules alike: + +```json package.json icon="file-json" +{ + "name": "my-app", + "overrides": { + "semver@<7.5.2": "7.5.2", + "webpack>terser@4": "4.8.1", + "terser@4": { + ".": "4.8.1" + } + } +} +``` + +`"terser@4": { ".": "4.8.1" }` is the object spelling of `"terser@4": "4.8.1"`; a selector on a child key, as in `{ "webpack": { "terser@4": "4.8.1" } }`, scopes that nested rule the same way. + +The selector is compared with the range each dependent _declares_ for that dependency, not with the version that would be installed, and the rule applies to every edge whose declared range overlaps the selector. `"semver@<7.5.2"` rewrites an edge declared as `^7.3.0`, since that range could still pick `7.3.x`, but leaves an edge declared as `^7.5.2` alone. Edges declared with a dist-tag such as `latest`, or with a `catalog:`, `workspace:`, git, or URL specifier, never match a selector. An `npm:` prefix inside a selector is ignored. + +## Limitations + +- Only one parent level is supported. `a>b>c`, `a/b/c`, and objects nested more than one level deep are ignored with a warning. +- `"pkg@"` (pnpm's convergence override, an empty selector) is not supported and is skipped with a warning. +- pnpm's `"-"` value, which removes a dependency, is not supported. Bun currently reads `"-"` as a dist-tag named `-`, and the dependency fails to resolve. +- A nested rule can't point at a `file:` path outside the project. +- A `bun.lock` that contains nested or version-scoped rules is written as `lockfileVersion` 3, which older versions of Bun refuse to read (run `bun upgrade`). Projects without such rules keep `lockfileVersion` 2 unchanged. diff --git a/docs/runtime/bunfig.mdx b/docs/runtime/bunfig.mdx index af34fadf6f59..0a4557fd0aab 100644 --- a/docs/runtime/bunfig.mdx +++ b/docs/runtime/bunfig.mdx @@ -430,7 +430,7 @@ peer = true Whether `bun install` runs in "production mode". Default `false`. -In production mode, `"devDependencies"` are not installed. The `--production` CLI flag overrides this setting. +In production mode, `"devDependencies"` are not installed, and `bun.lock` is treated as frozen, the same as setting [`install.frozenLockfile`](#install-frozenlockfile) to `true`. The `--production` CLI flag turns production mode on for a single install; there is no flag to turn it off when it is enabled here. ```toml title="bunfig.toml" icon="settings" [install] diff --git a/docs/snippets/cli/add.mdx b/docs/snippets/cli/add.mdx index 5db517d56400..c12f274153b1 100644 --- a/docs/snippets/cli/add.mdx +++ b/docs/snippets/cli/add.mdx @@ -43,6 +43,10 @@ bun add <@version> --catalog=NAME targets catalogs.NAME + + Add the package(s) to the matching workspaces instead of the current package. Alias: -F + + ### Project Files & Lockfiles diff --git a/docs/snippets/cli/remove.mdx b/docs/snippets/cli/remove.mdx index 789f6bd7abff..b423e83100d9 100644 --- a/docs/snippets/cli/remove.mdx +++ b/docs/snippets/cli/remove.mdx @@ -30,6 +30,10 @@ bun remove Add to trustedDependencies in the project's package.json and install the package(s) + + Remove the package(s) from the matching workspaces instead of the current package. Alias: -F + + ### Lockfile Behavior diff --git a/docs/snippets/cli/update.mdx b/docs/snippets/cli/update.mdx index ca176bfb97b9..9fe4013e019f 100644 --- a/docs/snippets/cli/update.mdx +++ b/docs/snippets/cli/update.mdx @@ -16,10 +16,6 @@ bun update @ ### Dependency Scope - - Don't install devDependencies. Alias: -p - - Install globally. Alias: -g diff --git a/src/install/PackageManager.rs b/src/install/PackageManager.rs index eb0d78093f2b..dc75bf2ab592 100644 --- a/src/install/PackageManager.rs +++ b/src/install/PackageManager.rs @@ -59,6 +59,8 @@ pub mod command_line_arguments; pub mod install_with_manager; #[path = "PackageManager/PackageJSONEditor.rs"] pub mod package_json_editor; +#[path = "PackageManager/package_json_write_back.rs"] +pub mod package_json_write_back; #[path = "PackageManager/PackageManagerDirectories.rs"] pub mod package_manager_directories; #[path = "PackageManager/PackageManagerEnqueue.rs"] @@ -419,9 +421,12 @@ pub struct PackageManager { // `bun update -r`/`--filter`: workspaces whose deps update. None = cwd only. pub(crate) update_target_workspaces: Option>, - // `bun add/remove --filter`: package.json edits written as soon as the lockfile is saved. + // bun add --filter: which target received which request; consumed by bind_update_requests and package_json_write_back. pub(crate) pending_filtered_write: Option>, + // package.json cache entries that differ from disk; written by package_json_write_back::flush. + pub(crate) edited_package_jsons: Vec, + pub(crate) patched_dependencies_to_remove: ArrayHashMap, @@ -664,7 +669,8 @@ impl WorkspaceFilter { #[derive(Default)] pub struct PackageUpdateInfo { pub(crate) original_version_literal: Box<[u8]>, - pub(crate) is_alias: bool, + // set by the post-install write-back; the install summary still needs the entry + pub(crate) written_back: bool, pub(crate) original_version_string_buf: Box<[u8]>, pub(crate) original_version: Option, } @@ -674,8 +680,7 @@ pub struct CatalogUpdateInfo { pub catalog_name: Box<[u8]>, pub dep_name: Box<[u8]>, pub original_version_literal: Box<[u8]>, - pub is_alias: bool, - /// Set by `Lockfile::clean_with_logger`; `None` leaves the entry as written. + /// Set by package_json_editor::resolve_catalog_literals; None leaves the entry as written. pub new_version_literal: Option>, } @@ -1453,7 +1458,15 @@ pub(crate) fn get() -> *mut PackageManager { // init // ────────────────────────────────────────────────────────────────────────── -/// bunfig beats npmrc per field; a registry npmrc left at bunfig's URL is kept since npmrc attached credentials to it. +fn registry_has_credentials(registry: &Api::NpmRegistry) -> bool { + !registry.token.is_empty() || !registry.username.is_empty() || !registry.password.is_empty() +} + +/// bunfig beats npmrc per field; a credential-less bunfig registry left at the same URL is kept since npmrc attached credentials to it. +fn bunfig_registry_wins(current: Option<&Api::NpmRegistry>, bunfig: &Api::NpmRegistry) -> bool { + current.is_none_or(|current| current.url != bunfig.url) || registry_has_credentials(bunfig) +} + fn overlay_bunfig_install(install: &mut Api::BunInstall, bunfig: Api::BunInstall) { let Api::BunInstall { default_registry, @@ -1492,11 +1505,7 @@ fn overlay_bunfig_install(install: &mut Api::BunInstall, bunfig: Api::BunInstall } = bunfig; if let Some(registry) = default_registry { - if install - .default_registry - .as_ref() - .is_none_or(|current| current.url != registry.url) - { + if bunfig_registry_wins(install.default_registry.as_ref(), ®istry) { install.default_registry = Some(registry); } } @@ -1504,10 +1513,7 @@ fn overlay_bunfig_install(install: &mut Api::BunInstall, bunfig: Api::BunInstall if let Some(bunfig_scopes) = scoped { let scopes = &mut install.scoped.get_or_insert_with(Default::default).scopes; for (name, registry) in bunfig_scopes.scopes.iter() { - if scopes - .get(name) - .is_none_or(|current| current.url != registry.url) - { + if bunfig_registry_wins(scopes.get(name), registry) { scopes.insert(name, registry.clone()); } } @@ -1851,7 +1857,7 @@ pub fn init( &json_source, prop.loc, None, - true, + Package::WorkspaceMap::MissingWorkspace::Skip, ) { Ok(v) => v, Err(_) => break, @@ -2211,6 +2217,7 @@ pub fn init( wr!(updating_catalogs, Vec::new()); wr!(update_target_workspaces, None); wr!(pending_filtered_write, None); + wr!(edited_package_jsons, Vec::new()); wr!(patched_dependencies_to_remove, ArrayHashMap::default()); wr!(last_reported_slow_lifecycle_script_at, 0); wr!(cached_tick_for_slow_lifecycle_script_logging, 0); @@ -2652,6 +2659,7 @@ fn init_with_runtime_once( wr!(updating_catalogs, Vec::new()); wr!(update_target_workspaces, None); wr!(pending_filtered_write, None); + wr!(edited_package_jsons, Vec::new()); wr!(patched_dependencies_to_remove, ArrayHashMap::default()); wr!(last_reported_slow_lifecycle_script_at, 0); wr!(cached_tick_for_slow_lifecycle_script_logging, 0); diff --git a/src/install/PackageManager/CommandLineArguments.rs b/src/install/PackageManager/CommandLineArguments.rs index 63e2e5260c3c..51e8cc45f16f 100644 --- a/src/install/PackageManager/CommandLineArguments.rs +++ b/src/install/PackageManager/CommandLineArguments.rs @@ -293,7 +293,9 @@ const AUDIT_PARAMS: &[ParamType] = &[ clap::param!( "--audit-level Only print advisories with severity greater than or equal to \\ (low, moderate, high, critical)" ), - clap::param!("--ignore ... Ignore specific CVE IDs from audit"), + clap::param!( + "--ignore ... Ignore advisories by GHSA or numeric advisory ID (repeatable)" + ), ]; static AUDIT_PARAMS_FULL: &[ParamType] = concat_params![SHARED_PARAMS, AUDIT_PARAMS]; @@ -1466,6 +1468,17 @@ Full documentation is available at https://bun.com/docs/pm/cli/prune cli.latest = args.flag(b"--latest"); cli.interactive = args.flag(b"--interactive"); cli.recursive = args.flag(b"--recursive"); + if cli.production { + Output::err_generic("--production cannot be used with bun update\n", ()); + Global::crash(); + } + if cli.positionals.len() > 1 && (cli.recursive || !cli.filters.is_empty()) { + Output::err_generic( + "--recursive and --filter cannot be combined with package names\n", + (), + ); + Global::crash(); + } } let specified_backend: Option = 'brk: { diff --git a/src/install/PackageManager/PackageJSONEditor.rs b/src/install/PackageManager/PackageJSONEditor.rs index e55f006f5297..06d357f6152c 100644 --- a/src/install/PackageManager/PackageJSONEditor.rs +++ b/src/install/PackageManager/PackageJSONEditor.rs @@ -3,7 +3,7 @@ use std::io::Write as _; use bun_ast as js_ast; use bun_ast::{E, Expr, G}; -use bun_core::strings; +use bun_core::{Global, Output, strings}; use bun_semver as semver; use bun_install::dependency::{self, TagExt as _}; @@ -11,7 +11,6 @@ use bun_install::lockfile::CatalogMap; use bun_install::lockfile::package::PackageColumns as _; use bun_install::{Dependency, INVALID_PACKAGE_ID, Lockfile, resolution}; use bun_install_types::{DependencyGroup, PackageNameHash}; -use bun_semver::ExternalString; use super::package_manager_options::{Do, Enable}; use super::{CatalogUpdateInfo, PackageManager, PackageUpdateInfo, Subcommand, UpdateRequest}; @@ -28,7 +27,6 @@ const DEPENDENCY_GROUPS: [DependencyGroup; 4] = [ #[derive(Default, Clone, Copy)] pub(crate) struct EditOptions { pub exact_versions: bool, - pub add_trusted_dependencies: bool, pub before_install: bool, } @@ -41,6 +39,82 @@ fn arena_dup<'a>(arena: &'a bun_alloc::Arena, bytes: &[u8]) -> &'a [u8] { arena.alloc_slice_copy(bytes) } +/// `npm:@foo/bar@~1.2.3` -> (`npm:@foo/bar`, `~1.2.3`); `npm:foo` -> (`npm:foo`, `""`). +fn split_npm_alias(literal: &[u8]) -> Option<(&[u8], &[u8])> { + let literal = strings::trim(literal, &strings::WHITESPACE_CHARS); + let name = literal.strip_prefix(b"npm:")?; + let scope_len = usize::from(name.starts_with(b"@")); + if name.len() == scope_len { + return None; + } + match strings::index_of_char_usize(&name[scope_len..], b'@') { + Some(i) => { + let (alias, version) = literal.split_at(b"npm:".len() + scope_len + i); + Some((alias, &version[1..])) + } + None => Some((literal, b"")), + } +} + +/// `version_literal` behind `from`'s `npm:@` (if any), unless it already names a target. +fn with_alias_of<'a>( + arena: &'a bun_alloc::Arena, + from: &[u8], + version_literal: &'a [u8], +) -> &'a [u8] { + match split_npm_alias(from) { + Some((alias, _)) if split_npm_alias(version_literal).is_none() => { + let mut v = Vec::new(); + write!( + &mut v, + "{}@{}", + bstr::BStr::new(alias), + bstr::BStr::new(version_literal) + ) + .expect("infallible: in-memory write"); + arena_str(arena, &v) + } + _ => version_literal, + } +} + +/// `resolved` in the pin style of `original_version_literal`, behind its `npm:@` if any. +fn updated_version_literal( + original_version_literal: &[u8], + resolved: semver::Version, + resolved_buf: &[u8], + exact_versions: bool, +) -> Vec { + let mut v = Vec::new(); + let version_literal = match split_npm_alias(original_version_literal) { + Some((alias, version_literal)) => { + write!(&mut v, "{}@", bstr::BStr::new(alias)).expect("infallible: in-memory write"); + version_literal + } + None => original_version_literal, + }; + + // `=1.0.0` round-trips as `=2.0.0`; `which_version_is_pinned` skips the `=` and reports Patch. + let exact_prefix = + if strings::trim(version_literal, &strings::WHITESPACE_CHARS).starts_with(b"=") { + "=" + } else { + "" + }; + let range_prefix = if exact_versions { + exact_prefix + } else { + match semver::Version::which_version_is_pinned(version_literal) { + semver::PinnedVersion::Patch => exact_prefix, + semver::PinnedVersion::Minor => "~", + semver::PinnedVersion::Major => "^", + } + }; + write!(&mut v, "{}{}", range_prefix, resolved.fmt(resolved_buf)) + .expect("infallible: in-memory write"); + v +} + /// Shallow-copy a `G::Property` for the JSON-editing path. Only `key`/`value` /// (both `Option`, `Copy`) are populated by the JSON parser; the rest /// (`ts_decorators`, `class_static_block`, …) are always default for parsed @@ -300,7 +374,7 @@ pub(crate) fn edit_update_no_args_in( let version_literal = value .as_utf8_string_literal() .unwrap_or_else(|| bun_core::out_of_memory()); - let mut tag = dependency::Tag::infer(version_literal); + let tag = dependency::Tag::infer(version_literal); // npm versions only (and dist-tags with --latest); `catalog:` is handled by edit_catalogs_*. if tag != dependency::Tag::Npm @@ -309,25 +383,6 @@ pub(crate) fn edit_update_no_args_in( continue; } - let mut alias_at_index: Option = None; - if strings::trim(version_literal, &strings::WHITESPACE_CHARS) - .starts_with(b"npm:") - { - // negative because the real package might have a scope - // e.g. "dep": "npm:@foo/bar@1.2.3" - if let Some(at_index) = - strings::last_index_of_char(version_literal, b'@') - { - tag = dependency::Tag::infer(&version_literal[at_index + 1..]); - if tag != dependency::Tag::Npm - && (tag != dependency::Tag::DistTag || !update_to_latest) - { - continue; - } - alias_at_index = Some(at_index); - } - } - let key_str = key.as_utf8_string_literal().expect("unreachable"); // Capture the literal as an owned // copy before borrowing `updating_packages` mutably. @@ -343,26 +398,13 @@ pub(crate) fn edit_update_no_args_in( *entry.value_ptr = PackageUpdateInfo { original_version_literal: version_literal_owned, - is_alias: alias_at_index.is_some(), + written_back: false, original_version_string_buf: Box::default(), original_version: None, }; if update_to_latest { - // is it an aliased package - let temp_version: &[u8] = if let Some(at_index) = alias_at_index { - let mut v = Vec::new(); - write!( - &mut v, - "{}@latest", - bstr::BStr::new(&version_literal[0..at_index]) - ) - .unwrap(); - arena_str(arena, &v) - } else { - b"latest" - }; - + let temp_version = with_alias_of(arena, version_literal, b"latest"); dep.value = Some(Expr::allocate( arena, E::EString::init(temp_version), @@ -414,11 +456,12 @@ pub(crate) fn edit_update_no_args_in( .unwrap_or_else(|| bun_core::out_of_memory()); 'updated: { - // fetchSwapRemove because we want to update the first dependency with a matching - // name, or none at all - if let Some(entry) = updating_packages.fetch_swap_remove(key_str) { - let is_alias = entry.value.is_alias; - let dep_name = &*entry.key; + // Only the first dependency group naming the package is rewritten. + if let Some(entry) = updating_packages.get_mut(key_str) { + if entry.written_back { + break 'updated; + } + entry.written_back = true; debug_assert_eq!( workspace_deps.len(), workspace_resolution_ids.len() @@ -436,7 +479,7 @@ pub(crate) fn edit_update_no_args_in( } let workspace_dep_name = workspace_dep.name.slice(string_buf); - if !strings::eql_long(workspace_dep_name, dep_name, true) { + if !strings::eql_long(workspace_dep_name, key_str, true) { continue; } @@ -444,90 +487,18 @@ pub(crate) fn edit_update_no_args_in( .resolve_catalog_dependency(workspace_dep) .unwrap_or_else(|| workspace_dep.version.clone()); if let Some(npm_version) = resolved_version.try_npm() { - // It's possible we inserted a dependency that won't update (version is an exact version). - // If we find one, skip to keep the original version literal. + // an exact pin is not moved by a plain `bun update` if !update_to_latest && npm_version.version.is_exact() { break 'updated; } } - let new_version: Vec = 'new_version: { - // `resolution.tag == Npm` checked above. - let version_fmt = resolution.npm().version.fmt(string_buf); - if options.exact_versions { - let mut v = Vec::new(); - write!(&mut v, "{}", version_fmt) - .expect("infallible: in-memory write"); - break 'new_version v; - } - - let version_literal: &[u8] = 'version_literal: { - if !is_alias { - break 'version_literal &entry - .value - .original_version_literal; - } - if let Some(at_index) = strings::last_index_of_char( - &entry.value.original_version_literal, - b'@', - ) { - break 'version_literal &entry - .value - .original_version_literal[at_index + 1..]; - } - &entry.value.original_version_literal - }; - - let pinned_version = - semver::Version::which_version_is_pinned( - version_literal, - ); - let mut v = Vec::new(); - match pinned_version { - semver::PinnedVersion::Patch => { - write!(&mut v, "{}", version_fmt) - .expect("infallible: in-memory write") - } - semver::PinnedVersion::Minor => { - write!(&mut v, "~{}", version_fmt) - .expect("infallible: in-memory write") - } - semver::PinnedVersion::Major => { - write!(&mut v, "^{}", version_fmt) - .expect("infallible: in-memory write") - } - } - v - }; - - if is_alias { - let dep_literal = - workspace_dep.version.literal.slice(string_buf); - - // negative because the real package might have a scope - // e.g. "dep": "npm:@foo/bar@1.2.3" - if let Some(at_index) = - strings::last_index_of_char(dep_literal, b'@') - { - let mut v = Vec::new(); - write!( - &mut v, - "{}@{}", - bstr::BStr::new(&dep_literal[0..at_index]), - bstr::BStr::new(&new_version) - ) - .unwrap(); - dep.value = Some(Expr::allocate( - arena, - E::EString::init(arena_str(arena, &v)), - bun_ast::Loc::EMPTY, - )); - break 'updated; - } - - // fallthrough and replace entire version. - } - + let new_version = updated_version_literal( + &entry.original_version_literal, + resolution.npm().version, + string_buf, + options.exact_versions, + ); dep.value = Some(Expr::allocate( arena, E::EString::init(arena_str(arena, &new_version)), @@ -623,16 +594,7 @@ pub(crate) fn edit_catalogs_before_update( let version_literal = value .as_utf8_string_literal() .unwrap_or_else(|| bun_core::out_of_memory()); - let mut tag = dependency::Tag::infer(version_literal); - - let mut alias_at_index: Option = None; - if strings::trim(version_literal, &strings::WHITESPACE_CHARS).starts_with(b"npm:") { - // last '@' handles scoped aliases like "npm:@foo/bar@1.2.3" - if let Some(at_index) = strings::last_index_of_char(version_literal, b'@') { - tag = dependency::Tag::infer(&version_literal[at_index + 1..]); - alias_at_index = Some(at_index); - } - } + let tag = dependency::Tag::infer(version_literal); // same tag rule as direct dependencies if tag != dependency::Tag::Npm && (tag != dependency::Tag::DistTag || !update_to_latest) @@ -648,24 +610,11 @@ pub(crate) fn edit_catalogs_before_update( catalog_name: Box::from(catalog_name), dep_name: Box::from(key_str), original_version_literal: Box::from(version_literal), - is_alias: alias_at_index.is_some(), new_version_literal: None, }); if update_to_latest { - let temp_version: &[u8] = if let Some(at_index) = alias_at_index { - let mut v = Vec::new(); - write!( - &mut v, - "{}@latest", - bstr::BStr::new(&version_literal[0..at_index]) - ) - .expect("infallible: in-memory write"); - arena_str(arena, &v) - } else { - b"latest" - }; - + let temp_version = with_alias_of(arena, version_literal, b"latest"); dep.value = Some(Expr::allocate( arena, E::EString::init(temp_version), @@ -679,7 +628,7 @@ pub(crate) fn edit_catalogs_before_update( Ok(!manager.updating_catalogs.is_empty()) } -/// Writes the literals `resolve_catalog_updates` recorded into the root's catalog entries (unresolved ones are restored); returns `changed`. +/// Writes each recorded catalog entry's resolved literal (unresolved ones are restored) into the root AST; returns `changed`. pub(crate) fn edit_catalogs_after_update( manager: &mut PackageManager, root_package_json: &Expr, @@ -687,10 +636,18 @@ pub(crate) fn edit_catalogs_after_update( // see note in `edit_update_no_args` — always avoid the store let _guard = ExprDisabler::scope(); - let infos = core::mem::take(&mut manager.updating_catalogs); + let mut infos = core::mem::take(&mut manager.updating_catalogs); if infos.is_empty() { return Ok(false); } + let index = CatalogInfoIndex::init(&infos)?; + resolve_catalog_literals( + &manager.lockfile, + &mut infos, + &index, + manager.options.do_.contains(Do::UPDATE_TO_LATEST), + manager.options.enable.exact_versions(), + ); let arena = &manager.ast_arena; let mut changed = false; @@ -713,10 +670,11 @@ pub(crate) fn edit_catalogs_after_update( .as_utf8_string_literal() .unwrap_or_else(|| bun_core::out_of_memory()); - let Some(info) = infos.iter().find(|info| { - strings::eql_long(&info.dep_name, key_str, true) - && strings::eql_long(&info.catalog_name, catalog_name, true) - }) else { + let Some(info) = index + .candidates(key_str) + .and_then(|candidates| CatalogInfoIndex::pick(candidates, &infos, catalog_name)) + .map(|i| &infos[i]) + else { continue; }; @@ -741,71 +699,45 @@ pub(crate) fn edit_catalogs_after_update( Ok(changed) } -/// Called from `Lockfile::clean_with_logger` on the cleaned lockfile: records each entry's new literal and writes it into `lockfile.catalogs` so bun.lock matches package.json. -pub(crate) fn resolve_catalog_updates( - lockfile: &mut Lockfile, - manager: &mut PackageManager, - exact_versions: bool, -) -> crate::Result<()> { - let mut infos = core::mem::take(&mut manager.updating_catalogs); - let update_to_latest = manager.options.do_.contains(Do::UPDATE_TO_LATEST); - resolve_catalog_literals(lockfile, &mut infos, update_to_latest, exact_versions); - let result = rewrite_lockfile_catalogs(lockfile, manager, &infos); - manager.updating_catalogs = infos; - result -} +/// Indices into `updating_catalogs` keyed by dependency name. +struct CatalogInfoIndex(StringArrayHashMap>); -fn rewrite_lockfile_catalogs( - lockfile: &mut Lockfile, - manager: &mut PackageManager, - infos: &[CatalogUpdateInfo], -) -> crate::Result<()> { - let literals = || { - infos - .iter() - .filter_map(|info| info.new_version_literal.as_deref()) - }; - if literals().next().is_none() { - return Ok(()); +impl CatalogInfoIndex { + fn init(infos: &[CatalogUpdateInfo]) -> Result { + let mut map = StringArrayHashMap::>::with_capacity(infos.len()); + for (i, info) in infos.iter().enumerate() { + map.get_or_put(&info.dep_name)?.value_ptr.push(i); + } + Ok(CatalogInfoIndex(map)) } - let (mut builder, lf) = lockfile.string_builder_split(); - for literal in literals() { - builder.count(literal); + fn candidates(&self, dep_name: &[u8]) -> Option<&[usize]> { + self.0.get(dep_name).map(Vec::as_slice) } - builder.allocate()?; - for info in infos { - let Some(literal) = info.new_version_literal.as_deref() else { - continue; - }; - let external = builder.append::(literal); - let string_bytes = builder.string_bytes.as_slice(); - let sliced = external.value.sliced(string_bytes); - let Some(entry) = lf - .catalogs - .find_mut(string_bytes, &info.catalog_name, &info.dep_name) - else { - continue; - }; - let (entry_name, entry_name_hash) = (entry.name, entry.name_hash); - if let Some(version) = dependency::parse( - entry_name, - entry_name_hash, - sliced.slice, - &sliced, - None, - &mut *manager, - ) { - entry.version = version; - } + + /// An entry spelled exactly like `catalog_name` wins over the `catalog:` / `catalog:default` equivalence. + fn pick( + candidates: &[usize], + infos: &[CatalogUpdateInfo], + catalog_name: &[u8], + ) -> Option { + candidates + .iter() + .copied() + .find(|&i| &*infos[i].catalog_name == catalog_name) + .or_else(|| { + candidates + .iter() + .copied() + .find(|&i| CatalogMap::same_name(&infos[i].catalog_name, catalog_name)) + }) } - builder.clamp(); - Ok(()) } fn resolve_catalog_literals( lockfile: &Lockfile, infos: &mut [CatalogUpdateInfo], + by_name: &CatalogInfoIndex, update_to_latest: bool, exact_versions: bool, ) { @@ -829,15 +761,17 @@ fn resolve_catalog_literals( continue; } - let dep_name = dep.name.slice(string_buf); - let catalog_name = dep.version.catalog().slice(string_buf); - let find = |same_catalog: fn(&[u8], &[u8]) -> bool| { - infos.iter().position(|info| { - strings::eql_long(&info.dep_name, dep_name, true) - && same_catalog(&info.catalog_name, catalog_name) - }) + let Some(candidates) = by_name.candidates(dep.name.slice(string_buf)) else { + continue; }; - let Some(index) = find(|a, b| a == b).or_else(|| find(CatalogMap::same_name)) else { + if candidates + .iter() + .all(|&i| infos[i].new_version_literal.is_some()) + { + continue; + } + let catalog_name = dep.version.catalog().slice(string_buf); + let Some(index) = CatalogInfoIndex::pick(candidates, infos, catalog_name) else { continue; }; if infos[index].new_version_literal.is_some() { @@ -861,67 +795,17 @@ fn resolve_catalog_literals( } } - let info = &infos[index]; - let version_fmt = resolution.npm().version.fmt(string_buf); - let new_version: Vec = 'new_version: { - if exact_versions { - let mut v = Vec::new(); - write!(&mut v, "{}", version_fmt).expect("infallible: in-memory write"); - break 'new_version v; - } - - let version_literal: &[u8] = 'version_literal: { - if !info.is_alias { - break 'version_literal &info.original_version_literal; - } - if let Some(at_index) = - strings::last_index_of_char(&info.original_version_literal, b'@') - { - break 'version_literal &info.original_version_literal[at_index + 1..]; - } - &info.original_version_literal - }; - - let pinned_version = semver::Version::which_version_is_pinned(version_literal); - let mut v = Vec::new(); - match pinned_version { - semver::PinnedVersion::Patch => { - write!(&mut v, "{}", version_fmt).expect("infallible: in-memory write") - } - semver::PinnedVersion::Minor => { - write!(&mut v, "~{}", version_fmt).expect("infallible: in-memory write") - } - semver::PinnedVersion::Major => { - write!(&mut v, "^{}", version_fmt).expect("infallible: in-memory write") - } - } - v - }; - - let new_literal = if info.is_alias { - let dep_literal = &info.original_version_literal; - if let Some(at_index) = strings::last_index_of_char(dep_literal, b'@') { - let mut v = Vec::new(); - write!( - &mut v, - "{}@{}", - bstr::BStr::new(&dep_literal[0..at_index]), - bstr::BStr::new(&new_version) - ) - .expect("infallible: in-memory write"); - v - } else { - new_version - } - } else { - new_version - }; + let new_literal = updated_version_literal( + &infos[index].original_version_literal, + resolution.npm().version, + string_buf, + exact_versions, + ); infos[index].new_version_literal = Some(new_literal.into_boxed_slice()); } } -/// edits dependencies and trusted dependencies -/// if options.add_trusted_dependencies is true, gets list from PackageManager.trusted_deps_to_add_to_package_json +/// Edits the dependency lists for `updates`; `trustedDependencies` is added later by `package_json_write_back::flush`. pub(crate) fn edit( manager: &mut PackageManager, // Pointer-to-slice whose `.len` is shrunk in place. @@ -937,10 +821,27 @@ pub(crate) fn edit( // Process-lifetime arena for AST // nodes that must outlive `Expr.Data.Store.reset()`. See `PackageManager.ast_arena`. - // `arena` is a disjoint-field borrow held across - // the `&mut manager.{updating_packages,trusted_deps_to_add_to_package_json}` accesses below. + // `arena` is a disjoint-field borrow held across the `&mut manager.updating_packages` accesses below. let arena = &manager.ast_arena; + let update_to_latest = manager.subcommand == Subcommand::Update + && manager.options.do_.contains(Do::UPDATE_TO_LATEST); + if update_to_latest && options.before_install { + if let Some(request) = updates.iter().find(|request| { + !request + .version + .literal + .slice(request.version_buf()) + .is_empty() + }) { + Output::err_generic( + "--latest cannot be combined with a version: {}\n", + (bstr::BStr::new(request.version_buf()),), + ); + Global::crash(); + } + } + let mut remaining = updates.len(); let mut replacing: usize = 0; let only_add_missing = manager.options.enable.contains(Enable::ONLY_MISSING); @@ -951,27 +852,6 @@ pub(crate) fn edit( // 3. There is a "dependencies" (or equivalent list), and the package name exists in multiple lists // Try to use the existing spot in the dependencies list if possible { - if options.add_trusted_dependencies { - if let Some(query) = current_package_json.as_property(TRUSTED_DEPENDENCIES_STRING) { - if let bun_ast::ExprData::EArray(arr) = query.expr.data { - // Iterate backwards to avoid index issues when removing items - let mut i: usize = manager.trusted_deps_to_add_to_package_json.len(); - while i > 0 { - i -= 1; - let trusted_package_name = &manager.trusted_deps_to_add_to_package_json[i]; - for item in arr.items.slice() { - if let bun_ast::ExprData::EString(s) = &item.data { - if s.eql_bytes(trusted_package_name) { - let _ = - manager.trusted_deps_to_add_to_package_json.swap_remove(i); - break; - } - } - } - } - } - } - } { let mut i: usize = 0; 'loop_: while i < updates.len() { @@ -1009,8 +889,7 @@ pub(crate) fn edit( else { break 'add_packages_to_update; }; - let mut tag = - dependency::Tag::infer(version_literal); + let tag = dependency::Tag::infer(version_literal); if tag != dependency::Tag::Npm && tag != dependency::Tag::DistTag @@ -1031,38 +910,9 @@ pub(crate) fn edit( break 'add_packages_to_update; } - // `get_or_put` default-initializes the slot, - // so the `npm:`-alias bailout path below - // (later read by `fetchSwapRemove`) is - // well-defined. - let mut is_alias = false; - if strings::trim( - &version_literal_owned, - &strings::WHITESPACE_CHARS, - ) - .starts_with(b"npm:") - { - if let Some(at_index) = - strings::last_index_of_char( - &version_literal_owned, - b'@', - ) - { - tag = dependency::Tag::infer( - &version_literal_owned[at_index + 1..], - ); - if tag != dependency::Tag::Npm - && tag != dependency::Tag::DistTag - { - break 'add_packages_to_update; - } - is_alias = true; - } - } - *entry.value_ptr = PackageUpdateInfo { original_version_literal: version_literal_owned, - is_alias, + written_back: false, original_version_string_buf: Box::default(), original_version: None, }; @@ -1147,6 +997,17 @@ pub(crate) fn edit( } } + // `bun update ` never adds ``: a name this file does not declare only moves in the lockfile. + let update_in_place = manager.subcommand == Subcommand::Update; + if update_in_place { + remaining -= updates + .iter() + .filter(|request| { + request.e_string.is_none() && request.package_id == INVALID_PACKAGE_ID + }) + .count(); + } + if remaining != 0 { let mut new_dependencies: Vec = { let mut dependencies: Vec = Vec::new(); @@ -1164,64 +1025,10 @@ pub(crate) fn edit( dependencies }; - let mut trusted_dependencies: &[Expr] = &[]; - if options.add_trusted_dependencies { - if let Some(query) = current_package_json.as_property(TRUSTED_DEPENDENCIES_STRING) { - if let bun_ast::ExprData::EArray(arr) = &query.expr.data { - // SAFETY: arena-backed slice; see note in `edit_trusted_dependencies`. - trusted_dependencies = unsafe { bun_ptr::detach_lifetime(arr.items.slice()) }; - } - } - } - - let trusted_dependencies_to_add = manager.trusted_deps_to_add_to_package_json.len(); - let new_trusted_deps: js_ast::ExprNodeList = 'brk: { - if !options.add_trusted_dependencies || trusted_dependencies_to_add == 0 { - break 'brk bun_alloc::AstAlloc::vec(); - } - - let mut deps = - vec![Expr::EMPTY; trusted_dependencies.len() + trusted_dependencies_to_add] - .into_boxed_slice(); - deps[0..trusted_dependencies.len()].copy_from_slice(trusted_dependencies); - // tail already initialized to Expr::EMPTY - - for package_name in &manager.trusted_deps_to_add_to_package_json { - #[cfg(debug_assertions)] - { - let mut has_missing = false; - for dep in deps.iter() { - if matches!(dep.data, bun_ast::ExprData::EMissing(_)) { - has_missing = true; - } - } - debug_assert!(has_missing); - } - - let mut i = deps.len(); - while i > 0 { - i -= 1; - if matches!(deps[i].data, bun_ast::ExprData::EMissing(_)) { - deps[i] = Expr::allocate( - arena, - E::EString::init(arena_dup(arena, package_name)), - bun_ast::Loc::EMPTY, - ); - break; - } - } - } - - #[cfg(debug_assertions)] - for dep in deps.iter() { - debug_assert!(!matches!(dep.data, bun_ast::ExprData::EMissing(_))); - } - - js_ast::ExprNodeList::from_owned_slice(deps) - }; - for request in updates.iter_mut() { - if request.e_string.is_some() { + if request.e_string.is_some() + || (update_in_place && request.package_id == INVALID_PACKAGE_ID) + { continue; } @@ -1314,40 +1121,6 @@ pub(crate) fn edit( } } - let mut needs_new_trusted_dependencies_list = true; - let mut trusted_dependencies_array: Expr = 'brk: { - if !options.add_trusted_dependencies || trusted_dependencies_to_add == 0 { - needs_new_trusted_dependencies_list = false; - break 'brk Expr::EMPTY; - } - if let Some(query) = current_package_json.as_property(TRUSTED_DEPENDENCIES_STRING) { - if matches!(query.expr.data, bun_ast::ExprData::EArray(_)) { - needs_new_trusted_dependencies_list = false; - break 'brk query.expr; - } - } - - Expr::allocate( - arena, - E::Array { - items: js_ast::ExprNodeList::from_slice(new_trusted_deps.slice()), - ..Default::default() - }, - bun_ast::Loc::EMPTY, - ) - }; - - if options.add_trusted_dependencies && trusted_dependencies_to_add > 0 { - let arr = trusted_dependencies_array - .data - .e_array_mut() - .expect("infallible: variant checked"); - arr.items = new_trusted_deps; - if arr.items.len_u32() > 1 { - arr.alphabetize_strings(); - } - } - if !matches!(current_package_json.data, bun_ast::ExprData::EObject(_)) || current_package_json .data @@ -1357,13 +1130,7 @@ pub(crate) fn edit( .len_u32() == 0 { - let n = if options.add_trusted_dependencies { - 2 - } else { - 1 - }; - let mut root_properties: Vec = Vec::with_capacity(n); - root_properties.push(G::Property { + let root_properties: Vec = vec![G::Property { key: Some(Expr::allocate( arena, E::EString::init(arena_dup(arena, dependency_list)), @@ -1371,19 +1138,7 @@ pub(crate) fn edit( )), value: Some(dependencies_object), ..Default::default() - }); - - if options.add_trusted_dependencies { - root_properties.push(G::Property { - key: Some(Expr::allocate( - arena, - E::EString::init(TRUSTED_DEPENDENCIES_STRING), - bun_ast::Loc::EMPTY, - )), - value: Some(trusted_dependencies_array), - ..Default::default() - }); - } + }]; *current_package_json = Expr::allocate( arena, @@ -1393,79 +1148,33 @@ pub(crate) fn edit( }, bun_ast::Loc::EMPTY, ); - } else { - if needs_new_dependency_list && needs_new_trusted_dependencies_list { - let obj = current_package_json - .data - .e_object() - .expect("infallible: variant checked"); - let old_props = obj.properties.slice(); - let mut root_properties: Vec = Vec::with_capacity(old_props.len() + 2); - for p in old_props { - root_properties.push(copy_property(p)); - } - root_properties.push(G::Property { - key: Some(Expr::allocate( - arena, - E::EString::init(arena_dup(arena, dependency_list)), - bun_ast::Loc::EMPTY, - )), - value: Some(dependencies_object), - ..Default::default() - }); - root_properties.push(G::Property { - key: Some(Expr::allocate( - arena, - E::EString::init(TRUSTED_DEPENDENCIES_STRING), - bun_ast::Loc::EMPTY, - )), - value: Some(trusted_dependencies_array), - ..Default::default() - }); - *current_package_json = Expr::allocate( + } else if needs_new_dependency_list { + let obj = current_package_json + .data + .e_object() + .expect("infallible: variant checked"); + let old_props = obj.properties.slice(); + let mut root_properties: Vec = Vec::with_capacity(old_props.len() + 1); + for p in old_props { + root_properties.push(copy_property(p)); + } + root_properties.push(G::Property { + key: Some(Expr::allocate( arena, - E::Object { - properties: G::PropertyList::move_from_list(root_properties), - ..Default::default() - }, + E::EString::init(arena_dup(arena, dependency_list)), bun_ast::Loc::EMPTY, - ); - } else if needs_new_dependency_list || needs_new_trusted_dependencies_list { - let obj = current_package_json - .data - .e_object() - .expect("infallible: variant checked"); - let old_props = obj.properties.slice(); - let mut root_properties: Vec = Vec::with_capacity(old_props.len() + 1); - for p in old_props { - root_properties.push(copy_property(p)); - } - root_properties.push(G::Property { - key: Some(Expr::allocate( - arena, - E::EString::init(if needs_new_dependency_list { - arena_dup(arena, dependency_list) - } else { - TRUSTED_DEPENDENCIES_STRING - }), - bun_ast::Loc::EMPTY, - )), - value: Some(if needs_new_dependency_list { - dependencies_object - } else { - trusted_dependencies_array - }), + )), + value: Some(dependencies_object), + ..Default::default() + }); + *current_package_json = Expr::allocate( + arena, + E::Object { + properties: G::PropertyList::move_from_list(root_properties), ..Default::default() - }); - *current_package_json = Expr::allocate( - arena, - E::Object { - properties: G::PropertyList::move_from_list(root_properties), - ..Default::default() - }, - bun_ast::Loc::EMPTY, - ); - } + }, + bun_ast::Loc::EMPTY, + ); } } @@ -1502,159 +1211,92 @@ pub(crate) fn edit( if request.package_id as usize >= resolutions.len() || resolutions[request.package_id as usize].tag == resolution::Tag::Uninitialized { - e_string.data = 'uninitialized: { - if manager.subcommand == Subcommand::Update - && manager.options.do_.contains(Do::UPDATE_TO_LATEST) - { - break 'uninitialized b"latest".into(); - } - - if manager.subcommand != Subcommand::Update - || !options.before_install - || e_string.is_blank() - || request.version.tag == dependency::Tag::Npm - { - break 'uninitialized match request.version.tag { - dependency::Tag::Uninitialized => b"latest".into(), - _ => arena_dup( - arena, - request.version.literal.slice(request.version_buf()), - ) - .into(), - }; - } else { - break 'uninitialized e_string.data; - } + // The entry `bun update` is updating keeps its alias target whatever gets resolved. + let existing: Option<&[u8]> = (manager.subcommand == Subcommand::Update + && options.before_install + && !e_string.is_blank()) + .then(|| e_string.data.slice()); + let mut version_literal: &[u8] = match existing { + Some(existing) if request.version.tag != dependency::Tag::Npm => existing, + _ => match request.version.tag { + dependency::Tag::Uninitialized => b"latest", + _ => request.version.literal.slice(request.version_buf()), + }, }; + if let Some(existing) = existing { + version_literal = with_alias_of(arena, existing, version_literal); + } + if update_to_latest { + version_literal = with_alias_of(arena, version_literal, b"latest"); + } + e_string.data = arena_dup(arena, version_literal).into(); continue; } e_string.data = bun_ast::StoreStr::new(match resolutions[request.package_id as usize].tag { resolution::Tag::Npm => 'npm: { + let installed = request.version.literal.slice(request.version_buf()); + let resolved = resolutions[request.package_id as usize].npm().version; + let string_buf = manager.lockfile.buffers.string_bytes.as_slice(); + // `bun update ` keeps a dist-tag literal as written unless --latest, like the bare path. if manager.subcommand == Subcommand::Update - && (request.version.tag == dependency::Tag::DistTag - || request.version.tag == dependency::Tag::Npm) + && request.version.tag == dependency::Tag::DistTag + && !update_to_latest { - if let Some(entry) = - manager.updating_packages.fetch_swap_remove(request.name) - { - let new_version: Vec = 'new_version: { - let version_fmt = resolutions[request.package_id as usize] - .npm() - .version - .fmt(manager.lockfile.buffers.string_bytes.as_slice()); - if options.exact_versions { - let mut v = Vec::new(); - write!(&mut v, "{}", version_fmt) - .expect("infallible: in-memory write"); - break 'new_version v; - } - - let version_literal: &[u8] = 'version_literal: { - if !entry.value.is_alias { - break 'version_literal &entry - .value - .original_version_literal; - } - if let Some(at_index) = strings::last_index_of_char( - &entry.value.original_version_literal, - b'@', - ) { - break 'version_literal &entry - .value - .original_version_literal[at_index + 1..]; - } - - &entry.value.original_version_literal - }; - - let pinned_version = - semver::Version::which_version_is_pinned(version_literal); - let mut v = Vec::new(); - match pinned_version { - semver::PinnedVersion::Patch => { - write!(&mut v, "{}", version_fmt) - .expect("infallible: in-memory write") - } - semver::PinnedVersion::Minor => { - write!(&mut v, "~{}", version_fmt) - .expect("infallible: in-memory write") - } - semver::PinnedVersion::Major => { - write!(&mut v, "^{}", version_fmt) - .expect("infallible: in-memory write") - } - } - v + break 'npm arena_dup(arena, installed); + } + if manager.subcommand == Subcommand::Update + && matches!( + request.version.tag, + dependency::Tag::DistTag | dependency::Tag::Npm + ) + { + if let Some(entry) = manager.updating_packages.get(request.name) { + let original: &[u8] = &entry.original_version_literal; + let original = match split_npm_alias(installed) { + Some(_) => with_alias_of( + arena, + installed, + split_npm_alias(original) + .map_or(original, |(_, version)| version), + ), + None => original, }; - - if entry.value.is_alias { - let dep_literal = &entry.value.original_version_literal; - - if let Some(at_index) = - strings::last_index_of_char(dep_literal, b'@') - { - let mut v = Vec::new(); - write!( - &mut v, - "{}@{}", - bstr::BStr::new(&dep_literal[0..at_index]), - bstr::BStr::new(&new_version) - ) - .unwrap(); - break 'npm arena_str(arena, &v); - } - } - + let new_version = updated_version_literal( + original, + resolved, + string_buf, + options.exact_versions, + ); break 'npm arena_str(arena, &new_version); } } + // `foo@npm:bar` (no version part) is saved like `foo` would be: `npm:bar@^`. + let bare_alias = split_npm_alias(installed) + .is_some_and(|(_, version)| version.is_empty()); if request.version.tag == dependency::Tag::DistTag + || bare_alias || (manager.subcommand == Subcommand::Update && request.version.tag == dependency::Tag::Npm && !request.version.npm().version.is_exact()) { - let new_version: Vec = { - // `tag == Npm` matched at the top of this arm. - let version_fmt = resolutions[request.package_id as usize] - .npm() - .version - .fmt(request.version_buf()); - let mut v = Vec::new(); - if options.exact_versions { - write!(&mut v, "{}", version_fmt) - .expect("infallible: in-memory write"); - } else { - write!(&mut v, "^{}", version_fmt) - .expect("infallible: in-memory write"); - } - v - }; - - if request.version.tag == dependency::Tag::Npm - && request.version.npm().is_alias - { - let dep_literal = - request.version.literal.slice(request.version_buf()); - if let Some(at_index) = strings::index_of_char(dep_literal, b'@') { - let at_index = at_index as usize; - let mut v = Vec::new(); - write!( - &mut v, - "{}@{}", - bstr::BStr::new(&dep_literal[0..at_index]), - bstr::BStr::new(&new_version) - ) - .unwrap(); - break 'npm arena_str(arena, &v); - } - } - - break 'npm arena_str(arena, &new_version); + let mut new_version = Vec::new(); + write!( + &mut new_version, + "{}{}", + if options.exact_versions { "" } else { "^" }, + resolved.fmt(string_buf) + ) + .expect("infallible: in-memory write"); + break 'npm with_alias_of( + arena, + installed, + arena_str(arena, &new_version), + ); } - arena_dup(arena, request.version.literal.slice(request.version_buf())) + arena_dup(arena, installed) } resolution::Tag::Workspace => b"workspace:*", diff --git a/src/install/PackageManager/PackageManagerEnqueue.rs b/src/install/PackageManager/PackageManagerEnqueue.rs index ca23f272fd55..45c3d5fe89ae 100644 --- a/src/install/PackageManager/PackageManagerEnqueue.rs +++ b/src/install/PackageManager/PackageManagerEnqueue.rs @@ -710,7 +710,7 @@ pub fn enqueue_dependency_with_main_and_success_fn( && (dependency.version.tag != dependency::version::Tag::Npm || !dependency.version.npm().is_alias) { - if let Some(new) = this.lockfile.overrides.get(name_hash) { + if let Some(new) = this.lockfile.overrides.get(&this.lockfile, id, name_hash) { bun_output::scoped_log!( PackageManager, "override: {} -> {}", @@ -2567,6 +2567,12 @@ fn get_or_put_resolved_package( } }; + let find_result = if version_was_replaced { + find_result + } else { + keep_locked_if_ahead(this, dependency, version, manifest, find_result) + }; + // reshaped for borrowck — `manifest`/`find_result` // borrow `this.manifests`; detach via `BackRef` so the `&mut *this` // call can proceed (`this.manifests` is not mutated by the callee). @@ -2812,6 +2818,43 @@ fn get_or_put_resolved_package( } } +/// `bun update --latest` rewrote this row to a dist-tag; a locked version already ahead of the tag (a prerelease) is kept rather than downgraded. +fn keep_locked_if_ahead<'m>( + this: &PackageManager, + dependency: &Dependency, + version: &dependency::Version, + manifest: &'m Npm::PackageManifest, + found: Npm::FindResult<'m>, +) -> Npm::FindResult<'m> { + if version.tag != dependency::version::Tag::DistTag + || !this.to_update + || !this + .options + .do_ + .contains(crate::package_manager::options::Do::UPDATE_TO_LATEST) + { + return found; + } + let Some(entry) = this + .updating_packages + .get(this.lockfile.str(&dependency.name)) + else { + return found; + }; + let Some(locked) = entry.original_version else { + return found; + }; + if found.version.order( + locked, + &manifest.string_buf, + &entry.original_version_string_buf, + ) != core::cmp::Ordering::Less + { + return found; + } + manifest.find_by_version(locked).unwrap_or(found) +} + fn resolution_satisfies_dependency( this: &PackageManager, resolution: &Resolution, diff --git a/src/install/PackageManager/PackageManagerOptions.rs b/src/install/PackageManager/PackageManagerOptions.rs index e40e7a2d5824..c5eb262fb788 100644 --- a/src/install/PackageManager/PackageManagerOptions.rs +++ b/src/install/PackageManager/PackageManagerOptions.rs @@ -34,7 +34,7 @@ pub struct Options { pub dry_run: bool, pub(crate) link_workspace_packages: bool, pub(crate) remote_package_features: Features, - pub(crate) local_package_features: Features, + pub local_package_features: Features, pub(crate) patch_features: PatchFeatures, pub filter_patterns: &'static [&'static [u8]], diff --git a/src/install/PackageManager/UpdateRequest.rs b/src/install/PackageManager/UpdateRequest.rs index 90c124c6fc70..75c102f35c1d 100644 --- a/src/install/PackageManager/UpdateRequest.rs +++ b/src/install/PackageManager/UpdateRequest.rs @@ -77,9 +77,17 @@ impl UpdateRequest { name_hash: PackageNameHash, name: &[u8], ) -> bool { + Self::index_of_name(requests, name_hash, name).is_some() + } + + pub(crate) fn index_of_name( + requests: &[UpdateRequest], + name_hash: PackageNameHash, + name: &[u8], + ) -> Option { requests .iter() - .any(|r| r.name_hash == name_hash && (r.name.is_empty() || r.name == name)) + .position(|r| r.name_hash == name_hash && (r.name.is_empty() || r.name == name)) } /// Borrow the backing string buffer. diff --git a/src/install/PackageManager/add_catalog.rs b/src/install/PackageManager/add_catalog.rs index 621f29a95be6..671d1bcf9cf7 100644 --- a/src/install/PackageManager/add_catalog.rs +++ b/src/install/PackageManager/add_catalog.rs @@ -4,17 +4,15 @@ use bstr::BStr; use bun_alloc::AllocError; use bun_ast::{E, Expr, ExprData, Loc, StoreStr}; use bun_collections::VecExt as _; -use bun_core::{Global, Output, ZStr}; -use bun_semver::ExternalString; -use bun_sys::{Fd, File}; +use bun_core::{Global, Output}; use bun_install::dependency; use bun_install::lockfile::CatalogMap; use bun_install::lockfile::package::PackageColumns as _; -use bun_install::{INVALID_PACKAGE_ID, Lockfile, resolution}; +use bun_install::{INVALID_PACKAGE_ID, Lockfile, PackageID, PackageNameHash, resolution}; use super::update_package_json_and_install::print_package_json_into_cache_entry; -use super::workspace_package_json_cache::{GetJSONOptions, GetResult, MapEntry}; +use super::workspace_package_json_cache::MapEntry; use super::{PackageManager, UpdateRequest}; type ExprDisabler = bun_ast::expr::Disabler; @@ -212,11 +210,15 @@ pub(crate) fn edit_root_entry_before_install( Ok(()) } +/// Runs after `clean_with_logger`: replaces the dist-tag seeds in the root's catalog entries with the resolved range; the lockfile catalog is re-derived from the file by `package_json_write_back`. pub(crate) fn edit_root_after_install( manager: &PackageManager, root_package_json: &Expr, updates: &[UpdateRequest], ) -> Result { + if updates.is_empty() { + return Ok(false); + } let _guard = ExprDisabler::scope(); let name = catalog_name(manager); let Some(mut entries) = entries_object(root_package_json, name, None) else { @@ -224,14 +226,73 @@ pub(crate) fn edit_root_after_install( }; let arena = &manager.ast_arena; + let exact = manager.options.enable.exact_versions(); let lockfile: &Lockfile = &manager.lockfile; - let string_bytes = lockfile.buffers.string_bytes.as_slice(); + let buf = lockfile.buffers.string_bytes.as_slice(); + let resolutions = lockfile.packages.items_resolution(); + + let mut resolved: Vec<(PackageNameHash, PackageID)> = Vec::with_capacity(updates.len()); + for (dep, &pkg_id) in lockfile + .buffers + .dependencies + .iter() + .zip(lockfile.buffers.resolutions.iter()) + { + if dep.version.tag != dependency::Tag::Catalog + || pkg_id == INVALID_PACKAGE_ID + || (pkg_id as usize) >= resolutions.len() + || resolutions[pkg_id as usize].tag != resolution::Tag::Npm + || !updates + .iter() + .any(|request| request.name_hash == dep.name_hash) + || resolved + .iter() + .any(|&(name_hash, _)| name_hash == dep.name_hash) + || !CatalogMap::same_name(dep.version.catalog().slice(buf), name) + { + continue; + } + resolved.push((dep.name_hash, pkg_id)); + if resolved.len() == updates.len() { + break; + } + } + if resolved.is_empty() { + return Ok(false); + } + let mut changed = false; for request in updates { - let Some(dep) = lockfile.catalogs.find(string_bytes, name, request.name) else { + let Some(&(_, pkg_id)) = resolved + .iter() + .find(|&&(name_hash, _)| name_hash == request.name_hash) + else { + continue; + }; + // `request.version` was rebound to the `catalog:` row by `bind_update_requests`, so the dist-tag seed is read back from the lockfile catalog. + let Some(seed) = lockfile.catalogs.find(buf, name, request.name) else { continue; }; - let new_literal = dep.version.literal.slice(string_bytes); + if seed.version.tag != dependency::Tag::DistTag { + continue; + } + let mut literal = Vec::new(); + if seed.version.literal.slice(buf).starts_with(b"npm:") { + write!( + &mut literal, + "npm:{}@", + BStr::new(seed.version.dist_tag().name.slice(buf)) + ) + .expect("infallible: in-memory write"); + } + write!( + &mut literal, + "{}{}", + if exact { "" } else { "^" }, + resolutions[pkg_id as usize].npm().version.fmt(buf) + ) + .expect("infallible: in-memory write"); + let obj = entries .data .e_object_mut() @@ -239,143 +300,11 @@ pub(crate) fn edit_root_after_install( let Some(q) = obj.as_property(request.name) else { continue; }; - if q.expr.as_utf8_string_literal() == Some(new_literal) { + if q.expr.as_utf8_string_literal() == Some(&literal[..]) { continue; } - obj.properties.slice_mut()[q.i as usize].value = Some(estring(arena, new_literal)); + obj.properties.slice_mut()[q.i as usize].value = Some(estring(arena, &literal)); changed = true; } Ok(changed) } - -pub(crate) fn write_root_after_install( - manager: &mut PackageManager, - root_package_json_path: &ZStr, - updates: &[UpdateRequest], -) -> Result<(), crate::Error> { - if updates.is_empty() { - return Ok(()); - } - let entry_ptr: *mut MapEntry = match manager.workspace_package_json_cache.get_with_path( - manager.log_mut(), - root_package_json_path.as_bytes(), - GetJSONOptions { - guess_indentation: true, - ..Default::default() - }, - ) { - GetResult::ParseErr(err) => { - let _ = manager - .log_mut() - .print(std::ptr::from_mut(Output::error_writer())); - Output::err_generic( - "failed to parse package.json \"{s}\": {s}", - (BStr::new(root_package_json_path.as_bytes()), err.name()), - ); - Global::crash(); - } - GetResult::ReadErr(err) => { - Output::err_generic( - "failed to read package.json \"{s}\": {s}", - (BStr::new(root_package_json_path.as_bytes()), err.name()), - ); - Global::crash(); - } - GetResult::Entry(entry) => core::ptr::from_mut(entry), - }; - // SAFETY: the cache is not touched again while `entry` is live; `edit_root_after_install` only reads disjoint manager fields. - let entry: &mut MapEntry = unsafe { &mut *entry_ptr }; - - let root = entry.root; - if edit_root_after_install(&*manager, &root, updates)? { - print_package_json_into_cache_entry(entry, root); - } - - let file = File::openat(Fd::cwd(), root_package_json_path, bun_sys::O::RDWR, 0) - .map_err(crate::Error::from)?; - file.pwrite_all(&entry.source.contents, 0) - .map_err(crate::Error::from)?; - let _ = bun_sys::ftruncate(file.handle, entry.source.contents.len() as i64); - let _ = file.close(); - Ok(()) -} - -pub(crate) fn rewrite_lockfile_entries( - lockfile: &mut Lockfile, - manager: &mut PackageManager, - updates: &[UpdateRequest], -) -> crate::Result<()> { - let name = catalog_name(manager); - let exact = manager.options.enable.exact_versions(); - - let mut rewrites: Vec<(&[u8], Vec)> = Vec::new(); - { - let buf = lockfile.buffers.string_bytes.as_slice(); - let resolutions = lockfile.packages.items_resolution(); - for request in updates { - if request.version.tag != dependency::Tag::DistTag { - continue; - } - let found = lockfile - .buffers - .dependencies - .iter() - .zip(lockfile.buffers.resolutions.iter()) - .find(|&(dep, &pkg_id)| { - dep.version.tag == dependency::Tag::Catalog - && dep.name_hash == request.name_hash - && CatalogMap::same_name(dep.version.catalog().slice(buf), name) - && pkg_id != INVALID_PACKAGE_ID - && (pkg_id as usize) < resolutions.len() - && resolutions[pkg_id as usize].tag == resolution::Tag::Npm - }); - let Some((_, &pkg_id)) = found else { - continue; - }; - let version = resolutions[pkg_id as usize].npm().version.fmt(buf); - let request_literal = request.version.literal.slice(request.version_buf()); - let mut literal = Vec::new(); - if request_literal.starts_with(b"npm:") { - write!( - &mut literal, - "npm:{}@", - BStr::new(request.version.dist_tag().name.slice(request.version_buf())) - ) - .expect("infallible: in-memory write"); - } - write!(&mut literal, "{}{}", if exact { "" } else { "^" }, version) - .expect("infallible: in-memory write"); - rewrites.push((request.name, literal)); - } - } - if rewrites.is_empty() { - return Ok(()); - } - - let (mut builder, lf) = lockfile.string_builder_split(); - for (_, literal) in &rewrites { - builder.count(literal); - } - builder.allocate()?; - for (dep_name, literal) in &rewrites { - let external = builder.append::(literal); - let string_bytes = builder.string_bytes.as_slice(); - let sliced = external.value.sliced(string_bytes); - let Some(entry) = lf.catalogs.find_mut(string_bytes, name, dep_name) else { - continue; - }; - let (entry_name, entry_name_hash) = (entry.name, entry.name_hash); - if let Some(version) = dependency::parse( - entry_name, - entry_name_hash, - sliced.slice, - &sliced, - None, - &mut *manager, - ) { - entry.version = version; - } - } - builder.clamp(); - Ok(()) -} diff --git a/src/install/PackageManager/add_remove_with_filter.rs b/src/install/PackageManager/add_remove_with_filter.rs index 9e55dbb74e0d..73f4ec958dea 100644 --- a/src/install/PackageManager/add_remove_with_filter.rs +++ b/src/install/PackageManager/add_remove_with_filter.rs @@ -3,7 +3,7 @@ use bstr::BStr; use crate::Error; use crate::bun_fs::FileSystem; use crate::lockfile_real::package::value_loc_of; -use crate::lockfile_real::package::workspace_map::{NamesArray, WorkspaceMap}; +use crate::lockfile_real::package::workspace_map::{MissingWorkspace, NamesArray, WorkspaceMap}; use bun_core::{Global, Output, strings}; use bun_install::dependency; use bun_install::{Lockfile, PackageID, PackageNameHash}; @@ -15,6 +15,7 @@ use super::add_catalog; use super::install_with_manager::install_with_manager; use super::options::Do; use super::package_json_editor::{self as PackageJSONEditor, EditOptions}; +use super::package_json_write_back; use super::update_package_json_and_install::{ print_package_json_into_cache_entry, remove_dependencies_from_package_json, remove_leftover_node_modules, @@ -22,6 +23,7 @@ use super::update_package_json_and_install::{ use super::workspace_package_json_cache::{GetJSONOptions, GetResult, MapEntry}; use super::{Command, PackageManager, Subcommand, UpdateRequest, WorkspaceFilter}; +#[derive(Clone)] pub(crate) struct WorkspaceTarget { pub(crate) name: Box<[u8]>, /// `None` = the workspace root. @@ -29,7 +31,7 @@ pub(crate) struct WorkspaceTarget { pub(crate) package_json_path: Box<[u8]>, } -fn root_package_json_path() -> Box<[u8]> { +pub(crate) fn root_package_json_path() -> Box<[u8]> { let top_level = strings::without_trailing_slash(FileSystem::instance().top_level_dir()); let mut buf = path_buffer_pool::get(); let path: Box<[u8]> = @@ -119,7 +121,7 @@ pub(crate) fn select_targets( &root_source, loc, None, - false, + MissingWorkspace::Error, ) { if log.has_errors() { let _ = log.print(std::ptr::from_mut(Output::error_writer())); @@ -244,7 +246,10 @@ fn quote_patterns(patterns: &[&[u8]]) -> Vec { out } -fn fetch_entry<'a>(manager: &'a mut PackageManager, target: &WorkspaceTarget) -> &'a mut MapEntry { +pub(crate) fn fetch_entry<'a>( + manager: &'a mut PackageManager, + target: &WorkspaceTarget, +) -> &'a mut MapEntry { let log = manager.log_mut(); match manager.workspace_package_json_cache.get_with_path( log, @@ -265,28 +270,28 @@ fn fetch_entry<'a>(manager: &'a mut PackageManager, target: &WorkspaceTarget) -> } } -fn fetch_entry_root(manager: &mut PackageManager, target: &WorkspaceTarget) -> bun_ast::Expr { +pub(crate) fn fetch_entry_root( + manager: &mut PackageManager, + target: &WorkspaceTarget, +) -> bun_ast::Expr { fetch_entry(manager, target).root } -fn store_entry( +pub(crate) fn store_entry( manager: &mut PackageManager, target: &WorkspaceTarget, root: bun_ast::Expr, - reparse: bool, ) { let log = manager.log_mut(); let entry = fetch_entry(manager, target); print_package_json_into_cache_entry(entry, root); - if reparse { - if let Err(err) = entry.reparse_root(log) { - bun_core::pretty_errorln!("package.json failed to parse due to error {}", err.name()); - Global::crash(); - } + if let Err(err) = entry.reparse_root(log) { + bun_core::pretty_errorln!("package.json failed to parse due to error {}", err.name()); + Global::crash(); } } -fn write_target(manager: &mut PackageManager, target: &WorkspaceTarget) -> bool { +pub(crate) fn write_target(manager: &mut PackageManager, target: &WorkspaceTarget) -> bool { let entry = fetch_entry(manager, target); let mut zbuf = path_buffer_pool::get(); let path = resolve_path::z(&target.package_json_path, &mut zbuf); @@ -302,7 +307,7 @@ fn write_target(manager: &mut PackageManager, target: &WorkspaceTarget) -> bool } } -fn reset_e_strings(updates: &mut [UpdateRequest]) { +pub(crate) fn reset_e_strings(updates: &mut [UpdateRequest]) { // `e_string` points into the previous target's AST; `edit` skips requests that already have one. for request in updates.iter_mut() { request.e_string = None; @@ -439,28 +444,13 @@ fn assign_requests( (requests, assigned) } -/// The targets whose package.json changed and, for `add`, the requests each one received. +/// The `add`/`link --filter` targets and the requests each one received; edited again once resolved. pub(crate) struct PendingWrite { targets: Vec<(WorkspaceTarget, Box<[PackageNameHash]>)>, - subcommand: Subcommand, catalog_mode: bool, root_target: WorkspaceTarget, } -/// Runs once: from `install_with_manager` right after the lockfile is saved, else after it returns. -pub(crate) fn flush_pending_write(manager: &mut PackageManager) -> Result<(), Error> { - let Some(pending) = manager.pending_filtered_write.take() else { - return Ok(()); - }; - if !manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) { - return Ok(()); - } - let mut updates: Box<[UpdateRequest]> = core::mem::take(&mut manager.update_requests); - let result = pending.write(manager, &mut updates); - manager.update_requests = updates; - result -} - impl PendingWrite { /// Package ids of the targets that received `request`; `clean_with_logger` resolves it from these. pub(crate) fn workspace_ids_receiving( @@ -478,34 +468,18 @@ impl PendingWrite { .collect() } - fn write( + /// Writes the resolved versions into every target's cache entry; `flush` puts them on disk. + pub(crate) fn edit_entries( &self, manager: &mut PackageManager, updates: &mut [UpdateRequest], ) -> Result<(), Error> { - let mut any_failed = false; - if self.subcommand == Subcommand::Remove { - for (target, _) in &self.targets { - any_failed |= !write_target(manager, target); - } - if any_failed { - Global::exit(1); - } - return Ok(()); - } - let dependency_list: &'static [u8] = manager.options.update.prop; let exact_versions = manager.options.enable.exact_versions(); - let add_trusted_dependencies = manager - .options - .do_ - .contains(Do::TRUST_DEPENDENCIES_FROM_ARGS); - let trusted_snapshot = manager.trusted_deps_to_add_to_package_json.clone(); let summary_order: Vec = updates.iter().map(|r| r.name_hash).collect(); for (target, received) in &self.targets { let kept = move_to_front(updates, received); - manager.trusted_deps_to_add_to_package_json = trusted_snapshot.clone(); let mut root = fetch_entry_root(manager, target); reset_e_strings(updates); let mut slice: &mut [UpdateRequest] = &mut updates[..kept]; @@ -516,28 +490,21 @@ impl PendingWrite { dependency_list, EditOptions { exact_versions, - add_trusted_dependencies, ..Default::default() }, )?; if self.catalog_mode { add_catalog::rewrite_references(manager, &updates[..kept]); } - let is_catalog_root = self.catalog_mode && target.name_hash.is_none(); - store_entry(manager, target, root, is_catalog_root); - if !is_catalog_root { - any_failed |= !write_target(manager, target); - } + store_entry(manager, target, root); } updates.sort_by_key(|r| summary_order.iter().position(|&h| h == r.name_hash)); - if any_failed { - Global::exit(1); - } + if self.catalog_mode { - let mut zbuf = path_buffer_pool::get(); - let root_package_json_path = - resolve_path::z(&self.root_target.package_json_path, &mut zbuf); - add_catalog::write_root_after_install(manager, root_package_json_path, updates)?; + let root = fetch_entry_root(manager, &self.root_target); + if add_catalog::edit_root_after_install(manager, &root, updates)? { + store_entry(manager, &self.root_target, root); + } } Ok(()) } @@ -607,9 +574,12 @@ pub(super) fn update_filtered_workspaces_and_install( } updates[..kept].iter().map(|r| r.name_hash).collect() }; - store_entry(manager, &target, root, true); + store_entry(manager, &target, root); changed.push((target, received)); } + for (target, _) in &changed { + package_json_write_back::record(manager, target.clone(), subcommand != Subcommand::Remove); + } let any_changed = !changed.is_empty(); if subcommand != Subcommand::Remove { @@ -619,33 +589,35 @@ pub(super) fn update_filtered_workspaces_and_install( .any(|(_, received)| received.contains(&r.name_hash)) }); } - if catalog_mode { + if catalog_mode && !updates.is_empty() { let root = fetch_entry_root(manager, &root_target); add_catalog::edit_root_before_install(manager, &root, &updates)?; - store_entry(manager, &root_target, root, true); + store_entry(manager, &root_target, root); + package_json_write_back::record(manager, root_target.clone(), false); } // The install summary is printed from this workspace's point of view. let summary_target = changed .iter() .find(|(_, received)| received.len() == updates.len()) - .or(changed.first()); + .or_else(|| changed.first()); manager.workspace_name_hash = summary_target.and_then(|(target, _)| target.name_hash); manager.to_update = false; manager.update_requests = updates.into_boxed_slice(); - manager.pending_filtered_write = Some(Box::new(PendingWrite { - targets: changed, - subcommand, - catalog_mode, - root_target, - })); + if subcommand != Subcommand::Remove { + manager.pending_filtered_write = Some(Box::new(PendingWrite { + targets: changed, + catalog_mode, + root_target, + })); + } { let mut zbuf = path_buffer_pool::get(); let root_package_json_path = resolve_path::z(&root_package_json_path, &mut zbuf); install_with_manager(manager, ctx, root_package_json_path, original_cwd)?; } - flush_pending_write(manager)?; + package_json_write_back::flush(manager)?; if subcommand == Subcommand::Remove && manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) { if !any_changed { diff --git a/src/install/PackageManager/install_with_manager.rs b/src/install/PackageManager/install_with_manager.rs index 4535882fcc02..ecf4319f319a 100644 --- a/src/install/PackageManager/install_with_manager.rs +++ b/src/install/PackageManager/install_with_manager.rs @@ -13,6 +13,7 @@ use crate::Subcommand; use crate::dependency::{DependencyExt as _, Tag as DependencyVersionTag}; use crate::lockfile::{self, Lockfile}; use crate::resolution::Tag as ResolutionTag; +use crate::update_transitive::{DirectDependencies, TransitiveUpdate, redirect_moved_edges}; use crate::{ Dependency, DependencyID, Features, PackageID, PackageNameHash, PatchTask, Resolution, invalid_package_id, @@ -113,9 +114,19 @@ pub fn install_with_manager( crate::dedupe::dedupe_before_install(manager, &load_result)?; } + let transitive = if manager.subcommand == Subcommand::Update + && manager.update_requests.is_empty() + && matches!(load_result, lockfile::LoadResult::Ok { .. }) + { + TransitiveUpdate::plan(manager)? + } else { + TransitiveUpdate::default() + }; + // this defaults to false // but we force allowing updates to the lockfile when you do bun add let mut had_any_diffs = false; + let mut direct_deps_before = DirectDependencies::default(); manager.progress = Default::default(); match &load_result { @@ -256,6 +267,9 @@ pub fn install_with_manager( }; had_any_diffs = manager.summary.has_diffs(); + if manager.summary.changes_resolutions() { + direct_deps_before = DirectDependencies::snapshot(&manager.lockfile); + } // Split-borrow `manager.lockfile` so the `StringBuilder` // (which owns `buffers.string_bytes` + `string_pool`) and the @@ -317,29 +331,15 @@ pub fn install_with_manager( lockfile::PackageIDSlice::new(off, len); builder.allocate()?; - let all_name_hashes: Vec = 'brk: { - if !summary.overrides_changed { - break 'brk Vec::new(); - } - let hashes_len = lf.overrides.map.len() + lockfile.overrides.map.len(); - if hashes_len == 0 { - break 'brk Vec::new(); - } - let mut all_name_hashes: Vec = - Vec::with_capacity(hashes_len); - all_name_hashes.extend_from_slice(lf.overrides.map.keys()); - all_name_hashes.extend_from_slice(lockfile.overrides.map.keys()); - let mut i = lf.overrides.map.len(); - while i < all_name_hashes.len() { - if all_name_hashes[..i].contains(&all_name_hashes[i]) { - let last = all_name_hashes.len() - 1; - all_name_hashes[i] = all_name_hashes[last]; - all_name_hashes.truncate(last); - } else { - i += 1; - } - } - break 'brk all_name_hashes; + let all_name_hashes: Vec = if !summary.overrides_changed { + Vec::new() + } else { + let mut v = Vec::new(); + lf.overrides.append_overridden_name_hashes(&mut v); + lockfile.overrides.append_overridden_name_hashes(&mut v); + v.sort_unstable(); + v.dedup(); + v }; *lf.overrides = lockfile.overrides.clone( @@ -491,7 +491,7 @@ pub fn install_with_manager( for dependency_i in 0..dependencies_len { let dependency = manager.lockfile.buffers.dependencies[dependency_i].clone(); - if all_name_hashes.contains(&dependency.name_hash) { + if all_name_hashes.binary_search(&dependency.name_hash).is_ok() { manager.lockfile.buffers.resolutions[dependency_i] = invalid_package_id; if let Err(err) = enqueue_dependency_with_main( @@ -508,12 +508,22 @@ pub fn install_with_manager( } if manager.summary.catalogs_changed { + let mut catalog_overridden: Vec = Vec::new(); + manager + .lockfile + .overrides + .append_catalog_valued_name_hashes(&mut catalog_overridden); + catalog_overridden.sort_unstable(); + catalog_overridden.dedup(); let dependencies_len = manager.lockfile.buffers.dependencies.len(); for _dep_id in 0..dependencies_len { let dep_id: DependencyID = u32::try_from(_dep_id).expect("int cast"); let dep = manager.lockfile.buffers.dependencies[dep_id as usize].clone(); - if dep.version.tag != DependencyVersionTag::Catalog { + if dep.version.tag != DependencyVersionTag::Catalog + && (catalog_overridden.is_empty() + || catalog_overridden.binary_search(&dep.name_hash).is_err()) + { continue; } @@ -531,34 +541,6 @@ pub fn install_with_manager( } } - // `bun update `: drop and re-enqueue every `` slot, not just root-level. - if manager.to_update && !manager.update_requests.is_empty() { - let dependencies_len = manager.lockfile.buffers.dependencies.len(); - for dependency_i in 0..dependencies_len { - let dependency = - manager.lockfile.buffers.dependencies[dependency_i].clone(); - if UpdateRequest::contains_name( - &manager.update_requests, - dependency.name_hash, - dependency - .name - .slice(manager.lockfile.buffers.string_bytes.as_slice()), - ) { - manager.lockfile.buffers.resolutions[dependency_i] = - invalid_package_id; - if let Err(err) = enqueue_dependency_with_main( - manager, - dependency_i as u32, - &dependency, - invalid_package_id, - false, - ) { - add_dependency_error(manager, &dependency, err); - } - } - } - } - // Split this into two passes because the below may allocate memory or invalidate pointers if manager.summary.add > 0 || manager.summary.update > 0 { let changes = mapping.len() as PackageID; @@ -598,11 +580,22 @@ pub fn install_with_manager( _ => {} } - if !manager.audit_fix_pins.is_empty() && !needs_new_lockfile { - crate::audit_fix::enqueue_planned_fixes(manager)?; + let named_update = manager.to_update && !manager.update_requests.is_empty(); + let mut named_moves: Vec<(DependencyID, PackageID)> = Vec::new(); + if !needs_new_lockfile { + if named_update { + named_moves = enqueue_named_updates(manager); + } + transitive.enqueue(manager)?; + if !manager.audit_fix_pins.is_empty() { + crate::audit_fix::enqueue_planned_fixes(manager)?; + } } if needs_new_lockfile { + if named_update { + reject_unknown_update_requests(manager, |_, request| request.e_string.is_none()); + } root = create_new_lockfile_and_enqueue( manager, &load_result, @@ -625,6 +618,10 @@ pub fn install_with_manager( resolve_pending_tasks(manager, &root, log_level)?; } + direct_deps_before.redirect_dependents(&mut manager.lockfile); + transitive.redirect_dependents(&mut manager.lockfile); + redirect_moved_edges(&mut manager.lockfile, &named_moves); + let had_errors_before_cleaning_lockfile = manager.log_mut().has_errors(); manager .log_mut() @@ -644,13 +641,11 @@ pub fn install_with_manager( // reborrows under one tag (PORTING.md §Aliasing-split-borrow). unsafe { let log = (*mgr).log; - let exact_versions = (*mgr).options.enable.exact_versions(); Lockfile::clean_with_logger( &mut (*mgr).lockfile, &mut *mgr, &mut (*mgr).update_requests, &mut *log, - exact_versions, log_level, )? } @@ -674,6 +669,8 @@ pub fn install_with_manager( if manager.options.security_scanner.is_some() { run_security_scanner(manager, ctx, original_cwd); } + + super::package_json_write_back::edit_after_resolve(manager)?; } // append scripts to lockfile before generating new metahash @@ -875,9 +872,7 @@ pub fn install_with_manager( // It's unnecessary work to re-save the lockfile if there are no changes. // A loaded text lockfile is never re-saved just to bump its version: an // existing `bun.lock` keeps the version it was written with. - let should_save_lockfile = (matches!(load_result, lockfile::LoadResult::Ok { .. }) - && load_result.ok().format == lockfile::Format::Binary - && save_format == lockfile::Format::Text) + let should_save_lockfile = saves_migrated_lockfile(&load_result, save_format) // check `save_lockfile` after checking if loaded from binary and save format is text // because `save_lockfile` is set to false for `--frozen-lockfile` || (manager.options.do_.save_lockfile() @@ -903,7 +898,7 @@ pub fn install_with_manager( } // Before root lifecycle scripts, which exit the process on failure. - super::add_remove_with_filter::flush_pending_write(manager)?; + super::package_json_write_back::flush(manager)?; if needs_new_lockfile { manager.summary.add = manager.lockfile.packages.len() as u32; @@ -1452,6 +1447,101 @@ fn report_lockfile_load_error( Ok(()) } +/// `bun update …`: every row resolving to `` (by alias or real package name), at any depth, re-resolves within its own range; rows the differ already re-enqueued are left to it, and peer/bundled rows only follow the moved package via `redirect_moved_edges`, as in the bare update. +#[cold] +#[inline(never)] +fn enqueue_named_updates(manager: &mut PackageManager) -> Vec<(DependencyID, PackageID)> { + let mut matched = vec![false; manager.update_requests.len()]; + let mut moved: Vec<(DependencyID, PackageID)> = Vec::new(); + let dependencies_len = manager.lockfile.buffers.dependencies.len(); + for dependency_i in 0..dependencies_len { + let dependency = manager.lockfile.buffers.dependencies[dependency_i].clone(); + let package_id = manager.lockfile.buffers.resolutions[dependency_i]; + let Some(request) = index_of_named_update(manager, &dependency, package_id) else { + continue; + }; + matched[request] = true; + if package_id == invalid_package_id + || dependency.behavior.is_peer() + || dependency.behavior.is_bundled() + { + continue; + } + manager.lockfile.buffers.resolutions[dependency_i] = invalid_package_id; + moved.push((dependency_i as DependencyID, package_id)); + if let Err(err) = enqueue_dependency_with_main( + manager, + dependency_i as DependencyID, + &dependency, + invalid_package_id, + false, + ) { + add_dependency_error(manager, &dependency, err); + } + } + + reject_unknown_update_requests(manager, |i, _| !matched[i]); + moved +} + +fn index_of_named_update( + manager: &PackageManager, + dependency: &Dependency, + package_id: PackageID, +) -> Option { + let requests = &manager.update_requests; + let buf = manager.lockfile.buffers.string_bytes.as_slice(); + if let Some(i) = + UpdateRequest::index_of_name(requests, dependency.name_hash, dependency.name.slice(buf)) + { + return Some(i); + } + if package_id != invalid_package_id { + let name_hash = manager.lockfile.packages.items_name_hash()[package_id as usize]; + if name_hash == dependency.name_hash { + return None; + } + let name = manager.lockfile.packages.items_name()[package_id as usize].slice(buf); + return UpdateRequest::index_of_name(requests, name_hash, name); + } + let realname = dependency.realname(); + if realname.eql(dependency.name, buf, buf) { + return None; + } + let realname = realname.slice(buf); + UpdateRequest::index_of_name( + requests, + bun_semver::string::Builder::string_hash(realname), + realname, + ) +} + +#[cold] +#[inline(never)] +fn reject_unknown_update_requests( + manager: &PackageManager, + is_unknown: impl Fn(usize, &UpdateRequest) -> bool, +) { + let unknown: Vec<&UpdateRequest> = manager + .update_requests + .iter() + .enumerate() + .filter_map(|(i, request)| is_unknown(i, request).then_some(request)) + .collect(); + if unknown.is_empty() { + return; + } + for request in unknown { + Output::err_generic( + "\"{}\" is not in the lockfile, so there is nothing to update", + (bstr::BStr::new(request.get_name()),), + ); + } + bun_core::note!("to add a dependency, use bun add"); + Output::flush(); + Global::exit(1); +} + #[cold] #[inline(never)] fn record_updating_package_versions(manager: &mut PackageManager) { @@ -1758,6 +1848,19 @@ fn run_security_scanner(manager: &mut PackageManager, ctx: Command::Context, ori } } +// bun.lockb / package-lock.json / yarn.lock / pnpm-lock.yaml -> bun.lock is written even under --frozen-lockfile. +fn saves_migrated_lockfile( + load_result: &lockfile::LoadResult, + save_format: lockfile::Format, +) -> bool { + save_format == lockfile::Format::Text + && matches!( + load_result, + lockfile::LoadResult::Ok(ok) + if ok.format == lockfile::Format::Binary || ok.migrated != lockfile::Migrated::None + ) +} + #[cold] #[inline(never)] #[allow(clippy::too_many_arguments)] @@ -1771,9 +1874,9 @@ fn save_lockfile_only( packages_len_before_install: usize, log_level: Options::LogLevel, ) -> crate::Result<()> { - let migrating_to_text = - load_result.loaded_from_binary_lockfile() && save_format == lockfile::Format::Text; - if manager.options.enable.frozen_lockfile() && !migrating_to_text { + if manager.options.enable.frozen_lockfile() + && !saves_migrated_lockfile(load_result, save_format) + { Output::flush(); return Ok(()); } diff --git a/src/install/PackageManager/package_json_write_back.rs b/src/install/PackageManager/package_json_write_back.rs new file mode 100644 index 000000000000..610a726ec073 --- /dev/null +++ b/src/install/PackageManager/package_json_write_back.rs @@ -0,0 +1,430 @@ +use bun_core::{Global, strings}; +use bun_paths::path_buffer_pool; +use bun_paths::resolve_path::{join_abs_string_buf, platform}; + +use crate::bun_fs::FileSystem; +use crate::dependency::DependencyExt as _; +use crate::lockfile::package::PackageColumns as _; +use crate::lockfile::{Lockfile, Package}; +use crate::resolution::Tag as ResolutionTag; +use crate::{Dependency, Features, PackageID, PackageNameHash, invalid_package_id}; + +use super::add_catalog; +use super::add_remove_with_filter::{ + WorkspaceTarget, fetch_entry, fetch_entry_root, root_package_json_path, store_entry, + write_target, +}; +use super::options::Do; +use super::package_json_editor::{self as PackageJSONEditor, EditOptions}; +use super::update_package_json_and_install::print_package_json_into_cache_entry; +use super::{PackageManager, Subcommand, UpdateRequest}; + +/// A package.json whose cache entry differs from disk; `target.name_hash == None` is the root. +pub(crate) struct EditedPackageJson { + pub(crate) target: WorkspaceTarget, + /// The command's positionals were applied to this file's dependency lists. + pub(crate) received_requests: bool, +} + +fn push(edited: &mut Vec, target: WorkspaceTarget, received_requests: bool) { + match edited + .iter_mut() + .find(|e| e.target.name_hash == target.name_hash) + { + Some(existing) => existing.received_requests |= received_requests, + None => edited.push(EditedPackageJson { + target, + received_requests, + }), + } +} + +pub(crate) fn record( + manager: &mut PackageManager, + target: WorkspaceTarget, + received_requests: bool, +) { + push(&mut manager.edited_package_jsons, target, received_requests); +} + +fn root_target() -> WorkspaceTarget { + WorkspaceTarget { + name: Box::default(), + name_hash: None, + package_json_path: root_package_json_path(), + } +} + +/// Phase 1 (before bun.lock is saved): write the resolved versions into the edited package.json entries and re-derive bun.lock's declared columns from them. +#[inline] +pub(crate) fn edit_after_resolve(manager: &mut PackageManager) -> crate::Result<()> { + if manager.pending_filtered_write.is_none() + && manager.update_target_workspaces.is_none() + && !manager + .edited_package_jsons + .iter() + .any(|e| e.received_requests) + { + return Ok(()); + } + edit_after_resolve_slow(manager) +} + +#[inline(never)] +fn edit_after_resolve_slow(manager: &mut PackageManager) -> crate::Result<()> { + let mut edited: Vec = core::mem::take(&mut manager.edited_package_jsons); + let mut updates: Box<[UpdateRequest]> = core::mem::take(&mut manager.update_requests); + let exact = manager.options.enable.exact_versions(); + let cwd = edited.iter().position(|e| e.received_requests); + + let result = if let Some(pending) = manager.pending_filtered_write.take() { + let result = pending.edit_entries(manager, &mut updates); + manager.pending_filtered_write = Some(pending); + result + } else if let Some(targets) = manager.update_target_workspaces.take() { + let result = edit_update_targets(manager, &targets, &mut edited, exact); + manager.update_target_workspaces = Some(targets); + result + } else if let Some(i) = cwd { + edit_cwd(manager, &mut edited, i, &mut updates, exact) + } else { + Ok(()) + } + .and_then(|()| sync_lockfile(manager, &edited)); + + if result.is_ok() && !updates.is_empty() { + manager.lockfile.bind_update_requests( + manager.pending_filtered_write.as_deref(), + manager.workspace_name_hash, + &mut updates, + ); + } + manager.update_requests = updates; + manager.edited_package_jsons = edited; + result +} + +/// `bun update -r` / `bun update --filter`: every targeted root/workspace package.json. +fn edit_update_targets( + manager: &mut PackageManager, + targets: &[super::UpdateTargetWorkspace], + edited: &mut Vec, + exact: bool, +) -> crate::Result<()> { + let top_level = strings::without_trailing_slash(FileSystem::instance().top_level_dir()); + let mut selected: Vec = Vec::new(); + { + let lockfile: &Lockfile = &manager.lockfile; + let buf = lockfile.buffers.string_bytes.as_slice(); + let resolutions = lockfile.packages.items_resolution(); + let name_hashes = lockfile.packages.items_name_hash(); + let names = lockfile.packages.items_name(); + let mut path_buf = path_buffer_pool::get(); + for pkg_id in 0..resolutions.len() { + let res = resolutions[pkg_id]; + let (name_hash, rel): (Option, &[u8]) = match res.tag { + ResolutionTag::Root => (None, b""), + ResolutionTag::Workspace => (Some(name_hashes[pkg_id]), res.workspace().slice(buf)), + _ => continue, + }; + let name = names[pkg_id].slice(buf); + if !targets + .iter() + .any(|t| t.matches(name_hash.is_none(), name_hashes[pkg_id], name)) + { + continue; + } + selected.push(WorkspaceTarget { + name: Box::from(name), + name_hash, + package_json_path: join_abs_string_buf::( + top_level, + &mut path_buf.0, + &[rel, b"package.json"], + ) + .into(), + }); + } + } + + let update_to_latest = manager.options.do_.contains(Do::UPDATE_TO_LATEST); + for target in selected { + let mut ast = fetch_entry_root(manager, &target); + let mut updating = bun_collections::StringArrayHashMap::default(); + for options in [ + EditOptions { + exact_versions: true, + before_install: true, + }, + EditOptions { + exact_versions: exact, + ..Default::default() + }, + ] { + PackageJSONEditor::edit_update_no_args_in( + &manager.lockfile, + &manager.ast_arena, + &mut updating, + target.name_hash, + update_to_latest, + &mut ast, + options, + )?; + } + store_entry(manager, &target, ast); + push(edited, target, false); + } + + if !manager.updating_catalogs.is_empty() { + let root = root_target(); + let root_ast = fetch_entry_root(manager, &root); + if PackageJSONEditor::edit_catalogs_after_update(manager, &root_ast)? { + store_entry(manager, &root, root_ast); + push(edited, root, false); + } + } + Ok(()) +} + +/// `bun add` / `bun update [names]` / `bun link` in the cwd's package.json, plus the root when its catalogs changed. +fn edit_cwd( + manager: &mut PackageManager, + edited: &mut Vec, + cwd_index: usize, + updates: &mut [UpdateRequest], + exact: bool, +) -> crate::Result<()> { + let cwd_target = edited[cwd_index].target.clone(); + let in_root = cwd_target.name_hash.is_none(); + let options = EditOptions { + exact_versions: exact, + ..Default::default() + }; + let mut ast = fetch_entry_root(manager, &cwd_target); + if updates.is_empty() { + if manager.subcommand == Subcommand::Update { + PackageJSONEditor::edit_update_no_args(manager, &mut ast, options)?; + } + if in_root && !manager.updating_catalogs.is_empty() { + PackageJSONEditor::edit_catalogs_after_update(manager, &ast)?; + } + } else { + let dependency_list: &'static [u8] = manager.options.update.prop; + let mut slice: &mut [UpdateRequest] = &mut updates[..]; + PackageJSONEditor::edit(manager, &mut slice, &mut ast, dependency_list, options)?; + } + let catalog_mode = manager.options.add_catalog.is_some(); + if catalog_mode { + add_catalog::rewrite_references(manager, updates); + if in_root { + add_catalog::edit_root_after_install(manager, &ast, updates)?; + } + } + store_entry(manager, &cwd_target, ast); + if in_root { + return Ok(()); + } + + let root = root_target(); + let root_ast = fetch_entry_root(manager, &root); + let mut changed = catalog_mode && !updates.is_empty(); + if !manager.updating_catalogs.is_empty() { + changed |= PackageJSONEditor::edit_catalogs_after_update(manager, &root_ast)?; + } + if catalog_mode { + add_catalog::edit_root_after_install(manager, &root_ast, updates)?; + } + if changed { + store_entry(manager, &root, root_ast); + push(edited, root, false); + } + Ok(()) +} + +/// Package id of each edited file's row in `lockfile` (`invalid_package_id` when the workspace has none). +fn target_package_ids(lockfile: &Lockfile, edited: &[EditedPackageJson]) -> Vec { + let mut ids = vec![invalid_package_id; edited.len()]; + for (i, e) in edited.iter().enumerate() { + if e.target.name_hash.is_none() { + ids[i] = 0; + } + } + let resolutions = lockfile.packages.items_resolution(); + let name_hashes = lockfile.packages.items_name_hash(); + for pkg_id in 0..resolutions.len() { + if resolutions[pkg_id].tag != ResolutionTag::Workspace { + continue; + } + for (i, e) in edited.iter().enumerate() { + if ids[i] == invalid_package_id && e.target.name_hash == Some(name_hashes[pkg_id]) { + ids[i] = pkg_id as PackageID; + } + } + } + ids +} + +/// Re-parses the edited files the way `bun install` would and copies every declared literal that differs (and, for the root, `overrides` + `catalogs`) into `manager.lockfile`, so the next install's differ sees no change. +fn sync_lockfile(manager: &mut PackageManager, edited: &[EditedPackageJson]) -> crate::Result<()> { + let mut scratch = Lockfile::default(); + let mut log = bun_ast::Log::init(); + let mut resolver: () = (); + + // Always parsed: it fills `scratch.workspace_paths`, which `workspace:` rows in every file resolve through. + let (root_source, root_json) = { + let entry = fetch_entry(manager, &root_target()); + (entry.source.clone(), entry.root) + }; + let mut root_pkg = Package::default(); + root_pkg.parse_with_json::<()>( + &mut scratch, + manager, + &mut log, + &root_source, + root_json, + &mut resolver, + Features::main(), + )?; + + let mut root_pkg = Some(root_pkg); + let mut parsed: Vec<(usize, Package)> = Vec::with_capacity(edited.len()); + for (i, e) in edited.iter().enumerate() { + if e.target.name_hash.is_none() { + parsed.extend(root_pkg.take().map(|pkg| (i, pkg))); + continue; + } + let (source, json) = { + let entry = fetch_entry(manager, &e.target); + (bun_ptr::ParentRef::new(&entry.source), entry.root) + }; + let mut pkg = Package::default(); + // Unlike the root's workspaces walk, a `Features::WORKSPACE` parse never grows the cache, so the entry stays put. + pkg.parse_with_json::<()>( + &mut scratch, + manager, + &mut log, + source.get(), + json, + &mut resolver, + Features::WORKSPACE, + )?; + parsed.push((i, pkg)); + } + + let target_ids = target_package_ids(&manager.lockfile, edited); + let sbuf = scratch.buffers.string_bytes.as_slice(); + for (i, pkg) in &parsed { + let target_id = target_ids[*i]; + if target_id == invalid_package_id { + continue; + } + let is_root = edited[*i].target.name_hash.is_none(); + let row = manager.lockfile.packages.items_dependencies()[target_id as usize]; + let scratch_deps = pkg + .dependencies + .get(scratch.buffers.dependencies.as_slice()); + + let changed: Vec<(usize, usize)> = { + let lbuf = manager.lockfile.buffers.string_bytes.as_slice(); + let row_deps = row.get(manager.lockfile.buffers.dependencies.as_slice()); + // Empty while every scratch dep so far matched the row at its own index. + let mut claimed: Vec = Vec::new(); + let mut changed = Vec::new(); + for (si, s) in scratch_deps.iter().enumerate() { + let same_index = si < row_deps.len() + && claimed.get(si).is_none_or(|&taken| !taken) + && same_row(s, &row_deps[si]); + let ti = if same_index { + if let Some(taken) = claimed.get_mut(si) { + *taken = true; + } + si + } else { + if claimed.is_empty() { + claimed = vec![false; row_deps.len()]; + claimed[..si.min(row_deps.len())].fill(true); + } + let Some(ti) = + (0..row_deps.len()).find(|&ti| !claimed[ti] && same_row(s, &row_deps[ti])) + else { + continue; + }; + claimed[ti] = true; + ti + }; + if row_deps[ti].version.literal.slice(lbuf) != s.version.literal.slice(sbuf) { + changed.push((ti, si)); + } + } + changed + }; + + let sync_maps = is_root + && (!scratch.overrides.is_empty() + || scratch.catalogs.has_any() + || !manager.lockfile.overrides.is_empty() + || manager.lockfile.catalogs.has_any()); + if changed.is_empty() && !sync_maps { + continue; + } + + let known = &mut manager.known_npm_aliases; + let (mut builder, lf) = manager.lockfile.string_builder_split(); + for &(_, si) in &changed { + scratch_deps[si].count(sbuf, &mut builder); + } + if sync_maps { + scratch.overrides.count(sbuf, &mut builder); + scratch.catalogs.count(sbuf, &mut builder); + } + builder.allocate()?; + let rows = row.mut_(lf.dependencies.as_mut_slice()); + for &(ti, si) in &changed { + rows[ti] = scratch_deps[si].clone_in(known, sbuf, &mut builder)?; + } + if sync_maps { + *lf.overrides = scratch.overrides.clone(known, sbuf, &mut builder)?; + *lf.catalogs = scratch.catalogs.clone(known, sbuf, &mut builder)?; + } + builder.clamp(); + } + Ok(()) +} + +fn same_row(scratch: &Dependency, row: &Dependency) -> bool { + row.name_hash == scratch.name_hash && row.behavior == scratch.behavior +} + +/// Phase 2 (after bun.lock is saved): add `trustedDependencies` learned during the install and write every edited entry to disk. +pub(crate) fn flush(manager: &mut PackageManager) -> Result<(), crate::Error> { + if manager.edited_package_jsons.is_empty() + || !manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) + { + return Ok(()); + } + let edited = core::mem::take(&mut manager.edited_package_jsons); + let mut trusted: Vec> = if manager + .options + .do_ + .contains(Do::TRUST_DEPENDENCIES_FROM_ARGS) + { + core::mem::take(&mut manager.trusted_deps_to_add_to_package_json) + } else { + Vec::new() + }; + + let mut any_failed = false; + for e in &edited { + if e.received_requests && !trusted.is_empty() { + let entry = fetch_entry(manager, &e.target); + let mut root = entry.root; + PackageJSONEditor::edit_trusted_dependencies(&mut root, &mut trusted)?; + print_package_json_into_cache_entry(entry, root); + } + any_failed |= !write_target(manager, &e.target); + } + if any_failed { + Global::exit(1); + } + Ok(()) +} diff --git a/src/install/PackageManager/updatePackageJSONAndInstall.rs b/src/install/PackageManager/updatePackageJSONAndInstall.rs index 5b61771871cc..f2b3b1d653d9 100644 --- a/src/install/PackageManager/updatePackageJSONAndInstall.rs +++ b/src/install/PackageManager/updatePackageJSONAndInstall.rs @@ -8,7 +8,6 @@ use bstr::BStr; use crate::Error; use crate::ShellCompletions; use crate::bun_fs::FileSystem; -use crate::bun_json as json; use bun_core::{Global, Output}; use bun_core::{ZStr, strings}; use bun_js_printer as js_printer; @@ -16,6 +15,7 @@ use bun_paths::{self, PathBuffer}; use bun_sys::{self, Fd, File}; use super::add_catalog; +use super::add_remove_with_filter::WorkspaceTarget; use super::command_line_arguments::CommandLineArguments; use super::package_json_editor as PackageJSONEditor; use super::update_request::Array as UpdateRequestArray; @@ -58,71 +58,38 @@ pub(super) fn remove_dependencies_from_package_json( package_json: &mut bun_ast::Expr, updates: &[UpdateRequest], ) -> bool { + const LISTS: [&[u8]; 4] = [ + b"dependencies", + b"devDependencies", + b"optionalDependencies", + b"peerDependencies", + ]; let mut any_changes = false; - // if we're removing, they don't have to specify where it is installed in the dependencies list - // they can even put it multiple times and we will just remove all of them for request in updates.iter() { - const LISTS: [&[u8]; 4] = [ - b"dependencies", - b"devDependencies", - b"optionalDependencies", - b"peerDependencies", - ]; for list in LISTS { - if let Some(query) = package_json.as_property(list) { - if query.expr.data.is_e_object() { - // reshaped for borrowck — - // `StoreRef` is `Copy` and derefs to a raw arena - // pointer, so taking it once works across writes to both the - // inner list and the parent object. - let mut e_object = query.expr.data.as_e_object(); - let dependencies = e_object.properties.slice_mut(); - let mut i: usize = 0; - let mut new_len = dependencies.len(); - // `G::Property` is not `Copy`, - // so we `swap` instead of copy-from-tail — but the swapped-out - // matched element - // lands in the truncated tail and MUST NOT be revisited (it would - // match again and over-truncate). Bounding by `new_len` yields the - // correct result for the unique-key case package.json guarantees. - while i < new_len { - let key = dependencies[i].key.unwrap(); - if key.data.is_e_string() { - if key.data.as_e_string().unwrap().eql_bytes(request.name) { - if new_len > 1 { - dependencies.swap(i, new_len - 1); - new_len -= 1; - } else { - new_len = 0; - } - - any_changes = true; - } - } - i += 1; - } - - let changed = new_len != dependencies.len(); - if changed { - e_object.properties.truncate(new_len); - - // If the dependencies list is now empty, remove it from the package.json - // since we're swapRemove, we have to re-sort it - if e_object.properties.len_u32() == 0 { - // TODO: Theoretically we could change these two lines to - // `.orderedRemove(query.i)`, but would that change user-facing - // behavior? - let _ = package_json - .data - .as_e_object_mut() - .properties - .swap_remove(query.i as usize); - package_json.data.as_e_object_mut().package_json_sort(); - } else { - e_object.alphabetize_properties(); - } - } - } + let Some(query) = package_json.as_property(list) else { + continue; + }; + let Some(mut e_object) = query.expr.data.e_object() else { + continue; + }; + let before = e_object.properties.len(); + e_object.properties.retain(|property| { + !property + .key + .and_then(|key| key.data.e_string()) + .is_some_and(|key| key.eql_bytes(request.name)) + }); + if e_object.properties.len() == before { + continue; + } + any_changes = true; + if e_object.properties.is_empty() { + let root = package_json.data.as_e_object_mut(); + let _ = root.properties.swap_remove(query.i as usize); + root.package_json_sort(); + } else { + e_object.alphabetize_properties(); } } } @@ -476,7 +443,7 @@ fn update_package_json_and_install_with_manager_with_updates( buffer_writer.append_newline = preserve_trailing_newline_at_eof_for_package_json; let mut package_json_writer = js_printer::BufferPrinter::init(buffer_writer); - let mut written = match js_printer::print_json( + if let Err(e) = js_printer::print_json( &mut package_json_writer, current_package_json_root, ¤t_package_json.source, @@ -486,12 +453,9 @@ fn update_package_json_and_install_with_manager_with_updates( ..Default::default() }, ) { - Ok(n) => n, - Err(e) => { - bun_core::pretty_errorln!("package.json failed to write due to error {}", e.name(),); - Global::crash(); - } - }; + bun_core::pretty_errorln!("package.json failed to write due to error {}", e.name(),); + Global::crash(); + } // There are various tradeoffs with how we commit updates when you run `bun add` or `bun remove` // The one we chose here is to effectively pretend a human did: @@ -503,13 +467,11 @@ fn update_package_json_and_install_with_manager_with_updates( // The Smarter™ approach is you resolve ahead of time and write to disk once! // But, turns out that's slower in any case where more than one package has to be resolved (most of the time!) // Concurrent network requests are faster than doing one and then waiting until the next batch - let mut new_package_json_source: Vec = package_json_writer + let new_package_json_source: Vec = package_json_writer .ctx .written_without_trailing_zero() .to_vec(); - // The cache entry (`Cow<'static, [u8]>`) outlives this stack frame, and - // `new_package_json_source` is reassigned below on the add/update/link path, so we - // must store an *owning* copy to avoid a dangling borrow. + // The cache entry (`Cow<'static, [u8]>`) outlives this stack frame, so it needs its own copy. current_package_json.source.contents = Cow::Owned(new_package_json_source.clone()); // The edits above went into a promoted copy // (`current_package_json_root`), so re-parse the @@ -520,7 +482,21 @@ fn update_package_json_and_install_with_manager_with_updates( Global::crash(); } - let mut editing_catalogs = false; + if matches!( + subcommand, + Subcommand::Add | Subcommand::Update | Subcommand::Link + ) && manager.update_target_workspaces.is_none() + { + super::package_json_write_back::record( + manager, + WorkspaceTarget { + name: Box::default(), + name_hash: manager.workspace_name_hash, + package_json_path: manager.original_package_json_path.as_bytes().into(), + }, + true, + ); + } // may or may not be the package json we are editing let top_level_dir_without_trailing_slash = @@ -630,19 +606,17 @@ fn update_package_json_and_install_with_manager_with_updates( && root_is_targeted { let root_package_json_root: bun_ast::Expr = root_package_json.root; - if PackageJSONEditor::edit_catalogs_before_update(manager, &root_package_json_root)? { - editing_catalogs = true; - - if manager.options.do_.contains(Do::UPDATE_TO_LATEST) { - // entries now hold a temporary `latest`; refresh the cache so install resolves those. - print_package_json_into_cache_entry(root_package_json, root_package_json_root); - if let Err(err) = root_package_json.reparse_root(manager.log_mut()) { - bun_core::pretty_errorln!( - "package.json failed to parse due to error {}", - err.name(), - ); - Global::crash(); - } + if PackageJSONEditor::edit_catalogs_before_update(manager, &root_package_json_root)? + && manager.options.do_.contains(Do::UPDATE_TO_LATEST) + { + // entries now hold a temporary `latest`; refresh the cache so install resolves those. + print_package_json_into_cache_entry(root_package_json, root_package_json_root); + if let Err(err) = root_package_json.reparse_root(manager.log_mut()) { + bun_core::pretty_errorln!( + "package.json failed to parse due to error {}", + err.name(), + ); + Global::crash(); } } } @@ -660,188 +634,14 @@ fn update_package_json_and_install_with_manager_with_updates( install_with_manager::install_with_manager(manager, ctx, root_package_json_path, original_cwd)?; - // reshaped for borrowck — see assignment above. `install_with_manager` - // is the only writer to `manager.update_requests` between the assignment and - // here, so taking it back yields exactly the slice assigned above. - let mut updates: Box<[UpdateRequest]> = core::mem::take(&mut manager.update_requests); - - if subcommand == Subcommand::Update - || subcommand == Subcommand::Add - || subcommand == Subcommand::Link - { - for request in updates.iter() { - if request.failed { - Global::exit(1); - } - } - - let source = - bun_ast::Source::init_path_string(&b"package.json"[..], &new_package_json_source[..]); - - // Now, we _re_ parse our in-memory edited package.json - // so we can commit the version we changed from the lockfile - let json_arena = bun_alloc::Arena::new(); - let mut new_package_json: bun_ast::Expr = - match json::parse_package_json_utf8(&source, manager.log_mut(), &json_arena) { - Ok(v) => v, - Err(err) => { - bun_core::pretty_errorln!( - "package.json failed to parse due to error {}", - err.name(), - ); - Global::crash(); - } - }; - - if updates.is_empty() { - if manager.update_target_workspaces.is_none() { - PackageJSONEditor::edit_update_no_args( - manager, - &mut new_package_json, - EditOptions { - exact_versions: manager.options.enable.exact_versions(), - ..Default::default() - }, - )?; - } - - if editing_catalogs - && manager.workspace_name_hash.is_none() - && manager.update_target_workspaces.is_none() - { - // running from root: catalogs live in this file. - let _ = PackageJSONEditor::edit_catalogs_after_update(manager, &new_package_json)?; - } - } else { - let mut updates_slice: &mut [UpdateRequest] = &mut updates[..]; - PackageJSONEditor::edit( - manager, - &mut updates_slice, - &mut new_package_json, - dependency_list, - EditOptions { - exact_versions: manager.options.enable.exact_versions(), - add_trusted_dependencies: manager - .options - .do_ - .contains(Do::TRUST_DEPENDENCIES_FROM_ARGS), - ..Default::default() - }, - )?; - } - if manager.options.add_catalog.is_some() { - add_catalog::rewrite_references(manager, &updates[..]); - if manager.workspace_name_hash.is_none() { - let _ = - add_catalog::edit_root_after_install(manager, &new_package_json, &updates[..])?; - } - } - let mut buffer_writer_two = js_printer::BufferWriter::init(); - buffer_writer_two.buffer.list.reserve( - (source.contents.len() + 1).saturating_sub(buffer_writer_two.buffer.list.len()), - ); - buffer_writer_two.append_newline = preserve_trailing_newline_at_eof_for_package_json; - let mut package_json_writer_two = js_printer::BufferPrinter::init(buffer_writer_two); - - written = match js_printer::print_json( - &mut package_json_writer_two, - new_package_json, - &source, - js_printer::PrintJsonOptions { - indent: current_package_json_indent, - mangled_props: None, - ..Default::default() - }, - ) { - Ok(n) => n, - Err(e) => { - bun_core::pretty_errorln!("package.json failed to write due to error {}", e.name(),); - Global::crash(); - } - }; - - new_package_json_source = package_json_writer_two - .ctx - .written_without_trailing_zero() - .to_vec(); - } - - if editing_catalogs - && (manager.workspace_name_hash.is_some() || manager.update_target_workspaces.is_some()) - && manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) - { - // running from a workspace, or with -r/--filter: catalogs live in the root package.json. - let root_package_json_ptr: *mut MapEntry = - match manager.workspace_package_json_cache.get_with_path( - manager.log_mut(), - root_package_json_path.as_bytes(), - GetJSONOptions { - guess_indentation: true, - ..Default::default() - }, - ) { - GetResult::ParseErr(err) => { - let _ = manager - .log_mut() - .print(std::ptr::from_mut(Output::error_writer())); - Output::err_generic( - "failed to parse package.json \"{s}\": {s}", - (BStr::new(root_package_json_path.as_bytes()), err.name()), - ); - Global::crash(); - } - GetResult::ReadErr(err) => { - Output::err_generic( - "failed to read package.json \"{s}\": {s}", - (BStr::new(root_package_json_path.as_bytes()), err.name()), - ); - Global::crash(); - } - GetResult::Entry(entry) => core::ptr::from_mut(entry), - }; - // SAFETY: pointer into `manager.workspace_package_json_cache`, valid until - // the next `get_with_path`. `edit_catalogs_after_update` touches only - // disjoint manager fields. - let root_package_json: &mut MapEntry = unsafe { &mut *root_package_json_ptr }; - let root_package_json_root: bun_ast::Expr = root_package_json.root; - - let root_catalogs_changed = - PackageJSONEditor::edit_catalogs_after_update(manager, &root_package_json_root)?; - - if root_catalogs_changed { - print_package_json_into_cache_entry(root_package_json, root_package_json_root); - - // the targets loop below writes root (with deps + catalogs) in one pass. - if manager.update_target_workspaces.is_none() { - let root_package_json_file = - File::openat(Fd::cwd(), root_package_json_path, bun_sys::O::RDWR, 0) - .map_err(Error::from)?; - root_package_json_file - .pwrite_all(&root_package_json.source.contents, 0) - .map_err(Error::from)?; - let _ = bun_sys::ftruncate( - root_package_json_file.handle, - root_package_json.source.contents.len() as i64, - ); - let _ = root_package_json_file.close(); // close error is non-actionable - } - } - } - - if manager.options.add_catalog.is_some() - && manager.workspace_name_hash.is_some() - && manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) - { - add_catalog::write_root_after_install(manager, root_package_json_path, &updates[..])?; - } - - let _ = written; - - if let Some(targets) = manager.update_target_workspaces.take() { - if manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) { - write_resolved_versions_to_targets(manager, &targets)?; + if matches!( + subcommand, + Subcommand::Update | Subcommand::Add | Subcommand::Link + ) { + if manager.update_requests.iter().any(|request| request.failed) { + Global::exit(1); } - return Ok(()); + return super::package_json_write_back::flush(manager); } if manager.options.do_.contains(Do::WRITE_PACKAGE_JSON) { @@ -895,6 +695,7 @@ fn update_package_json_and_install_with_manager_with_updates( if !any_changes { Global::exit(0); } + let updates: Box<[UpdateRequest]> = core::mem::take(&mut manager.update_requests); remove_leftover_node_modules(manager, &updates); } } @@ -902,125 +703,19 @@ fn update_package_json_and_install_with_manager_with_updates( Ok(()) } -fn write_resolved_versions_to_targets( - manager: &mut PackageManager, - targets: &[super::UpdateTargetWorkspace], -) -> Result<(), Error> { - let top_level = strings::without_trailing_slash(FileSystem::instance().top_level_dir()); - let update_to_latest = manager.options.do_.contains(Do::UPDATE_TO_LATEST); - let exact_versions = manager.options.enable.exact_versions(); - let log = manager.log_mut(); - let mut any_failed = false; - - let packages = manager.lockfile.packages.slice(); - let pkg_resolutions = packages.items_resolution(); - let pkg_name_hashes = packages.items_name_hash(); - let pkg_names = packages.items_name(); - for pkg_id in 0..packages.len() { - let res = pkg_resolutions[pkg_id]; - let is_root = res.tag == crate::resolution::Tag::Root; - let (ws_name_hash, rel): (Option, &[u8]) = match res.tag { - crate::resolution::Tag::Root => (None, b""), - crate::resolution::Tag::Workspace => ( - Some(pkg_name_hashes[pkg_id]), - res.workspace() - .slice(manager.lockfile.buffers.string_bytes.as_slice()), - ), - _ => continue, - }; - let hash = pkg_name_hashes[pkg_id]; - let name = pkg_names[pkg_id].slice(manager.lockfile.buffers.string_bytes.as_slice()); - if !targets.iter().any(|t| t.matches(is_root, hash, name)) { - continue; - } - let mut path_buf = PathBuffer::uninit(); - let path: &[u8] = bun_paths::resolve_path::join_abs_string_buf::< - bun_paths::resolve_path::platform::Auto, - >(top_level, &mut path_buf.0, &[rel, b"package.json"]); - - let entry = match manager.workspace_package_json_cache.get_with_path( - log, - path, - GetJSONOptions { - guess_indentation: true, - ..Default::default() - }, - ) { - GetResult::Entry(e) => e, - GetResult::ParseErr(err) | GetResult::ReadErr(err) => { - Output::err_generic( - "failed to read/parse package.json for workspace '{s}': {s}", - (bstr::BStr::new(name), err.name()), - ); - any_failed = true; - continue; - } - }; - - let mut ast = entry.root; - let mut updating = bun_collections::StringArrayHashMap::default(); - PackageJSONEditor::edit_update_no_args_in( - &manager.lockfile, - &manager.ast_arena, - &mut updating, - ws_name_hash, - update_to_latest, - &mut ast, - EditOptions { - exact_versions: true, - before_install: true, - ..Default::default() - }, - )?; - PackageJSONEditor::edit_update_no_args_in( - &manager.lockfile, - &manager.ast_arena, - &mut updating, - ws_name_hash, - update_to_latest, - &mut ast, - EditOptions { - exact_versions, - ..Default::default() - }, - )?; - - print_package_json_into_cache_entry(entry, ast); - let mut path_zbuf = PathBuffer::uninit(); - let path_z = bun_paths::resolve_path::z(path, &mut path_zbuf); - if let Err(err) = File::write_file(Fd::cwd(), path_z, &entry.source.contents) { - Output::err_generic( - "failed to write package.json for workspace '{s}': {s}", - (bstr::BStr::new(name), bstr::BStr::new(err.name())), - ); - any_failed = true; - } - } - if any_failed { - Global::exit(1); - } - Ok(()) -} - pub(super) fn remove_leftover_node_modules( manager: &mut PackageManager, updates: &[UpdateRequest], ) { let cwd = bun_sys::Dir::cwd(); - // This is not exactly correct let mut node_modules_buf = PathBuffer::uninit(); node_modules_buf[..b"node_modules".len()].copy_from_slice(b"node_modules"); node_modules_buf[b"node_modules".len()] = bun_paths::SEP; let name_hashes = manager.lockfile.packages.items_name_hash(); for request in updates.iter() { - // If the package no longer exists in the updated lockfile, delete the directory - // This is not thorough. - // It does not handle nested dependencies - // This is a quick & dirty cleanup intended for when deleting top-level dependencies - if !name_hashes - .iter() - .any(|h| *h == bun_semver::semver_string::Builder::string_hash(request.name)) - { + // Only top-level folders are removed; nested copies are left alone. + let name_hash = bun_semver::semver_string::Builder::string_hash(request.name); + if !name_hashes.contains(&name_hash) { let offset_buf = &mut node_modules_buf[b"node_modules/".len()..]; offset_buf[..request.name.len()].copy_from_slice(request.name); let _ = @@ -1028,18 +723,14 @@ pub(super) fn remove_leftover_node_modules( } } - // This is where we clean dangling symlinks - // This could be slow if there are a lot of symlinks match bun_sys::open_dir_for_iteration(cwd.fd(), manager.options.bin_path.as_bytes()) { Ok(node_modules_bin) => { - // `defer node_modules_bin.close()` — explicit close below (Fd is Copy, no Drop). let mut iter = bun_sys::iterate_dir(node_modules_bin); 'iterator: loop { let Ok(Some(entry)) = iter.next() else { break }; match entry.kind { bun_sys::EntryKind::SymLink => { - // any symlinks which we are unable to open are assumed to be dangling - // note that using access won't work here, because access doesn't resolve symlinks + // access(2) does not follow symlinks, so open() is the dangling check. let name = entry.name.slice_u8(); node_modules_buf[..name.len()].copy_from_slice(name); node_modules_buf[name.len()] = 0; diff --git a/src/install/audit_fix.rs b/src/install/audit_fix.rs index 24db66761409..7084834782fe 100644 --- a/src/install/audit_fix.rs +++ b/src/install/audit_fix.rs @@ -95,7 +95,7 @@ fn fmt_version(version: Semver::Version, buf: &[u8]) -> Box<[u8]> { out.into_boxed_slice() } -fn vuln_word(n: u32) -> &'static str { +pub fn vuln_word(n: u32) -> &'static str { if n == 1 { "vulnerability" } else { @@ -254,7 +254,11 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat dependent.extend_from_slice(b"package.json"); } instances[instance as usize].edges.push(Edge { - range: crate::dedupe::effective_npm_range(lockfile, dep), + range: crate::dedupe::effective_npm_range( + lockfile, + dep_id as DependencyID, + dep, + ), literal: Box::from(dep.version.literal.slice(buf)), dependent: dependent.into_boxed_slice(), bundled: dep.behavior.is_bundled(), @@ -570,6 +574,8 @@ pub fn enqueue_planned_fixes(manager: &mut PackageManager) -> crate::Result<()> .options .enable .set(Enable::FORCE_SAVE_LOCKFILE, true); + // Counted as updates so clean drops the versions the peer slots skipped below still point at. + manager.summary.update += pins.len() as u32; let target_of: Vec = { let lockfile = &*manager.lockfile; @@ -606,36 +612,41 @@ pub fn enqueue_planned_fixes(manager: &mut PackageManager) -> crate::Result<()> { continue; } - let row = manager.lockfile.buffers.dependencies[dep_id].clone(); - if row.behavior.is_optional_peer() || row.behavior.is_bundled() { + let behavior = manager.lockfile.buffers.dependencies[dep_id].behavior; + if behavior.is_optional_peer() || behavior.is_bundled() { continue; } - let pkg_name = manager.lockfile.packages.items_name()[target as usize]; let pin = &pins[target_of[target as usize] as usize]; - let pinned = Dependency { - name: row.name, - name_hash: row.name_hash, - behavior: row.behavior.with(Behavior::PEER, false), - version: dependency::Version { - tag: DependencyVersionTag::Npm, - literal: Semver::String::default(), - value: dependency::Value { - npm: ManuallyDrop::new(dependency::NpmInfo { - name: pkg_name, - version: Group::from(pin.to_version), - is_alias: true, - }), - }, - }, - }; - manager.lockfile.buffers.resolutions[dep_id] = invalid_package_id; - enqueue_dependency_with_main( - manager, - dep_id as DependencyID, - &pinned, - invalid_package_id, - false, - )?; + enqueue_pinned(manager, dep_id as DependencyID, pin.to_version)?; } Ok(()) } + +/// Re-resolves the edge `dep_id` (which must currently resolve to an npm package) to exactly `to_version`. +pub(crate) fn enqueue_pinned( + manager: &mut PackageManager, + dep_id: DependencyID, + to_version: Semver::Version, +) -> crate::Result<()> { + let target = manager.lockfile.buffers.resolutions[dep_id as usize]; + let row = manager.lockfile.buffers.dependencies[dep_id as usize].clone(); + let pkg_name = manager.lockfile.packages.items_name()[target as usize]; + let pinned = Dependency { + name: row.name, + name_hash: row.name_hash, + behavior: row.behavior.with(Behavior::PEER, false), + version: dependency::Version { + tag: DependencyVersionTag::Npm, + literal: Semver::String::default(), + value: dependency::Value { + npm: ManuallyDrop::new(dependency::NpmInfo { + name: pkg_name, + version: Group::from(to_version), + is_alias: true, + }), + }, + }, + }; + manager.lockfile.buffers.resolutions[dep_id as usize] = invalid_package_id; + enqueue_dependency_with_main(manager, dep_id, &pinned, invalid_package_id, false) +} diff --git a/src/install/dedupe.rs b/src/install/dedupe.rs index cbeae2a061a0..d4569ae249b6 100644 --- a/src/install/dedupe.rs +++ b/src/install/dedupe.rs @@ -96,7 +96,7 @@ impl Pass<'_> { let range = if self.pinned[target as usize] || dep.behavior.is_bundled() { None } else { - effective_npm_range(lockfile, dep) + effective_npm_range(lockfile, dep_id, dep) }; match range { None => row[cur_c] = true, @@ -154,12 +154,11 @@ impl Pass<'_> { // Packages the pass itself removes must not vote (pnpm/pnpm#9213): shrink voters until the outcome agrees. fn settle(&mut self, cur: &[PackageID], live: &[bool]) -> Vec { - let dep_slices = self.lockfile.packages.items_dependencies(); let mut voters = live.to_vec(); let mut best: Option> = None; loop { let (next, voted) = self.vote(cur, live, &voters); - let after = reachable(dep_slices, &next); + let after = reachable(self.lockfile, &next); if (0..after.len()).any(|p| after[p] && !voters[p]) { return best.unwrap_or(next); } @@ -178,7 +177,7 @@ impl Pass<'_> { } } -fn label(lockfile: &Lockfile, id: PackageID) -> Vec { +pub(crate) fn label(lockfile: &Lockfile, id: PackageID) -> Vec { let buf = lockfile.buffers.string_bytes.as_slice(); let mut label = Vec::new(); let _ = write!( @@ -195,7 +194,6 @@ fn label(lockfile: &Lockfile, id: PackageID) -> Vec { pub fn dedupe_lockfile(lockfile: &mut Lockfile) -> Vec> { let pkg_res = lockfile.packages.items_resolution(); - let dep_slices = lockfile.packages.items_dependencies(); let has_patches = lockfile.patched_dependencies.count() > 0; let mut groups: Vec> = Vec::new(); @@ -228,9 +226,9 @@ pub fn dedupe_lockfile(lockfile: &mut Lockfile) -> Vec> { return Vec::new(); } - let initial = reachable(dep_slices, &lockfile.buffers.resolutions); + let initial = reachable(lockfile, &lockfile.buffers.resolutions); let mut live = initial.clone(); - let mut cur: Vec = lockfile.buffers.resolutions.to_vec(); + let mut cur: Vec = lockfile.buffers.resolutions.clone(); { let mut pass = Pass { lockfile, @@ -245,7 +243,7 @@ pub fn dedupe_lockfile(lockfile: &mut Lockfile) -> Vec> { }; loop { cur = pass.settle(&cur, &live); - let after = reachable(dep_slices, &cur); + let after = reachable(lockfile, &cur); if after == live { break; } @@ -285,15 +283,17 @@ pub fn dedupe_lockfile(lockfile: &mut Lockfile) -> Vec> { pub(crate) fn effective_npm_range( lockfile: &Lockfile, + dep_id: DependencyID, dep: &Dependency, ) -> Option { - let mut version = if dep.version.tag == DependencyVersionTag::Npm && dep.version.npm().is_alias + let mut version = if dep.behavior.is_workspace() + || (dep.version.tag == DependencyVersionTag::Npm && dep.version.npm().is_alias) { dep.version.clone() } else { lockfile .overrides - .get(dep.name_hash) + .get(lockfile, dep_id, dep.name_hash) .unwrap_or_else(|| dep.version.clone()) }; if version.tag == DependencyVersionTag::Catalog { @@ -306,33 +306,12 @@ pub(crate) fn effective_npm_range( } // Optional-peer edges are followed too: with an in-sync package.json `clean` runs with `keep_optional_peer_targets`. -fn reachable( - dep_slices: &[crate::lockfile::DependencySlice], - resolutions: &[PackageID], -) -> Vec { - let mut seen = vec![false; dep_slices.len()]; - if seen.is_empty() { - return seen; - } - seen[0] = true; - let mut worklist: Vec = vec![0]; - while let Some(pkg_id) = worklist.pop() { - let slice = dep_slices[pkg_id as usize]; - for i in slice.begin() as usize..slice.end() as usize { - let target = resolutions[i]; - if target == invalid_package_id { - continue; - } - let Some(slot) = seen.get_mut(target as usize) else { - continue; - }; - if !*slot { - *slot = true; - worklist.push(target); - } - } - } - seen +fn reachable(lockfile: &Lockfile, resolutions: &[PackageID]) -> Vec { + crate::lockfile::reachable::packages( + lockfile, + resolutions, + crate::lockfile::reachable::Options::all(0), + ) } fn load_step_verb(step: LoadStep) -> &'static str { diff --git a/src/install/isolated_install.rs b/src/install/isolated_install.rs index 9b94cdda2d30..f0b3ea2ea181 100644 --- a/src/install/isolated_install.rs +++ b/src/install/isolated_install.rs @@ -2005,6 +2005,7 @@ pub(crate) fn install_isolated_packages( // placeholder is never dereferenced installer: bun_ptr::BackRef::from(core::ptr::NonNull::dangling()), result: installer::Result::None, + relink: installer::Relink::Off, task: bun_threading::thread_pool::Task { callback: installer::Task::callback, node: Default::default(), @@ -2036,6 +2037,8 @@ pub(crate) fn install_isolated_packages( }, store: &store, tasks, + waiters_head: vec![store::entry::Id::INVALID; store.entries.len()].into_boxed_slice(), + next_waiter: vec![store::entry::Id::INVALID; store.entries.len()].into_boxed_slice(), trusted_dependencies_mutex: Default::default(), trusted_dependencies_from_update_requests, supported_backend: std::sync::atomic::AtomicU8::new( @@ -2086,6 +2089,47 @@ pub(crate) fn install_isolated_packages( ); } + let force_install = installer.manager().options.enable.force_install(); + let store_hash_path = paths::path_literal!("node_modules/.bun/.store-hash"); + let store_graph_hex: [u8; 16] = { + let mut hasher = Wyhash::init(0); + for _entry_id in 0..store.entries.len() { + let entry_id = store::entry::Id::from(u32::try_from(_entry_id).expect("int cast")); + { + let mut hw = WyhashWriter { + hasher: &mut hasher, + }; + write!( + hw, + "{}\0", + store::entry::fmt_store_path(entry_id, &store, lockfile_ro) + ) + .expect("unreachable"); + } + for dep in entry_dependencies[_entry_id].slice() { + hasher.update( + lockfile_ro.buffers.dependencies[dep.dep_id as usize] + .name + .slice(string_buf), + ); + hasher.update(b"\0"); + hasher.update(&dep.entry_id.get().to_le_bytes()); + } + } + let mut hex = [0u8; 16]; + write!(&mut hex[..], "{:016x}", hasher.final_()).expect("unreachable"); + hex + }; + let relink_needed = !is_new_bun_modules && !force_install && { + let mut stamp = [0u8; 17]; + let read = sys::File::openat(Fd::cwd(), store_hash_path, sys::O::RDONLY, 0) + .and_then(|file| file.read_all(&mut stamp)); + !matches!(read, Ok(n) if n == store_graph_hex.len() && stamp[..n] == store_graph_hex) + }; + if relink_needed { + let _ = sys::unlinkat(Fd::cwd(), store_hash_path); + } + // add the pending task count upfront installer .manager_mut() @@ -2295,6 +2339,10 @@ pub(crate) fn install_isolated_packages( if entry_hoisted[entry_id.get() as usize] { installer.link_to_hidden_node_modules(entry_id); } + if relink_needed && !uses_global_store { + installer.start_relink_task(entry_id); + continue; + } // .monotonic is okay because the task isn't running on another thread. entry_steps[entry_id.get() as usize] .store(installer::Step::Done as u32, Ordering::Relaxed); @@ -2634,6 +2682,11 @@ pub(crate) fn install_isolated_packages( debug_assert!(done); } + if installer.summary.fail == 0 && (relink_needed || is_new_bun_modules || force_install) { + let _ = sys::File::create(Fd::cwd(), store_hash_path, true) + .and_then(|file| file.write_all(&store_graph_hex)); + } + let mut summary = core::mem::take(&mut installer.summary); summary.successfully_installed = Some(core::mem::take(&mut installer.installed)); diff --git a/src/install/isolated_install/Installer.rs b/src/install/isolated_install/Installer.rs index edb25516b394..ae4737311b2c 100644 --- a/src/install/isolated_install/Installer.rs +++ b/src/install/isolated_install/Installer.rs @@ -113,6 +113,10 @@ pub struct Installer<'a> { /// the final step. The directory existing at its final path is the only /// completeness signal the warm-hit check needs. pub(crate) global_store_tmp_suffix: u64, + + /// Main-thread only: `waiters_head[dep]` starts the intrusive list of blocked entries waiting on `dep`, linked through `next_waiter`. + pub(crate) waiters_head: Box<[StoreEntryId]>, + pub(crate) next_waiter: Box<[StoreEntryId]>, } impl<'a> Installer<'a> { @@ -168,6 +172,15 @@ impl<'a> Installer<'a> { .schedule(thread_pool::Batch::from(&raw mut task.task)); } + /// Called from main thread, for an existing project-local store entry. + pub(crate) fn start_relink_task(&mut self, entry_id: StoreEntryId) { + self.tasks[entry_id.get() as usize].relink = Relink::Pending; + // .monotonic is okay because the task isn't running yet. + self.store.entries.items_step()[entry_id.get() as usize] + .store(Step::SymlinkDependencies as u32, Ordering::Relaxed); + self.start_task(entry_id); + } + pub(crate) fn on_package_extracted(&mut self, task_id: crate::package_manager_task::Id) { if let Some(removed) = self.manager_mut().task_queue.remove(&task_id) { let store = self.store; @@ -422,7 +435,7 @@ impl<'a> Installer<'a> { self.summary.fail += 1; self.decrement_pending_tasks(); - self.resume_unblocked_tasks(); + self.resume_unblocked_tasks(entry_id); } pub(crate) fn decrement_pending_tasks(&mut self) { @@ -441,26 +454,35 @@ impl<'a> Installer<'a> { let mut parent_dedupe: ArrayHashMap = ArrayHashMap::default(); - if !self.is_task_blocked(entry_id, &mut parent_dedupe) { - // .monotonic is okay because the task isn't running right now. - self.store.entries.items_step()[entry_id.get() as usize] - .store(Step::SymlinkDependencyBinaries as u32, Ordering::Relaxed); - self.start_task(entry_id); - return; + match self.is_task_blocked(entry_id, &mut parent_dedupe) { + None => { + // .monotonic is okay because the task isn't running right now. + self.store.entries.items_step()[entry_id.get() as usize] + .store(Step::SymlinkDependencyBinaries as u32, Ordering::Relaxed); + self.start_task(entry_id); + } + Some(blocked_on) => { + // .monotonic is okay because the task isn't running right now. + self.store.entries.items_step()[entry_id.get() as usize] + .store(Step::Blocked as u32, Ordering::Relaxed); + self.push_waiter(blocked_on, entry_id); + } } + } - // .monotonic is okay because the task isn't running right now. - self.store.entries.items_step()[entry_id.get() as usize] - .store(Step::Blocked as u32, Ordering::Relaxed); + /// Main thread only. + fn push_waiter(&mut self, dep: StoreEntryId, waiter: StoreEntryId) { + let head = &mut self.waiters_head[dep.get() as usize]; + self.next_waiter[waiter.get() as usize] = *head; + *head = waiter; } - /// Called from both the main thread (via `onTaskBlocked` and `resumeUnblockedTasks`) and the - /// task thread (via `run`). `parent_dedupe` should not be shared between threads. + /// Returns the first unfinished non-cyclic dependency; runs on main and task threads, so `parent_dedupe` must not be shared. fn is_task_blocked( &self, entry_id: StoreEntryId, parent_dedupe: &mut ArrayHashMap, - ) -> bool { + ) -> Option { let entries = &self.store.entries; let entry_deps = entries.items_dependencies(); let entry_steps = entries.items_step(); @@ -473,14 +495,18 @@ impl<'a> Installer<'a> { if self.store.is_cycle(entry_id, dep.entry_id, parent_dedupe) { continue; } - return true; + return Some(dep.entry_id); } } - false + None } /// Called from main thread pub(crate) fn on_task_complete(&mut self, entry_id: StoreEntryId, state: CompleteState) { + let state = match self.tasks[entry_id.get() as usize].relink { + Relink::Unchanged => CompleteState::Skipped, + Relink::Off | Relink::Pending | Relink::Changed => state, + }; if Environment::CI_ASSERT { // .monotonic is okay because we should have already synchronized with the completed // task thread by virtue of popping from the `UnboundedQueue`. @@ -491,7 +517,7 @@ impl<'a> Installer<'a> { } self.decrement_pending_tasks(); - self.resume_unblocked_tasks(); + self.resume_unblocked_tasks(entry_id); if let Some(node) = self.install_node.as_mut() { node.complete_one(); @@ -543,28 +569,36 @@ impl<'a> Installer<'a> { self.installed.set(pkg_id as usize); } - // This function runs only on the main thread. The installer tasks threads - // will be changing values in `entry_step`, but the blocked state is only - // set on the main thread, allowing the code between - // `entry_steps[entry_id.get() as usize].load(.monotonic)` - // and - // `entry_steps[entry_id.get() as usize].store(.symlink_dependency_binaries, .monotonic)` - pub(crate) fn resume_unblocked_tasks(&mut self) { - let entries = &self.store.entries; - let entry_steps = entries.items_step(); + /// Main thread only: `completed` just reached `Step::Done`; re-check every entry waiting on it. + pub(crate) fn resume_unblocked_tasks(&mut self, completed: StoreEntryId) { + let entry_steps = self.store.entries.items_step(); + if Environment::CI_ASSERT { + assert!( + entry_steps[completed.get() as usize].load(Ordering::Relaxed) == Step::Done as u32 + ); + } let mut parent_dedupe: ArrayHashMap = ArrayHashMap::default(); - for id_int in 0..self.store.entries.len() { - let entry_id = StoreEntryId::from(u32::try_from(id_int).expect("int cast")); - - // .monotonic is okay because only the main thread sets this to `.blocked`. - let entry_step = entry_steps[entry_id.get() as usize].load(Ordering::Relaxed); - if entry_step != Step::Blocked as u32 { - continue; + let mut waiter = core::mem::replace( + &mut self.waiters_head[completed.get() as usize], + StoreEntryId::INVALID, + ); + while waiter != StoreEntryId::INVALID { + let entry_id = waiter; + waiter = core::mem::replace( + &mut self.next_waiter[entry_id.get() as usize], + StoreEntryId::INVALID, + ); + if Environment::CI_ASSERT { + assert!( + entry_steps[entry_id.get() as usize].load(Ordering::Relaxed) + == Step::Blocked as u32 + ); } - if self.is_task_blocked(entry_id, &mut parent_dedupe) { + if let Some(blocked_on) = self.is_task_blocked(entry_id, &mut parent_dedupe) { + self.push_waiter(blocked_on, entry_id); continue; } @@ -583,6 +617,14 @@ pub enum CompleteState { Fail, } +#[derive(Clone, Copy, PartialEq, Eq)] +pub enum Relink { + Off, + Pending, + Unchanged, + Changed, +} + fn download_error_reason(e: crate::Error) -> &'static [u8] { match e { crate::Error::TarballHTTP400 => b"400 Bad Request", @@ -615,6 +657,7 @@ pub struct Task { pub(crate) next: bun_threading::Link, // INTRUSIVE: bun.UnboundedQueue(Task, .next) link pub(crate) result: Result, + pub(crate) relink: Relink, } // SAFETY: `next` is the sole intrusive link for `UnboundedQueue`. @@ -765,7 +808,13 @@ impl Task { Step::LinkPackage => Step::SymlinkDependencies, Step::SymlinkDependencies => Step::CheckIfBlocked, Step::CheckIfBlocked => Step::SymlinkDependencyBinaries, - Step::SymlinkDependencyBinaries => Step::RunPreinstall, + Step::SymlinkDependencyBinaries => { + if self.relink == Relink::Off { + Step::RunPreinstall + } else { + Step::Done + } + } Step::RunPreinstall => Step::Binaries, Step::Binaries => Step::RunPostInstallAndPrePostPrepare, Step::RunPostInstallAndPrePostPrepare => Step::Done, @@ -843,7 +892,6 @@ impl Task { let entries = &installer.store.entries; let entry_node_ids = entries.items_node_id(); - let entry_dependencies = entries.items_dependencies(); let entry_steps = entries.items_step(); let entry_scripts = entries.items_scripts(); let entry_hoisted = entries.items_hoisted(); @@ -1451,112 +1499,31 @@ impl Task { Step::SymlinkDependencies => { let current_step = Step::SymlinkDependencies; - let string_buf = lockfile.buffers.string_bytes.as_slice(); - let dependencies = lockfile.buffers.dependencies.as_slice(); - - for dep in entry_dependencies[self.entry_id.get() as usize].slice() { - let dep_name = dependencies[dep.dep_id as usize].name.slice(string_buf); - - let mut dest = AutoPath::init_top_level_dir(); - - installer.append_real_store_node_modules_path( - &mut dest, - self.entry_id, - Which::Staging, - ); - - let _ = dest.append(dep_name); // OOM/capacity: fire-and-forget - - if let Some(entry_node_modules_name) = installer - .entry_store_node_modules_package_name( - dep_id, pkg_id, &pkg_res, pkg_names, - ) - { - if strings::eql_long(dep_name, entry_node_modules_name, true) { - // nest the dependency in another node_modules if the name is the same as the entry name - // in the store node_modules to avoid collision - let _ = dest.append(b"node_modules"); // OOM/capacity: fire-and-forget - let _ = dest.append(dep_name); // OOM/capacity: fire-and-forget - } - } + let relinking = self.relink != Relink::Off; + let strategy = if relinking + || matches!(pkg_res.tag, ResolutionTag::Root | ResolutionTag::Workspace) + { + symlinker::Strategy::ExpectExisting + } else { + symlinker::Strategy::ExpectMissing + }; - let mut dep_store_path = AutoAbsPath::init_top_level_dir(); - - // When this entry lives in the global virtual store, its - // dep symlinks must point at sibling *global* entries - // (relative `../../-/...`) so the entry stays - // valid for any project. Non-global parents (root, - // workspace) keep pointing at the project-local - // `.bun/` indirection so `node_modules/` - // remains a relative link into `node_modules/.bun/`. - if installer.entry_uses_global_store(self.entry_id) { - // The eligibility DFS + fixed-point pass guarantee - // every dep of a global entry is itself global; if - // that ever regressed the failure mode is a - // dangling symlink with no install-time error. - debug_assert!(installer.entry_uses_global_store(dep.entry_id)); - // Target the dep's *final* path: the relative - // `../..//...` link is computed against our - // staging directory but resolves identically once - // we're renamed (same parent), and the dep will - // have been (or will be) renamed into that final - // path by its own task. - installer.append_real_store_path( - &mut dep_store_path, - dep.entry_id, - Which::Final, - ); - } else { - installer.append_store_path(&mut dep_store_path, dep.entry_id); + let changed = match installer.symlink_dependencies(self.entry_id, strategy) { + sys::Result::Ok(changed) => changed, + sys::Result::Err(err) => { + return Ok(Yield::failure(TaskError::SymlinkDependencies(err))); } + }; - // A `dest.save()` `ResetScope` guard would hold `&mut dest`, - // which can't coexist with `dest.undo()/dest.relative()`. - // Capture the length and restore manually. - let dest_saved_len = dest.len(); - let target = { - dest.undo(1); - dest.relative(&dep_store_path) - }; - dest.set_length(dest_saved_len); - - let mut symlinker = Symlinker { - dest: dest.into_sep::<{ PathSeparators::ANY }>(), - target: target.into_sep::<{ PathSeparators::ANY }>(), - #[cfg(windows)] - fallback_junction_target: dep_store_path - .into_sep::<{ PathSeparators::ANY }>(), - }; - - let link_strategy: symlinker::Strategy = if matches!( - pkg_res.tag, - ResolutionTag::Root | ResolutionTag::Workspace - ) { - // root and workspace packages ensure their dependency symlinks - // exist unconditionally. To make sure it's fast, first readlink - // then create the symlink if necessary - symlinker::Strategy::ExpectExisting - } else { - // Global-store entries are built under a private - // per-process staging directory, so nothing else - // is touching this path. - symlinker::Strategy::ExpectMissing - }; - - match symlinker.ensure_symlink(link_strategy) { - sys::Result::Ok(()) => {} - sys::Result::Err(err) => { - return Ok(Yield::failure(TaskError::SymlinkDependencies(err))); - } + if relinking { + if !changed { + self.relink = Relink::Unchanged; + entry_steps[self.entry_id.get() as usize] + .store(Step::Done as u32, Ordering::Release); + return Ok(Yield::Done); } - } - - if installer.entry_uses_global_store(self.entry_id) { - // The entry now exists in the shared global virtual store. - // Project-local `node_modules/.bun/` becomes a - // symlink into it so that the relative `../../` links - // created above (which live inside the global entry) remain - // reachable from the project's node_modules. + self.relink = Relink::Changed; + } else if installer.entry_uses_global_store(self.entry_id) { match installer.link_project_to_global_store(self.entry_id) { sys::Result::Ok(()) => {} sys::Result::Err(err) => { @@ -1576,7 +1543,10 @@ impl Task { let mut parent_dedupe: ArrayHashMap = ArrayHashMap::default(); - if installer.is_task_blocked(self.entry_id, &mut parent_dedupe) { + if installer + .is_task_blocked(self.entry_id, &mut parent_dedupe) + .is_some() + { return Ok(Yield::Blocked); } @@ -2272,6 +2242,70 @@ impl<'a> Installer<'a> { None } + /// Ok(true) when at least one dependency link of the entry was written. + fn symlink_dependencies( + &self, + entry_id: StoreEntryId, + strategy: symlinker::Strategy, + ) -> sys::Result { + let lockfile = self.lockfile(); + let string_buf = lockfile.buffers.string_bytes.as_slice(); + let dependencies = lockfile.buffers.dependencies.as_slice(); + let pkg_names = lockfile.packages.items_name(); + let pkg_resolutions = lockfile.packages.items_resolution(); + + let node_id = self.store.entries.items_node_id()[entry_id.get() as usize]; + let pkg_id = self.store.nodes.items_pkg_id()[node_id.get() as usize]; + let dep_id = self.store.nodes.items_dep_id()[node_id.get() as usize]; + let pkg_res = &pkg_resolutions[pkg_id as usize]; + + let entry_node_modules_name = + self.entry_store_node_modules_package_name(dep_id, pkg_id, pkg_res, pkg_names); + let uses_global_store = self.entry_uses_global_store(entry_id); + + let mut dest = AutoPath::init_top_level_dir(); + self.append_real_store_node_modules_path(&mut dest, entry_id, Which::Staging); + let base_len = dest.len(); + + let mut changed = false; + for dep in self.store.entries.items_dependencies()[entry_id.get() as usize].slice() { + let dep_name = dependencies[dep.dep_id as usize].name.slice(string_buf); + + dest.set_length(base_len); + let _ = dest.append(dep_name); // OOM/capacity: fire-and-forget + if entry_node_modules_name.is_some_and(|name| strings::eql_long(dep_name, name, true)) { + // same name as the entry itself: nest one node_modules deeper to avoid the collision + let _ = dest.append(b"node_modules"); // OOM/capacity: fire-and-forget + let _ = dest.append(dep_name); // OOM/capacity: fire-and-forget + } + + let mut dep_store_path = AutoAbsPath::init_top_level_dir(); + if uses_global_store { + debug_assert!(self.entry_uses_global_store(dep.entry_id)); + self.append_real_store_path(&mut dep_store_path, dep.entry_id, Which::Final); + } else { + self.append_store_path(&mut dep_store_path, dep.entry_id); + } + + let dest_len = dest.len(); + dest.undo(1); + let target = dest.relative(&dep_store_path); + dest.set_length(dest_len); + + let mut symlinker = Symlinker { + dest: dest.into_sep::<{ PathSeparators::ANY }>(), + target: target.into_sep::<{ PathSeparators::ANY }>(), + #[cfg(windows)] + fallback_junction_target: dep_store_path.into_sep::<{ PathSeparators::ANY }>(), + }; + let result = symlinker.ensure_symlink(strategy); + dest = symlinker.dest.into_sep::<{ PathSeparators::AUTO }>(); + changed |= result?; + } + + Ok(changed) + } + pub(crate) fn link_dependency_bins(&self, parent_entry_id: StoreEntryId) -> crate::Result<()> { let lockfile = self.lockfile(); let store = self.store; diff --git a/src/install/isolated_install/Symlinker.rs b/src/install/isolated_install/Symlinker.rs index 2fed28adcc5e..52c90735abba 100644 --- a/src/install/isolated_install/Symlinker.rs +++ b/src/install/isolated_install/Symlinker.rs @@ -31,11 +31,12 @@ impl Symlinker { } } - pub(crate) fn ensure_symlink(&mut self, strategy: Strategy) -> bun_sys::Result<()> { + // Ok(true) when a link was written. + pub(crate) fn ensure_symlink(&mut self, strategy: Strategy) -> bun_sys::Result { match strategy { Strategy::ExpectMissing => { return match self.symlink() { - Ok(()) => Ok(()), + Ok(()) => Ok(true), Err(symlink_err1) => match symlink_err1.get_errno() { Errno::ENOENT => { let Some(dest_parent) = self.dest.dirname() else { @@ -43,11 +44,11 @@ impl Symlinker { }; let _ = Fd::cwd().make_path(dest_parent); - return self.symlink(); + return self.symlink().map(|()| true); } Errno::EEXIST => { let _ = Fd::cwd().delete_tree(self.dest.slice_z()); - return self.symlink(); + return self.symlink().map(|()| true); } _ => Err(symlink_err1), }, @@ -61,7 +62,7 @@ impl Symlinker { Err(readlink_err) => { return match readlink_err.get_errno() { Errno::ENOENT => match self.symlink() { - Ok(()) => Ok(()), + Ok(()) => Ok(true), Err(symlink_err) => match symlink_err.get_errno() { Errno::ENOENT => { let Some(dest_parent) = self.dest.dirname() else { @@ -69,7 +70,7 @@ impl Symlinker { }; let _ = Fd::cwd().make_path(dest_parent); - return self.symlink(); + return self.symlink().map(|()| true); } _ => Err(symlink_err), }, @@ -99,10 +100,10 @@ impl Symlinker { false }; if is_dir { - return Ok(()); + return Ok(false); } let _ = bun_sys::unlink(self.dest.slice_z()); - return self.symlink(); + return self.symlink().map(|()| true); } }; } @@ -113,14 +114,14 @@ impl Symlinker { current_link = strings::without_trailing_slash(current_link); if strings::eql_long(current_link, self.target.slice_z().as_bytes(), true) { - return Ok(()); + return Ok(false); } #[cfg(windows)] { if strings::eql_long(current_link, self.fallback_junction_target.slice(), true) { - return Ok(()); + return Ok(false); } // this existing link is pointing to the wrong package. @@ -142,7 +143,7 @@ impl Symlinker { let _ = bun_sys::unlink(self.dest.slice_z()); } - return self.symlink(); + return self.symlink().map(|()| true); } } } diff --git a/src/install/lib.rs b/src/install/lib.rs index c2e2204be780..4f156ed065de 100644 --- a/src/install/lib.rs +++ b/src/install/lib.rs @@ -125,6 +125,7 @@ pub mod pnpm; pub mod prune; #[path = "repository.rs"] pub mod repository_real; +pub mod update_transitive; pub mod yarn; /// `repository` — re-export of the file-backed `repository_real` module diff --git a/src/install/lockfile.rs b/src/install/lockfile.rs index 0716f97e6f05..0339b62f7996 100644 --- a/src/install/lockfile.rs +++ b/src/install/lockfile.rs @@ -58,10 +58,14 @@ pub mod catalog_map; pub mod lockfile_json_stringify_for_debugging; #[path = "lockfile/OverrideMap.rs"] pub mod override_map; +#[path = "lockfile/override_selector.rs"] +pub(crate) mod override_selector; #[path = "lockfile/Package.rs"] pub mod package; #[path = "lockfile/pruned_workspaces.rs"] pub(crate) mod pruned_workspaces; +#[path = "lockfile/reachable.rs"] +pub mod reachable; #[path = "lockfile/Tree.rs"] pub mod tree; #[path = "lockfile/printer"] @@ -692,162 +696,6 @@ impl Lockfile { dep.behavior.is_bundled() || !dep.behavior.is_enabled(features) } - fn preprocess_update_requests( - old: &mut Lockfile, - manager: &mut PackageManager, - updates: &mut [UpdateRequest], - exact_versions: bool, - ) -> Result<(), BunError> { - let workspace_ids: Vec> = updates - .iter() - .map(|update| { - let mut ids = old.update_request_workspace_ids(manager, update); - ids.retain(|&id| { - (old.packages.items_dependencies()[id as usize].off as usize) - < old.buffers.dependencies.len() - }); - ids - }) - .collect(); - if workspace_ids.iter().all(Vec::is_empty) { - return Ok(()); - } - - // `bun add x --dev` seeds only the dev slot with a dist-tag; the name's other groups keep their own ranges. - let only_dist_tag_deps = manager.subcommand != crate::Subcommand::Update; - - // Split-borrow: `string_builder!` takes only `string_bytes` + `string_pool`, leaving `packages`/`dependencies`/`resolutions` free. - let mut string_builder = string_builder!(old); - - { - let resolutions_of_yore: &[Resolution] = old.packages.items_resolution(); - let packages_len = old.packages.len(); - - for (update, workspace_ids) in updates.iter().zip(&workspace_ids) { - if update.package_id != invalid_package_id { - continue; - } - for &workspace_package_id in workspace_ids { - let root_deps: &[Dependency] = old.packages.items_dependencies() - [workspace_package_id as usize] - .get(old.buffers.dependencies.as_slice()); - let old_resolutions: &[PackageID] = old.packages.items_resolutions() - [workspace_package_id as usize] - .get(old.buffers.resolutions.as_slice()); - debug_assert_eq!(root_deps.len(), old_resolutions.len()); - for (dep, &old_resolution) in root_deps.iter().zip(old_resolutions.iter()) { - if dep.name_hash == SemverStringBuilder::string_hash(update.name) { - if old_resolution as usize >= packages_len { - continue; - } - let res = resolutions_of_yore[old_resolution as usize]; - if res.tag != ResolutionTag::Npm - || update.version.tag != dependency::Tag::DistTag - || dep.version.tag == dependency::Tag::Catalog - || (only_dist_tag_deps - && dep.version.tag != dependency::Tag::DistTag) - { - continue; - } - - // TODO(dylan-conway): this will need to handle updating dependencies (exact, ^, or ~) and aliases - - let npm_ver = res.npm().version; - let len = bun_core::fmt::count(format_args!( - "{}{}", - if exact_versions { "" } else { "^" }, - npm_ver.fmt(string_builder.string_bytes.as_slice()), - )); - - if len >= SemverString::MAX_INLINE_LEN { - string_builder.cap += len; - } - } - } - } - } - } - - string_builder.allocate()?; - // `clamp()` runs once after the second pass; nothing below can `?` out before it. - - { - let mut temp_buf = [0u8; 513]; - let packages_len = old.packages.len(); - - for (update, workspace_ids) in updates.iter_mut().zip(&workspace_ids) { - update.e_string = None; - if update.package_id != invalid_package_id { - continue; - } - for &workspace_package_id in workspace_ids { - let root_deps_list: DependencySlice = - old.packages.items_dependencies()[workspace_package_id as usize]; - let root_deps: &mut [Dependency] = - root_deps_list.mut_(old.buffers.dependencies.as_mut_slice()); - let old_resolutions: &[PackageID] = old.packages.items_resolutions() - [workspace_package_id as usize] - .get(old.buffers.resolutions.as_slice()); - let resolutions_of_yore: &[Resolution] = old.packages.items_resolution(); - debug_assert_eq!(root_deps.len(), old_resolutions.len()); - for (dep, &old_resolution) in root_deps.iter_mut().zip(old_resolutions.iter()) { - if dep.name_hash == SemverStringBuilder::string_hash(update.name) { - if old_resolution as usize >= packages_len { - continue; - } - let res = resolutions_of_yore[old_resolution as usize]; - if res.tag != ResolutionTag::Npm - || update.version.tag != dependency::Tag::DistTag - || dep.version.tag == dependency::Tag::Catalog - || (only_dist_tag_deps - && dep.version.tag != dependency::Tag::DistTag) - { - continue; - } - - // TODO(dylan-conway): this will need to handle updating dependencies (exact, ^, or ~) and aliases - - let npm_ver = res.npm().version; - let buf = { - let mut cursor: &mut [u8] = &mut temp_buf[..]; - let start_len = cursor.len(); - if write!( - cursor, - "{}{}", - if exact_versions { "" } else { "^" }, - npm_ver.fmt(string_builder.string_bytes.as_slice()), - ) - .is_err() - { - break; - } - let written = start_len - cursor.len(); - &temp_buf[..written] - }; - - let external_version = string_builder.append::(buf); - let sliced = external_version - .value - .sliced(string_builder.string_bytes.as_slice()); - dep.version = dependency::parse( - dep.name, - dep.name_hash, - sliced.slice, - &sliced, - None, - &mut *manager, - ) - .unwrap_or_default(); - } - } - } - } - } - - string_builder.clamp(); - Ok(()) - } - pub fn resolve_catalog_dependency(&self, dep: &Dependency) -> Option { if dep.version.tag != dependency::Tag::Catalog { return Some(dep.version.clone()); @@ -966,35 +814,66 @@ impl Lockfile { } } - /// Package ids of the workspaces whose package.json received `update`: the `--filter` targets, else the cwd workspace. - fn update_request_workspace_ids( + /// Re-runnable: package_json_write_back binds again after re-deriving the declared columns. + #[cold] + #[inline(never)] + pub(crate) fn bind_update_requests( &self, - manager: &mut PackageManager, - update: &UpdateRequest, - ) -> Vec { - match &manager.pending_filtered_write { - Some(pending) => pending.workspace_ids_receiving(self, update.name_hash), - None => vec![ - manager - .root_package_id - .get(self, manager.workspace_name_hash), - ], + pending: Option<&crate::package_manager_real::add_remove_with_filter::PendingWrite>, + workspace_name_hash: Option, + updates: &mut [UpdateRequest], + ) { + // `version_buf` is a raw (ptr, len) into `self.buffers.string_bytes`, finalized by now and outliving every `UpdateRequest`. + let string_buf = self.buffers.string_bytes.as_slice(); + let string_buf_ptr = bun_ptr::RawSlice::new(string_buf); + let slice = self.packages.slice(); + let cwd_workspace = [self.get_workspace_package_id(workspace_name_hash)]; + + 'request_updated: for update in updates.iter_mut() { + update.e_string = None; + let filtered: Vec; + let workspace_ids: &[PackageID] = match pending { + Some(pending) => { + filtered = pending.workspace_ids_receiving(self, update.name_hash); + &filtered + } + None => &cwd_workspace, + }; + for &workspace_package_id in workspace_ids { + let dep_list = slice.items_dependencies()[workspace_package_id as usize]; + let res_list = slice.items_resolutions()[workspace_package_id as usize]; + let workspace_deps: &[Dependency] = + dep_list.get(self.buffers.dependencies.as_slice()); + let resolved_ids: &[PackageID] = res_list.get(self.buffers.resolutions.as_slice()); + debug_assert_eq!(resolved_ids.len(), workspace_deps.len()); + for (&package_id, dep) in resolved_ids.iter().zip(workspace_deps.iter()) { + if update.matches(dep, string_buf) { + if package_id as usize > self.packages.len() { + continue; + } + update.version_buf = string_buf_ptr; + update.version = dep.version.clone(); + update.package_id = package_id; + + continue 'request_updated; + } + } + } } } // `#[inline(never)]` keeps the panic/format machinery from // `bun_core::output` (pulled in by the cold helpers below) out of callers; - // the hot copy/remap loop stays in this body while the three cold sections - // — update-request preprocessing, verbose timer reporting, and the - // trusted/patched-dependency migration — are outlined so a no-change - // `bun install` (install/fastify bench) does not page them in. + // the hot copy/remap loop stays in this body while the cold sections + // — update-request binding (`bind_update_requests`), verbose timer + // reporting, and the patched-dependency migration — are outlined so a + // no-change `bun install` (install/fastify bench) does not page them in. #[inline(never)] pub(crate) fn clean_with_logger( &mut self, manager: &mut PackageManager, updates: &mut [UpdateRequest], log: &mut bun_ast::Log, - exact_versions: bool, log_level: LogLevel, ) -> Result, BunError> { let old: &mut Lockfile = self; @@ -1020,10 +899,6 @@ impl Lockfile { let old_preinstall_state = preinstall_state.clone(); preinstall_state.fill(Install::PreinstallState::Unknown); - if !updates.is_empty() { - clean_preprocess_update_requests_cold(old, manager, updates, exact_versions)?; - } - // Caller owns the new lockfile; return `Box` so Drop reclaims // it (never `Box::leak` to satisfy a lifetime). let mut new: Box = Box::default(); @@ -1063,7 +938,7 @@ impl Lockfile { let clone_queue_ = PendingResolutions::new(); // Explicit `&mut *` reborrows so `old`/`manager`/`new` are // released back to this scope once `cloner` is dropped. - let keep_optional_peer_targets = !manager.summary.has_diffs(); + let keep_optional_peer_targets = !manager.summary.changes_resolutions(); let mut cloner = Cloner { old: &mut *old, lockfile: &mut *new, @@ -1189,63 +1064,12 @@ impl Lockfile { clean_migrate_patched_dependencies_cold(old, &mut new)?; } - // Read catalog resolutions from `new`: `old` still holds the stale dependency slots that `bun update` orphaned. - if !updates.is_empty() && manager.options.add_catalog.is_some() { - crate::package_manager_real::add_catalog::rewrite_lockfile_entries( - &mut new, manager, updates, - )?; - } - if !manager.updating_catalogs.is_empty() { - crate::package_manager_real::package_json_editor::resolve_catalog_updates( - &mut new, - manager, - exact_versions, - )?; - } - - // Don't allow invalid memory to happen if !updates.is_empty() { - // `UpdateRequest.version_buf` is a raw `*const [u8]` (PORTING.md - // type-map: `[]const u8` struct-field, ARENA-class). The slice - // points into `new.buffers.string_bytes`; `new` is *returned* to - // the caller below and `string_bytes` is finalized at this point - // (cloner.flush() and the patched-dep StringBuilder have both - // run), so the storage outlives every `UpdateRequest` the caller - // threads it through. No lifetime extension — store the raw - // (ptr, len) and let `UpdateRequest::version_buf()` reborrow at - // each read site. - let string_buf = new.buffers.string_bytes.as_slice(); - let string_buf_ptr = bun_ptr::RawSlice::new(string_buf); - let slice = new.packages.slice(); - - // updates might be applied to the root package.json or one - // of the workspace package.json files. - 'request_updated: for update in updates.iter_mut() { - if update.package_id != invalid_package_id { - continue; - } - for workspace_package_id in new.update_request_workspace_ids(manager, update) { - let dep_list = slice.items_dependencies()[workspace_package_id as usize]; - let res_list = slice.items_resolutions()[workspace_package_id as usize]; - let workspace_deps: &[Dependency] = - dep_list.get(new.buffers.dependencies.as_slice()); - let resolved_ids: &[PackageID] = - res_list.get(new.buffers.resolutions.as_slice()); - debug_assert_eq!(resolved_ids.len(), workspace_deps.len()); - for (&package_id, dep) in resolved_ids.iter().zip(workspace_deps.iter()) { - if update.matches(dep, string_buf) { - if package_id as usize > new.packages.len() { - continue; - } - update.version_buf = string_buf_ptr; - update.version = dep.version.clone(); - update.package_id = package_id; - - continue 'request_updated; - } - } - } - } + new.bind_update_requests( + manager.pending_filtered_write.as_deref(), + manager.workspace_name_hash, + updates, + ); } if log_level.is_verbose() { @@ -1259,21 +1083,10 @@ impl Lockfile { // ──────────────────────────────────────────────────────────────────────────── // clean_with_logger cold helpers — outlined so the hot copy/remap loop in the // main body is contiguous in `.text` and the install/fastify no-change bench -// does not fault in update-request rewriting, patched-dep migration, or the +// does not fault in update-request binding, patched-dep migration, or the // verbose timer/format machinery. // ──────────────────────────────────────────────────────────────────────────── -#[cold] -#[inline(never)] -fn clean_preprocess_update_requests_cold( - old: &mut Lockfile, - manager: &mut PackageManager, - updates: &mut [UpdateRequest], - exact_versions: bool, -) -> Result<(), BunError> { - Lockfile::preprocess_update_requests(old, manager, updates, exact_versions) -} - #[cold] #[inline(never)] fn clean_verbose_timer_start() -> Result { diff --git a/src/install/lockfile/CatalogMap.rs b/src/install/lockfile/CatalogMap.rs index 1cf5cde36ca7..3c5a6a5db952 100644 --- a/src/install/lockfile/CatalogMap.rs +++ b/src/install/lockfile/CatalogMap.rs @@ -59,17 +59,32 @@ impl CatalogMap { catalog_name: &[u8], dep_name: &[u8], ) -> Option<(Option, usize)> { + let has_default = self.default.count() > 0; + let has_groups = self.groups.count() > 0; + if !has_default && !has_groups { + return None; + } let dep_key = String::init(dep_name, dep_name); let dep_ctx = ArrayHashContext { arg_buf: dep_name, existing_buf: string_buf, }; + let dep_hash = dep_ctx.hash(dep_key); + let probe = |map: &Map| { + map.get_index_adapted_raw(dep_hash, |existing: &String, i| { + dep_ctx.eql(dep_key, *existing, i) + }) + }; let in_default = || { - self.default - .get_index_adapted(&dep_key, &dep_ctx) - .map(|i| (None, i)) + if !has_default { + return None; + } + probe(&self.default).map(|i| (None, i)) }; let in_group = |name: &[u8]| { + if !has_groups { + return None; + } let ctx = ArrayHashContext { arg_buf: name, existing_buf: string_buf, @@ -77,7 +92,7 @@ impl CatalogMap { let g = self .groups .get_index_adapted(&String::init(name, name), &ctx)?; - let i = self.groups.values()[g].get_index_adapted(&dep_key, &dep_ctx)?; + let i = probe(&self.groups.values()[g])?; Some((Some(g), i)) }; if catalog_name.is_empty() { @@ -100,20 +115,6 @@ impl CatalogMap { Some(&map.values()[i]) } - pub(crate) fn find_mut<'a>( - &'a mut self, - string_buf: &[u8], - catalog_name: &[u8], - dep_name: &[u8], - ) -> Option<&'a mut Dependency> { - let (group, i) = self.locate(string_buf, catalog_name, dep_name)?; - let map = match group { - Some(g) => &mut self.groups.values_mut()[g], - None => &mut self.default, - }; - Some(&mut map.values_mut()[i]) - } - pub(crate) fn get_ref<'a>( &'a self, string_buf: &[u8], diff --git a/src/install/lockfile/OverrideMap.rs b/src/install/lockfile/OverrideMap.rs index 126dd8a4a90c..072e15942b65 100644 --- a/src/install/lockfile/OverrideMap.rs +++ b/src/install/lockfile/OverrideMap.rs @@ -1,16 +1,30 @@ +use core::cell::RefCell; use core::cmp::Ordering; +use crate::DependencyID; use crate::Error; +use crate::package_manager::workspace_package_json_cache::{GetJSONOptions, GetResult}; +use crate::resolution::Tag as ResolutionTag; +use crate::{PackageID, invalid_package_id}; use bun_collections::ArrayHashMap; -use bun_core::strings; -use bun_install::dependency::{self, Behavior, Dependency, DependencyExt as _}; +use bun_install::dependency::{ + self, Behavior, Dependency, DependencyExt as _, NpmAliasRegistry, Tag as VersionTag, + VersionExt as _, +}; use bun_install::{PackageManager, PackageNameHash}; use bun_output::{declare_scope, scoped_log}; +use bun_paths::resolve_path; use bun_semver::String as SemverString; +use bun_semver::Version as SemverVersion; use bun_semver::string::Builder as SemverBuilder; -use super::package::value_loc_of; -use super::{StringBuilder, package::Package}; +use super::override_selector::{ + PackageSelector, Selector, SelectorError, parse_package_segment, parse_selector, +}; +use super::package::PackageColumns as _; +use super::package::workspace_map::WorkspaceMap; +use super::package::{DependencyGroup, value_loc_of}; +use super::{Lockfile, StringBuilder, package::Package}; // LAYERING NOTE: package.json is parsed by `bun_parsers::json` which // produces the T2 value-shaped `bun_ast::Expr` (aliased as // `crate::bun_json::Expr`), NOT the full T4 `bun_ast::Expr`. JSON parse @@ -19,29 +33,248 @@ use crate::bun_json::Expr; declare_scope!(OverrideMap, visible); +/// A rule that is not a plain name: scoped to a parent, to the range the dependent declares, or both. +pub struct ScopedOverride { + pub(crate) parent: Option, + pub(crate) target_range: dependency::Version, + pub(crate) dep: Dependency, +} + +impl ScopedOverride { + #[inline] + pub(crate) fn parent_has_range(&self) -> bool { + self.parent + .as_ref() + .is_some_and(|parent| parent.version.tag == VersionTag::Npm) + } + + #[inline] + pub(crate) fn has_target_range(&self) -> bool { + self.target_range.tag == VersionTag::Npm + } +} + +fn cmp_range_text(l: &ScopedOverride, r: &ScopedOverride, buf: &[u8]) -> Ordering { + let by_parent = match (&l.parent, &r.parent) { + (Some(a), Some(b)) if l.parent_has_range() && r.parent_has_range() => { + a.version.literal.order(b.version.literal, buf, buf) + } + _ => Ordering::Equal, + }; + by_parent.then_with(|| { + l.target_range + .literal + .order(r.target_range.literal, buf, buf) + }) +} + +/// dependency id -> owning package id, filled lazily because packages are only ever appended while a map is live. +#[derive(Default)] +struct OwnerIndex { + by_dep: Vec, + packages_indexed: usize, +} + #[derive(Default)] pub struct OverrideMap { // `ArrayHashMap` defaults to identity hashing for integer keys. pub(crate) map: ArrayHashMap, + pub(crate) scoped: Vec, + scoped_names: ArrayHashMap, + owner_index: RefCell, +} + +#[derive(Clone, Copy, PartialEq, Eq)] +enum Field { + Overrides, + Resolutions, +} + +impl Field { + fn label(self) -> &'static str { + match self { + Field::Overrides => "override", + Field::Resolutions => "resolution", + } + } + + fn json_name(self) -> &'static str { + match self { + Field::Overrides => "overrides", + Field::Resolutions => "resolutions", + } + } + + fn missing_name_message(self) -> &'static str { + match self { + Field::Overrides => "Missing overridden package name", + Field::Resolutions => "Missing resolution package name", + } + } +} + +struct Ambiguous; + +struct ParseContext<'a, 'b> { + field: Field, + pm: &'a mut PackageManager, + lockfile_dependencies: &'a [Dependency], + root_package: &'a Package, + log: &'a mut bun_ast::Log, + source: &'a bun_ast::Source, + workspace_names: &'a WorkspaceMap, + builder: &'a mut StringBuilder<'b>, } impl OverrideMap { - /// In the future, this `get` function should handle multi-level resolutions. This is difficult right - /// now because given a Dependency ID, there is no fast way to trace it to its package. - /// - /// A potential approach is to add another buffer to the lockfile that maps Dependency ID to Package ID, - /// and from there `OverrideMap.map` can have a union as the value, where the union is between "override all" - /// and "here is a list of overrides depending on the package that imported" similar to PackageIndex above. - pub(crate) fn get(&self, name_hash: PackageNameHash) -> Option { + /// Precedence: ranged parent > parent > none; within a tier a matching target range wins; ties go to the parent range text, then the target range text, that sorts first; then the flat map. + pub(crate) fn get( + &self, + lockfile: &Lockfile, + dependency_id: DependencyID, + name_hash: PackageNameHash, + ) -> Option { scoped_log!(OverrideMap, "looking up override for {:x}", name_hash); + if self.scoped.is_empty() { + return self.get_flat(name_hash); + } + if self.scoped_names.contains(&name_hash) { + if let Some(rule) = self.scoped_rule_for(lockfile, dependency_id, name_hash) { + return Some(rule.dep.version.clone()); + } + } + self.get_flat(name_hash) + } + + #[inline] + fn get_flat(&self, name_hash: PackageNameHash) -> Option { if self.map.count() == 0 { return None; } self.map.get(&name_hash).map(|dep| dep.version.clone()) } - /// Like `get().is_some()` but also compares the stored name so a hash - /// collision cannot produce a false positive. + fn scoped_rule_for<'s>( + &'s self, + lockfile: &Lockfile, + dependency_id: DependencyID, + name_hash: PackageNameHash, + ) -> Option<&'s ScopedOverride> { + let buf = lockfile.buffers.string_bytes.as_slice(); + let declared = &lockfile.buffers.dependencies[dependency_id as usize].version; + let mut owner: Option)>> = None; + let mut best: Option<(&'s ScopedOverride, u8)> = None; + + for rule in &self.scoped { + if rule.dep.name_hash != name_hash { + continue; + } + let mut tier: u8 = 0; + if let Some(parent) = &rule.parent { + let Some((owner_name_hash, owner_version)) = + *owner.get_or_insert_with(|| self.owner_of(lockfile, dependency_id)) + else { + continue; + }; + if owner_name_hash != parent.name_hash { + continue; + } + if rule.parent_has_range() { + let Some(owner_version) = owner_version else { + continue; + }; + if !parent + .version + .npm() + .version + .satisfies(owner_version, buf, buf) + { + continue; + } + tier = 4; + } else { + tier = 2; + } + } + if rule.has_target_range() { + if best.is_some_and(|(_, best_tier)| best_tier > tier + 1) { + continue; + } + if declared.tag != VersionTag::Npm + || !rule.target_range.npm().version.intersects( + buf, + &declared.npm().version, + buf, + ) + { + continue; + } + tier += 1; + } + let replace = match best { + None => true, + Some((best_rule, best_tier)) => { + tier > best_tier + || (tier == best_tier + && cmp_range_text(rule, best_rule, buf) == Ordering::Less) + } + }; + if replace { + best = Some((rule, tier)); + } + } + + best.map(|(rule, _)| rule) + } + + fn owner_of( + &self, + lockfile: &Lockfile, + dependency_id: DependencyID, + ) -> Option<(PackageNameHash, Option)> { + let owner_id = self.owner_package_id(lockfile, dependency_id); + if owner_id == invalid_package_id { + return None; + } + let owner_id = owner_id as usize; + let owner_name_hash = lockfile.packages.items_name_hash()[owner_id]; + let resolution = &lockfile.packages.items_resolution()[owner_id]; + let owner_version = match resolution.tag { + ResolutionTag::Npm => Some(resolution.npm().version), + ResolutionTag::Workspace => lockfile.workspace_versions.get(&owner_name_hash).copied(), + _ => None, + }; + Some((owner_name_hash, owner_version)) + } + + fn owner_package_id(&self, lockfile: &Lockfile, dependency_id: DependencyID) -> PackageID { + let dep_slices = lockfile.packages.items_dependencies(); + let dependencies_len = lockfile.buffers.dependencies.len(); + let mut index = self.owner_index.borrow_mut(); + let index = &mut *index; + if index.packages_indexed < dep_slices.len() { + for (pkg_id, slice) in dep_slices.iter().enumerate().skip(index.packages_indexed) { + let end = (slice.end() as usize).min(dependencies_len); + let begin = (slice.begin() as usize).min(end); + if end > index.by_dep.len() { + index.by_dep.resize(end, invalid_package_id); + } + index.by_dep[begin..end].fill(pkg_id as PackageID); + } + index.packages_indexed = dep_slices.len(); + } + let owner = index + .by_dep + .get(dependency_id as usize) + .copied() + .unwrap_or(invalid_package_id); + debug_assert!( + owner == invalid_package_id || dep_slices[owner as usize].contains(dependency_id) + ); + owner + } + + /// Plain rules only: a scoped rule does not make every edge of this name root-authored (trust checks in PackageManagerEnqueue.rs / PackageInstaller.rs). pub(crate) fn contains_name( &self, name_hash: PackageNameHash, @@ -56,6 +289,89 @@ impl OverrideMap { .is_some_and(|dep| dep.name.slice(buf) == name) } + #[inline] + pub(crate) fn is_empty(&self) -> bool { + self.map.count() == 0 && self.scoped.is_empty() + } + + #[inline] + pub(crate) fn has_scoped(&self) -> bool { + !self.scoped.is_empty() + } + + #[inline] + pub(crate) fn has_rule_for_name(&self, name_hash: PackageNameHash) -> bool { + self.map.contains(&name_hash) || self.scoped_names.contains(&name_hash) + } + + pub(crate) fn append_overridden_name_hashes(&self, out: &mut Vec) { + out.extend_from_slice(self.map.keys()); + out.extend_from_slice(self.scoped_names.keys()); + } + + pub(crate) fn append_catalog_valued_name_hashes(&self, out: &mut Vec) { + for (name_hash, dep) in self.map.iter() { + if dep.version.tag == VersionTag::Catalog { + out.push(*name_hash); + } + } + for rule in &self.scoped { + if rule.dep.version.tag == VersionTag::Catalog { + out.push(rule.dep.name_hash); + } + } + } + + pub(crate) fn changed( + from: &mut OverrideMap, + from_string_bytes: &[u8], + to: &mut OverrideMap, + to_string_bytes: &[u8], + ) -> bool { + if from.map.count() != to.map.count() || from.scoped.len() != to.scoped.len() { + return true; + } + if from.is_empty() { + return false; + } + + from.sort(from_string_bytes); + to.sort(to_string_bytes); + + let flat_changed = + from.map + .iter() + .zip(to.map.iter()) + .any(|((from_k, from_dep), (to_k, to_dep))| { + from_k != to_k || !from_dep.eql(to_dep, from_string_bytes, to_string_bytes) + }); + if flat_changed { + return true; + } + + from.scoped + .iter() + .zip(&to.scoped) + .any(|(from_rule, to_rule)| { + let parent_changed = match (&from_rule.parent, &to_rule.parent) { + (None, None) => false, + (Some(from_parent), Some(to_parent)) => { + !from_parent.eql(to_parent, from_string_bytes, to_string_bytes) + } + _ => true, + }; + parent_changed + || !from_rule.target_range.eql( + &to_rule.target_range, + from_string_bytes, + to_string_bytes, + ) + || !from_rule + .dep + .eql(&to_rule.dep, from_string_bytes, to_string_bytes) + }) + } + // Every caller already holds `&mut self` on `lockfile.overrides`, so // accept just the string buffer (the only lockfile field `sort` reads) // rather than the whole `Lockfile`. @@ -63,6 +379,28 @@ impl OverrideMap { self.map.sort(|_, deps: &[Dependency], l, r| { deps[l].name.order(deps[r].name, string_bytes, string_bytes) == Ordering::Less }); + self.scoped.sort_by(|l, r| { + l.parent + .is_some() + .cmp(&r.parent.is_some()) + .then_with(|| match (&l.parent, &r.parent) { + (Some(lp), Some(rp)) => lp + .name + .order(rp.name, string_bytes, string_bytes) + .then_with(|| { + lp.version + .literal + .order(rp.version.literal, string_bytes, string_bytes) + }), + _ => Ordering::Equal, + }) + .then_with(|| l.dep.name.order(r.dep.name, string_bytes, string_bytes)) + .then_with(|| { + l.target_range + .literal + .order(r.target_range.literal, string_bytes, string_bytes) + }) + }); } /// Accepts `lockfile.buffers.string_bytes` directly (rather than the whole @@ -72,6 +410,15 @@ impl OverrideMap { for dep in self.map.values() { dep.count(string_bytes, builder); } + for rule in &self.scoped { + if let Some(parent) = &rule.parent { + parent.count(string_bytes, builder); + } + if rule.has_target_range() { + builder.count(rule.target_range.literal.slice(string_bytes)); + } + rule.dep.count(string_bytes, builder); + } } /// The new-side buffer lives inside `new_builder`, so no separate @@ -79,7 +426,7 @@ impl OverrideMap { /// `pm` is generic over `NpmAliasRegistry` (not `&mut PackageManager`) so a /// caller already holding `&mut manager.lockfile` can pass /// `&mut manager.known_npm_aliases` instead of the whole manager. - pub(crate) fn clone( + pub(crate) fn clone( &self, pm: &mut PM, old_string_bytes: &[u8], @@ -93,34 +440,198 @@ impl OverrideMap { .put_assume_capacity(*k, v.clone_in(pm, old_string_bytes, new_builder)?); } + if !self.scoped.is_empty() { + new.scoped = Vec::with_capacity(self.scoped.len()); + new.scoped_names + .ensure_total_capacity(self.scoped_names.count())?; + for rule in &self.scoped { + let parent = match &rule.parent { + None => None, + Some(parent) => Some(parent.clone_in(pm, old_string_bytes, new_builder)?), + }; + let dep = rule.dep.clone_in(pm, old_string_bytes, new_builder)?; + let target_range = if rule.has_target_range() { + clone_range(pm, &dep, &rule.target_range, old_string_bytes, new_builder) + } else { + dependency::Version::default() + }; + new.push_scoped( + ScopedOverride { + parent, + target_range, + dep, + }, + new_builder.string_bytes.as_slice(), + ); + } + } + Ok(new) } + /// Replaces the rule with the same (parent, parent range, target, target range); `buf` is the buffer `rule` was appended into. + pub(crate) fn push_scoped(&mut self, rule: ScopedOverride, buf: &[u8]) { + if self.scoped_names.contains(&rule.dep.name_hash) { + if let Some(existing) = self.scoped.iter_mut().find(|existing| { + existing.dep.name_hash == rule.dep.name_hash + && match (&existing.parent, &rule.parent) { + (None, None) => true, + (Some(a), Some(b)) => { + a.name_hash == b.name_hash + && a.version.literal.eql(b.version.literal, buf, buf) + } + _ => false, + } + && existing + .target_range + .literal + .eql(rule.target_range.literal, buf, buf) + }) { + *existing = rule; + return; + } + } else { + self.scoped_names.insert(rule.dep.name_hash, ()); + } + self.scoped.push(rule); + } + + /// Row constructor for bun.lock / pnpm-lock.yaml; `Ok(false)` when `value`, the parent range or the target range does not parse. + pub(crate) fn put_lockfile_rule( + &mut self, + parent: Option>, + target: PackageSelector<'_>, + value: &[u8], + buf: &mut bun_semver::string::Buf<'_>, + log: &mut bun_ast::Log, + mut manager: Option<&mut PackageManager>, + ) -> Result { + let name_hash = SemverBuilder::string_hash(target.name); + let name = buf.append_with_hash(target.name, name_hash)?; + let Some(target_range) = lockfile_range( + name, + name_hash, + target.range, + buf, + log, + manager.as_deref_mut(), + )? + else { + return Ok(false); + }; + + let parent = match parent { + None => None, + Some(selector) => { + let parent_name_hash = SemverBuilder::string_hash(selector.name); + let parent_name = buf.append_with_hash(selector.name, parent_name_hash)?; + let Some(version) = lockfile_range( + parent_name, + parent_name_hash, + selector.range, + buf, + log, + manager.as_deref_mut(), + )? + else { + return Ok(false); + }; + Some(Dependency { + name: parent_name, + name_hash: parent_name_hash, + version, + behavior: Behavior::default(), + }) + } + }; + + let value = buf.append(value)?; + let sliced = value.sliced(buf.bytes.as_slice()); + let Some(version) = dependency::parse(name, name_hash, sliced.slice, &sliced, log, manager) + else { + return Ok(false); + }; + let dep = Dependency { + name, + name_hash, + version, + behavior: Behavior::default(), + }; + + match (parent, target_range.tag == VersionTag::Npm) { + (None, false) => self.map.put(name_hash, dep)?, + (parent, _) => self.push_scoped( + ScopedOverride { + parent, + target_range, + dep, + }, + buf.bytes.as_slice(), + ), + } + Ok(true) + } + // the rest of this struct is expression parsing code: - // No `lockfile` param: JSON strings are already UTF-8 here, and omitting - // it avoids the `&mut lockfile.overrides` / `&mut lockfile` alias at the - // only call site. - pub(crate) fn parse_count(&mut self, expr: Expr, builder: &mut StringBuilder) { - if let Some(overrides) = expr.as_property(b"overrides") { - overrides.expr.for_each_property(|key, _key_loc, value| { - builder.count(key); - if let Some(s) = value.as_utf8_string_literal() { - builder.count(s); - } else if let Some(dot) = value.as_property(b".") { - if let Some(s) = dot.expr.as_utf8_string_literal() { - builder.count(s); + pub(crate) fn parse_count( + &mut self, + pm: &mut PackageManager, + log: &mut bun_ast::Log, + json_source: &bun_ast::Source, + workspace_names: &WorkspaceMap, + expr: Expr, + builder: &mut StringBuilder, + ) { + let (field, field_expr) = if let Some(overrides) = expr.as_property(b"overrides") { + (Field::Overrides, overrides.expr) + } else if let Some(resolutions) = expr.as_property(b"resolutions") { + (Field::Resolutions, resolutions.expr) + } else { + return; + }; + + field_expr.for_each_property(|key, _key_loc, value| { + builder.count(key); + if let Some(value) = value.as_utf8_string_literal() { + if let Ok(Selector { parent, target }) = parse_selector(key) { + builder.count(target.name); + builder.count(target.range); + if let Some(parent) = parent { + builder.count(parent.name); + builder.count(parent.range); } } - }); - } else if let Some(resolutions) = expr.as_property(b"resolutions") { - resolutions.expr.for_each_property(|key, _key_loc, value| { - builder.count(key); - if let Some(v) = value.as_utf8_string_literal() { - builder.count(v); + builder.count(value); + count_ref_value(pm, log, json_source, workspace_names, &expr, value, builder); + return; + } + if field != Field::Overrides || !value.is_object() { + return; + } + if let Ok(parent) = parse_package_segment(key) { + builder.count(parent.name); + builder.count(parent.range); + } + value.for_each_property(|child_key, _child_key_loc, child_value| { + if let Ok(selector) = parse_selector(child_key) { + builder.count(selector.target.name); + builder.count(selector.target.range); + } + if let Some(child_value) = child_value.as_utf8_string_literal() { + builder.count(child_value); + count_ref_value( + pm, + log, + json_source, + workspace_names, + &expr, + child_value, + builder, + ); } }); - } + }); } /// Given a package json expression, detect and parse override configuration into the given override map. @@ -132,50 +643,51 @@ impl OverrideMap { root_package: &Package, log: &mut bun_ast::Log, json_source: &bun_ast::Source, + workspace_names: &WorkspaceMap, expr: Expr, builder: &mut StringBuilder, ) -> Result<(), Error> { - debug_assert!(self.map.count() == 0); // only call parse once - if let Some(overrides) = expr.as_property(b"overrides") { - self.parse_from_overrides( - pm, - lockfile_dependencies, - root_package, - json_source, - log, - overrides.expr, - builder, - )?; + debug_assert!(self.map.count() == 0 && self.scoped.is_empty()); // only call parse once + let (field, field_expr) = if let Some(overrides) = expr.as_property(b"overrides") { + (Field::Overrides, overrides.expr) } else if let Some(resolutions) = expr.as_property(b"resolutions") { - self.parse_from_resolutions( - pm, - lockfile_dependencies, - root_package, - json_source, - log, - resolutions.expr, - builder, - )?; + (Field::Resolutions, resolutions.expr) + } else { + return Ok(()); + }; + let mut ctx = ParseContext { + field, + pm, + lockfile_dependencies, + root_package, + log, + source: json_source, + workspace_names, + builder, + }; + match field { + Field::Overrides => self.parse_from_overrides(&mut ctx, field_expr)?, + Field::Resolutions => self.parse_from_resolutions(&mut ctx, field_expr)?, } - scoped_log!(OverrideMap, "parsed {} overrides", self.map.count()); + scoped_log!( + OverrideMap, + "parsed {} overrides, {} scoped", + self.map.count(), + self.scoped.len() + ); Ok(()) } /// https://docs.npmjs.com/cli/v9/configuring-npm/package-json#overrides fn parse_from_overrides( &mut self, - pm: &mut PackageManager, - lockfile_dependencies: &[Dependency], - root_package: &Package, - source: &bun_ast::Source, - log: &mut bun_ast::Log, + ctx: &mut ParseContext<'_, '_>, expr: Expr, - builder: &mut StringBuilder, ) -> Result<(), Error> { if !expr.is_object() { - log.add_warning_fmt( - Some(source), - value_loc_of(source, expr.loc), + ctx.log.add_warning_fmt( + Some(ctx.source), + value_loc_of(ctx.source, expr.loc), format_args!("\"overrides\" must be an object"), ); return Err(crate::Error::Invalid); @@ -185,98 +697,97 @@ impl OverrideMap { expr.try_for_each_property(|k, key_loc, value_expr| { if k.is_empty() { - log.add_warning_fmt( - Some(source), + ctx.log.add_warning_fmt( + Some(ctx.source), key_loc, - format_args!("Missing overridden package name"), + format_args!("{}", ctx.field.missing_name_message()), ); return Ok(()); } - let name_hash = SemverBuilder::string_hash(k); - - let value_expr_loc = - crate::bun_json::value_loc_of_property(&source.contents, key_loc, &value_expr); - let (value, value_loc): (Expr, _) = 'value: { - // for one level deep, we will only support a string and { ".": value } - if value_expr.data.is_e_string() { - break 'value (value_expr, value_expr_loc); - } else if value_expr.is_object() { - if let Some(dot) = value_expr.as_property(b".") { - if dot.expr.data.is_e_string() { - if value_expr.property_count() > 1 { - log.add_warning_fmt( - Some(source), - value_expr_loc, - format_args!( - "Bun currently does not support nested \"overrides\"" - ), - ); - } - break 'value ( - dot.expr, - crate::bun_json::value_loc_of_property( - &source.contents, - dot.loc, - &dot.expr, - ), - ); - } else { - log.add_warning_fmt( - Some(source), - value_expr_loc, - format_args!( - "Invalid override value for \"{}\"", - bstr::BStr::new(k) - ), - ); - return Ok(()); - } - } else { - log.add_warning_fmt( - Some(source), - value_expr_loc, - format_args!("Bun currently does not support nested \"overrides\""), - ); - return Ok(()); - } - } - log.add_warning_fmt( - Some(source), - value_expr_loc, + let value_loc = + crate::bun_json::value_loc_of_property(&ctx.source.contents, key_loc, &value_expr); + if let Some(value) = value_expr.as_utf8_string_literal() { + return self.parse_string_rule(ctx, k, key_loc, value, value_loc); + } + if !value_expr.is_object() { + ctx.log.add_warning_fmt( + Some(ctx.source), + value_loc, format_args!("Invalid override value for \"{}\"", bstr::BStr::new(k)), ); return Ok(()); + } + + let parent = match parse_package_segment(k) { + Ok(parent) => parent, + Err(err) => { + warn_selector_error(ctx, k, key_loc, err); + return Ok(()); + } }; - let version_str = value - .as_utf8_string_literal() - .expect("infallible: is_string checked"); - if version_str.starts_with(b"patch:") { - // TODO(dylan-conway): apply .patch files to packages - log.add_warning_fmt( - Some(source), - key_loc, - format_args!("Bun currently does not support patched package \"overrides\""), + value_expr.try_for_each_property(|child_key, child_key_loc, child_value_expr| { + let child_value_loc = crate::bun_json::value_loc_of_property( + &ctx.source.contents, + child_key_loc, + &child_value_expr, ); - return Ok(()); - } + if child_value_expr.is_object() { + warn_selector_error(ctx, child_key, child_key_loc, SelectorError::TooDeep); + return Ok(()); + } + let Some(child_value) = child_value_expr.as_utf8_string_literal() else { + ctx.log.add_warning_fmt( + Some(ctx.source), + child_value_loc, + format_args!( + "Invalid override value for \"{}\"", + bstr::BStr::new(child_key) + ), + ); + return Ok(()); + }; - if let Some(version) = parse_override_value( - "override", - lockfile_dependencies, - pm, - root_package, - source, - value_loc, - log, - k, - version_str, - builder, - )? { - self.map.put_assume_capacity(name_hash, version); - } - Ok(()) + if child_key == b"." { + return self.put_rule( + ctx, + None, + &PackageSelector { + name: parent.name, + range: parent.range, + }, + key_loc, + child_value, + child_value_loc, + ); + } + + let target = match parse_selector(child_key) { + Ok(Selector { + parent: None, + target, + }) => target, + Ok(Selector { + parent: Some(_), .. + }) => { + warn_selector_error(ctx, child_key, child_key_loc, SelectorError::TooDeep); + return Ok(()); + } + Err(err) => { + warn_selector_error(ctx, child_key, child_key_loc, err); + return Ok(()); + } + }; + self.put_rule( + ctx, + Some(&parent), + &target, + child_key_loc, + child_value, + child_value_loc, + ) + }) }) } @@ -284,39 +795,30 @@ impl OverrideMap { /// yarn berry: https://yarnpkg.com/configuration/manifest#resolutions fn parse_from_resolutions( &mut self, - pm: &mut PackageManager, - lockfile_dependencies: &[Dependency], - root_package: &Package, - source: &bun_ast::Source, - log: &mut bun_ast::Log, + ctx: &mut ParseContext<'_, '_>, expr: Expr, - builder: &mut StringBuilder, ) -> Result<(), Error> { if !expr.is_object() { - log.add_warning_fmt( - Some(source), - value_loc_of(source, expr.loc), + ctx.log.add_warning_fmt( + Some(ctx.source), + value_loc_of(ctx.source, expr.loc), format_args!("\"resolutions\" must be an object with string values"), ); return Ok(()); } self.map.ensure_unused_capacity(expr.property_count())?; - expr.try_for_each_property(|key, key_loc, value| { - let mut k = key; - if k.starts_with(b"**/") { - k = &k[3..]; - } + expr.try_for_each_property(|k, key_loc, value_expr| { if k.is_empty() { - log.add_warning_fmt( - Some(source), + ctx.log.add_warning_fmt( + Some(ctx.source), key_loc, - format_args!("Missing resolution package name"), + format_args!("{}", ctx.field.missing_name_message()), ); return Ok(()); } - let Some(version_str) = value.as_utf8_string_literal() else { - log.add_warning_fmt( - Some(source), + let Some(value) = value_expr.as_utf8_string_literal() else { + ctx.log.add_warning_fmt( + Some(ctx.source), key_loc, format_args!( "Expected string value for resolution \"{}\"", @@ -325,144 +827,324 @@ impl OverrideMap { ); return Ok(()); }; - // currently we only support one level deep, so we should error if there are more than one - // - "foo/bar": - // - "@namespace/hello/world" - if k[0] == b'@' { - let Some(first_slash) = strings::index_of_char(k, b'/') else { - log.add_warning_fmt( - Some(source), - key_loc, - format_args!("Invalid package name \"{}\"", bstr::BStr::new(k)), - ); - return Ok(()); - }; - if strings::index_of_char(&k[first_slash as usize + 1..], b'/').is_some() { - log.add_warning_fmt( - Some(source), - key_loc, - format_args!("Bun currently does not support nested \"resolutions\""), - ); - return Ok(()); - } - } else if strings::index_of_char(k, b'/').is_some() { - log.add_warning_fmt( - Some(source), - key_loc, - format_args!("Bun currently does not support nested \"resolutions\""), - ); - return Ok(()); - } + let value_loc = + crate::bun_json::value_loc_of_property(&ctx.source.contents, key_loc, &value_expr); + self.parse_string_rule(ctx, k, key_loc, value, value_loc) + }) + } - if version_str.starts_with(b"patch:") { - // TODO(dylan-conway): apply .patch files to packages - log.add_warning_fmt( - Some(source), - key_loc, - format_args!("Bun currently does not support patched package \"resolutions\""), - ); + fn parse_string_rule( + &mut self, + ctx: &mut ParseContext<'_, '_>, + key: &[u8], + key_loc: bun_ast::Loc, + value: &[u8], + value_loc: bun_ast::Loc, + ) -> Result<(), Error> { + let Selector { parent, target } = match parse_selector(key) { + Ok(selector) => selector, + Err(err) => { + warn_selector_error(ctx, key, key_loc, err); return Ok(()); } + }; + self.put_rule(ctx, parent.as_ref(), &target, key_loc, value, value_loc) + } - if let Some(version) = parse_override_value( - "resolution", - lockfile_dependencies, - pm, - root_package, - source, - crate::bun_json::value_loc_of_property(&source.contents, key_loc, &value), - log, - k, - version_str, - builder, - )? { - let name_hash = SemverBuilder::string_hash(k); - self.map.put_assume_capacity(name_hash, version); - } - Ok(()) - }) + fn put_rule( + &mut self, + ctx: &mut ParseContext<'_, '_>, + parent: Option<&PackageSelector<'_>>, + target: &PackageSelector<'_>, + key_loc: bun_ast::Loc, + value: &[u8], + value_loc: bun_ast::Loc, + ) -> Result<(), Error> { + if value.starts_with(b"patch:") { + // TODO(dylan-conway): apply .patch files to packages + ctx.log.add_warning_fmt( + Some(ctx.source), + key_loc, + format_args!( + "Bun currently does not support patched package \"{}\"", + ctx.field.json_name() + ), + ); + return Ok(()); + } + + let Some(dep) = parse_override_value(ctx, value_loc, target.name, value)? else { + return Ok(()); + }; + let Some(target_range) = parse_range(ctx, key_loc, dep.name, dep.name_hash, target.range) + else { + return Ok(()); + }; + let parent = match parent { + None => None, + Some(selector) => match parse_parent(ctx, key_loc, selector) { + Some(parent) => Some(parent), + None => return Ok(()), + }, + }; + + if parent.is_none() && target_range.tag != VersionTag::Npm { + self.map.put_assume_capacity(dep.name_hash, dep); + } else { + self.push_scoped( + ScopedOverride { + parent, + target_range, + dep, + }, + ctx.builder.string_bytes.as_slice(), + ); + } + Ok(()) } } -// `field` is only used in warning-message -// formatting, so a runtime `&'static str` is fine. -pub(crate) fn parse_override_value( - field: &'static str, - // Callers hold a live `StringBuilder` (which owns `&mut string_bytes`), so - // accept the dependency slice directly and read string-bytes through - // `builder.string_bytes` instead of taking the whole `Lockfile`. - lockfile_dependencies: &[Dependency], - package_manager: &mut PackageManager, - root_package: &Package, - source: &bun_ast::Source, - loc: bun_ast::Loc, +fn lockfile_range( + name: SemverString, + name_hash: PackageNameHash, + range: &[u8], + buf: &mut bun_semver::string::Buf<'_>, log: &mut bun_ast::Log, + manager: Option<&mut PackageManager>, +) -> Result, Error> { + if range.is_empty() { + return Ok(Some(dependency::Version::default())); + } + let range = buf.append(range)?; + let sliced = range.sliced(buf.bytes.as_slice()); + Ok( + match dependency::parse(name, name_hash, sliced.slice, &sliced, log, manager) { + Some(version) if version.tag == VersionTag::Npm => Some(version), + _ => None, + }, + ) +} + +/// Re-parses like `Dependency::clone_in` so a prerelease comparator does not keep pointing into the old buffer. +fn clone_range( + pm: &mut PM, + dep: &Dependency, + range: &dependency::Version, + old_string_bytes: &[u8], + new_builder: &mut StringBuilder, +) -> dependency::Version { + let literal = new_builder.append::(range.literal.slice(old_string_bytes)); + let sliced = literal.sliced(new_builder.string_bytes.as_slice()); + dependency::parse_with_tag( + dep.name, + Some(dep.name_hash), + sliced.slice, + VersionTag::Npm, + &sliced, + None, + Some(pm as &mut dyn NpmAliasRegistry), + ) + .unwrap_or_default() +} + +fn warn_selector_error( + ctx: &mut ParseContext<'_, '_>, + key: &[u8], + key_loc: bun_ast::Loc, + err: SelectorError, +) { + let source = Some(ctx.source); + match err { + SelectorError::EmptyName => ctx.log.add_warning_fmt( + source, + key_loc, + format_args!("{}", ctx.field.missing_name_message()), + ), + SelectorError::InvalidName => ctx.log.add_warning_fmt( + source, + key_loc, + format_args!("Invalid package name \"{}\"", bstr::BStr::new(key)), + ), + SelectorError::TooDeep => ctx.log.add_warning_fmt( + source, + key_loc, + format_args!( + "Bun currently only supports one level of nested \"{}\"", + ctx.field.json_name() + ), + ), + SelectorError::EmptyRange => ctx.log.add_warning_fmt( + source, + key_loc, + format_args!( + "Bun does not support an empty version selector (a pnpm convergence {}); {} \"{}\" will not apply", + ctx.field.label(), + ctx.field.label(), + bstr::BStr::new(key) + ), + ), + } +} + +fn parse_parent( + ctx: &mut ParseContext<'_, '_>, + key_loc: bun_ast::Loc, + selector: &PackageSelector<'_>, +) -> Option { + let name_hash = SemverBuilder::string_hash(selector.name); + let name = ctx + .builder + .append_with_hash::(selector.name, name_hash); + let version = parse_range(ctx, key_loc, name, name_hash, selector.range)?; + Some(Dependency { + name, + name_hash, + version, + behavior: Behavior::default(), + }) +} + +fn parse_range( + ctx: &mut ParseContext<'_, '_>, + key_loc: bun_ast::Loc, + name: SemverString, + name_hash: PackageNameHash, + range: &[u8], +) -> Option { + if range.is_empty() { + return Some(dependency::Version::default()); + } + let appended = ctx.builder.append::(range); + let sliced = appended.sliced(ctx.builder.string_bytes.as_slice()); + match dependency::parse( + name, + name_hash, + sliced.slice, + &sliced, + &mut *ctx.log, + &mut *ctx.pm, + ) { + Some(version) if version.tag == VersionTag::Npm => Some(version), + _ => { + ctx.log.add_warning_fmt( + Some(ctx.source), + key_loc, + format_args!( + "Invalid version range \"{}\" for \"{}\"", + bstr::BStr::new(range), + bstr::BStr::new(name.slice(ctx.builder.string_bytes.as_slice())) + ), + ); + None + } + } +} + +/// The rule is always stored under `key`; a `$ref` value only supplies the version spec (npm/pnpm semantics). +fn parse_override_value( + ctx: &mut ParseContext<'_, '_>, + loc: bun_ast::Loc, key: &[u8], value: &[u8], - builder: &mut StringBuilder, ) -> Result, Error> { + let field = ctx.field.label(); if value.is_empty() { - log.add_warning_fmt(Some(source), loc, format_args!("Missing {} value", field)); + ctx.log.add_warning_fmt( + Some(ctx.source), + loc, + format_args!("Missing {} value", field), + ); return Ok(None); } - - // "Overrides may also be defined as a reference to a spec for a direct dependency - // by prefixing the name of the package you wish the version to match with a `$`" - // https://docs.npmjs.com/cli/v9/configuring-npm/package-json#overrides - // This is why a `*Lockfile.Package` is needed here. - if value[0] == b'$' { - let ref_name = &value[1..]; - // This is fine for this string to not share the string pool, because it's only used for .eql() - let ref_name_str = SemverString::init(ref_name, ref_name); - let pkg_deps: &[Dependency] = root_package.dependencies.get(lockfile_dependencies); - for dep in pkg_deps { - if dep - .name - .eql(ref_name_str, builder.string_bytes.as_slice(), ref_name) - { - return Ok(Some(dep.clone())); - } - } - log.add_warning_fmt( - Some(source), + if value == b"-" { + ctx.log.add_warning_fmt( + Some(ctx.source), loc, format_args!( - "Could not resolve {} \"{}\" (you need \"{}\" in your dependencies)", + "{} \"{}\" removes the dependency ('-'), which bun does not support", field, - bstr::BStr::new(value), - bstr::BStr::new(ref_name), + bstr::BStr::new(key) ), ); return Ok(None); } - let literal_string = builder.append::(value); - // SAFETY: `string_bytes` was pre-reserved by `allocate()`; subsequent - // `append` calls don't realloc, so a detached view is sound here while we - // still need `&mut builder` for the next `append`. - let string_bytes = unsafe { bun_ptr::detach_lifetime(builder.string_bytes.as_slice()) }; - let literal_sliced = literal_string.sliced(string_bytes); - let name_hash = SemverBuilder::string_hash(key); - let name = builder.append_with_hash::(key, name_hash); + let name = ctx.builder.append_with_hash::(key, name_hash); + + // https://docs.npmjs.com/cli/v9/configuring-npm/package-json#overrides + let (literal, tag): (SemverString, Option) = if value[0] == b'$' { + let ref_name = &value[1..]; + let ref_name_str = SemverString::init(ref_name, ref_name); + let root_ref = ctx + .root_package + .dependencies + .get(ctx.lockfile_dependencies) + .iter() + .find(|dep| { + dep.name + .eql(ref_name_str, ctx.builder.string_bytes.as_slice(), ref_name) + }) + .map(|dep| (dep.version.literal, Some(dep.version.tag))); + + match root_ref { + Some(root_ref) => root_ref, + None => match workspace_ref_literal( + ctx.pm, + ctx.log, + ctx.source, + ctx.workspace_names, + ref_name, + ) { + Ok(Some(literal)) => (ctx.builder.append::(&literal), None), + Ok(None) => { + ctx.log.add_warning_fmt( + Some(ctx.source), + loc, + format_args!( + "Could not resolve {} \"{}\" (you need \"{}\" in your dependencies)", + field, + bstr::BStr::new(value), + bstr::BStr::new(ref_name), + ), + ); + return Ok(None); + } + Err(Ambiguous) => { + ctx.log.add_warning_fmt( + Some(ctx.source), + loc, + format_args!( + "Could not resolve {} \"{}\": workspaces declare different versions of \"{}\"", + field, + bstr::BStr::new(value), + bstr::BStr::new(ref_name), + ), + ); + return Ok(None); + } + }, + } + } else { + (ctx.builder.append::(value), None) + }; - let version = match dependency::parse( + let sliced = literal.sliced(ctx.builder.string_bytes.as_slice()); + let Some(version) = dependency::parse_with_optional_tag( name, name_hash, - literal_sliced.slice, - &literal_sliced, - &mut *log, - package_manager, - ) { - Some(v) => v, - None => { - log.add_warning_fmt( - Some(source), - loc, - format_args!("Invalid {} value \"{}\"", field, bstr::BStr::new(value)), - ); - return Ok(None); - } + sliced.slice, + tag, + &sliced, + &mut *ctx.log, + &mut *ctx.pm, + ) else { + ctx.log.add_warning_fmt( + Some(ctx.source), + loc, + format_args!("Invalid {} value \"{}\"", field, bstr::BStr::new(value)), + ); + return Ok(None); }; Ok(Some(Dependency { @@ -472,3 +1154,84 @@ pub(crate) fn parse_override_value( behavior: Behavior::default(), })) } + +/// First string-valued entry for `name` across the four dependency sections of a package.json object. +fn declared_dependency(root: &Expr, name: &[u8]) -> Option { + DependencyGroup::FOUR.iter().find_map(|group| { + root.get(group.prop) + .and_then(|deps| deps.get(name)) + .filter(|value| value.as_utf8_string_literal().is_some()) + }) +} + +fn count_ref_value( + pm: &mut PackageManager, + log: &mut bun_ast::Log, + source: &bun_ast::Source, + workspace_names: &WorkspaceMap, + root_json: &Expr, + value: &[u8], + builder: &mut StringBuilder, +) { + let Some(ref_name) = value.strip_prefix(b"$") else { + return; + }; + if let Some(declared) = declared_dependency(root_json, ref_name) { + if let Some(literal) = declared.as_utf8_string_literal() { + builder.count(literal); + } + return; + } + if let Ok(Some(literal)) = workspace_ref_literal(pm, log, source, workspace_names, ref_name) { + builder.count(&literal); + } +} + +fn workspace_ref_literal( + pm: &mut PackageManager, + log: &mut bun_ast::Log, + source: &bun_ast::Source, + workspace_names: &WorkspaceMap, + ref_name: &[u8], +) -> Result>, Ambiguous> { + if workspace_names.count() == 0 { + return Ok(None); + } + let root_dir: &[u8] = source.path.name().dir; + let mut path_buf = bun_paths::path_buffer_pool::get(); + let mut found: Option> = None; + for relative_dir in workspace_names.keys() { + let Some(abs_package_json_path) = + resolve_path::join_abs_string_buf_checked::( + root_dir, + &mut path_buf.0, + &[&relative_dir[..], b"package.json"], + ) + else { + continue; + }; + let GetResult::Entry(entry) = pm.workspace_package_json_cache.get_with_path( + log, + abs_package_json_path, + GetJSONOptions { + init_reset_store: false, + guess_indentation: true, + ..Default::default() + }, + ) else { + continue; + }; + let Some(declared) = declared_dependency(&entry.root, ref_name) else { + continue; + }; + let Some(literal) = declared.as_utf8_string_literal() else { + continue; + }; + match found.as_deref() { + None => found = Some(literal.to_vec()), + Some(first) if first == literal => {} + Some(_) => return Err(Ambiguous), + } + } + Ok(found) +} diff --git a/src/install/lockfile/Package.rs b/src/install/lockfile/Package.rs index 24e3c99efd5c..a27695bbfb6e 100644 --- a/src/install/lockfile/Package.rs +++ b/src/install/lockfile/Package.rs @@ -925,12 +925,17 @@ pub struct DiffSummary { impl DiffSummary { #[inline] - pub(crate) fn has_diffs(&self) -> bool { + pub(crate) fn changes_resolutions(&self) -> bool { self.add > 0 || self.remove > 0 || self.update > 0 || self.overrides_changed || self.catalogs_changed + } + + #[inline] + pub(crate) fn has_diffs(&self) -> bool { + self.changes_resolutions() || self.added_trusted_dependencies.count() > 0 || self.removed_trusted_dependencies.count() > 0 || self.patched_dependencies_changed @@ -983,51 +988,17 @@ impl Diff { let (from_deps, from_resolutions) = (from_deps.slice(), from_resolutions.slice()); let mut to_i: usize = 0; - if from_lockfile.overrides.map.count() != to_lockfile.overrides.map.count() { + if lockfile::OverrideMap::changed( + &mut from_lockfile.overrides, + from_lockfile.buffers.string_bytes.as_slice(), + &mut to_lockfile.overrides, + to_lockfile.buffers.string_bytes.as_slice(), + ) { summary.overrides_changed = true; if PackageManager::verbose_install() { bun_core::pretty_errorln!("Overrides changed since last install"); } - } else { - // `OverrideMap::sort` only reads `lockfile.buffers.string_bytes`, - // so split the borrow at the field. - lockfile::OverrideMap::sort( - &mut from_lockfile.overrides, - from_lockfile.buffers.string_bytes.as_slice(), - ); - lockfile::OverrideMap::sort( - &mut to_lockfile.overrides, - to_lockfile.buffers.string_bytes.as_slice(), - ); - debug_assert_eq!( - from_lockfile.overrides.map.keys().len(), - to_lockfile.overrides.map.keys().len() - ); - for (((from_k, from_override), to_k), to_override) in from_lockfile - .overrides - .map - .keys() - .iter() - .zip(from_lockfile.overrides.map.values()) - .zip(to_lockfile.overrides.map.keys()) - .zip(to_lockfile.overrides.map.values()) - { - if (from_k != to_k) - || (!Dependency::eql( - from_override, - to_override, - from_lockfile.buffers.string_bytes.as_slice(), - to_lockfile.buffers.string_bytes.as_slice(), - )) - { - summary.overrides_changed = true; - if PackageManager::verbose_install() { - bun_core::pretty_errorln!("Overrides changed since last install"); - } - break; - } - } } if is_root { @@ -1485,7 +1456,7 @@ impl Diff { ); } - !diff.has_diffs() + !diff.changes_resolutions() }; if update_mapping { @@ -2238,6 +2209,12 @@ impl Package { } } + let missing_workspace = if pm.options.enable.frozen_lockfile() { + workspace_map::MissingWorkspace::SkipIfInLockfile(&pm.lockfile) + } else { + workspace_map::MissingWorkspace::Error + }; + for group in &dependency_groups { if let Some(dependencies_q) = json.as_property(group.prop) { 'brk: { @@ -2264,7 +2241,7 @@ impl Package { source, dependencies_q.loc, Some(&mut string_builder), - pm.options.enable.frozen_lockfile(), + missing_workspace, )?; break 'brk; } @@ -2311,7 +2288,7 @@ impl Package { source, packages_loc, Some(&mut string_builder), - pm.options.enable.frozen_lockfile(), + missing_workspace, )?; } @@ -2394,7 +2371,14 @@ impl Package { } if FEATURES.is_main { - lockfile.overrides.parse_count(json, &mut string_builder); + lockfile.overrides.parse_count( + pm, + log, + source, + &workspace_names, + json, + &mut string_builder, + ); if let Some(workspaces_expr) = json.get(b"workspaces") { lockfile @@ -2932,6 +2916,7 @@ impl Package { self, log, source, + &workspace_names, json, &mut string_builder, )?; diff --git a/src/install/lockfile/Package/WorkspaceMap.rs b/src/install/lockfile/Package/WorkspaceMap.rs index 4a66ff4711ee..2f2d415ccd4d 100644 --- a/src/install/lockfile/Package/WorkspaceMap.rs +++ b/src/install/lockfile/Package/WorkspaceMap.rs @@ -8,7 +8,7 @@ use bun_paths as path; use bun_paths::resolve_path; use bun_paths::{MAX_PATH_BYTES, PathBuffer, SEP_STR}; -use crate::lockfile_real::StringBuilder; +use crate::lockfile_real::{Lockfile, StringBuilder, pruned_workspaces}; use crate::package_manager::workspace_package_json_cache::{ GetJSONOptions, WorkspacePackageJSONCache, }; @@ -115,6 +115,14 @@ impl<'a> NamesArray<'a> { } } +// What to do with a listed (non-glob) workspace whose package.json is missing. +#[derive(Clone, Copy)] +pub(crate) enum MissingWorkspace<'a> { + Error, + Skip, + SkipIfInLockfile(&'a Lockfile), +} + fn process_workspace_name( json_cache: &mut WorkspacePackageJSONCache, abs_package_json_path: &[u8], @@ -166,6 +174,28 @@ fn process_workspace_name( Ok(entry) } +fn workspace_dir_of(abs_package_json_path: &[u8]) -> &[u8] { + strings::without_suffix_comptime( + abs_package_json_path, + const_format::concatcp!(SEP_STR, "package.json").as_bytes(), + ) +} + +fn relative_workspace_path<'b>( + buf: &'b mut [u8], + root_dir: &[u8], + abs_workspace_dir: &[u8], +) -> &'b [u8] { + let len = resolve_path::relative_platform_buf::( + buf, + root_dir, + abs_workspace_dir, + ) + .len(); + resolve_path::platform_to_posix_in_place::(&mut buf[..len]); + &buf[..len] +} + impl WorkspaceMap { pub(crate) fn process_names_array( &mut self, @@ -175,7 +205,7 @@ impl WorkspaceMap { source: &bun_ast::Source, loc: bun_ast::Loc, mut string_builder: Option<&mut StringBuilder<'_>>, - skip_missing: bool, + missing_workspace: MissingWorkspace<'_>, ) -> crate::Result { let workspace_names = self; let item_count = arr.len(); @@ -189,6 +219,8 @@ impl WorkspaceMap { let mut filepath_buf_os: Box = Box::new(PathBuffer::uninit()); // Boxed to avoid a large stack frame. let filepath_buf: &mut [u8] = &mut filepath_buf_os.0[..]; + let mut rel_path_buf = path::path_buffer_pool::get(); + let root_dir: &[u8] = source.path.name().dir; let scratch = Arena::new(); @@ -216,16 +248,17 @@ impl WorkspaceMap { continue; } - let processed = match resolve_path::join_abs_string_buf_checked::( - source.path.name().dir, - filepath_buf, - &[input_path, b"package.json"], - ) { + let abs_package_json_path = resolve_path::join_abs_string_buf_checked::< + path::platform::Auto, + >( + root_dir, filepath_buf, &[input_path, b"package.json"] + ); + let processed = match abs_package_json_path { Some(abs_package_json_path) => { // skip root package.json if strings::eql_long( resolve_path::dirname::(abs_package_json_path), - source.path.name().dir, + root_dir, true, ) { continue; @@ -240,8 +273,25 @@ impl WorkspaceMap { let (abs_package_json_path, workspace_entry) = match processed { Ok(processed) => processed, Err(err) => { - if skip_missing && err == crate::Error::Sys(bun_errno::SystemErrno::ENOENT) { - continue; + if err == crate::Error::Sys(bun_errno::SystemErrno::ENOENT) { + let tolerated = match missing_workspace { + MissingWorkspace::Skip => true, + MissingWorkspace::Error => false, + MissingWorkspace::SkipIfInLockfile(lockfile) => abs_package_json_path + .is_some_and(|abs| { + pruned_workspaces::lockfile_lists_workspace_path( + lockfile, + relative_workspace_path( + &mut rel_path_buf.0, + root_dir, + workspace_dir_of(abs), + ), + ) + }), + }; + if tolerated { + continue; + } } if err == crate::Error::Sys(bun_errno::SystemErrno::EISDIR) || err == crate::Error::Sys(bun_errno::SystemErrno::EPERM) @@ -283,31 +333,11 @@ impl WorkspaceMap { continue; } - let rel_input_path = resolve_path::relative_platform::( - source.path.name().dir, - strings::without_suffix_comptime( - abs_package_json_path, - const_format::concatcp!(SEP_STR, "package.json").as_bytes(), - ), + let rel_input_path = relative_workspace_path( + &mut rel_path_buf.0, + root_dir, + workspace_dir_of(abs_package_json_path), ); - #[cfg(windows)] - let rel_input_path: &[u8] = { - // `rel_input_path` is a shared borrow into the thread-local - // `relative_to_common_path_buf()`. Deriving a `&mut` from - // `rel_input_path.as_ptr().cast_mut()` and writing through it is - // Stacked-Borrows UB (SharedReadOnly provenance), and the still-live - // shared ref would alias it. Instead capture the length, drop the - // shared borrow, take a single fresh `&mut` reborrow from the raw - // threadlocal pointer, mutate, then downgrade to `&[u8]`. - let len = rel_input_path.len(); - let _ = rel_input_path; - // SAFETY: thread-local scratch; this is the only live borrow on this - // thread for the remainder of this block. - let s: &mut [u8] = - &mut unsafe { &mut *resolve_path::relative_to_common_path_buf() }[0..len]; - path::dangerously_convert_path_to_posix_in_place::(s); - &*s - }; if let Some(builder) = string_builder.as_deref_mut() { builder.count(&workspace_entry.name); @@ -497,33 +527,11 @@ impl WorkspaceMap { continue; } - let abs_workspace_dir_path: &[u8] = - strings::without_suffix_comptime(abs_package_json_path, b"package.json"); - let workspace_path: &[u8] = - resolve_path::relative_platform::( - source.path.name().dir, - abs_workspace_dir_path, - ); - #[cfg(windows)] - let workspace_path: &[u8] = { - // `workspace_path` is a shared borrow into the thread-local - // `relative_to_common_path_buf()`. Deriving a `&mut` from - // `workspace_path.as_ptr().cast_mut()` and writing through it is - // Stacked-Borrows UB (SharedReadOnly provenance), and the - // still-live shared ref would alias it. Instead capture the - // length, drop the shared borrow, take a single fresh `&mut` - // reborrow from the raw threadlocal pointer, mutate, then - // downgrade to `&[u8]`. - let len = workspace_path.len(); - let _ = workspace_path; - // SAFETY: thread-local scratch; this is the only live borrow on - // this thread for the remainder of this block. - let s: &mut [u8] = - &mut unsafe { &mut *resolve_path::relative_to_common_path_buf() } - [0..len]; - path::dangerously_convert_path_to_posix_in_place::(s); - &*s - }; + let workspace_path: &[u8] = relative_workspace_path( + &mut rel_path_buf.0, + root_dir, + workspace_dir_of(abs_package_json_path), + ); if let Some(builder) = string_builder.as_deref_mut() { builder.count(&workspace_entry.name); diff --git a/src/install/lockfile/bun.lock.rs b/src/install/lockfile/bun.lock.rs index ee6399a19485..f01b8460724a 100644 --- a/src/install/lockfile/bun.lock.rs +++ b/src/install/lockfile/bun.lock.rs @@ -39,6 +39,8 @@ use bun_install_types::DependencyVersionTag; // this file is `crate::lockfile_real::bun_lock`; `super` is the // real `Lockfile` module, distinct from the `crate::lockfile` stub. use super::PackageIDSlice; +use super::override_map::ScopedOverride; +use super::override_selector::{PackageSelector, parse_package_segment}; use super::package::{Meta, PackageColumns as _, value_loc_of}; use super::{ CatalogMap, DependencySlice, LoadResult, Lockfile as BinaryLockfile, OverrideMap, Package, @@ -113,10 +115,13 @@ pub enum Version { /// check on a `github` tag is enforced at every version, since its /// download path has no checkout-time re-validation) V2 = 2, + + /// `overrides` values may be objects holding scoped rules (parent-scoped or `name@range` targets); stamped only while such rules exist + V3 = 3, } impl Version { - pub(crate) const CURRENT: Version = Version::V2; + pub(crate) const CURRENT: Version = Version::V3; #[inline] pub(crate) const fn current() -> Version { @@ -128,6 +133,7 @@ impl Version { 0 => Some(Version::V0), 1 => Some(Version::V1), 2 => Some(Version::V2), + 3 => Some(Version::V3), _ => None, } } @@ -179,8 +185,8 @@ impl Stringifier { /// existing `bun.lock` to a newer format. `text_lockfile_version` holds the /// parsed version when the lockfile was loaded from text, and defaults to /// `Version::CURRENT` otherwise (a fresh install, or a migration from - /// another lockfile format), which is the "no version previously" case that - /// does get the current version. + /// another lockfile format), the "no version previously" case whose stamp + /// is decided by the walk below. /// /// The one version that is *not* preserved is v0: v0→v1 was a content-format /// change (v1 stopped listing a workspace package's dependencies as a @@ -191,35 +197,35 @@ impl Stringifier { /// v1→v2, by contrast, only added parse-time strictness on identical /// content, so v1 is preserved as-is. /// - /// When the target is the current version (v2), it is stamped only if every - /// serialized package satisfies the v2 invariants. v2 added parse-time - /// checks that reject entries older versions tolerated: an off-registry npm - /// tarball without a supported integrity hash, and an unsafe git `.bun-tag`. - /// The writer emits those fields verbatim (no backfill), so stamping v2 on a - /// lockfile that still carries such an entry — possible for a migrated - /// lockfile — would make the *next* parse reject it. Those stay at v1 so the - /// file round-trips (load → save → load) cleanly, across machines too, since - /// a lockfile is committed and shared. That decision is made without - /// consulting the writer's registry config: whether the *reader* will accept - /// the file must not depend on the writer's `~/.npmrc` / scoped registries. + /// v3 is stamped only while parent-scoped overrides exist (older readers + /// cannot parse the object rows, so the upgrade is forced like v0→v1); a + /// lockfile without them keeps its loaded v1/v2, and a fresh one — or a v3 + /// whose nested rules were removed — is walked down to v2, or to v1 when a + /// serialized package violates a v2 invariant (an off-registry npm tarball + /// without a supported integrity hash, or an unsafe git `.bun-tag`), which + /// the writer emits verbatim and a v2 reader would reject on the next parse. + /// A loaded v3 with nested rules stays v3 without the walk, so the walk only + /// runs for fresh lockfiles and v2↔v3 transitions. The v2 decision must not + /// depend on the writer's `~/.npmrc` / scoped registries, since the reader + /// may not share them. /// /// Walks the package tree the same way the writer does — only packages that /// are actually serialized are considered, not every entry in the in-memory /// `pkg_resolutions` buffer (migration can leave pruned/unreferenced entries /// there that never reach the written `packages` object). fn version_to_write(lockfile: &BinaryLockfile) -> Version { - // An older on-disk lockfile keeps its version; only a no-prior-version - // lockfile (the `Version::CURRENT` default) is a candidate for v2. v0 is - // the exception: the writer can't emit v0-format workspace entries, so a - // v0 lockfile is upgraded to v1 rather than preserved verbatim. let loaded = lockfile.text_lockfile_version; - if !loaded.at_least(Version::CURRENT) { + let has_scoped = lockfile.overrides.has_scoped(); + if !has_scoped && !loaded.at_least(Version::V3) { return if loaded.at_least(Version::V1) { loaded } else { Version::V1 }; } + if has_scoped && loaded == Version::V3 { + return Version::V3; + } let buf = lockfile.buffers.string_bytes.as_slice(); let deps_buf = lockfile.buffers.dependencies.as_slice(); @@ -281,7 +287,7 @@ impl Stringifier { } } } - Version::CURRENT + if has_scoped { Version::V3 } else { Version::V2 } } fn save_from_binary_inner( @@ -558,7 +564,7 @@ impl Stringifier { writer.write_all(b"},\n")?; } - if lockfile.overrides.map.count() > 0 { + if !lockfile.overrides.is_empty() { lockfile .overrides .sort(lockfile.buffers.string_bytes.as_slice()); @@ -566,17 +572,21 @@ impl Stringifier { Self::write_indent(writer, *indent)?; writer.write_all(b"\"overrides\": {\n")?; *indent += 1; - for override_dep in lockfile.overrides.map.values() { - Self::write_indent(writer, *indent)?; - writeln!( - writer, - "{}: {},", - override_dep.name.fmt_json(buf, Default::default()), - override_dep - .version - .literal - .fmt_json(buf, Default::default()), - )?; + if !lockfile.overrides.has_scoped() { + let mut key_buf: Vec = Vec::new(); + for override_dep in lockfile.overrides.map.values() { + Self::write_override_rule( + writer, + *indent, + &mut key_buf, + override_dep.name.slice(buf), + b"", + override_dep.version.literal, + buf, + )?; + } + } else { + Self::write_override_rules(writer, *indent, &lockfile.overrides, buf)?; } Self::dec_indent(writer, indent)?; @@ -1393,6 +1403,193 @@ impl Stringifier { Ok(()) } + /// `name` when `range` is empty, else `name@range` built in `key_buf`. + fn override_selector_key<'a>( + key_buf: &'a mut Vec, + name: &'a [u8], + range: &[u8], + ) -> &'a [u8] { + if range.is_empty() { + return name; + } + key_buf.clear(); + key_buf.extend_from_slice(name); + key_buf.push(b'@'); + key_buf.extend_from_slice(range); + key_buf.as_slice() + } + + fn write_override_rule( + writer: &mut Writer, + indent: u32, + key_buf: &mut Vec, + name: &[u8], + range: &[u8], + value: String, + buf: &[u8], + ) -> Result<(), WriteError> { + Self::write_indent(writer, indent)?; + writeln!( + writer, + "{}: {},", + bun_core::fmt::format_json_string_utf8( + Self::override_selector_key(key_buf, name, range), + Default::default() + ), + value.fmt_json(buf, Default::default()), + )?; + Ok(()) + } + + /// Scoped rules live in selector-keyed objects; a flat rule folds into its name's unranged group as `"."`. + fn write_override_rules( + writer: &mut Writer, + indent: u32, + overrides: &OverrideMap, + buf: &[u8], + ) -> Result<(), WriteError> { + struct GroupKey<'a> { + name_hash: PackageNameHash, + name: &'a [u8], + literal: &'a [u8], + } + + fn order_keys(lhs: &GroupKey<'_>, rhs: &GroupKey<'_>) -> core::cmp::Ordering { + strings::order(lhs.name, rhs.name) + .then_with(|| strings::order(lhs.literal, rhs.literal)) + } + + let flat: &[Dependency] = overrides.map.values(); + let scoped: &[ScopedOverride] = overrides.scoped.as_slice(); + let solo_end = scoped + .iter() + .position(|rule| rule.parent.is_some()) + .unwrap_or(scoped.len()); + let mut key_buf: Vec = Vec::new(); + let mut f = 0usize; + let mut s = 0usize; + let mut g = solo_end; + + while f < flat.len() || s < solo_end || g < scoped.len() { + let solo_key = (s < solo_end).then(|| GroupKey { + name_hash: scoped[s].dep.name_hash, + name: scoped[s].dep.name.slice(buf), + literal: scoped[s].target_range.literal.slice(buf), + }); + let parented_key = scoped.get(g).map(|rule| { + let parent = rule + .parent + .as_ref() + .expect("OverrideMap::sort places parent-less rules first"); + GroupKey { + name_hash: parent.name_hash, + name: parent.name.slice(buf), + literal: parent.version.literal.slice(buf), + } + }); + let group = match (solo_key, parented_key) { + (Some(solo), Some(parented)) => Some(if order_keys(&parented, &solo).is_lt() { + parented + } else { + solo + }), + (solo, parented) => solo.or(parented), + }; + + if let Some(flat_dep) = flat.get(f) { + let flat_first = match &group { + None => true, + Some(group) => match strings::order(flat_dep.name.slice(buf), group.name) { + core::cmp::Ordering::Less => true, + core::cmp::Ordering::Equal => !group.literal.is_empty(), + core::cmp::Ordering::Greater => false, + }, + }; + if flat_first { + Self::write_override_rule( + writer, + indent, + &mut key_buf, + flat_dep.name.slice(buf), + b"", + flat_dep.version.literal, + buf, + )?; + f += 1; + continue; + } + } + + let Some(group) = group else { + break; + }; + + Self::write_indent(writer, indent)?; + writeln!( + writer, + "{}: {{", + bun_core::fmt::format_json_string_utf8( + Self::override_selector_key(&mut key_buf, group.name, group.literal), + Default::default() + ), + )?; + + let dot: Option = if group.literal.is_empty() { + match flat.get(f) { + Some(flat_dep) if flat_dep.name_hash == group.name_hash => { + f += 1; + Some(flat_dep.version.literal) + } + _ => None, + } + } else { + match (s < solo_end).then(|| &scoped[s]) { + Some(rule) + if rule.dep.name_hash == group.name_hash + && rule.target_range.literal.slice(buf) == group.literal => + { + s += 1; + Some(rule.dep.version.literal) + } + _ => None, + } + }; + if let Some(value) = dot { + Self::write_indent(writer, indent + 1)?; + writeln!( + writer, + "\".\": {},", + value.fmt_json(buf, Default::default()) + )?; + } + + while let Some(rule) = scoped.get(g) { + let in_group = rule.parent.as_ref().is_some_and(|parent| { + parent.name_hash == group.name_hash + && parent.version.literal.slice(buf) == group.literal + }); + if !in_group { + break; + } + Self::write_override_rule( + writer, + indent + 1, + &mut key_buf, + rule.dep.name.slice(buf), + rule.target_range.literal.slice(buf), + rule.dep.version.literal, + buf, + )?; + g += 1; + } + + Self::write_indent(writer, indent)?; + writer.write_all(b"},\n")?; + } + + Ok(()) + } + fn write_indent(writer: &mut Writer, indent: u32) -> Result<(), WriteError> { const INDENT: &[u8] = b" "; // " " ** indent_scalar (2) const _: () = assert!(INDENT.len() == Stringifier::INDENT_SCALAR); @@ -1784,48 +1981,91 @@ pub(crate) fn parse_into_binary_lockfile( return Err(ParseError::InvalidOverridesObject); } - let name_hash = StringBuilder::string_hash(name_str); - let name = sbuf!(lockfile).append_with_hash(name_str, name_hash)?; + if let Some(version_str) = row.value.as_str() { + let ok = lockfile + .overrides + .put_lockfile_rule( + None, + PackageSelector { + name: name_str, + range: b"", + }, + version_str, + &mut sbuf!(lockfile), + &mut *log, + manager.as_deref_mut(), + ) + .map_err(|_| ParseError::OutOfMemory)?; + if !ok { + log.add_error( + Some(source), + value_loc_of(source, row.key_loc), + b"Invalid override version", + ); + return Err(ParseError::InvalidOverridesObject); + } + continue; + } - // TODO(dylan-conway) also accept object when supported - let Some(version_str) = row.value.as_str() else { + let Some(group_obj) = row.value.as_object() else { log.add_error( Some(source), value_loc_of(source, row.key_loc), - b"Expected a string", + b"Expected a string or an object", ); return Err(ParseError::InvalidOverridesObject); }; - let version_hash = StringBuilder::string_hash(version_str); - let version = sbuf!(lockfile).append_with_hash(version_str, version_hash)?; - let version_sliced = version.sliced(lockfile.buffers.string_bytes.as_slice()); + let Ok(parent) = parse_package_segment(name_str) else { + log.add_error(Some(source), row.key_loc, b"Invalid override key"); + return Err(ParseError::InvalidOverridesObject); + }; - let dep = Dependency { - name, - name_hash, - version: match dependency::parse( - name, - name_hash, - version_sliced.slice, - &version_sliced, - &mut *log, - manager.as_deref_mut(), - ) { - Some(v) => v, - None => { - log.add_error( - Some(source), - value_loc_of(source, row.key_loc), - b"Invalid override version", - ); + for child in group_obj.properties() { + let child_key = child.key.slice(); + let Some(version_str) = child.value.as_str() else { + log.add_error( + Some(source), + value_loc_of(source, child.key_loc), + b"Expected a string", + ); + return Err(ParseError::InvalidOverridesObject); + }; + + let group_selector = PackageSelector { + name: parent.name, + range: parent.range, + }; + let (rule_parent, rule_target) = if child_key == b"." { + (None, group_selector) + } else { + let Ok(target) = parse_package_segment(child_key) else { + log.add_error(Some(source), child.key_loc, b"Invalid override key"); return Err(ParseError::InvalidOverridesObject); - } - }, - ..Default::default() - }; + }; + (Some(group_selector), target) + }; - lockfile.overrides.map.insert(name_hash, dep); + let ok = lockfile + .overrides + .put_lockfile_rule( + rule_parent, + rule_target, + version_str, + &mut sbuf!(lockfile), + &mut *log, + manager.as_deref_mut(), + ) + .map_err(|_| ParseError::OutOfMemory)?; + if !ok { + log.add_error( + Some(source), + value_loc_of(source, child.key_loc), + b"Invalid override version", + ); + return Err(ParseError::InvalidOverridesObject); + } + } } } @@ -3137,7 +3377,7 @@ fn resolve_peer_dep_version_based( } else { name_hash }; - if overridable && overrides.get(override_name_hash).is_some() { + if overridable && overrides.has_rule_for_name(override_name_hash) { return None; } diff --git a/src/install/lockfile/bun.lockb.rs b/src/install/lockfile/bun.lockb.rs index b9b9bf1a554e..28d719ae958a 100644 --- a/src/install/lockfile/bun.lockb.rs +++ b/src/install/lockfile/bun.lockb.rs @@ -9,6 +9,7 @@ use bun_io::Write as _; // `lockfile_real` module (this file is `lockfile_real::bun_lockb`). The // `bun_install::lockfile::*` path is the stub surface and lacks these items. use super::PatchedDep; +use super::override_map::ScopedOverride; use super::{ FormatVersion, Lockfile, Scratch, Stream, StringPool, buffers, package, package_index as PackageIndex, @@ -16,6 +17,7 @@ use super::{ use crate::ALIGNMENT_BYTES_TO_REPEAT_BUFFER; use crate::config_version::ConfigVersion; use crate::dependency; +use crate::dependency::{Behavior, Dependency}; use crate::package_manager_real::Options as PackageManagerOptions; use crate::resolution_real::Tag as ResolutionTag; use bun_ast::Log; @@ -37,6 +39,7 @@ const HAS_EMPTY_TRUSTED_DEPENDENCIES_TAG: u64 = u64::from_ne_bytes(*b"eMpTrUsT") const HAS_OVERRIDES_TAG: u64 = u64::from_ne_bytes(*b"oVeRriDs"); const HAS_CATALOGS_TAG: u64 = u64::from_ne_bytes(*b"cAtAlOgS"); const HAS_CONFIG_VERSION_TAG: u64 = u64::from_ne_bytes(*b"cNfGvRsN"); +const HAS_SCOPED_OVERRIDES_TAG: u64 = u64::from_ne_bytes(*b"sCoPdOvR"); /// Wraps a growing `Vec` to provide both positional-write semantics /// (`get_pos`/`pwrite`) and append semantics (`write_all`/`write_int_*`) for @@ -341,6 +344,37 @@ pub(crate) fn save( let config_version: ConfigVersion = options.config_version.unwrap_or(ConfigVersion::CURRENT); stream.write_int_le::(config_version as u64)?; + if this.overrides.has_scoped() { + stream.write_all(&HAS_SCOPED_OVERRIDES_TAG.to_ne_bytes())?; + + let scoped = &this.overrides.scoped; + let mut externals: Vec = Vec::with_capacity(scoped.len()); + for rule in scoped { + externals.push(rule.parent.as_ref().map_or_else( + || dependency::to_external(&Dependency::default()), + dependency::to_external, + )); + } + write_array::(&mut stream, &externals, PREFIX_DEP_EXTERNAL)?; + + externals.clear(); + for rule in scoped { + externals.push(dependency::to_external(&Dependency { + name: rule.dep.name, + name_hash: rule.dep.name_hash, + version: rule.target_range.clone(), + behavior: Behavior::default(), + })); + } + write_array::(&mut stream, &externals, PREFIX_DEP_EXTERNAL)?; + + externals.clear(); + for rule in scoped { + externals.push(dependency::to_external(&rule.dep)); + } + write_array::(&mut stream, &externals, PREFIX_DEP_EXTERNAL)?; + } + *total_size = stream.get_pos()?; stream.write_all(&ALIGNMENT_BYTES_TO_REPEAT_BUFFER)?; @@ -737,6 +771,43 @@ pub(crate) fn load( } } + { + let remaining_in_buffer = total_buffer_size.saturating_sub(stream.pos as u64); + + if remaining_in_buffer > 8 && total_buffer_size <= stream.buffer.len() as u64 { + let next_num = stream.read_int_le::()?; + if next_num == HAS_SCOPED_OVERRIDES_TAG { + let parents: Vec = buffers::read_array(stream)?; + let ranges: Vec = buffers::read_array(stream)?; + let deps: Vec = buffers::read_array(stream)?; + debug_assert_eq!(parents.len(), ranges.len()); + debug_assert_eq!(parents.len(), deps.len()); + + let Lockfile { + buffers, overrides, .. + } = &mut *lockfile; + let string_bytes: &[u8] = buffers.string_bytes.as_slice(); + overrides.scoped.reserve(parents.len()); + for ((parent, range), dep) in parents.iter().zip(ranges.iter()).zip(deps.iter()) { + let mut context = dependency::Context { + log: &mut *log, + buffer: string_bytes, + package_manager: manager.as_deref_mut(), + }; + let parent = dependency::to_dependency(*parent, &mut context); + let rule = ScopedOverride { + parent: (!parent.name.is_empty()).then_some(parent), + target_range: dependency::to_dependency(*range, &mut context).version, + dep: dependency::to_dependency(*dep, &mut context), + }; + overrides.push_scoped(rule, string_bytes); + } + } else { + stream.pos -= 8; + } + } + } + lockfile.scratch = Scratch::init(); lockfile.package_index = PackageIndex::Map::default(); lockfile.string_pool = StringPool::default(); diff --git a/src/install/lockfile/override_selector.rs b/src/install/lockfile/override_selector.rs new file mode 100644 index 000000000000..f0d08aa83257 --- /dev/null +++ b/src/install/lockfile/override_selector.rs @@ -0,0 +1,149 @@ +use bun_core::strings; + +use crate::dependency::{Dependency, DependencyExt as _}; + +#[derive(Clone, Copy)] +pub(crate) struct PackageSelector<'a> { + pub name: &'a [u8], + /// Empty when the rule applies to every version of `name`. + pub range: &'a [u8], +} + +pub(crate) struct Selector<'a> { + pub parent: Option>, + pub target: PackageSelector<'a>, +} + +#[derive(Copy, Clone, Eq, PartialEq, Debug)] +pub(crate) enum SelectorError { + EmptyName, + InvalidName, + TooDeep, + EmptyRange, +} + +fn is_bare_scope(name: &[u8]) -> bool { + name.starts_with(b"@") && !strings::contains_char(name, b'/') +} + +pub(crate) fn parse_package_segment(segment: &[u8]) -> Result, SelectorError> { + if segment.is_empty() { + return Err(SelectorError::EmptyName); + } + let (name, range) = Dependency::split_name_and_maybe_version(segment); + if name.is_empty() { + return Err(SelectorError::EmptyName); + } + if is_bare_scope(name) { + return Err(SelectorError::InvalidName); + } + let range = strings::without_prefix(range.unwrap_or(b""), b"npm:"); + if range.is_empty() && segment.len() > name.len() { + return Err(SelectorError::EmptyRange); + } + Ok(PackageSelector { name, range }) +} + +pub(crate) fn parse_selector(key: &[u8]) -> Result, SelectorError> { + if key.starts_with(b"//") { + return Ok(Selector { + parent: None, + target: PackageSelector { + name: key, + range: b"", + }, + }); + } + if let Some(delimiter) = pnpm_delimiter(key) { + return parse_pnpm(key, delimiter); + } + parse_yarn_path(key) +} + +/// pnpm's `parent>child` delimiter: a `>` that does not continue a range (`@>1`, `|| >1`). +fn pnpm_delimiter(key: &[u8]) -> Option { + let mut from = 1; + while from < key.len() { + let i = from + strings::index_of_char_usize(&key[from..], b'>')?; + match key[..i].trim_ascii_end().last() { + None | Some(b'|' | b'@') => from = i + 1, + Some(_) => return Some(i), + } + } + None +} + +fn parse_pnpm(key: &[u8], delimiter: usize) -> Result, SelectorError> { + let parent = key[..delimiter].trim_ascii(); + let name = key[delimiter + 1..].trim_ascii(); + if name.is_empty() { + return Err(SelectorError::EmptyName); + } + if pnpm_delimiter(name).is_some() { + return Err(SelectorError::TooDeep); + } + let parent = parse_package_segment(parent)?; + Ok(Selector { + parent: Some(parent), + target: parse_package_segment(name)?, + }) +} + +/// Next `/`-delimited token of `key` starting at `pos`, and the position after its delimiter. +fn next_token(key: &[u8], pos: usize) -> (&[u8], Option) { + match strings::index_of_char_usize(&key[pos..], b'/') { + Some(i) => (&key[pos..pos + i], Some(pos + i + 1)), + None => (&key[pos..], None), + } +} + +fn parse_yarn_path(key: &[u8]) -> Result, SelectorError> { + let mut segments: [&[u8]; 2] = [b"", b""]; + let mut count = 0usize; + let mut cursor = Some(0usize); + while let Some(start) = cursor { + let (token, next) = next_token(key, start); + cursor = next; + if token.is_empty() { + return Err(SelectorError::EmptyName); + } + if token == b"**" { + if next.is_none() { + return Err(SelectorError::EmptyName); + } + continue; + } + let mut end = start + token.len(); + if token.starts_with(b"@") { + let Some(name_start) = next else { + return Err(SelectorError::InvalidName); + }; + let (name, after) = next_token(key, name_start); + cursor = after; + if name.is_empty() { + return Err(SelectorError::EmptyName); + } + end = name_start + name.len(); + } + if count == segments.len() { + return Err(SelectorError::TooDeep); + } + segments[count] = &key[start..end]; + count += 1; + } + + match count { + 0 => Err(SelectorError::EmptyName), + 1 => Ok(Selector { + parent: None, + target: parse_package_segment(segments[0])?, + }), + _ => { + let parent = parse_package_segment(segments[0])?; + Ok(Selector { + parent: Some(parent), + target: parse_package_segment(segments[1])?, + }) + } + } +} diff --git a/src/install/lockfile/printer/tree_printer.rs b/src/install/lockfile/printer/tree_printer.rs index 9dad918b9f25..72b9516f29e8 100644 --- a/src/install/lockfile/printer/tree_printer.rs +++ b/src/install/lockfile/printer/tree_printer.rs @@ -334,6 +334,47 @@ where Ok(()) } +fn print_installed_update_request( + writer: &mut W, + dependency: &Dependency, + resolution: &Resolution, + string_buf: &[u8], + has_binaries: bool, +) -> Result<(), crate::Error> +where + W: Write, +{ + bun_core::write_pretty!( + writer, + ENABLE_ANSI_COLORS, + "installed {s}", + bstr::BStr::new(dependency.name.slice(string_buf)), + )?; + + if let Some(npm) = dependency.version.try_npm().filter(|npm| npm.is_alias) { + bun_core::write_pretty!( + writer, + ENABLE_ANSI_COLORS, + "@npm:{s}", + bstr::BStr::new(npm.name.slice(string_buf)), + )?; + } + + bun_core::write_pretty!( + writer, + ENABLE_ANSI_COLORS, + "@{f}", + resolution.fmt(string_buf, PathSep::Posix), + )?; + writer.write_str(if has_binaries { + " with binaries:\n" + } else { + "\n" + })?; + + Ok(()) +} + /// - Prints an empty newline with no diffs /// - Prints a leading and trailing blank newline with diffs pub(crate) fn print( @@ -500,22 +541,17 @@ where had_printed_new_install = true; } - let name = dependencies_buffer[dependency_id as usize].name; + let dependency = &dependencies_buffer[dependency_id as usize]; let package_id = resolutions_buffer[dependency_id as usize]; let bin = bins[package_id as usize]; - - let package_name = name.slice(string_buf); + let resolution = &resolved[package_id as usize]; match bin.tag { bin::Tag::None | bin::Tag::Dir => { printed_installed_update_request = true; - bun_core::write_pretty!( - writer, - ENABLE_ANSI_COLORS, - "installed {s}@{f}\n", - bstr::BStr::new(package_name), - resolved[package_id as usize].fmt(string_buf, PathSep::Posix), + print_installed_update_request::( + writer, dependency, resolution, string_buf, false, )?; } bin::Tag::Map | bin::Tag::File | bin::Tag::NamedFile => { @@ -526,7 +562,7 @@ where i: 0, done: false, dir_iterator: None, - package_name: name, + package_name: dependency.name, // Never read on the .map/.file/.named_file paths this arm covers. destination_node_modules: Fd::INVALID, buf: bun_paths::PathBuffer::uninit(), @@ -534,15 +570,9 @@ where extern_string_buf: this.lockfile.buffers.extern_strings.as_slice(), }; - { - bun_core::write_pretty!( - writer, - ENABLE_ANSI_COLORS, - "installed {s}@{f} with binaries:\n", - bstr::BStr::new(package_name), - resolved[package_id as usize].fmt(string_buf, PathSep::Posix), - )?; - } + print_installed_update_request::( + writer, dependency, resolution, string_buf, true, + )?; { if matches!(manager.track_installed_bin, TrackInstalledBin::Pending) { diff --git a/src/install/lockfile/pruned_workspaces.rs b/src/install/lockfile/pruned_workspaces.rs index 96fbab00ed5e..399f58fd460f 100644 --- a/src/install/lockfile/pruned_workspaces.rs +++ b/src/install/lockfile/pruned_workspaces.rs @@ -16,3 +16,12 @@ pub(crate) fn workspace_is_missing_on_disk( let _ = package_json_path.append(b"package.json"); !bun_sys::exists_z(package_json_path.slice_z()) } + +pub(crate) fn lockfile_lists_workspace_path(lockfile: &Lockfile, workspace_path: &[u8]) -> bool { + let string_bytes = lockfile.buffers.string_bytes.as_slice(); + lockfile + .workspace_paths + .values() + .iter() + .any(|path| path.slice(string_bytes) == workspace_path) +} diff --git a/src/install/lockfile/reachable.rs b/src/install/lockfile/reachable.rs new file mode 100644 index 000000000000..0fb72ee8b983 --- /dev/null +++ b/src/install/lockfile/reachable.rs @@ -0,0 +1,99 @@ +use crate::lockfile::package::PackageColumns as _; +use crate::lockfile_real::Lockfile; +use crate::npm::{Architecture, OperatingSystem}; +use crate::{PackageID, PackageManager}; + +#[derive(Clone, Copy)] +pub struct Options { + pub root: PackageID, + pub dev: bool, + pub optional: bool, + pub peer: bool, + pub optional_peer: bool, + pub bundled: bool, + pub platform: Option<(Architecture, OperatingSystem)>, +} + +impl Options { + pub fn all(root: PackageID) -> Options { + Options { + root, + dev: true, + optional: true, + peer: true, + optional_peer: true, + bundled: true, + platform: None, + } + } + + // What `bun install` would link with the manager's `--production` / `--omit` / os / cpu settings. + pub(crate) fn install(manager: &PackageManager) -> Options { + let features = manager.options.local_package_features; + Options { + root: 0, + dev: features.dev_dependencies, + optional: features.optional_dependencies, + peer: features.peer_dependencies, + optional_peer: features.peer_dependencies, + bundled: false, + platform: Some((manager.options.cpu, manager.options.os)), + } + } +} + +// `resolutions` is passed separately so callers can walk a candidate resolution buffer (dedupe). +pub fn packages(lockfile: &Lockfile, resolutions: &[PackageID], options: Options) -> Vec { + let pkgs = lockfile.packages.slice(); + let dep_slices = pkgs.items_dependencies(); + let metas = pkgs.items_meta(); + let deps = lockfile.buffers.dependencies.as_slice(); + + let mut seen = vec![false; dep_slices.len()]; + if (options.root as usize) >= seen.len() { + return seen; + } + seen[options.root as usize] = true; + let follow_all = + options.dev && options.optional && options.peer && options.optional_peer && options.bundled; + let mut worklist: Vec = vec![options.root]; + while let Some(parent) = worklist.pop() { + let slice = dep_slices[parent as usize]; + for dep_id in slice.begin() as usize..slice.end() as usize { + let followed = follow_all || { + let behavior = deps[dep_id].behavior; + if behavior.is_bundled() && !options.bundled { + false + } else if behavior.is_optional_peer() { + options.optional_peer + } else if behavior.is_peer() { + options.peer + } else if behavior.is_optional() { + options.optional + } else if behavior.is_dev() { + options.dev + } else { + true + } + }; + if !followed { + continue; + } + let target = resolutions[dep_id]; + let Some(slot) = seen.get_mut(target as usize) else { + continue; + }; + if *slot { + continue; + } + if let Some((cpu, os)) = options.platform + && metas[target as usize].is_disabled(cpu, os) + { + continue; + } + *slot = true; + worklist.push(target); + } + } + seen +} diff --git a/src/install/migration.rs b/src/install/migration.rs index f00f50d900c4..544120706f2d 100644 --- a/src/install/migration.rs +++ b/src/install/migration.rs @@ -21,7 +21,7 @@ use crate::lockfile::{ Migrated, PackageListEntry, }; use crate::lockfile_real::package::PackageColumns as _; -use crate::lockfile_real::package::workspace_map::{NamesArray, WorkspaceMap}; +use crate::lockfile_real::package::workspace_map::{MissingWorkspace, NamesArray, WorkspaceMap}; use crate::npm::{self as Npm}; use crate::pnpm; use crate::pnpm::MigratePnpmLockfileError; @@ -326,7 +326,7 @@ fn migrate_npm_lockfile<'a>( &json_src, wksp_loc, None, - false, + MissingWorkspace::Error, )?; debug!("found {} workspace packages", workspace_packages_count); num_deps += workspace_packages_count; diff --git a/src/install/pnpm.rs b/src/install/pnpm.rs index df6dd8e327de..6a8829341baa 100644 --- a/src/install/pnpm.rs +++ b/src/install/pnpm.rs @@ -570,36 +570,33 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( bstr::BStr::new(name_str) ), ); - } else if Dependency::split_name_and_maybe_version(name_str) - .1 - .is_some() - || strings::contains_char(name_str, b'>') - { - log.add_warning_fmt( - None, - bun_ast::Loc::EMPTY, - format_args!( - "pnpm-lock.yaml override '{}' is scoped to a version range or parent package, which bun does not support; it will not apply", - bstr::BStr::new(name_str) - ), - ); } - let name_hash = semver::string::Builder::string_hash(name_str); - let name = sbuf!(lockfile).append_with_hash(name_str, name_hash)?; - - let version_hash = semver::string::Builder::string_hash(version_str); - let version = sbuf!(lockfile).append_with_hash(version_str, version_hash)?; - let version_sliced = version.sliced(string_bytes!(lockfile)); + let sel = match crate::lockfile_real::override_selector::parse_selector(name_str) { + Ok(sel) => sel, + Err(_) => { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml override '{}' uses a selector bun does not support; it will not apply", + bstr::BStr::new(name_str) + ), + ); + continue; + } + }; - let Some(version) = Dependency::parse( - name, - name_hash, - version_sliced.slice, - &version_sliced, - Some(&mut *log), + let ok = crate::lockfile_real::OverrideMap::put_lockfile_rule( + &mut lockfile.overrides, + sel.parent, + sel.target, + version_str, + &mut sbuf!(lockfile), + log, Some(&mut *manager), - ) else { + )?; + if !ok { log.add_error_fmt( None, bun_ast::Loc::EMPTY, @@ -610,16 +607,7 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( ), ); return Err(invalid_pnpm_lockfile()); - }; - - let dep = Dependency { - name, - name_hash, - version, - ..Default::default() - }; - - lockfile.overrides.map.put(name_hash, dep)?; + } } } diff --git a/src/install/prune.rs b/src/install/prune.rs index 440bacc3f6f5..9ff7c4a610b0 100644 --- a/src/install/prune.rs +++ b/src/install/prune.rs @@ -1,15 +1,17 @@ +use core::cell::{Cell, RefCell}; +use core::ops::Range; use std::io::Write as _; use bstr::BStr; use bun_core::{Global, Output, ZStr, strings}; use bun_install_types::NodeLinker::NodeLinker; use bun_paths::SEP; -use bun_sys::{self as sys, Dir, E, EntryKind}; +use bun_sys::{self as sys, Dir, E, EntryKind, O}; use crate::config_version::ConfigVersion; use crate::lockfile::package::PackageColumns as _; use crate::lockfile::tree::is_filtered_dependency_or_workspace; -use crate::lockfile::{LoadResult, Lockfile, tree}; +use crate::lockfile::{LoadResult, Lockfile, reachable, tree}; use crate::package_manager::Options::LogLevel; use crate::{PackageID, PackageManager, ResolutionTag, invalid_package_id}; @@ -125,13 +127,14 @@ pub fn prune(manager: &mut PackageManager) -> crate::Result<()> { let quiet = manager.options.log_level == LogLevel::Silent; let dry_run = manager.options.dry_run; - let load = manager.load_lockfile_from_cwd::(); - let loaded = match &load { - LoadResult::NotFound => Err(None), - LoadResult::Err(cause) => Err(Some(cause.value.name())), - LoadResult::Ok(_) => Ok(load.choose_config_version().0), + let loaded = { + let load = manager.load_lockfile_from_cwd::(); + match &load { + LoadResult::NotFound => Err(None), + LoadResult::Err(cause) => Err(Some(cause.value.name())), + LoadResult::Ok(_) => Ok(load.choose_config_version().0), + } }; - drop(load); let config_version = match loaded { Ok(config_version) => config_version, Err(None) => { @@ -276,6 +279,195 @@ fn hoist_filtered(manager: &mut PackageManager) { } } +struct TreeFolder { + path: Range, + expected: Range, +} + +struct HoistedTree<'a> { + lockfile: &'a Lockfile, + trees: &'a [tree::Tree], + folders: Vec, + paths: Vec, + expected: Vec<(&'a [u8], PackageID)>, + workspace_names: &'a [Box<[u8]>], + quiet: bool, + kept_mismatched: Cell, + verified: RefCell>, +} + +impl<'a> HoistedTree<'a> { + fn init( + lockfile: &'a Lockfile, + workspace_names: &'a [Box<[u8]>], + quiet: bool, + ) -> HoistedTree<'a> { + let trees = lockfile.buffers.trees.as_slice(); + let deps = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + let buf = lockfile.buffers.string_bytes.as_slice(); + + let mut folders: Vec = Vec::with_capacity(trees.len()); + folders.resize_with(trees.len(), || TreeFolder { + path: 0..0, + expected: 0..0, + }); + let mut paths: Vec = Vec::new(); + let mut expected: Vec<(&'a [u8], PackageID)> = + Vec::with_capacity(lockfile.buffers.hoisted_dependencies.len()); + + let mut it = tree::Iterator::<{ tree::IteratorPathStyle::NodeModules }>::init(lockfile); + while let Some(folder) = it.next(None) { + let path_start = paths.len(); + paths.extend_from_slice(folder.relative_path.as_bytes()); + + let start = expected.len(); + expected.extend(folder.dependencies.iter().map(|&dep_id| { + ( + deps[dep_id as usize].name.slice(buf), + resolutions[dep_id as usize], + ) + })); + expected[start..].sort_unstable(); + let mut len = start; + for i in start..expected.len() { + if len == start || expected[len - 1].0 != expected[i].0 { + expected[len] = expected[i]; + len += 1; + } + } + expected.truncate(len); + + folders[folder.tree_id as usize] = TreeFolder { + path: path_start as u32..paths.len() as u32, + expected: start as u32..len as u32, + }; + } + + HoistedTree { + lockfile, + trees, + folders, + paths, + expected, + workspace_names, + quiet, + kept_mismatched: Cell::new(false), + verified: RefCell::new(Vec::new()), + } + } + + fn path(&self, tree_id: usize) -> &[u8] { + let range = &self.folders[tree_id].path; + &self.paths[range.start as usize..range.end as usize] + } + + fn expected(&self, tree_id: usize) -> &[(&'a [u8], PackageID)] { + let range = &self.folders[tree_id].expected; + &self.expected[range.start as usize..range.end as usize] + } + + fn expected_in(&self, tree_id: tree::Id, alias: &[u8]) -> Option<(&'a [u8], PackageID)> { + if (tree_id as usize) >= self.folders.len() { + return None; + } + let expected = self.expected(tree_id as usize); + expected + .binary_search_by(|(name, _)| (*name).cmp(alias)) + .ok() + .map(|i| expected[i]) + } + + fn removable(&self, from: tree::Id, alias: &[u8], entry_folder: &[u8]) -> bool { + let mut id = from; + while (id as usize) < self.trees.len() { + if let Some((alias, pkg_id)) = self.expected_in(id, alias) { + if self.verified_installed(id, alias, pkg_id) { + return true; + } + self.kept_mismatched.set(true); + if !self.quiet { + bun_core::warn!( + "{} is not the version bun.lock installs there; keeping {}", + BStr::new(&join(self.path(id as usize), alias)), + BStr::new(&join(entry_folder, alias)) + ); + } + return false; + } + id = self.trees[id as usize].parent; + } + true + } + + fn verified_installed(&self, tree_id: tree::Id, alias: &'a [u8], pkg_id: PackageID) -> bool { + if let Some(&(_, _, matches)) = self + .verified + .borrow() + .iter() + .find(|(id, name, _)| *id == tree_id && *name == alias) + { + return matches; + } + let matches = self.installed_matches(tree_id, alias, pkg_id); + self.verified.borrow_mut().push((tree_id, alias, matches)); + matches + } + + fn installed_matches(&self, tree_id: tree::Id, alias: &[u8], pkg_id: PackageID) -> bool { + let buf = self.lockfile.buffers.string_bytes.as_slice(); + let deps = self.lockfile.buffers.dependencies.as_slice(); + let Some(res) = self + .lockfile + .packages + .items_resolution() + .get(pkg_id as usize) + else { + return false; + }; + let Some(folder) = open_tree_folder(self.trees, deps, buf, tree_id, self.workspace_names) + else { + return false; + }; + let Some(package) = descend(&folder, alias, false) else { + return false; + }; + match res.tag { + ResolutionTag::Npm => { + let Ok(bytes) = sys::File::read_from(package.fd(), b"package.json") else { + return false; + }; + crate::initialize_store(); + let source = bun_ast::Source::init_path_string_owned(b"package.json", bytes); + let mut log = bun_ast::Log::init(); + let mut checker = + crate::bun_json::PackageJSONVersionChecker::init(&source, &mut log); + if checker.parse().is_err() + || checker.has_errors() + || !checker.has_found_name + || !checker.has_found_version + { + return false; + } + let expected = res.npm().version.fmt(buf).to_string(); + without_build(checker.found_version()) == without_build(expected.as_bytes()) + && checker.found_name() + == self.lockfile.packages.items_name()[pkg_id as usize].slice(buf) + } + ResolutionTag::Git | ResolutionTag::Github => { + sys::File::read_from(package.fd(), b".bun-tag") + .is_ok_and(|tag| tag.as_slice() == res.repository().resolved.slice(buf)) + } + _ => false, + } + } +} + +// https://github.com/oven-sh/bun/issues/13563 +fn without_build(version: &[u8]) -> &[u8] { + &version[..strings::last_index_of_char(version, b'+').unwrap_or(version.len())] +} + fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], plan: &mut Plan) { let keep_workspaces = |entry: &Entry| contains(workspace_names, entry.alias); if manager.lockfile.packages.len() == 0 { @@ -287,7 +479,9 @@ fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], pla hoist_filtered(manager); + let quiet = manager.options.log_level == LogLevel::Silent; let lockfile: &Lockfile = &manager.lockfile; + let hoisted = HoistedTree::init(lockfile, workspace_names, quiet); let buf = lockfile.buffers.string_bytes.as_slice(); let deps = lockfile.buffers.dependencies.as_slice(); let resolutions = lockfile.buffers.resolutions.as_slice(); @@ -308,10 +502,13 @@ fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], pla nested_trees.sort_unstable(); let mut visited = vec![false; pkg_res.len()]; - let mut expected: Vec<(&[u8], PackageID)> = Vec::new(); - let mut it = tree::Iterator::<{ tree::IteratorPathStyle::NodeModules }>::init(lockfile); - while let Some(folder) = it.next(None) { - let owner: PackageID = match trees[folder.tree_id as usize].dependency_id { + for tree_idx in 0..hoisted.folders.len() { + let folder_path = hoisted.path(tree_idx); + if folder_path.is_empty() { + continue; + } + let tree_id = tree_idx as tree::Id; + let owner: PackageID = match trees[tree_idx].dependency_id { tree::ROOT_DEP_ID => 0, dep_id => resolutions[dep_id as usize], }; @@ -326,24 +523,15 @@ fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], pla } } - expected.clear(); - expected.extend(folder.dependencies.iter().map(|&dep_id| { - ( - deps[dep_id as usize].name.slice(buf), - resolutions[dep_id as usize], - ) - })); - expected.sort_unstable(); - expected.dedup_by_key(|(alias, _)| *alias); + let expected = hoisted.expected(tree_idx); - let Some(dir) = open_tree_folder(trees, deps, buf, folder.tree_id, workspace_names) else { + let Some(dir) = open_tree_folder(trees, deps, buf, tree_id, workspace_names) else { continue; }; - let folder_path = folder.relative_path.as_bytes(); - for &(alias, pkg_id) in &expected { + for &(alias, pkg_id) in expected { if (pkg_id as usize) >= pkg_res.len() - || nested_trees.binary_search(&(folder.tree_id, alias)).is_ok() + || nested_trees.binary_search(&(tree_id, alias)).is_ok() { continue; } @@ -364,9 +552,19 @@ fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], pla continue; }; let nested_path = join(&join(folder_path, alias), b"node_modules"); - scan_folder(nested, &nested_path, false, &keep_workspaces, plan); + scan_folder( + nested, + &nested_path, + false, + &|entry: &Entry| { + contains(workspace_names, entry.alias) + || !hoisted.removable(tree_id, entry.alias, &nested_path) + }, + plan, + ); } + let parent = trees[tree_idx].parent; scan_folder( dir, folder_path, @@ -376,6 +574,7 @@ fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], pla .binary_search_by(|(name, _)| (*name).cmp(entry.alias)) .is_ok() || contains(workspace_names, entry.alias) + || !hoisted.removable(parent, entry.alias, folder_path) }, plan, ); @@ -396,9 +595,24 @@ fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], pla } let folder_path = join(path, b"node_modules"); if let Ok(dir) = Dir::open(&folder_path) { - scan_folder(dir, &folder_path, false, &keep_workspaces, plan); + scan_folder( + dir, + &folder_path, + false, + &|entry: &Entry| { + contains(workspace_names, entry.alias) + || !hoisted.removable(0, entry.alias, &folder_path) + }, + plan, + ); } } + + if hoisted.kept_mismatched.get() && !quiet { + bun_core::note!( + "run 'bun install' with the same flags to install the versions bun.lock expects, then run 'bun prune' again" + ); + } } fn open_tree_folder( @@ -444,9 +658,11 @@ fn open_real_subdir(dir: &Dir, name: &[u8]) -> Option { if lstat_kind(dir, name) != EntryKind::Directory { return None; } - dir.open_at(name).ok() + dir.open_at_with(name, O::RDONLY | O::CLOEXEC | O::NOFOLLOW) + .ok() } +#[cfg(not(windows))] fn lstat_kind(dir: &Dir, name: &[u8]) -> EntryKind { match sys::lstatat(dir.fd(), ZStr::from_slice_with_nul(&zname(name))) { Ok(st) => sys::kind_from_mode(st.st_mode as sys::Mode), @@ -454,6 +670,24 @@ fn lstat_kind(dir: &Dir, name: &[u8]) -> EntryKind { } } +// `sys::lstatat` fstats the opened reparse point, which reports junctions as directories. +#[cfg(windows)] +fn lstat_kind(dir: &Dir, name: &[u8]) -> EntryKind { + let mut dir_buf = bun_paths::path_buffer_pool::get(); + let Ok(dir_path) = dir.get_fd_path(&mut dir_buf) else { + return EntryKind::Unknown; + }; + let mut path_buf = bun_paths::path_buffer_pool::get(); + let path = bun_paths::resolve_path::join_string_buf_z::( + &mut path_buf[..], + &[&*dir_path, name], + ); + match sys::lstat(path) { + Ok(st) => sys::kind_from_mode(st.st_mode as sys::Mode), + Err(_) => EntryKind::Unknown, + } +} + fn entry_kind(dir: &Dir, name: &[u8], kind: EntryKind) -> EntryKind { if kind != EntryKind::Unknown { return kind; @@ -528,40 +762,11 @@ fn scan_folder( fn wanted_packages(manager: &PackageManager) -> Vec { let lockfile: &Lockfile = &manager.lockfile; - let resolutions = lockfile.buffers.resolutions.as_slice(); - let dep_slices = lockfile.packages.items_dependencies(); - let mut wanted = vec![false; dep_slices.len()]; - if wanted.is_empty() { - return wanted; - } - wanted[0] = true; - let mut worklist: Vec = vec![0]; - while let Some(parent) = worklist.pop() { - let slice = dep_slices[parent as usize]; - for dep_id in slice.begin()..slice.end() { - if is_filtered_dependency_or_workspace( - dep_id, - parent, - &[], - true, - manager, - lockfile, - resolutions, - ) { - continue; - } - let target = resolutions[dep_id as usize]; - if target == invalid_package_id - || (target as usize) >= wanted.len() - || wanted[target as usize] - { - continue; - } - wanted[target as usize] = true; - worklist.push(target); - } - } - wanted + reachable::packages( + lockfile, + lockfile.buffers.resolutions.as_slice(), + reachable::Options::install(manager), + ) } fn store_keys(lockfile: &Lockfile, wanted: &[bool]) -> Vec> { @@ -619,7 +824,7 @@ fn strip_peer_hash(name: &[u8]) -> Option<&[u8]> { (suffix[0] == b'+' && suffix[1..].iter().all(u8::is_ascii_hexdigit)).then_some(base) } -fn direct_aliases(manager: &PackageManager, pkg_id: PackageID, filtered: bool) -> Vec> { +fn direct_aliases(manager: &PackageManager, pkg_id: PackageID) -> Vec> { let lockfile: &Lockfile = &manager.lockfile; let buf = lockfile.buffers.string_bytes.as_slice(); let deps = lockfile.buffers.dependencies.as_slice(); @@ -627,22 +832,20 @@ fn direct_aliases(manager: &PackageManager, pkg_id: PackageID, filtered: bool) - let slice = lockfile.packages.items_dependencies()[pkg_id as usize]; let mut direct: Vec> = Vec::new(); for dep_id in slice.begin()..slice.end() { - if filtered { - if is_filtered_dependency_or_workspace( - dep_id, - pkg_id, - &[], - true, - manager, - lockfile, - resolutions, - ) { - continue; - } - let target = resolutions[dep_id as usize]; - if target == invalid_package_id || (target as usize) >= lockfile.packages.len() { - continue; - } + if is_filtered_dependency_or_workspace( + dep_id, + pkg_id, + &[], + true, + manager, + lockfile, + resolutions, + ) { + continue; + } + let target = resolutions[dep_id as usize]; + if target == invalid_package_id || (target as usize) >= lockfile.packages.len() { + continue; } direct.push(deps[dep_id as usize].name.slice(buf).into()); } @@ -700,24 +903,18 @@ fn plan_isolated(manager: &PackageManager, workspace_names: &[Box<[u8]>], plan: let Ok(dir) = Dir::open(&folder_path) else { continue; }; - let direct = direct_aliases(manager, pkg_id as PackageID, true); - let declared = direct_aliases(manager, pkg_id as PackageID, false); + let direct = direct_aliases(manager, pkg_id as PackageID); let folder_idx = scan_folder( dir, &folder_path, store_touched, &|entry| { - let known = match entry.kind { - EntryKind::SymLink => { - contains(&declared, entry.alias) - || store_link_target(entry.dir, entry.name) - .is_some_and(|target| contains(&removed_store, &target)) - } - _ => contains(&direct, entry.alias), - }; - known + contains(&direct, entry.alias) || contains(workspace_names, entry.alias) || public_hoist_matches(manager, entry.alias) + || (entry.kind == EntryKind::SymLink + && store_link_target(entry.dir, entry.name) + .is_some_and(|target| contains(&removed_store, &target))) }, plan, ); @@ -807,7 +1004,7 @@ fn rmdir(dir: &Dir, name: &[u8]) { fn is_dangling(dir: &Dir, name: &[u8]) -> bool { let z = zname(name); - match sys::exists_at_type(dir.fd(), ZStr::from_slice_with_nul(&z)) { + match sys::fstatat(dir.fd(), ZStr::from_slice_with_nul(&z)) { Ok(_) => false, Err(err) => matches!(err.get_errno(), E::ENOENT | E::ENOTDIR), } diff --git a/src/install/update_transitive.rs b/src/install/update_transitive.rs new file mode 100644 index 000000000000..c07f9ee73822 --- /dev/null +++ b/src/install/update_transitive.rs @@ -0,0 +1,453 @@ +use core::cmp::Ordering; +use std::io::Write as _; + +use bstr::BStr; +use bun_core::{Output, prettyln}; +use bun_semver as Semver; + +use crate::audit_fix; +use crate::dedupe; +use crate::dependency::{self, Behavior}; +use crate::lockfile::Lockfile; +use crate::lockfile::package::PackageColumns as _; +use crate::npm::PackageManifest; +use crate::package_manager::Options::LogLevel; +use crate::package_manager_real::populate_manifest_cache::{self, Packages}; +use crate::{ + DependencyID, DependencyVersionTag, ManifestLoad, PackageID, PackageManager, PackageNameHash, + ResolutionTag, invalid_package_id, +}; + +/// Root/workspace dependency rows as loaded from bun.lock, taken before the differ re-enqueues them. +#[derive(Default)] +pub struct DirectDependencies { + owners: Vec<(PackageID, u32, u32)>, + rows: Vec<(PackageNameHash, Behavior, PackageID)>, +} + +impl DirectDependencies { + pub fn snapshot(lockfile: &Lockfile) -> DirectDependencies { + let pkg_res = lockfile.packages.items_resolution(); + let dep_slices = lockfile.packages.items_dependencies(); + let res_slices = lockfile.packages.items_resolutions(); + let deps = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + + let mut out = DirectDependencies::default(); + for owner in 0..pkg_res.len() { + if !matches!( + pkg_res[owner].tag, + ResolutionTag::Root | ResolutionTag::Workspace + ) { + continue; + } + let start = out.rows.len(); + out.rows.extend( + dep_slices[owner] + .get(deps) + .iter() + .zip(res_slices[owner].get(resolutions)) + .map(|(dep, &resolved)| (dep.name_hash, dep.behavior, resolved)), + ); + out.owners.push(( + owner as PackageID, + start as u32, + (out.rows.len() - start) as u32, + )); + } + out + } + + /// Edges still resolving to the previous package of a direct dependency that moved follow it when their range allows. + pub fn redirect_dependents(&self, lockfile: &mut Lockfile) { + if self.owners.is_empty() || lockfile.loaded_package_count == 0 { + return; + } + redirect(lockfile, self.moved_pairs(lockfile)); + } + + fn moved_pairs(&self, lockfile: &Lockfile) -> Vec<(PackageID, PackageID)> { + let packages_len = lockfile.packages.len(); + let pkg_res = lockfile.packages.items_resolution(); + let name_hashes = lockfile.packages.items_name_hash(); + let dep_slices = lockfile.packages.items_dependencies(); + let res_slices = lockfile.packages.items_resolutions(); + let deps = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + + let mut pairs: Vec<(PackageID, PackageID)> = Vec::new(); + for &(owner, start, len) in &self.owners { + let owner = owner as usize; + if owner >= packages_len { + continue; + } + let rows = &self.rows[start as usize..(start + len) as usize]; + let mut claimed: Option> = None; + let current = dep_slices[owner] + .get(deps) + .iter() + .zip(res_slices[owner].get(resolutions)); + for (i, (dep, &new)) in current.enumerate() { + let same = |row: &(PackageNameHash, Behavior, PackageID)| { + row.0 == dep.name_hash && row.1 == dep.behavior + }; + let index = if claimed.is_none() && rows.get(i).is_some_and(same) { + i + } else { + // Every row before the first miss was a same-index hit. + let taken = claimed.get_or_insert_with(|| { + let mut taken = vec![false; rows.len()]; + taken[..i.min(rows.len())].fill(true); + taken + }); + let hit = if !taken.get(i).copied().unwrap_or(true) && same(&rows[i]) { + Some(i) + } else { + (0..rows.len()).find(|&k| !taken[k] && same(&rows[k])) + }; + let Some(k) = hit else { + continue; + }; + taken[k] = true; + k + }; + let old = rows[index].2; + if old == new + || (old as usize) >= packages_len + || (new as usize) >= packages_len + || pkg_res[new as usize].tag != ResolutionTag::Npm + || name_hashes[old as usize] != name_hashes[new as usize] + { + continue; + } + pairs.push((old, new)); + } + } + pairs + } +} + +/// Every edge still resolving to an `old` package whose range accepts its `new` npm package is re-pointed at it. +fn redirect(lockfile: &mut Lockfile, pairs: Vec<(PackageID, PackageID)>) { + if pairs.is_empty() { + return; + } + let mut new_of: Vec = vec![invalid_package_id; lockfile.packages.len()]; + for (old, new) in pairs { + let slot = &mut new_of[old as usize]; + if *slot == invalid_package_id { + *slot = new; + } + } + + let moved: Vec<(usize, PackageID)> = { + let lockfile: &Lockfile = lockfile; + let buf = lockfile.buffers.string_bytes.as_slice(); + let pkg_res = lockfile.packages.items_resolution(); + let deps = lockfile.buffers.dependencies.as_slice(); + let mut moved = Vec::new(); + for (j, &target) in lockfile.buffers.resolutions.iter().enumerate() { + let Some(&new) = new_of.get(target as usize) else { + continue; + }; + if new == invalid_package_id { + continue; + } + let dep = &deps[j]; + if dep.behavior.is_bundled() { + continue; + } + let Some(range) = dedupe::effective_npm_range(lockfile, j as DependencyID, dep) else { + continue; + }; + if range + .npm() + .version + .satisfies(pkg_res[new as usize].npm().version, buf, buf) + { + moved.push((j, new)); + } + } + moved + }; + for (j, new) in moved { + lockfile.buffers.resolutions[j] = new; + } +} + +struct Pin { + dep_id: DependencyID, + from: PackageID, + to: Semver::Version, +} + +/// The transitive half of a bare `bun update`: every edge owned by a non-workspace package moves to the newest release its own range allows. +#[derive(Default)] +pub struct TransitiveUpdate { + pins: Vec, +} + +impl TransitiveUpdate { + /// Runs on the loaded lockfile, before the differ; prints the plan. + pub fn plan(manager: &mut PackageManager) -> crate::Result { + let (pins, rows) = plan_edges(manager)?; + if !rows.is_empty() && manager.options.log_level != LogLevel::Silent { + print_plan(&rows, manager.options.dry_run); + } + Ok(TransitiveUpdate { pins }) + } + + /// Runs after the differ has enqueued the direct dependencies; edges the differ already invalidated are left to it. + pub fn enqueue(&self, manager: &mut PackageManager) -> crate::Result<()> { + if self.pins.is_empty() { + return Ok(()); + } + let _ = manager.get_cache_directory(); + let _ = manager.get_temporary_directory(); + for pin in &self.pins { + if manager.lockfile.buffers.resolutions[pin.dep_id as usize] != pin.from { + continue; + } + audit_fix::enqueue_pinned(manager, pin.dep_id, pin.to)?; + manager.summary.update += 1; + } + Ok(()) + } + + /// Once everything is resolved, the edges the plan skipped (peers, other workspaces' pins) follow a moved package when their range allows, as they do for a moved direct dependency. + pub fn redirect_dependents(&self, lockfile: &mut Lockfile) { + let moved: Vec<(DependencyID, PackageID)> = + self.pins.iter().map(|pin| (pin.dep_id, pin.from)).collect(); + redirect_moved_edges(lockfile, &moved); + } +} + +/// `moved` pairs an invalidated edge with the package it used to resolve to; every other edge still on that package follows it to the edge's new npm resolution when its range allows. +pub(crate) fn redirect_moved_edges(lockfile: &mut Lockfile, moved: &[(DependencyID, PackageID)]) { + let pairs: Vec<(PackageID, PackageID)> = { + let pkg_res = lockfile.packages.items_resolution(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + moved + .iter() + .map(|&(dep_id, from)| (from, resolutions[dep_id as usize])) + .filter(|&(from, to)| { + to != from + && (to as usize) < pkg_res.len() + && pkg_res[to as usize].tag == ResolutionTag::Npm + }) + .collect() + }; + redirect(lockfile, pairs); +} + +type Row = (Box<[u8]>, Box<[u8]>, Box<[u8]>); + +fn print_plan(rows: &[Row], dry_run: bool) { + prettyln!("updating:"); + for (name, from, to) in rows { + prettyln!( + " {}@{} → {}", + BStr::new(name), + BStr::new(from), + BStr::new(to) + ); + } + prettyln!(""); + if dry_run { + let n = rows.len(); + prettyln!( + "Would update {} {}", + n, + if n == 1 { "package" } else { "packages" } + ); + } + Output::flush(); +} + +struct Want { + literal: Option, + range: dependency::Version, + dep_ids: Vec, +} + +struct Instance { + pkg_id: PackageID, + current: Semver::Version, + wants: Vec, +} + +fn workspace_owned_dependencies(lockfile: &Lockfile) -> Vec { + let mut owned = vec![false; lockfile.buffers.dependencies.len()]; + let pkg_res = lockfile.packages.items_resolution(); + for (owner, slice) in lockfile.packages.items_dependencies().iter().enumerate() { + if matches!( + pkg_res[owner].tag, + ResolutionTag::Root | ResolutionTag::Workspace + ) { + owned[slice.begin() as usize..slice.end() as usize].fill(true); + } + } + owned +} + +fn plan_edges(manager: &mut PackageManager) -> crate::Result<(Vec, Vec)> { + let mut instances: Vec = Vec::new(); + { + let lockfile = &*manager.lockfile; + let res = lockfile.packages.items_resolution(); + let has_patches = lockfile.patched_dependencies.count() > 0; + + let mut instance_of: Vec = vec![u32::MAX; res.len()]; + for pkg_id in 0..res.len() { + if res[pkg_id].tag != ResolutionTag::Npm { + continue; + } + if has_patches + && lockfile + .patched_dependencies + .contains(&Semver::string::Builder::string_hash(&dedupe::label( + lockfile, + pkg_id as PackageID, + ))) + { + continue; + } + instance_of[pkg_id] = instances.len() as u32; + instances.push(Instance { + pkg_id: pkg_id as PackageID, + current: res[pkg_id].npm().version, + wants: Vec::new(), + }); + } + if instances.is_empty() { + return Ok((Vec::new(), Vec::new())); + } + + let workspace_owned = workspace_owned_dependencies(lockfile); + let no_overrides = lockfile.overrides.is_empty(); + let deps = lockfile.buffers.dependencies.as_slice(); + for (dep_id, &target) in lockfile.buffers.resolutions.iter().enumerate() { + let Some(&instance) = instance_of.get(target as usize) else { + continue; + }; + let dep = &deps[dep_id]; + if instance == u32::MAX + || workspace_owned[dep_id] + || dep.behavior.is_peer() + || dep.behavior.is_bundled() + { + continue; + } + let dep_id = dep_id as DependencyID; + let inst = &mut instances[instance as usize]; + let literal = (no_overrides + && dep.version.tag == DependencyVersionTag::Npm + && !dep.version.npm().is_alias) + .then_some(dep.version.literal); + if let Some(want) = inst + .wants + .iter_mut() + .find(|want| want.literal.is_some() && want.literal == literal) + { + want.dep_ids.push(dep_id); + continue; + } + let Some(range) = dedupe::effective_npm_range(lockfile, dep_id, dep) else { + continue; + }; + inst.wants.push(Want { + literal, + range, + dep_ids: vec![dep_id], + }); + } + } + instances.retain(|inst| !inst.wants.is_empty()); + if instances.is_empty() { + return Ok((Vec::new(), Vec::new())); + } + + let ids: Vec = instances.iter().map(|inst| inst.pkg_id).collect(); + populate_manifest_cache::populate_manifest_cache(manager, Packages::Exact(&ids))?; + manager + .log_mut() + .print(core::ptr::from_mut(Output::error_writer()))?; + manager.log_mut().reset(); + + let cache_ctx = manager.manifest_disk_cache_ctx(); + let min_age = manager.options.minimum_release_age_ms; + let excludes = manager.options.minimum_release_age_excludes; + let buf = manager.lockfile.buffers.string_bytes.as_slice(); + let pkg_names = manager.lockfile.packages.items_name(); + + let mut pins: Vec = Vec::new(); + let mut rows: Vec = Vec::new(); + for inst in &instances { + let name = pkg_names[inst.pkg_id as usize].slice(buf); + let mut expired = false; + let scope = manager.options.scope_for_package_name(name); + let Some(manifest) = manager.manifests.by_name_allow_expired( + cache_ctx, + scope, + name, + Some(&mut expired), + ManifestLoad::LoadFromMemoryFallbackToDisk, + min_age.is_some(), + ) else { + continue; + }; + let manifest: &PackageManifest = manifest; + let manifest_buf: &[u8] = &manifest.string_buf; + let releases = manifest.pkg.releases.keys.get(&manifest.versions); + let release_pkgs = manifest.pkg.releases.values.get(&manifest.package_versions); + let age_limit = min_age.filter(|_| !manifest.should_exclude_from_age_filter(excludes)); + + let mut from: Vec = Vec::new(); + let _ = write!(from, "{}", inst.current.fmt(buf)); + for want in &inst.wants { + let target = releases + .iter() + .enumerate() + .rev() + .take_while(|(_, v)| v.order(inst.current, manifest_buf, buf) == Ordering::Greater) + .find_map(|(i, &v)| { + if v.tag.has_build() + || !want.range.npm().version.satisfies(v, buf, manifest_buf) + { + return None; + } + if let Some(limit) = age_limit + && PackageManifest::is_package_version_too_recent(&release_pkgs[i], limit) + { + return None; + } + Some(v) + }); + let Some(v) = target else { + continue; + }; + let to = Semver::Version { + major: v.major, + minor: v.minor, + patch: v.patch, + ..Default::default() + }; + pins.extend(want.dep_ids.iter().map(|&dep_id| Pin { + dep_id, + from: inst.pkg_id, + to, + })); + let mut to_text: Vec = Vec::new(); + let _ = write!(to_text, "{}", v.fmt(manifest_buf)); + rows.push(( + Box::from(name), + from.clone().into_boxed_slice(), + to_text.into_boxed_slice(), + )); + } + } + + rows.sort_unstable(); + rows.dedup(); + Ok((pins, rows)) +} diff --git a/src/install/yarn.rs b/src/install/yarn.rs index 1fcfd3ce50ea..e56117ec3d22 100644 --- a/src/install/yarn.rs +++ b/src/install/yarn.rs @@ -804,25 +804,32 @@ pub(crate) fn migrate_yarn_lockfile<'a>( scripts: Default::default(), })?; - if let Some(resolutions) = package_json.as_property(b"resolutions") { + if package_json.as_property(b"resolutions").is_some() { let root_package = *this.packages.get(0); let (mut string_builder, lf) = this.string_builder_split(); - if let bun_ast::ExprData::EObject(e_object) = &resolutions.expr.data { - string_builder.cap += e_object.properties.len_u32() as usize * 128; - } - if string_builder.cap > 0 { - string_builder.allocate()?; - } + let workspace_names = + crate::lockfile_real::package::workspace_map::WorkspaceMap::init(); + lf.overrides.parse_count( + manager, + log, + &package_json_source, + &workspace_names, + package_json, + &mut string_builder, + ); + string_builder.allocate()?; lf.overrides.parse_append( manager, lf.dependencies.as_slice(), &root_package, log, &package_json_source, + &workspace_names, package_json, &mut string_builder, )?; + string_builder.clamp(); this.packages.set(0, root_package); } } diff --git a/src/runtime/cli/audit_command.rs b/src/runtime/cli/audit_command.rs index d39a45e2974f..190dca81d9fb 100644 --- a/src/runtime/cli/audit_command.rs +++ b/src/runtime/cli/audit_command.rs @@ -8,10 +8,11 @@ use bun_core::{MutableString, strings}; use bun_http::{self as http, HeaderBuilder}; use bun_install::audit_fix::{self, Advisory}; use bun_install::lockfile::package::PackageColumns as _; +use bun_install::lockfile::reachable; use bun_install::package_manager_real::command_line_arguments::AuditLevel; use bun_install::package_manager_real::{ROOT_PACKAGE_JSON_PATH, install_with_manager}; use bun_install::resolution::Tag as ResolutionTag; -use bun_install::{CommandLineArguments, Lockfile, PackageID, PackageManager, Subcommand}; +use bun_install::{CommandLineArguments, PackageManager, Subcommand}; use bun_libdeflate_sys::libdeflate; use bun_parsers::json as bun_json; use bun_url::URL; @@ -67,7 +68,6 @@ impl AuditCommand { let cli = CommandLineArguments::parse(Subcommand::Audit)?; // Note: `init` consumes `cli`; capture the fields read after it. let audit_level = cli.audit_level; - let production = cli.production; let audit_ignore_list = cli.audit_ignore_list; let fix = cli.positionals.len() > 1 && cli.positionals[1] == b"fix"; if fix && cli.positionals.len() > 2 { @@ -106,26 +106,15 @@ impl AuditCommand { return Self::audit_fix(ctx, manager, audit_level, audit_ignore_list, &original_cwd); } - let code = Self::audit( - ctx, - manager, - json_output, - audit_level, - production, - audit_ignore_list, - )?; + let code = Self::audit(ctx, manager, json_output, audit_level, audit_ignore_list)?; Global::exit(code); } - /// Returns the exit code of the command. 0 if no vulnerabilities were found, 1 if vulnerabilities were found. - /// The exception is when you pass --json, it will simply return 0 as that was considered a successful "request - /// for the audit information" fn audit( _ctx: Command::Context, pm: &mut PackageManager, json_output: bool, audit_level: Option, - audit_prod_only: bool, ignore_list: &[&[u8]], ) -> Result { bun_core::pretty_error!( @@ -144,7 +133,7 @@ impl AuditCommand { let dependency_tree = build_dependency_tree(pm)?; - let packages_result = collect_packages_for_audit(pm, audit_prod_only)?; + let packages_result = collect_packages_for_audit(pm, true)?; let response_text = send_audit_request(pm, &packages_result.audit_body)?; @@ -152,33 +141,19 @@ impl AuditCommand { let _ = Output::writer().write_all(&response_text); let _ = Output::writer().write_all(b"\n"); - if !response_text.is_empty() { - let source = - bun_ast::Source::init_path_string(b"audit-response.json", &response_text[..]); - let mut log = bun_ast::Log::init(); - - let parsed = match bun_json::ParsedJson::parse_json(&source, &mut log) { - Ok(e) => e, - Err(_) => { - bun_core::pretty_errorln!( - "error: audit request failed to parse json. Is the registry down?" - ); - return Ok(1); // If we can't parse then safe to assume a similar failure - } - }; - - // If the response is an empty object, no vulnerabilities - if let ExprData::EObjectJSON(obj) = &parsed.root.data { - if obj.get().properties().is_empty() { - return Ok(0); - } - } - - // If there's any content in the response, there are vulnerabilities - return Ok(1); + if response_text.is_empty() { + return Ok(0); } - return Ok(0); + return match collect_vulnerabilities(&response_text, audit_level, ignore_list)? { + Some(vulnerabilities) => Ok(u32::from(!vulnerabilities.is_empty())), + None => { + bun_core::pretty_errorln!( + "error: audit request failed to parse json. Is the registry down?" + ); + Ok(1) + } + }; } else if !response_text.is_empty() { let exit_code = print_enhanced_audit_report( &response_text, @@ -342,41 +317,6 @@ fn build_dependency_tree( Ok(dependency_tree) } -fn build_production_package_set(lockfile: &Lockfile, root_id: PackageID) -> Vec { - let packages = lockfile.packages.slice(); - let pkg_dependencies = packages.items_dependencies(); - let pkg_resolutions = packages.items_resolutions(); - let dependencies = lockfile.buffers.dependencies.as_slice(); - let resolutions = lockfile.buffers.resolutions.as_slice(); - - let mut prod_set = vec![false; packages.len()]; - let mut queue: std::collections::VecDeque = std::collections::VecDeque::new(); - if (root_id as usize) < packages.len() { - prod_set[root_id as usize] = true; - queue.push_back(root_id); - } - - while let Some(current_pkg_id) = queue.pop_front() { - let dep_slice = pkg_dependencies[current_pkg_id as usize].get(dependencies); - let res_slice = pkg_resolutions[current_pkg_id as usize].get(resolutions); - - for (dep, &resolved_pkg_id) in dep_slice.iter().zip(res_slice.iter()) { - if dep.behavior.is_dev() || dep.behavior.is_optional_peer() { - continue; - } - let Some(seen) = prod_set.get_mut(resolved_pkg_id as usize) else { - continue; - }; - if !*seen { - *seen = true; - queue.push_back(resolved_pkg_id); - } - } - } - - prod_set -} - struct CollectPackagesResult { audit_body: Box<[u8]>, skipped_packages: Vec>, @@ -389,15 +329,33 @@ struct PackageVersions { fn collect_packages_for_audit( pm: &mut PackageManager, - prod_only: bool, + apply_omit: bool, ) -> Result { let root_id = pm.root_package_id.get(&pm.lockfile, pm.workspace_name_hash); let mut packages_list: Vec = Vec::new(); let mut skipped_packages: Vec> = Vec::new(); - let prod_packages: Option> = - prod_only.then(|| build_production_package_set(&pm.lockfile, root_id)); + let features = pm.options.local_package_features; + let omits_something = apply_omit + && !(features.dev_dependencies + && features.optional_dependencies + && features.peer_dependencies); + let wanted_packages: Option> = omits_something.then(|| { + reachable::packages( + &pm.lockfile, + pm.lockfile.buffers.resolutions.as_slice(), + reachable::Options { + root: root_id, + dev: features.dev_dependencies, + optional: features.optional_dependencies, + peer: features.peer_dependencies, + optional_peer: false, + bundled: true, + platform: None, + }, + ) + }); let options = &pm.options; let default_url_hash = options.scope.url_hash; @@ -414,7 +372,7 @@ fn collect_packages_for_audit( continue; } - if prod_packages.as_ref().is_some_and(|prod| !prod[idx]) { + if wanted_packages.as_ref().is_some_and(|wanted| !wanted[idx]) { continue; } @@ -1036,7 +994,7 @@ fn print_enhanced_audit_report( let total = vuln_counts.low + vuln_counts.moderate + vuln_counts.high + vuln_counts.critical; if total > 0 { - pretty!("{} vulnerabilities (", total); + pretty!("{} {} (", total, audit_fix::vuln_word(total)); let mut has_previous = false; if vuln_counts.critical > 0 { @@ -1065,6 +1023,9 @@ fn print_enhanced_audit_report( } prettyln!(")"); + prettyln!(""); + prettyln!("To upgrade only the vulnerable packages, within their declared ranges:"); + prettyln!(" bun audit fix"); prettyln!(""); prettyln!("To update all dependencies to the latest compatible versions:"); prettyln!(" bun update"); diff --git a/src/runtime/cli/dedupe_command.rs b/src/runtime/cli/dedupe_command.rs index 3fe02ce380d7..590d613fc877 100644 --- a/src/runtime/cli/dedupe_command.rs +++ b/src/runtime/cli/dedupe_command.rs @@ -25,7 +25,7 @@ impl DedupeCommand { let (manager, original_cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Dedupe) { Ok(v) => v, - Err(err) if err == bun_install::Error::MissingPackageJSON => { + Err(bun_install::Error::MissingPackageJSON) => { Output::err_generic("missing package.json, nothing to dedupe", ()); Global::exit(1); } @@ -34,7 +34,7 @@ impl DedupeCommand { if manager.options.should_print_command_name() { bun_core::prettyln!( - "bun dedupe v{}\n\n", + "bun dedupe v{}\n", Global::package_json_version_with_sha, ); Output::flush(); diff --git a/src/runtime/cli/pm_licenses_command.rs b/src/runtime/cli/pm_licenses_command.rs index d5ff539121d9..2a7ed4b65f79 100644 --- a/src/runtime/cli/pm_licenses_command.rs +++ b/src/runtime/cli/pm_licenses_command.rs @@ -6,9 +6,8 @@ use bun_ast::{Expr, Log, Source}; use bun_collections::StringHashMap; use bun_core::fmt::PathSep; use bun_core::{FileKind, Global, Output, strings}; -use bun_install::lockfile::{Lockfile, package::PackageColumns as _, tree}; -use bun_install::npm::{Architecture, OperatingSystem}; -use bun_install::{PackageID, PackageManager, Resolution, ResolutionTag}; +use bun_install::lockfile::{Lockfile, package::PackageColumns as _, reachable, tree}; +use bun_install::{PackageManager, Resolution, ResolutionTag}; use bun_parsers::json as JSON; use bun_paths::AutoAbsPath; use bun_sys::{self, Dir, Fd, File}; @@ -50,7 +49,7 @@ impl PmLicensesCommand { pub(crate) fn exec( pm: &mut PackageManager, positionals: &[&[u8]], - production: bool, + _production: bool, ) -> crate::Result<()> { if positionals.len() > 1 && !strings::eql_comptime(positionals[1], b"list") @@ -65,6 +64,7 @@ impl PmLicensesCommand { PackageManagerCommand::handle_load_lockfile_errors(&load, log_level); let json_output = pm.options.json_output; + let features = pm.options.local_package_features; let root_id = pm.root_package_id.get(&pm.lockfile, pm.workspace_name_hash); let lockfile: &Lockfile = &pm.lockfile; @@ -77,8 +77,19 @@ impl PmLicensesCommand { } path.set_length(top_len); - let wanted = - reachable_packages(lockfile, root_id, production, pm.options.cpu, pm.options.os); + let wanted = reachable::packages( + lockfile, + lockfile.buffers.resolutions.as_slice(), + reachable::Options { + root: root_id, + dev: features.dev_dependencies, + optional: features.optional_dependencies, + peer: features.peer_dependencies, + optional_peer: features.peer_dependencies, + bundled: true, + platform: Some((pm.options.cpu, pm.options.os)), + }, + ); let locations = tree_locations(lockfile); let packages = lockfile.packages.slice(); @@ -166,11 +177,11 @@ impl PmLicensesCommand { } if missing > 0 { - Output::warn(format_args!( + bun_core::warn!( "omitted {} {} from the lockfile not found in node_modules", missing, - if missing == 1 { "package" } else { "packages" }, - )); + if missing == 1 { "package" } else { "packages" } + ); } entries.sort_by(|a, b| { @@ -204,58 +215,6 @@ fn sort_key(e: &Entry) -> (bool, &[u8], &[u8]) { ) } -fn reachable_packages( - lockfile: &Lockfile, - root_id: PackageID, - production: bool, - cpu: Architecture, - os: OperatingSystem, -) -> Vec { - let packages = lockfile.packages.slice(); - let pkg_resolution = packages.items_resolution(); - let pkg_metas = packages.items_meta(); - let pkg_dependencies = packages.items_dependencies(); - let pkg_resolutions = packages.items_resolutions(); - let dependencies = lockfile.buffers.dependencies.as_slice(); - let resolutions = lockfile.buffers.resolutions.as_slice(); - let len = packages.len(); - - let mut marked = vec![false; len]; - if (root_id as usize) >= len { - return marked; - } - marked[root_id as usize] = true; - let mut work: Vec = vec![root_id]; - - while let Some(pkg) = work.pop() { - let skip_dev = production - && matches!( - pkg_resolution[pkg as usize].tag, - ResolutionTag::Root | ResolutionTag::Workspace - ); - let dep_slice = pkg_dependencies[pkg as usize].get(dependencies); - let res_slice = pkg_resolutions[pkg as usize].get(resolutions); - - for (dep, &dep_pkg_id) in dep_slice.iter().zip(res_slice.iter()) { - if (dep_pkg_id as usize) >= len { - continue; - } - if skip_dev && dep.behavior.is_dev() { - continue; - } - if pkg_metas[dep_pkg_id as usize].is_disabled(cpu, os) { - continue; - } - if !marked[dep_pkg_id as usize] { - marked[dep_pkg_id as usize] = true; - work.push(dep_pkg_id); - } - } - } - - marked -} - fn tree_locations(lockfile: &Lockfile) -> Vec>> { let len = lockfile.packages.len(); let mut out: Vec>> = vec![None; len]; @@ -443,7 +402,7 @@ impl BunStore { ); } - let i = entries.partition_point(|e| &e[..] < &key[..]); + let i = entries.partition_point(|e| e[..] < key[..]); let entry = entries.get(i)?; let exact = entry[..] == key[..]; let peer_suffixed = @@ -455,16 +414,16 @@ impl BunStore { path.set_length(top_len); let _ = path.append(b"node_modules"); let _ = path.append(b".bun"); - let mut names: Vec> = list_dir(path.slice()) + let names: Vec> = list_dir(path.slice()) .into_iter() .map(|(name, _)| name) .collect(); path.set_length(top_len); - names.sort_unstable(); names } } +/// Sorted by name: readdir order differs per filesystem and `DiskIndex` keeps the first copy it sees. fn list_dir(path: &[u8]) -> Vec<(Box<[u8]>, FileKind)> { let mut out: Vec<(Box<[u8]>, FileKind)> = Vec::new(); let Ok(dir) = Dir::open(path) else { @@ -474,6 +433,7 @@ fn list_dir(path: &[u8]) -> Vec<(Box<[u8]>, FileKind)> { while let Ok(Some(entry)) = iter.next() { out.push((entry.name.slice_u8().into(), entry.kind)); } + out.sort_unstable_by(|a, b| a.0.cmp(&b.0)); out } diff --git a/src/runtime/cli/prune_command.rs b/src/runtime/cli/prune_command.rs index 26a59feaa1ce..4849f4a7c08b 100644 --- a/src/runtime/cli/prune_command.rs +++ b/src/runtime/cli/prune_command.rs @@ -22,7 +22,7 @@ impl PruneCommand { let (manager, _original_cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Prune) { Ok(v) => v, - Err(err) if err == bun_install::Error::MissingPackageJSON => { + Err(bun_install::Error::MissingPackageJSON) => { Output::err_generic("missing package.json, nothing to prune", ()); Global::exit(1); } diff --git a/src/runtime/cli/update_interactive_command.rs b/src/runtime/cli/update_interactive_command.rs index 1421ac1c7923..5a7316ba4e9b 100644 --- a/src/runtime/cli/update_interactive_command.rs +++ b/src/runtime/cli/update_interactive_command.rs @@ -11,9 +11,10 @@ use bun_core::{Global, Output}; use bun_install::dependency::{self, Behavior}; use bun_install::lockfile::package::PackageColumns as _; use bun_install::lockfile::{LoadResult, LoadStep}; +use bun_install::package_manager::options::Do; use bun_install::package_manager::{ - LogLevel, ManifestLoad, ROOT_PACKAGE_JSON_PATH, Subcommand, WorkspaceFilter, - install_with_manager, populate_manifest_cache, + LogLevel, ManifestLoad, Subcommand, WorkspaceFilter, populate_manifest_cache, + update_package_json_and_install_with_manager, }; use bun_install::{ CommandLineArguments, GetJsonOptions, GetJsonResult, INVALID_PACKAGE_ID, PackageID, @@ -585,6 +586,9 @@ impl UpdateInteractiveCommand { // Collect all package updates with full information let mut package_updates: Vec = Vec::new(); + // Becomes `options.positionals` so the install runs as `bun update `. + let mut positionals: Vec<&'static [u8]> = vec![&b"update"[..]]; + // Process selected packages debug_assert_eq!(outdated_packages.len(), selected.len()); for (pkg, &is_selected) in outdated_packages.iter().zip(selected.iter()) { @@ -603,6 +607,13 @@ impl UpdateInteractiveCommand { continue; } + if !positionals[1..] + .iter() + .any(|name| strings::eql(name, &pkg.name)) + { + positionals.push(crate::cli::cli_dupe(&pkg.name)); + } + // For catalog dependencies, we need to collect them separately // to update the catalog definitions in the root or workspace package.json if pkg.is_catalog { @@ -710,23 +721,14 @@ impl UpdateInteractiveCommand { Self::update_package_json_files_from_updates(manager, &package_updates)?; } - manager.to_update = true; - // Reset the timer to show actual install time instead of total command time ctx.start_time = bun_core::time::nano_timestamp(); - // SAFETY: `ROOT_PACKAGE_JSON_PATH` is set once during - // `PackageManager::init` (single-threaded CLI startup). - let root_pkg_json = unsafe { ROOT_PACKAGE_JSON_PATH.read() }; - // `install_with_manager` takes the original cwd path slice. - // Snapshot before the `&mut manager` borrow. - let root_dir_path: &'static [u8] = manager.root_dir.dir; - install_with_manager::install_with_manager( - manager, - &mut *ctx, - root_pkg_json, - root_dir_path, - )?; + // The chosen versions are already in package.json; the flag would re-pin every selection to `latest`, ignoring per-package `l` toggles. + manager.options.do_.remove(Do::UPDATE_TO_LATEST); + manager.options.positionals = + crate::cli::cli_arena().alloc_slice_copy(&positionals); + update_package_json_and_install_with_manager(manager, &mut *ctx, original_cwd)?; } } Ok(()) diff --git a/src/semver/intersects.rs b/src/semver/intersects.rs new file mode 100644 index 000000000000..736fe6905318 --- /dev/null +++ b/src/semver/intersects.rs @@ -0,0 +1,112 @@ +use core::cmp::Ordering; + +use crate::Version; +use crate::query::{Group, Query}; +use crate::range::{Comparator, Op}; + +#[derive(Clone, Copy)] +struct Bound<'a> { + version: Version, + buf: &'a [u8], + inclusive: bool, +} + +#[derive(Clone, Copy, Default)] +struct Interval<'a> { + lower: Option>, + upper: Option>, +} + +impl<'a> Interval<'a> { + fn raise(&mut self, b: &Bound<'a>) { + let replace = match self.lower { + None => true, + Some(cur) => match b.version.order_without_build(cur.version, b.buf, cur.buf) { + Ordering::Greater => true, + Ordering::Equal => !b.inclusive, + Ordering::Less => false, + }, + }; + if replace { + self.lower = Some(*b); + } + } + + fn cap(&mut self, b: &Bound<'a>) { + let replace = match self.upper { + None => true, + Some(cur) => match b.version.order_without_build(cur.version, b.buf, cur.buf) { + Ordering::Less => true, + Ordering::Equal => !b.inclusive, + Ordering::Greater => false, + }, + }; + if replace { + self.upper = Some(*b); + } + } + + fn narrow(&mut self, c: Comparator, buf: &'a [u8]) { + let bound = |inclusive: bool| Bound { + version: c.version, + buf, + inclusive, + }; + match c.op { + Op::Unset => {} + Op::Eql => { + self.raise(&bound(true)); + self.cap(&bound(true)); + } + Op::Gt => self.raise(&bound(false)), + Op::Gte => self.raise(&bound(true)), + Op::Lt => self.cap(&bound(false)), + Op::Lte => self.cap(&bound(true)), + } + } + + fn and_query(&mut self, query: &Query, buf: &'a [u8]) { + let mut cur = Some(query); + while let Some(q) = cur { + self.narrow(q.range.left, buf); + self.narrow(q.range.right, buf); + cur = q.next.as_deref(); + } + } + + fn is_non_empty(&self) -> bool { + match (self.lower, self.upper) { + (Some(l), Some(u)) => match l.version.order_without_build(u.version, l.buf, u.buf) { + Ordering::Less => true, + Ordering::Equal => l.inclusive && u.inclusive, + Ordering::Greater => false, + }, + _ => true, + } + } +} + +impl Group { + /// Whether some version satisfies both groups; prerelease-exclusion rules are not modelled, comparators are compared directly. + pub fn intersects(&self, self_buf: &[u8], other: &Group, other_buf: &[u8]) -> bool { + let mut a = Some(&self.head); + while let Some(list_a) = a { + a = list_a.next.as_deref(); + let mut base = Interval::default(); + base.and_query(&list_a.head, self_buf); + if !base.is_non_empty() { + continue; + } + let mut b = Some(&other.head); + while let Some(list_b) = b { + b = list_b.next.as_deref(); + let mut i = base; + i.and_query(&list_b.head, other_buf); + if i.is_non_empty() { + return true; + } + } + } + false + } +} diff --git a/src/semver/lib.rs b/src/semver/lib.rs index 43495dae1e6e..aadba242a72f 100644 --- a/src/semver/lib.rs +++ b/src/semver/lib.rs @@ -9,6 +9,7 @@ pub use crate::semver_query::Query; pub use crate::semver_range::Range; pub use crate::sliced_string::SlicedString; +mod intersects; #[path = "SemverQuery.rs"] pub mod semver_query; #[path = "SemverRange.rs"] diff --git a/test/cli/install/bun-add-catalog.test.ts b/test/cli/install/bun-add-catalog.test.ts index 6098c02d4e01..a325d6ac25c7 100644 --- a/test/cli/install/bun-add-catalog.test.ts +++ b/test/cli/install/bun-add-catalog.test.ts @@ -16,6 +16,24 @@ afterAll(() => { const PKG1 = JSON.stringify({ name: "pkg1", version: "1.0.0" }); +// CI's per-file BUN_INSTALL_CACHE_DIR overrides bunfig's cache; concurrent installs sharing it race on Windows. +function envFor(packageDir: string) { + return { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(packageDir, ".bun-cache") }; +} + +async function spawnBun(cwd: string, args: string[], env: Record) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...args], + cwd, + env, + stdout: "pipe", + stderr: "pipe", + stdin: "ignore", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + async function createDir( root: Record | string, pkg1: Record | string = PKG1, @@ -33,9 +51,15 @@ async function createDir( const rootPath = join(packageDir, "package.json"); const pkg1Path = join(packageDir, "packages", "pkg1", "package.json"); const pkg2Path = join(packageDir, "packages", "pkg2", "package.json"); + const env = envFor(packageDir); return { packageDir, + env, + run: (cwd: string, args: string[], spawnEnv: Record = env) => + spawnBun(cwd, args, spawnEnv), + add: (cwd: string, ...args: string[]) => spawnBun(cwd, ["add", ...args], env), + install: (options: Parameters[2] = {}) => runBunInstall(env, packageDir, options), pkg1Dir: join(packageDir, "packages", "pkg1"), pkg2Dir: join(packageDir, "packages", "pkg2"), rootPath, @@ -59,23 +83,6 @@ const withPkg2 = (pkg2: Record | string = PKG2) => ({ "packages/pkg2/package.json": typeof pkg2 === "string" ? pkg2 : JSON.stringify(pkg2), }); -async function run(cwd: string, args: string[], env: Record = bunEnv) { - await using proc = Bun.spawn({ - cmd: [bunExe(), ...args], - cwd, - env, - stdout: "pipe", - stderr: "pipe", - stdin: "ignore", - }); - const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); - return { stdout, stderr, exitCode }; -} - -function runAdd(cwd: string, ...args: string[]) { - return run(cwd, ["add", ...args]); -} - function expectOk({ stderr, exitCode }: { stderr: string; exitCode: number }) { expect(stderr).not.toContain("error:"); expect(stderr).not.toContain("panic:"); @@ -91,7 +98,7 @@ describe.concurrent("bun add --catalog", () => { test("default catalog from a workspace member", async () => { const dir = await createDir(workspacesObject({ catalog: {} })); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); @@ -102,14 +109,14 @@ describe.concurrent("bun add --catalog", () => { expect(lock.catalog).toEqual({ "no-deps": "^2.0.0" }); expect(lock.packages["no-deps"][0]).toBe("no-deps@2.0.0"); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); test("named catalog creates the catalogs object", async () => { const dir = await createDir(workspacesObject({ catalog: { "no-deps": "1.0.0" } })); - expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=testing")); + expectOk(await dir.add(dir.pkg1Dir, "a-dep", "--catalog=testing")); expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:testing" }); expect((await dir.root()).workspaces).toEqual({ @@ -128,7 +135,7 @@ describe.concurrent("bun add --catalog", () => { test("workspaces object gets workspaces.catalog", async () => { const dir = await createDir(workspacesObject()); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect(await dir.root()).toEqual({ name: "root", @@ -140,7 +147,7 @@ describe.concurrent("bun add --catalog", () => { test("workspaces array gets a top-level catalog", async () => { const dir = await createDir({ name: "root", workspaces: ["packages/*"] }); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect(await dir.root()).toEqual({ name: "root", @@ -157,7 +164,7 @@ describe.concurrent("bun add --catalog", () => { test("default catalog", async () => { const dir = await createDir(topLevel); - expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "a-dep", "--catalog")); const root = await dir.root(); expect(root).toEqual({ @@ -172,7 +179,7 @@ describe.concurrent("bun add --catalog", () => { test("named catalog", async () => { const dir = await createDir(topLevel); - expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=x")); + expectOk(await dir.add(dir.pkg1Dir, "a-dep", "--catalog=x")); expect(await dir.root()).toEqual({ name: "root", @@ -193,7 +200,7 @@ describe.concurrent("bun add --catalog", () => { test(`bun add ${args.join(" ")} writes ${JSON.stringify(entry)}`, async () => { const dir = await createDir(workspacesObject({ catalog: {} })); - expectOk(await runAdd(dir.pkg1Dir, ...args)); + expectOk(await dir.add(dir.pkg1Dir, ...args)); expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": entry }); @@ -210,10 +217,10 @@ describe.concurrent("bun add --catalog", () => { dependencies: { "no-deps": "catalog:" }, }); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.1.0"); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); expect((await dir.installed("no-deps")).version).toBe("2.0.0"); @@ -228,7 +235,7 @@ describe.concurrent("bun add --catalog", () => { test("run from the workspace root", async () => { const dir = await createDir(workspacesObject({ catalog: {} })); - expectOk(await runAdd(dir.packageDir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.packageDir, "no-deps", "--catalog")); expect(await dir.root()).toEqual({ name: "root", @@ -242,14 +249,14 @@ describe.concurrent("bun add --catalog", () => { expect(lock.workspaces[""].dependencies).toEqual({ "no-deps": "catalog:" }); expect(lock.catalog).toEqual({ "no-deps": "^2.0.0" }); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); test("bun install --catalog --dev", async () => { const dir = await createDir(workspacesObject({ catalog: {} })); - expectOk(await run(dir.pkg1Dir, ["install", "no-deps", "--catalog", "--dev"])); + expectOk(await dir.run(dir.pkg1Dir, ["install", "no-deps", "--catalog", "--dev"])); const pkg1 = await dir.pkg1(); expect(pkg1.devDependencies).toEqual({ "no-deps": "catalog:" }); @@ -261,7 +268,7 @@ describe.concurrent("bun add --catalog", () => { test("several packages into a named catalog", async () => { const dir = await createDir(workspacesObject()); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "a-dep", "--catalog=libs")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "a-dep", "--catalog=libs")); expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:libs", "no-deps": "catalog:libs" }); const root = await dir.root(); @@ -278,7 +285,7 @@ describe.concurrent("bun add --catalog", () => { const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); const cwd = from === "member" ? dir.pkg1Dir : dir.packageDir; - const { stderr, exitCode } = await runAdd(cwd, "no-deps", "--catalog", "--dry-run"); + const { stderr, exitCode } = await dir.add(cwd, "no-deps", "--catalog", "--dry-run"); expect(stderr).not.toContain("error:"); expect(await dir.rootText()).toBe(rootBefore); @@ -296,7 +303,7 @@ describe.concurrent("bun add --catalog", () => { peerDependencies: { "no-deps": ">=1" }, }); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog", "--dev")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog", "--dev")); expect(await dir.pkg1()).toEqual({ name: "pkg1", @@ -319,8 +326,8 @@ describe.concurrent("bun add --catalog", () => { ]); const [plainResult, catalogResult] = await Promise.all([ - runAdd(plain.pkg1Dir, "no-deps", "--dev"), - runAdd(catalog.pkg1Dir, "no-deps", "--dev", "--catalog"), + plain.add(plain.pkg1Dir, "no-deps", "--dev"), + catalog.add(catalog.pkg1Dir, "no-deps", "--dev", "--catalog"), ]); expectOk(plainResult); expectOk(catalogResult); @@ -335,7 +342,7 @@ describe.concurrent("bun add --catalog", () => { test("--peer", async () => { const dir = await createDir(workspacesObject({ catalog: {} })); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog", "--peer")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog", "--peer")); expect(await dir.pkg1()).toEqual({ name: "pkg1", version: "1.0.0", peerDependencies: { "no-deps": "catalog:" } }); expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); @@ -347,10 +354,10 @@ describe.concurrent("bun add --catalog", () => { name: "pkg1", dependencies: { "no-deps": "catalog:testing" }, }); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.0.0"); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); expect((await dir.root()).workspaces).toEqual({ @@ -365,7 +372,7 @@ describe.concurrent("bun add --catalog", () => { expect(lock.catalog).toEqual({ "no-deps": "^2.0.0" }); expect(lock.catalogs).toEqual({ testing: { "no-deps": "1.0.0" } }); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); @@ -373,28 +380,28 @@ describe.concurrent("bun add --catalog", () => { test("alias@npm:pkg is written verbatim, like plain add", async () => { const dir = await createDir(workspacesObject({ catalog: {} })); - expectOk(await runAdd(dir.pkg1Dir, "foo@npm:no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "foo@npm:no-deps", "--catalog")); expect((await dir.pkg1()).dependencies).toEqual({ foo: "catalog:" }); expect((await dir.root()).workspaces.catalog).toEqual({ foo: "npm:no-deps" }); expect((await dir.lock()).catalog).toEqual({ foo: "npm:no-deps" }); expect(await dir.installed("foo")).toMatchObject({ name: "no-deps", version: "2.0.0" }); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); test("alias@npm:pkg@dist-tag gets the resolved range", async () => { const dir = await createDir(workspacesObject({ catalog: {} })); - expectOk(await runAdd(dir.pkg1Dir, "foo@npm:no-deps@latest", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "foo@npm:no-deps@latest", "--catalog")); expect((await dir.pkg1()).dependencies).toEqual({ foo: "catalog:" }); expect((await dir.root()).workspaces.catalog).toEqual({ foo: "npm:no-deps@^2.0.0" }); expect((await dir.lock()).catalog).toEqual({ foo: "npm:no-deps@^2.0.0" }); expect(await dir.installed("foo")).toMatchObject({ name: "no-deps", version: "2.0.0" }); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); @@ -402,7 +409,7 @@ describe.concurrent("bun add --catalog", () => { const dir = await createDir(workspacesObject({ catalog: {} })); const tarball = `${registry.registryUrl()}no-deps/-/no-deps-1.0.0.tgz`; - expectOk(await runAdd(dir.pkg1Dir, `no-deps@${tarball}`, "dep-with-tags@pre-1", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, `no-deps@${tarball}`, "dep-with-tags@pre-1", "--catalog")); expect((await dir.pkg1()).dependencies).toEqual({ "dep-with-tags": "catalog:", "no-deps": "catalog:" }); const catalog = { "dep-with-tags": "^1.0.1", "no-deps": tarball }; @@ -411,14 +418,14 @@ describe.concurrent("bun add --catalog", () => { expect((await dir.installed("no-deps")).version).toBe("1.0.0"); expect((await dir.installed("dep-with-tags")).version).toBe("1.0.1"); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); test("scoped package into a named catalog", async () => { const dir = await createDir(workspacesObject()); - expectOk(await runAdd(dir.pkg1Dir, "@types/no-deps", "--catalog=types")); + expectOk(await dir.add(dir.pkg1Dir, "@types/no-deps", "--catalog=types")); expect((await dir.pkg1()).dependencies).toEqual({ "@types/no-deps": "catalog:types" }); expect((await dir.root()).workspaces.catalogs).toEqual({ types: { "@types/no-deps": "^2.0.0" } }); @@ -432,7 +439,7 @@ describe.concurrent("bun add --catalog", () => { const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); - const { stderr, exitCode } = await runAdd(dir.pkg1Dir, "pkg2", "--catalog"); + const { stderr, exitCode } = await dir.add(dir.pkg1Dir, "pkg2", "--catalog"); expect(stderr).toContain("error:"); expect(await dir.rootText()).toBe(rootBefore); @@ -447,7 +454,7 @@ describe.concurrent("bun add --catalog", () => { const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); const cwd = from === "member" ? dir.pkg1Dir : dir.packageDir; - const { stderr, exitCode } = await runAdd(cwd, "pkg2@workspace:*", "--catalog"); + const { stderr, exitCode } = await dir.add(cwd, "pkg2@workspace:*", "--catalog"); expect(stderr).toContain('error: --catalog cannot add a workspace package, but got "pkg2@workspace:*"'); expect(await dir.rootText()).toBe(rootBefore); @@ -461,7 +468,7 @@ describe.concurrent("bun add --catalog", () => { const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); - const { stderr, exitCode } = await runAdd(dir.packageDir, "pkg2@workspace:*", "--catalog", "--filter", "pkg1"); + const { stderr, exitCode } = await dir.add(dir.packageDir, "pkg2@workspace:*", "--catalog", "--filter", "pkg1"); expect(stderr).toContain('error: --catalog cannot add a workspace package, but got "pkg2@workspace:*"'); expect(await dir.rootText()).toBe(rootBefore); @@ -475,9 +482,9 @@ describe.concurrent("bun add --catalog", () => { test("edits only the filtered member and the root catalog", async () => { const pkg1 = JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:" } }); const dir = await createDir(workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), pkg1, withPkg2()); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); - expectOk(await runAdd(dir.packageDir, "a-dep", "--catalog", "--filter", "pkg2")); + expectOk(await dir.add(dir.packageDir, "a-dep", "--catalog", "--filter", "pkg2")); expect((await dir.pkg2()).dependencies).toEqual({ "a-dep": "catalog:" }); expect(await dir.pkg1Text()).toBe(pkg1); @@ -488,14 +495,14 @@ describe.concurrent("bun add --catalog", () => { expect((await dir.lock()).catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); expect((await dir.installed("a-dep")).version).toBe("1.0.10"); - expectOk(await runAdd(dir.packageDir, "a-dep", "--catalog", "--filter", "*")); + expectOk(await dir.add(dir.packageDir, "a-dep", "--catalog", "--filter", "*")); expect((await dir.root()).dependencies).toEqual({ "a-dep": "catalog:" }); expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); expect((await dir.pkg2()).dependencies).toEqual({ "a-dep": "catalog:" }); expect((await dir.root()).workspaces.catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); @@ -504,7 +511,7 @@ describe.concurrent("bun add --catalog", () => { const dir = await createDir(workspacesObject({ catalog: {} }), pkg1, withPkg2()); expectOk( - await runAdd(dir.packageDir, "no-deps", "--catalog", "--only-missing", "--filter", "pkg1", "--filter", "pkg2"), + await dir.add(dir.packageDir, "no-deps", "--catalog", "--only-missing", "--filter", "pkg1", "--filter", "pkg2"), ); expect(await dir.pkg1()).toEqual(pkg1); @@ -516,7 +523,7 @@ describe.concurrent("bun add --catalog", () => { const pkg1 = { name: "pkg1", dependencies: { "no-deps": "^1.0.0" } }; const dir = await createDir(workspacesObject({ catalog: {} }), pkg1); - const { stderr, exitCode } = await runAdd( + const { stderr, exitCode } = await dir.add( dir.packageDir, "no-deps", "--catalog", @@ -537,7 +544,7 @@ describe.concurrent("bun add --catalog", () => { const dir = await createDir(workspacesObject({ catalog: {} }), pkg1); const rootBefore = await dir.rootText(); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog", "--only-missing")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog", "--only-missing")); expect(await dir.pkg1()).toEqual(pkg1); expect(await dir.rootText()).toBe(rootBefore); @@ -552,10 +559,10 @@ describe.concurrent("bun add --catalog", () => { member("pkg1"), withPkg2(member("pkg2")), ); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); expect((await dir.installed("no-deps")).version).toBe("2.0.0"); - expectOk(await runAdd(dir.pkg1Dir, "no-deps@1.0.0", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps@1.0.0", "--catalog")); expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "1.0.0" }); expect(await dir.pkg2Text()).toBe(member("pkg2")); @@ -565,7 +572,7 @@ describe.concurrent("bun add --catalog", () => { expect(lockText).not.toContain("no-deps@2.0.0"); expect((await dir.lock()).catalog).toEqual({ "no-deps": "1.0.0" }); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); @@ -575,7 +582,7 @@ describe.concurrent("bun add --catalog", () => { dependencies: { "a-dep": "1.0.1" }, }); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "1.0.1", "no-deps": "catalog:" }); expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); @@ -588,10 +595,10 @@ describe.concurrent("bun add --catalog", () => { name: "pkg1", dependencies: { "no-deps": "^1.0.0" }, }); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.1.0"); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect(await dir.pkg1Text()).toBe( JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:" } }, null, 2), @@ -603,10 +610,10 @@ describe.concurrent("bun add --catalog", () => { test("re-running the same add is idempotent", async () => { const dir = await createDir(workspacesObject({ catalog: {} })); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); const [rootAfter, pkg1After, lockAfter] = await Promise.all([dir.rootText(), dir.pkg1Text(), dir.lockText()]); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect(await dir.rootText()).toBe(rootAfter); expect(await dir.pkg1Text()).toBe(pkg1After); @@ -618,7 +625,7 @@ describe.concurrent("bun add --catalog", () => { workspacesObject({ catalogs: { other: { "a-dep": "1.0.1" }, testing: { "no-deps": "1.0.0" } } }), ); - expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=testing")); + expectOk(await dir.add(dir.pkg1Dir, "a-dep", "--catalog=testing")); const catalogs = { other: { "a-dep": "1.0.1" }, testing: { "a-dep": "^1.0.10", "no-deps": "1.0.0" } }; const root = await dir.root(); @@ -628,7 +635,7 @@ describe.concurrent("bun add --catalog", () => { expect((await dir.lock()).catalogs).toEqual(catalogs); expect((await dir.installed("a-dep")).version).toBe("1.0.10"); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); @@ -636,7 +643,7 @@ describe.concurrent("bun add --catalog", () => { const dir = await createDir(workspacesObject({ catalog: {} })); const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); - const { stderr, exitCode } = await runAdd(dir.pkg1Dir, "no-deps", "this-package-does-not-exist-xyz", "--catalog"); + const { stderr, exitCode } = await dir.add(dir.pkg1Dir, "no-deps", "this-package-does-not-exist-xyz", "--catalog"); expect(stderr).toContain("error:"); expect(await dir.rootText()).toBe(rootBefore); @@ -651,7 +658,7 @@ describe.concurrent("bun add --catalog", () => { const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); const cwd = from === "member" ? dir.pkg1Dir : dir.packageDir; - expectOk(await runAdd(cwd, "no-deps", "--catalog", "--no-save")); + expectOk(await dir.add(cwd, "no-deps", "--catalog", "--no-save")); expect(await dir.rootText()).toBe(rootBefore); expect(await dir.pkg1Text()).toBe(pkg1Before); @@ -669,7 +676,7 @@ describe.concurrent("bun add --catalog", () => { ) + "\n"; const dir = await createDir(rootBefore); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); const rootText = await dir.rootText(); expect(rootText).toBe( @@ -691,10 +698,10 @@ describe.concurrent("bun add --catalog", () => { member("pkg1"), withPkg2(member("pkg2", "catalog:default")), ); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.0.0"); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect((await dir.root()).workspaces).toEqual({ packages: ["packages/*"], @@ -709,7 +716,7 @@ describe.concurrent("bun add --catalog", () => { expect(lock.catalogs).toEqual({ default: { "no-deps": "^2.0.0" } }); expect(lock.packages["no-deps"][0]).toBe("no-deps@2.0.0"); - const { stderr, exitCode } = await run(dir.packageDir, ["install", "--frozen-lockfile"]); + const { stderr, exitCode } = await dir.run(dir.packageDir, ["install", "--frozen-lockfile"]); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); }); @@ -720,10 +727,10 @@ describe.concurrent("bun add --catalog", () => { PKG1, withPkg2(member("pkg2")), ); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.1.0"); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog=default")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog=default")); expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:default" }); expect((await dir.root()).workspaces).toEqual({ @@ -738,14 +745,14 @@ describe.concurrent("bun add --catalog", () => { expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:default" }); expect(await dir.lockText()).not.toContain("no-deps@1.1.0"); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); test("--catalog=default with no catalog defined creates the singular catalog", async () => { const dir = await createDir(workspacesObject()); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog=default")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog=default")); expect(await dir.root()).toEqual({ name: "root", @@ -758,17 +765,17 @@ describe.concurrent("bun add --catalog", () => { test("bun update --latest refreshes a catalogs.default entry referenced as catalog:", async () => { const dir = await createDir(workspacesObject({ catalogs: { default: { "no-deps": "^1.0.0" } } }), member("pkg1")); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.1.0"); - expectOk(await run(dir.packageDir, ["update", "--latest"])); + expectOk(await dir.run(dir.packageDir, ["update", "--latest"])); expect((await dir.root()).workspaces.catalogs).toEqual({ default: { "no-deps": "^2.0.0" } }); expect(await dir.pkg1Text()).toBe(member("pkg1")); expect((await dir.installed("no-deps")).version).toBe("2.0.0"); expect((await dir.lock()).catalogs).toEqual({ default: { "no-deps": "^2.0.0" } }); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); @@ -778,9 +785,9 @@ describe.concurrent("bun add --catalog", () => { version: "1.0.0", peerDependencies: { "no-deps": "catalog:" }, }); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); - const { stderr, exitCode } = await run(dir.pkg1Dir, ["pm", "pack"]); + const { stderr, exitCode } = await dir.run(dir.pkg1Dir, ["pm", "pack"]); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -800,12 +807,12 @@ describe.concurrent("bun add --catalog", () => { version: "1.0.0", peerDependencies: { "no-deps": "catalog:" }, }); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); - expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=x")); + expectOk(await dir.add(dir.pkg1Dir, "a-dep", "--catalog=x")); expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:x" }); - const { stderr, exitCode } = await run(dir.pkg1Dir, ["pm", "pack"]); + const { stderr, exitCode } = await dir.run(dir.pkg1Dir, ["pm", "pack"]); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -827,9 +834,9 @@ describe.concurrent("bun add --catalog", () => { withPkg2(member("pkg2")), ); - expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog=libs")); - expectOk(await run(dir.pkg1Dir, ["remove", "no-deps"])); - expectOk(await run(dir.pkg2Dir, ["remove", "no-deps"])); + expectOk(await dir.add(dir.pkg1Dir, "a-dep", "--catalog=libs")); + expectOk(await dir.run(dir.pkg1Dir, ["remove", "no-deps"])); + expectOk(await dir.run(dir.pkg2Dir, ["remove", "no-deps"])); expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:libs" }); expect((await dir.pkg2()).dependencies).toBeUndefined(); @@ -842,7 +849,7 @@ describe.concurrent("bun add --catalog", () => { expect(lock.catalog).toEqual({ "no-deps": "1.0.0" }); expect(lock.catalogs).toEqual({ libs: { "a-dep": "^1.0.10" } }); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); @@ -851,9 +858,9 @@ describe.concurrent("bun add --catalog", () => { test(`--frozen-lockfile passes after the add and fails once the entry is edited (from ${from})`, async () => { const dir = await createDir(workspacesObject({ catalog: {} })); - expectOk(await runAdd(from === "member" ? dir.pkg1Dir : dir.packageDir, "no-deps", "--catalog")); + expectOk(await dir.add(from === "member" ? dir.pkg1Dir : dir.packageDir, "no-deps", "--catalog")); - const frozen = await run(dir.packageDir, ["install", "--frozen-lockfile"]); + const frozen = await dir.run(dir.packageDir, ["install", "--frozen-lockfile"]); expect(frozen.stderr).not.toContain("error:"); expect(frozen.exitCode).toBe(0); @@ -861,7 +868,7 @@ describe.concurrent("bun add --catalog", () => { root.workspaces.catalog["no-deps"] = "1.0.0"; await write(dir.rootPath, JSON.stringify(root)); - const { stderr, exitCode } = await run(dir.packageDir, ["install", "--frozen-lockfile"]); + const { stderr, exitCode } = await dir.run(dir.packageDir, ["install", "--frozen-lockfile"]); expect(stderr).toContain("error:"); expect(stderr).toContain("frozen-lockfile"); expect(exitCode).toBe(1); @@ -875,14 +882,14 @@ describe.concurrent("bun add --catalog", () => { { ...workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), overrides: { "no-deps": "1.0.0" } }, pkg1, ); - await runBunInstall(bunEnv, dir.packageDir); + await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.0.0"); for (const args of [["update"], ["update", "--recursive"], ["update", "no-deps"], ["update", "no-deps@1.1.0"]]) { - expectOk(await run(dir.pkg1Dir, args)); + expectOk(await dir.run(dir.pkg1Dir, args)); expect(await dir.pkg1()).toEqual(pkg1); } - expectOk(await runAdd(dir.pkg1Dir, "a-dep", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "a-dep", "--catalog")); expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); const lock = await dir.lock(); @@ -890,7 +897,7 @@ describe.concurrent("bun add --catalog", () => { expect(lock.catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); expect((await dir.installed("no-deps")).version).toBe("1.0.0"); - const { stderr, exitCode } = await run(dir.packageDir, ["install", "--frozen-lockfile"]); + const { stderr, exitCode } = await dir.run(dir.packageDir, ["install", "--frozen-lockfile"]); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); }); @@ -899,14 +906,14 @@ describe.concurrent("bun add --catalog", () => { test("an override decides what --catalog records", async () => { const dir = await createDir({ ...workspacesObject({ catalog: {} }), overrides: { "no-deps": "1.0.0" } }); - expectOk(await runAdd(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^1.0.0" }); expect((await dir.lock()).catalog).toEqual({ "no-deps": "^1.0.0" }); expect((await dir.installed("no-deps")).version).toBe("1.0.0"); - const { err } = await runBunInstall(bunEnv, dir.packageDir, { savesLockfile: false }); + const { err } = await dir.install({ savesLockfile: false }); expect(err).not.toContain("Saved lockfile"); }); @@ -918,7 +925,7 @@ describe.concurrent("bun add --catalog", () => { }); const before = await file(join(packageDir, "package.json")).text(); - const { stderr, exitCode } = await runAdd(packageDir, "no-deps", "--catalog"); + const { stderr, exitCode } = await spawnBun(packageDir, ["add", "no-deps", "--catalog"], envFor(packageDir)); expect(stderr).toContain('error: --catalog requires a "workspaces" field in the root package.json'); expect(await file(join(packageDir, "package.json")).text()).toBe(before); @@ -931,7 +938,7 @@ describe.concurrent("bun add --catalog", () => { const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); const tarball = `${registry.registryUrl()}no-deps/-/no-deps-1.0.0.tgz`; - const { stderr, exitCode } = await runAdd(dir.packageDir, "--catalog", tarball); + const { stderr, exitCode } = await dir.add(dir.packageDir, "--catalog", tarball); expect(stderr).toContain(`error: --catalog can only add packages by name, but got "${tarball}"`); expect(await dir.rootText()).toBe(rootBefore); @@ -943,7 +950,7 @@ describe.concurrent("bun add --catalog", () => { const dir = await createDir(workspacesObject({ catalog: {} })); const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); - const { stderr, exitCode } = await run(dir.packageDir, ["install", "--catalog"]); + const { stderr, exitCode } = await dir.run(dir.packageDir, ["install", "--catalog"]); expect(stderr).toContain("error: --catalog requires at least one package to add"); expect(await dir.rootText()).toBe(rootBefore); @@ -957,8 +964,8 @@ describe.concurrent("bun add --catalog", () => { const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); const globalDir = join(dir.packageDir, ".bun-global"); - const { stderr, exitCode } = await run(dir.pkg1Dir, ["add", "no-deps", "--catalog", "-g"], { - ...bunEnv, + const { stderr, exitCode } = await dir.run(dir.pkg1Dir, ["add", "no-deps", "--catalog", "-g"], { + ...dir.env, BUN_INSTALL: globalDir, BUN_INSTALL_GLOBAL_DIR: join(globalDir, "install", "global"), BUN_INSTALL_BIN: join(globalDir, "bin"), diff --git a/test/cli/install/bun-add-filter.test.ts b/test/cli/install/bun-add-filter.test.ts index abc36ec57244..e4cd03482fe5 100644 --- a/test/cli/install/bun-add-filter.test.ts +++ b/test/cli/install/bun-add-filter.test.ts @@ -1,8 +1,9 @@ import { file, write } from "bun"; import { afterAll, beforeAll, expect, test } from "bun:test"; +import { existsSync } from "fs"; import { exists, mkdir } from "fs/promises"; import { VerdaccioRegistry, bunEnv, bunExe } from "harness"; -import { join } from "path"; +import { dirname, join } from "path"; const registry = new VerdaccioRegistry(); @@ -40,12 +41,26 @@ async function makeMonorepo(extra: Partial> = return packageDir; } +// CI exports BUN_INSTALL_CACHE_DIR (one per test file), which overrides the per-test-dir bunfig `cache`; concurrent cases racing on one cache fail on Windows. +function envFor(cwd: string) { + let root = cwd; + while (!existsSync(join(root, "bunfig.toml"))) { + const parent = dirname(root); + if (parent === root) { + root = cwd; + break; + } + root = parent; + } + return { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(root, ".bun-cache") }; +} + // `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. async function run(args: string[], cwd: string, linker?: Linker) { await using proc = Bun.spawn({ cmd: [bunExe(), ...args, ...(linker ? ["--linker", linker] : [])], cwd, - env: bunEnv, + env: envFor(cwd), stdout: "pipe", stderr: "pipe", }); @@ -761,7 +776,7 @@ test.concurrent("--filter with --global is rejected", async () => { await using proc = Bun.spawn({ cmd: [bunExe(), "add", "no-deps", "-g", "--filter", "api"], cwd: dir, - env: { ...bunEnv, BUN_INSTALL: globalDir, BUN_INSTALL_GLOBAL_DIR: join(globalDir, "install", "global") }, + env: { ...envFor(dir), BUN_INSTALL: globalDir, BUN_INSTALL_GLOBAL_DIR: join(globalDir, "install", "global") }, stdout: "pipe", stderr: "pipe", }); diff --git a/test/cli/install/bun-add.test.ts b/test/cli/install/bun-add.test.ts index 2a0f346eb12b..d1c51013feee 100644 --- a/test/cli/install/bun-add.test.ts +++ b/test/cli/install/bun-add.test.ts @@ -1,5 +1,6 @@ +import type { BunLockFile } from "bun"; import { file, spawn } from "bun"; -import { afterAll, afterEach, beforeAll, beforeEach, expect, it, setDefaultTimeout } from "bun:test"; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, setDefaultTimeout, test } from "bun:test"; import { access, appendFile, copyFile, mkdir, readlink, rm, writeFile } from "fs/promises"; import { bunExe, bunEnv as env, readdirSorted, tmpdirSync, toBeValidBin, toBeWorkspaceLink, toHaveBins } from "harness"; import { join, relative, resolve } from "path"; @@ -914,7 +915,7 @@ it("should add dependency with package.json in it and http tarball", async () => "", expect.stringContaining("+ booop@http://"), "", - "installed bap@0.0.5 with binaries:", + "installed bap@npm:baz@0.0.5 with binaries:", " - baz-run", "", "2 packages installed", @@ -1206,7 +1207,7 @@ it("should add aliased dependency (npm)", async () => { expect(out.replace(/\s*\[[0-9\.]+m?s\]\s*$/, "").split(/\r?\n/)).toEqual([ expect.stringContaining("bun add v1."), "", - "installed bar@0.0.3 with binaries:", + "installed bar@npm:baz@0.0.3 with binaries:", " - baz-run", "", "1 package installed", @@ -1241,6 +1242,149 @@ it("should add aliased dependency (npm)", async () => { await access(join(package_dir, "bun.lockb")); }); +describe("npm aliases", () => { + type TestCase = { + args: string[]; + resolved: { name: string; version: string; tarballVersion?: string; binaries?: boolean }; + expected: Omit; + }; + const packageJSON = { name: "foo", version: "0.0.1" }; + const registryVersions = { + "0.0.3": { bin: { "baz-run": "index.js" } }, + "0.0.5-rc.123456789": { as: "0.0.5" }, + latest: "0.0.3", + }; + let urls: string[]; + + beforeEach(async () => { + urls = []; + const registry = dummyRegistry(urls, registryVersions); + setHandler(async request => { + const response = await registry(request); + if (request.url.endsWith(".tgz")) return response; + const manifest = await response.json(); + manifest["dist-tags"].rc = "0.0.5-rc.123456789"; + return Response.json(manifest); + }); + await writeFile(join(package_dir, "package.json"), JSON.stringify(packageJSON)); + }); + + const scoped = { name: "@scope/baz", version: "0.0.3", binaries: true }; + const testCases: TestCase[] = [ + { + args: ["format@npm:baz"], + resolved: { name: "baz", version: "0.0.3", binaries: true }, + expected: { dependencies: { format: "npm:baz@^0.0.3" } }, + }, + { + args: ["bar@npm:@scope/baz"], + resolved: scoped, + expected: { dependencies: { bar: "npm:@scope/baz@^0.0.3" } }, + }, + { + args: ["bar@npm:@scope/baz@latest"], + resolved: scoped, + expected: { dependencies: { bar: "npm:@scope/baz@^0.0.3" } }, + }, + { + args: ["bar@npm:@scope/baz@rc"], + resolved: { name: "@scope/baz", version: "0.0.5-rc.123456789", tarballVersion: "0.0.5" }, + expected: { dependencies: { bar: "npm:@scope/baz@^0.0.5-rc.123456789" } }, + }, + { + args: ["--exact", "bar@npm:@scope/baz"], + resolved: scoped, + expected: { dependencies: { bar: "npm:@scope/baz@0.0.3" } }, + }, + { + args: ["bar@npm:@scope/baz@0.0.3"], + resolved: scoped, + expected: { dependencies: { bar: "npm:@scope/baz@0.0.3" } }, + }, + { + args: ["bar@npm:@scope/baz@^0.0.3"], + resolved: scoped, + expected: { dependencies: { bar: "npm:@scope/baz@^0.0.3" } }, + }, + { + args: ["bar@npm:@scope/baz@~0.0.2"], + resolved: scoped, + expected: { dependencies: { bar: "npm:@scope/baz@~0.0.2" } }, + }, + { + args: ["bar@npm:@scope/baz@>=0.0.2"], + resolved: scoped, + expected: { dependencies: { bar: "npm:@scope/baz@>=0.0.2" } }, + }, + { + args: ["--dev", "bar@npm:@scope/baz"], + resolved: scoped, + expected: { devDependencies: { bar: "npm:@scope/baz@^0.0.3" } }, + }, + { + args: ["--optional", "bar@npm:@scope/baz"], + resolved: scoped, + expected: { optionalDependencies: { bar: "npm:@scope/baz@^0.0.3" } }, + }, + { + args: ["--peer", "bar@npm:@scope/baz"], + resolved: scoped, + expected: { peerDependencies: { bar: "npm:@scope/baz@^0.0.3" } }, + }, + ]; + + test.each(testCases.map(testCase => ({ ...testCase, command: `bun add ${testCase.args.join(" ")}` })))( + "$command", + async ({ args, resolved, expected }) => { + const [section] = Object.values(expected) as Record[]; + const [alias] = Object.keys(section); + + const { stdout, stderr, exited } = spawn({ + cmd: [bunExe(), "add", "--save-text-lockfile", ...args], + cwd: package_dir, + stdout: "pipe", + stdin: "pipe", + stderr: "pipe", + env, + }); + const [out, err, exitCode] = await Promise.all([stdout.text(), stderr.text(), exited]); + expect(err).not.toContain("error:"); + expect(err).toContain("Saved lockfile"); + expect(out).toContain( + `installed ${alias}@npm:${resolved.name}@${resolved.version}${resolved.binaries ? " with binaries:" : ""}`, + ); + expect(exitCode).toBe(0); + expect(urls.sort()).toEqual([ + `${root_url}/${resolved.name.replace("/", "%2f")}`, + `${root_url}/${resolved.name}-${resolved.tarballVersion ?? resolved.version}.tgz`, + ]); + expect(await file(join(package_dir, "package.json")).json()).toEqual({ ...packageJSON, ...expected }); + + const lockfileText = await file(join(package_dir, "bun.lock")).text(); + const lockfile = Bun.JSONC.parse(lockfileText) as BunLockFile; + expect(lockfile.workspaces[""]).toEqual({ name: packageJSON.name, ...expected }); + expect(lockfile.packages[alias][0]).toBe(`${resolved.name}@${resolved.version}`); + + const frozen = spawn({ + cmd: [bunExe(), "install", "--frozen-lockfile"], + cwd: package_dir, + stdout: "pipe", + stdin: "pipe", + stderr: "pipe", + env, + }); + const [, frozenErr, frozenExitCode] = await Promise.all([ + frozen.stdout.text(), + frozen.stderr.text(), + frozen.exited, + ]); + expect(frozenErr).not.toContain("error:"); + expect(frozenExitCode).toBe(0); + expect(await file(join(package_dir, "bun.lock")).text()).toBe(lockfileText); + }, + ); +}); + it("should add aliased dependency (GitHub)", async () => { const urls: string[] = []; setHandler(dummyRegistry(urls)); diff --git a/test/cli/install/bun-audit-fix.test.ts b/test/cli/install/bun-audit-fix.test.ts deleted file mode 100644 index fec7838ac31b..000000000000 --- a/test/cli/install/bun-audit-fix.test.ts +++ /dev/null @@ -1,1020 +0,0 @@ -import { file, write } from "bun"; -import { afterAll, beforeAll, expect, test } from "bun:test"; -import { exists, readlink } from "fs/promises"; -import { - VerdaccioRegistry, - bunEnv, - bunExe, - gunzipJsonRequest, - normalizeBunSnapshot, - runBunInstall, - tempDir, -} from "harness"; -import { join } from "path"; - -const verdaccio = new VerdaccioRegistry(); - -beforeAll(async () => { - await verdaccio.start(); -}); - -afterAll(() => { - verdaccio.stop(); -}); - -type Advisory = { id: number; title: string; severity: string; url: string; vulnerable_versions: string }; - -function adv(range: string, id = 1): Advisory { - return { - id, - title: "test advisory", - severity: "high", - url: "https://example.invalid/advisory/" + id, - vulnerable_versions: range, - }; -} - -type RegistryOptions = { - // Serve the bulk response verbatim instead of filtering advisories by the submitted versions. - bulkResponse?: unknown; - bulkStatus?: number; - // Package names whose manifest requests answer 404; mutable so a test can break the registry after installing. - denyManifests?: Set; -}; - -// Answers the bulk-advisory endpoint itself and proxies everything else to verdaccio. -function startRegistry(advisories: Record, options: RegistryOptions = {}) { - return Bun.serve({ - port: 0, - async fetch(req) { - const url = new URL(req.url); - if (req.method === "POST" && url.pathname === "/-/npm/v1/security/advisories/bulk") { - if (options.bulkStatus) return new Response("registry exploded", { status: options.bulkStatus }); - if (options.bulkResponse !== undefined) return Response.json(options.bulkResponse); - const body: Record = await gunzipJsonRequest(req); - const out: Record = {}; - for (const [name, versions] of Object.entries(body)) { - const matching = (advisories[name] ?? []).filter(a => - versions.some(v => Bun.semver.satisfies(v, a.vulnerable_versions)), - ); - if (matching.length > 0) out[name] = matching; - } - return Response.json(out); - } - - if (options.denyManifests?.has(decodeURIComponent(url.pathname.slice(1)))) { - return new Response("not found", { status: 404 }); - } - - const up = await fetch(new URL(url.pathname + url.search, verdaccio.registryUrl()), { - method: req.method, - headers: { accept: req.headers.get("accept") ?? "*/*" }, - }); - return new Response(up.body, { - status: up.status, - headers: { "content-type": up.headers.get("content-type") ?? "application/octet-stream" }, - }); - }, - }); -} - -type Registry = ReturnType; - -function writeBunfig(dir: string, server: Registry) { - return write( - join(dir, "bunfig.toml"), - Bun.TOML.stringify({ - install: { cache: join(dir, ".bun-cache"), registry: server.url.href, saveTextLockfile: true }, - }), - ); -} - -async function setup(server: Registry, pkgJson: object, extraFiles: Record = {}) { - const dir = tempDir("audit-fix-", { "package.json": JSON.stringify(pkgJson), ...extraFiles }); - await writeBunfig(dir, server); - await runBunInstall(bunEnv, dir); - return dir; -} - -async function reinstall(dir: string, pkgJson: object) { - await write(join(dir, "package.json"), JSON.stringify(pkgJson)); - await runBunInstall(bunEnv, dir); -} - -async function run(dir: string, args: string[]) { - await using proc = Bun.spawn({ - cmd: [bunExe(), ...args], - env: bunEnv, - cwd: dir, - stdout: "pipe", - stderr: "pipe", - }); - const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); - return { stdout, stderr, exitCode }; -} - -function auditFix(dir: string, ...args: string[]) { - return run(dir, ["audit", "fix", ...args]); -} - -function audit(dir: string, ...args: string[]) { - return run(dir, ["audit", ...args]); -} - -function lock(dir: string) { - return file(join(dir, "bun.lock")).text(); -} - -async function installedVersion(dir: string, ...segments: string[]) { - return (await file(join(dir, "node_modules", ...segments, "package.json")).json()).version; -} - -// a-dep@1.0.2 stays installed after the range is widened because the still-satisfied edge is not re-resolved. -async function setupVulnerableADep(server: Registry) { - const dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2" } }); - await reinstall(dir, { name: "foo", dependencies: { "a-dep": "^1.0.2" } }); - expect(await lock(dir)).toContain('"a-dep@1.0.2"'); - return dir; -} - -test.concurrent("fixes a direct dependency to the lowest safe version, not the newest", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); - using dir = await setupVulnerableADep(server); - const pkgJsonBefore = await file(join(dir, "package.json")).text(); - - const { stdout, stderr, exitCode } = await auditFix(dir); - expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); - expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); - expect(stdout).not.toContain("remaining"); - expect(stderr).toContain("Saved lockfile"); - expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); - - const lockfile = await lock(dir); - expect(lockfile).toContain('"a-dep@1.0.4"'); - expect(lockfile).not.toContain('"a-dep@1.0.2"'); - expect(lockfile).not.toContain('"a-dep@1.0.10"'); - expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); - expect(await file(join(dir, "package.json")).text()).toBe(pkgJsonBefore); - - const recheck = await audit(dir); - expect(recheck.stdout).toBe("No vulnerabilities found\n"); - expect(recheck.exitCode).toBe(0); - - await runBunInstall(bunEnv, dir, { frozenLockfile: true }); -}); - -test.concurrent("fixes a transitive dependency and leaves its dependent alone", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup(server, { name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" } }); - await reinstall(dir, { name: "foo", dependencies: { "one-range-dep": "1.0.0" } }); - let lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.0"'); - expect(lockfile).not.toContain('"no-deps@1.1.0"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); - expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); - expect(exitCode).toBe(0); - - lockfile = await lock(dir); - expect(lockfile).toContain('"one-range-dep@1.0.0"'); - expect(lockfile).toContain('"no-deps@1.0.1"'); - expect(lockfile).not.toContain('"no-deps@1.0.0"'); - expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); - - await runBunInstall(bunEnv, dir, { frozenLockfile: true }); -}); - -test.concurrent("reports a fix that would violate a dependent's range and changes nothing", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); - using dir = await setup(server, { name: "foo", dependencies: { "one-dep": "1.0.0" } }); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps@1.0.1"'); - - const { stdout, stderr, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: - no-deps@1.0.1 → 1.1.0 - one-dep@1.0.0 depends on no-deps@1.0.1 - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(stderr).not.toContain("Saved lockfile"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("--dry-run prints the plan and writes nothing", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); - using dir = await setupVulnerableADep(server); - const lockBefore = await lock(dir); - - const { stdout, stderr, exitCode } = await auditFix(dir, "--dry-run"); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "fixing: - a-dep@1.0.2 → 1.0.4 - - Would fix 1 vulnerability in 1 package" - `); - expect(stderr).not.toContain("Saved lockfile"); - expect(exitCode).toBe(0); - expect(await lock(dir)).toBe(lockBefore); - expect(await installedVersion(dir, "a-dep")).toBe("1.0.2"); -}); - -test.concurrent("no fix available", async () => { - await using server = startRegistry({ "no-deps": [adv(">=2.0.0")] }); - using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "^2.0.0" } }); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps@2.0.0"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "no fix available: - no-deps@2.0.0 - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("no vulnerabilities", async () => { - await using server = startRegistry({}); - using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); - const lockBefore = await lock(dir); - - const { stdout, stderr, exitCode } = await auditFix(dir); - expect(stdout).toBe("No vulnerabilities found\n"); - expect(stderr).toContain("bun audit fix v"); - expect(exitCode).toBe(0); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("peer dependency edges constrain the fix and are re-pointed", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup(server, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0", "no-deps": "1.0.0" } }); - await reinstall(dir, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0", "no-deps": "^1.0.0" } }); - let lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.0"'); - expect(lockfile).toContain('"peer-deps-fixed@1.0.0"'); - - const { stdout, stderr, exitCode } = await auditFix(dir); - expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); - expect(stderr).not.toContain("incorrect peer dependency"); - expect(exitCode).toBe(0); - - lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.1"'); - expect(lockfile).not.toContain('"no-deps@1.0.0"'); - - await runBunInstall(bunEnv, dir, { frozenLockfile: true }); -}); - -test.concurrent("instances of the same package are planned independently", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); - using dir = await setup(server, { name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "1.0.0" } }); - await reinstall(dir, { name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "^1.0.0" } }); - let lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.0"'); - expect(lockfile).toContain('"no-deps@1.0.1"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("fixing:"); - expect(stdout).toContain("no-deps@1.0.0 → 1.1.0"); - expect(stdout).toContain("requires a semver-major update:"); - expect(stdout).toContain("no-deps@1.0.1 → 1.1.0"); - expect(stdout).toContain("one-dep@1.0.0 depends on no-deps@1.0.1"); - // pnpm#10646: the advisory still applies to no-deps@1.0.1, so it is remaining, not fixed, and `bun audit` agrees. - expect(stdout).toContain("Fixed 0 vulnerabilities in 1 package"); - expect(stdout).toContain("1 vulnerability remaining"); - expect(exitCode).toBe(1); - - lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.1.0"'); - expect(lockfile).toContain('"no-deps@1.0.1"'); - expect(lockfile).not.toContain('"no-deps@1.0.0"'); - expect(await installedVersion(dir, "no-deps")).toBe("1.1.0"); - - const recheck = await audit(dir); - expect(recheck.stdout).toContain("1 vulnerabilities (1 high)"); - expect(recheck.exitCode).toBe(1); - - await runBunInstall(bunEnv, dir, { frozenLockfile: true }); -}); - -// pnpm#10646: one advisory hitting two installed versions is one vulnerability, as `bun audit` counts it. -test.concurrent("one advisory across two fixable versions counts once", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); - const workspace = (name: string, range: string) => JSON.stringify({ name, dependencies: { "no-deps": range } }); - const root = { name: "root", workspaces: ["packages/*"] }; - using dir = await setup(server, root, { - "packages/a/package.json": workspace("a", "1.0.0"), - "packages/b/package.json": workspace("b", "1.0.1"), - }); - await write(join(dir, "packages", "a", "package.json"), workspace("a", "1.0.0 || >=1.1.0")); - await write(join(dir, "packages", "b", "package.json"), workspace("b", "^1.0.1")); - await runBunInstall(bunEnv, dir); - let lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.0"'); - expect(lockfile).toContain('"no-deps@1.0.1"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("fixing:\n no-deps@1.0.0 → 1.1.0\n no-deps@1.0.1 → 1.1.0\n"); - expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); - expect(stdout).not.toContain("remaining"); - expect(exitCode).toBe(0); - - lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.1.0"'); - expect(lockfile).not.toContain('"no-deps@1.0.0"'); - expect(lockfile).not.toContain('"no-deps@1.0.1"'); - - const recheck = await audit(dir); - expect(recheck.stdout).toBe("No vulnerabilities found\n"); - expect(recheck.exitCode).toBe(0); -}); - -// pnpm#13605: an optional peer that only a devDependency brought in is not a production dependency. -test.concurrent("bun audit --prod skips a dev-only optional peer of a production package", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup(server, { - name: "foo", - dependencies: { "one-optional-peer-dep": "1.0.2" }, - devDependencies: { "no-deps": "1.0.0" }, - }); - expect(await lock(dir)).toContain('"no-deps@1.0.0"'); - - const all = await audit(dir); - expect(all.stdout).toContain("no-deps"); - expect(all.exitCode).toBe(1); - - const prod = await audit(dir, "--prod"); - expect(prod.stdout).toBe("No vulnerabilities found\n"); - expect(prod.exitCode).toBe(0); -}); - -// pnpm#13605: production status is per installed version, not per name. -test.concurrent( - "bun audit --prod skips a dev-only version of a name that is also a production dependency", - async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup(server, { - name: "foo", - dependencies: { "one-dep": "1.0.0" }, - devDependencies: { "no-deps": "1.0.0" }, - }); - const lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.0"'); - expect(lockfile).toContain('"no-deps@1.0.1"'); - - const all = await audit(dir); - expect(all.stdout).toContain("no-deps"); - expect(all.exitCode).toBe(1); - - const prod = await audit(dir, "--prod"); - expect(prod.stdout).toBe("No vulnerabilities found\n"); - expect(prod.exitCode).toBe(0); - }, -); - -test.concurrent( - "bun audit --prod still reports the production version of a name that also has a dev version", - async () => { - await using server = startRegistry({ "no-deps": [adv("1.0.1")] }); - using dir = await setup(server, { - name: "foo", - dependencies: { "one-dep": "1.0.0" }, - devDependencies: { "no-deps": "1.0.0" }, - }); - - const prod = await audit(dir, "--prod"); - expect(prod.stdout).toContain("no-deps"); - expect(prod.stdout).toContain("1 vulnerabilities (1 high)"); - expect(prod.exitCode).toBe(1); - }, -); - -// pnpm#8943: a patch release on the current line wins over the next major that the range would also allow. -test.concurrent("prefers an in-line patch over a major that the range also allows", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); - await reinstall(dir, { name: "foo", dependencies: { "no-deps": ">=1.0.0" } }); - expect(await lock(dir)).toContain('"no-deps@1.0.0"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); - expect(exitCode).toBe(0); - - const lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.1"'); - expect(lockfile).not.toContain('"no-deps@2.0.0"'); -}); - -// pnpm#12651 / #13824: an advisory with no released fix must not invent a version or leave bun.lock unusable. -test.concurrent("an advisory covering the newest release leaves a lockfile that still installs frozen", async () => { - await using server = startRegistry({ "a-dep": [adv("<=1.0.10")] }); - using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "^1.0.0" } }); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"a-dep@1.0.10"'); - - const { stdout, stderr, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "no fix available: - a-dep@1.0.10 - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(stderr).not.toContain("Saved lockfile"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); - - await runBunInstall(bunEnv, dir, { frozenLockfile: true }); -}); - -// pnpm#11101: a workspace package sharing a name with an advised npm package is not audited. -test.concurrent("a workspace package is never matched against an advisory for its name", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup( - server, - { name: "root", workspaces: ["packages/*"] }, - { "packages/no-deps/package.json": JSON.stringify({ name: "no-deps", version: "1.0.0" }) }, - ); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps@workspace:packages/no-deps"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toBe("No vulnerabilities found\n"); - expect(exitCode).toBe(0); - expect(await lock(dir)).toBe(lockBefore); -}); - -// pnpm#10486 / #12487: a package kept alive only by a peer edge is still upgraded. -test.concurrent("fixes a package reachable only through a peer dependency edge", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup(server, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0", "no-deps": "1.0.0" } }); - await reinstall(dir, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0" } }); - let lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.0"'); - expect(lockfile).not.toContain('"no-deps@1.1.0"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); - expect(exitCode).toBe(0); - - lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.1"'); - expect(lockfile).not.toContain('"no-deps@1.0.0"'); - - await runBunInstall(bunEnv, dir, { frozenLockfile: true }); -}); - -test.concurrent("honours catalog ranges", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup( - server, - { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "1.0.0" } }, - { "packages/a/package.json": JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }) }, - ); - await reinstall(dir, { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "^1.0.0" } }); - let lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.0"'); - expect(lockfile).not.toContain('"no-deps@1.0.1"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); - expect(exitCode).toBe(0); - - lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.1"'); - expect(lockfile).not.toContain('"no-deps@1.0.0"'); -}); - -test.concurrent("--ignore and --audit-level filter what gets fixed", async () => { - await using server = startRegistry({ "a-dep": [{ ...adv("<1.0.4", 7), severity: "low" }] }); - using dir = await setupVulnerableADep(server); - const lockBefore = await lock(dir); - - const ignored = await auditFix(dir, "--ignore", "7"); - expect(ignored.stdout).toBe("No vulnerabilities found\n"); - expect(ignored.exitCode).toBe(0); - expect(await lock(dir)).toBe(lockBefore); - - const belowLevel = await auditFix(dir, "--audit-level", "high"); - expect(belowLevel.stdout).toBe("No vulnerabilities found\n"); - expect(belowLevel.exitCode).toBe(0); - expect(await lock(dir)).toBe(lockBefore); - - const fixed = await auditFix(dir); - expect(fixed.stdout).toContain("Fixed 1 vulnerability in 1 package"); - expect(fixed.exitCode).toBe(0); - expect(await lock(dir)).toContain('"a-dep@1.0.4"'); -}); - -test.concurrent("rejects --json and extra arguments", async () => { - await using server = startRegistry({}); - using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); - const lockBefore = await lock(dir); - - const json = await auditFix(dir, "--json"); - expect(json.stderr).toContain("error: --json is not supported by bun audit fix"); - expect(json.exitCode).toBe(1); - - const extra = await auditFix(dir, "extra"); - expect(extra.stderr).toContain("error: bun audit fix does not take arguments"); - expect(extra.exitCode).toBe(1); - - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("refuses to run against a frozen lockfile", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); - using dir = await setupVulnerableADep(server); - const lockBefore = await lock(dir); - - for (const flag of ["--frozen-lockfile", "--production"]) { - const { stderr, exitCode } = await auditFix(dir, flag); - expect(stderr).toContain("error: bun audit fix needs to write bun.lock, but the lockfile is frozen"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); - } - - const dryRun = await auditFix(dir, "--frozen-lockfile", "--dry-run"); - expect(dryRun.stdout).toContain("Would fix 1 vulnerability in 1 package"); - expect(dryRun.exitCode).toBe(0); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("an optional peer edge does not keep the vulnerable version alive", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup(server, { name: "foo", dependencies: { "optional-peer-deps": "1.0.0", "no-deps": "1.0.0" } }); - await reinstall(dir, { name: "foo", dependencies: { "optional-peer-deps": "1.0.0", "no-deps": "^1.0.0" } }); - expect(await lock(dir)).toContain('"no-deps@1.0.0"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); - expect(exitCode).toBe(0); - - const lockfile = await lock(dir); - expect(lockfile).toContain('"no-deps@1.0.1"'); - expect(lockfile).not.toContain('"no-deps@1.0.0"'); - - const recheck = await audit(dir); - expect(recheck.stdout).toBe("No vulnerabilities found\n"); - expect(recheck.exitCode).toBe(0); - - await runBunInstall(bunEnv, dir, { frozenLockfile: true }); -}); - -test.concurrent("an advisory for an installed prerelease is matched and reported", async () => { - await using server = startRegistry({}, { bulkResponse: { "no-deps-backward-tags": [adv("<1.1.0")] } }); - using dir = await setup(server, { name: "foo", dependencies: { "no-deps-backward-tags": "1.0.0-rc.1" } }); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps-backward-tags@1.0.0-rc.1"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: - no-deps-backward-tags@1.0.0-rc.1 → 1.1.0 - foo depends on no-deps-backward-tags@1.0.0-rc.1 - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("advisories that match no installed version are listed, not just counted", async () => { - await using server = startRegistry( - {}, - { bulkResponse: { "no-deps": [adv(">=5.0.0"), adv(">=5.0.0", 2), adv("not a range", 3)] } }, - ); - using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); - const lockBefore = await lock(dir); - - const { stdout, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "not matched to an installed version: - no-deps@>=5.0.0 - no-deps@not a range - - No fixable vulnerabilities - 3 vulnerabilities remaining" - `); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("an unparsable advisory does not hide a real fix for the same package", async () => { - await using server = startRegistry({}, { bulkResponse: { "a-dep": [adv("<1.0.4"), adv("not a range", 2)] } }); - using dir = await setupVulnerableADep(server); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); - expect(stdout).toContain("a-dep@not a range"); - expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); - expect(stdout).toContain("1 vulnerability remaining"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toContain('"a-dep@1.0.4"'); -}); - -test.concurrent("a bundled dependency is never claimed as fixed", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup(server, { name: "foo", dependencies: { "bundled-1": "1.0.0" } }); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps@1.0.0"'); - - const { stdout, stderr, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: - no-deps@1.0.0 → 1.0.1 - bundled-1@1.0.0 bundles no-deps@1.0.0 - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(stderr).not.toContain("Saved lockfile"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); - expect(await installedVersion(dir, "bundled-1", "node_modules", "no-deps")).toBe("1.0.0"); -}); - -test.concurrent("multiple advisories on one instance are cleared together", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4", 1), adv("<1.0.6", 2)] }); - using dir = await setupVulnerableADep(server); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("a-dep@1.0.2 → 1.0.6"); - expect(stdout).toContain("Fixed 2 vulnerabilities in 1 package"); - expect(stdout).not.toContain("remaining"); - expect(exitCode).toBe(0); - - const lockfile = await lock(dir); - expect(lockfile).toContain('"a-dep@1.0.6"'); - expect(lockfile).not.toContain('"a-dep@1.0.4"'); - - const recheck = await audit(dir); - expect(recheck.stdout).toBe("No vulnerabilities found\n"); - expect(recheck.exitCode).toBe(0); -}); - -// pnpm fixtures/update-multiple: two advisories for one name with disjoint ranges. -test.concurrent("disjoint advisory ranges for one package are all avoided", async () => { - await using server = startRegistry({ "no-deps": [adv(">=1.0.0 <1.0.1", 1), adv(">=1.1.0 <2.0.0", 2)] }); - using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); - await reinstall(dir, { name: "foo", dependencies: { "no-deps": ">=1.0.0" } }); - expect(await lock(dir)).toContain('"no-deps@1.0.0"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); - expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); - expect(exitCode).toBe(0); - expect(await lock(dir)).toContain('"no-deps@1.0.1"'); -}); - -// pnpm fixtures/update-single-depth-2/responses/unfixable-vulnerability.json: npm-style advisory objects carry -// fields (findings, patched_versions, ...) that must be ignored, and `>=0.0.0` is unfixable. -test.concurrent("ignores unknown advisory fields and treats >=0.0.0 as unfixable", async () => { - await using server = startRegistry( - {}, - { - bulkResponse: { - "no-deps": [ - { - ...adv(">=0.0.0", 1234), - findings: [{ version: "1.0.0", paths: ["no-deps"] }], - patched_versions: "<0.0.0", - recommendation: "None", - cwe: ["CWE-1"], - cvss: { score: 0, vectorString: null }, - }, - ], - }, - }, - ); - using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "^1.0.0" } }); - const lockBefore = await lock(dir); - - const { stdout, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "no fix available: - no-deps@1.1.0 - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -// Ported verbatim from pnpm's test/audit/utils/responses/all-vulnerabilities-response.json (51 packages, 111 advisories). -test.concurrent("parses a real bulk response and only plans installed packages", async () => { - const bulkResponse = await file( - join(import.meta.dir, "registry/fixtures/audit/pnpm-all-vulnerabilities-response.json"), - ).json(); - await using server = startRegistry({}, { bulkResponse }); - using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); - const lockBefore = await lock(dir); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).not.toContain("fixing:"); - expect(stdout).not.toContain("no-deps"); - expect(stdout).toContain("not matched to an installed version:"); - expect(stdout).toContain(" axios@<0.21.2\n"); - expect(stdout).toContain(" semver@>=2.0.0-alpha <5.7.2\n"); - expect(stdout).toContain("111 vulnerabilities remaining"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("fixes two packages in one run and leaves the rest of the lockfile alone", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4")], "no-deps": [adv("<1.0.1", 2)] }); - using dir = await setup(server, { - name: "foo", - dependencies: { "a-dep": "1.0.2", "one-range-dep": "1.0.0", "no-deps": "1.0.0", "@types/is-number": "1.0.0" }, - }); - await reinstall(dir, { - name: "foo", - dependencies: { "a-dep": "^1.0.2", "one-range-dep": "1.0.0", "@types/is-number": "1.0.0" }, - }); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"a-dep@1.0.2"'); - expect(lockBefore).toContain('"no-deps@1.0.0"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("fixing:\n a-dep@1.0.2 → 1.0.4\n no-deps@1.0.0 → 1.0.1\n"); - expect(stdout).toContain("Fixed 2 vulnerabilities in 2 packages"); - expect(exitCode).toBe(0); - - const lockAfter = await lock(dir); - const packageRows = lockBefore.split("\n").filter(line => /^ "[^"]+": \["/.test(line)); - const untouched = packageRows.filter(line => !line.includes('"a-dep@') && !line.includes('"no-deps@')); - expect(untouched.map(line => line.split('"')[1]).sort()).toEqual(["@types/is-number", "one-range-dep"]); - for (const line of untouched) expect(lockAfter).toContain(line); - expect(lockAfter).toContain('"a-dep@1.0.4"'); - expect(lockAfter).toContain('"no-deps@1.0.1"'); -}); - -test.concurrent("fixes a scoped package", async () => { - await using server = startRegistry({ "@types/is-number": [adv("<2.0.0")] }); - using dir = await setup(server, { name: "foo", dependencies: { "@types/is-number": "1.0.0" } }); - await reinstall(dir, { name: "foo", dependencies: { "@types/is-number": ">=1.0.0" } }); - expect(await lock(dir)).toContain('"@types/is-number@1.0.0"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("@types/is-number@1.0.0 → 2.0.0"); - expect(exitCode).toBe(0); - - const lockfile = await lock(dir); - expect(lockfile).toContain('"@types/is-number@2.0.0"'); - expect(lockfile).not.toContain('"@types/is-number@1.0.0"'); - expect(await installedVersion(dir, "@types", "is-number")).toBe("2.0.0"); -}); - -test.concurrent("fixes an npm: alias pointing at a vulnerable package", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); - using dir = await setup(server, { name: "foo", dependencies: { nd: "npm:a-dep@1.0.2" } }); - await reinstall(dir, { name: "foo", dependencies: { nd: "npm:a-dep@^1.0.2" } }); - expect(await lock(dir)).toContain('"a-dep@1.0.2"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); - expect(exitCode).toBe(0); - - const lockfile = await lock(dir); - expect(lockfile).toContain('"a-dep@1.0.4"'); - expect(lockfile).not.toContain('"a-dep@1.0.2"'); - expect(await installedVersion(dir, "nd")).toBe("1.0.4"); - - await runBunInstall(bunEnv, dir, { frozenLockfile: true }); -}); - -test.concurrent("an overrides pin is reported as the blocker", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); - using dir = await setup(server, { - name: "foo", - dependencies: { "a-dep": "^1.0.2" }, - overrides: { "a-dep": "1.0.2" }, - }); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"a-dep@1.0.2"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: - a-dep@1.0.2 → 1.0.4 - foo depends on a-dep@1.0.2 - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -// pnpm fixtures/update-workspace-catalog-pinned: a pinned catalog entry blocks the fix. -test.concurrent("a pinned catalog entry is reported as the blocker", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); - using dir = await setup( - server, - { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "1.0.0" } }, - { "packages/a/package.json": JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }) }, - ); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps@1.0.0"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: - no-deps@1.0.0 → 1.0.1 - a depends on no-deps@1.0.0 - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -// mismatched-peer-deps-lvl1 declares peer no-deps@<=1.0.1; its own dependency lvl2 declares peer no-deps@1.0.0, -// so the installs warn about an incorrect peer and cannot go through runBunInstall. -test.concurrent("a peer range is a blocker and is labelled with the peer dependent", async () => { - await using server = startRegistry({ "no-deps": [adv("<=1.0.1")] }); - const pkgJson = (noDeps: string) => - JSON.stringify({ name: "foo", dependencies: { "mismatched-peer-deps-lvl1": "1.0.0", "no-deps": noDeps } }); - using dir = tempDir("audit-fix-", { "package.json": pkgJson("1.0.1") }); - await writeBunfig(dir, server); - expect((await run(dir, ["install"])).exitCode).toBe(0); - await write(join(dir, "package.json"), pkgJson("^1.0.0")); - expect((await run(dir, ["install"])).exitCode).toBe(0); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps@1.0.1"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("no-deps@1.0.1 → 1.1.0"); - expect(stdout).toContain("mismatched-peer-deps-lvl1@1.0.0 depends on no-deps@<=1.0.1"); - expect(stdout).not.toContain("foo depends on"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("a depth-3 blocker names the immediate dependent", async () => { - await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); - using dir = await setup(server, { name: "foo", dependencies: { "one-one-dep": "1.0.0" } }); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps@1.0.1"'); - - const { stdout, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: - no-deps@1.0.1 → 1.1.0 - one-dep@1.0.0 depends on no-deps@1.0.1 - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -// Workspaces default to the isolated linker, so the linker is pinned to keep node_modules paths predictable. -test.concurrent("fixes a workspace member's dependency when run from the member directory", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); - const member = (range: string) => JSON.stringify({ name: "a", dependencies: { "a-dep": range } }); - const rootPkgJson = JSON.stringify({ name: "root", workspaces: ["packages/*"] }); - using dir = tempDir("audit-fix-", { "package.json": rootPkgJson, "packages/a/package.json": member("1.0.2") }); - await writeBunfig(dir, server); - expect((await run(dir, ["install", "--linker", "hoisted"])).exitCode).toBe(0); - await write(join(dir, "packages", "a", "package.json"), member("^1.0.2")); - expect((await run(dir, ["install", "--linker", "hoisted"])).exitCode).toBe(0); - expect(await lock(dir)).toContain('"a-dep@1.0.2"'); - - const { stdout, exitCode } = await run(join(dir, "packages", "a"), ["audit", "fix", "--linker", "hoisted"]); - expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); - expect(exitCode).toBe(0); - - const lockfile = await lock(dir); - expect(lockfile).toContain('"a-dep@1.0.4"'); - expect(lockfile).not.toContain('"a-dep@1.0.2"'); - expect(await exists(join(dir, "packages", "a", "bun.lock"))).toBeFalse(); - expect(await file(join(dir, "package.json")).text()).toBe(rootPkgJson); - expect(await file(join(dir, "packages", "a", "package.json")).text()).toBe(member("^1.0.2")); - expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); - - const frozen = await run(dir, ["install", "--frozen-lockfile", "--linker", "hoisted"]); - expect(frozen.stderr).not.toContain("error:"); - expect(frozen.exitCode).toBe(0); -}); - -// `--linker isolated` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. -test.concurrent("isolated linker layout", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); - using dir = tempDir("audit-fix-", { - "package.json": JSON.stringify({ name: "foo", dependencies: { "a-dep": "1.0.2" } }), - }); - await writeBunfig(dir, server); - expect((await run(dir, ["install", "--linker", "isolated"])).exitCode).toBe(0); - await write(join(dir, "package.json"), JSON.stringify({ name: "foo", dependencies: { "a-dep": "^1.0.2" } })); - expect((await run(dir, ["install", "--linker", "isolated"])).exitCode).toBe(0); - expect(await lock(dir)).toContain('"a-dep@1.0.2"'); - expect(await readlink(join(dir, "node_modules", "a-dep"))).toContain("a-dep@1.0.2"); - - const { stdout, exitCode } = await auditFix(dir, "--linker", "isolated"); - expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); - expect(exitCode).toBe(0); - - expect(await lock(dir)).toContain('"a-dep@1.0.4"'); - expect(await readlink(join(dir, "node_modules", "a-dep"))).toContain("a-dep@1.0.4"); - expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); - - const frozen = await run(dir, ["install", "--frozen-lockfile", "--linker", "isolated"]); - expect(frozen.stderr).not.toContain("error:"); - expect(frozen.exitCode).toBe(0); -}); - -// Every a-dep release was published in 2023, so a 100-year minimum age gates all of them. -test.concurrent("a fix gated by --minimum-release-age is reported distinctly", async () => { - await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); - using dir = await setupVulnerableADep(server); - const lockBefore = await lock(dir); - - const gated = await auditFix(dir, "--minimum-release-age", "3153600000"); - expect(normalizeBunSnapshot(gated.stdout)).toMatchInlineSnapshot(` - "no fix available: - a-dep@1.0.2 (1.0.4 is newer than --minimum-release-age) - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(gated.exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); - - const fixed = await auditFix(dir, "--minimum-release-age", "60"); - expect(fixed.stdout).toContain("a-dep@1.0.2 → 1.0.4"); - expect(fixed.exitCode).toBe(0); - expect(await lock(dir)).toContain('"a-dep@1.0.4"'); -}); - -test.concurrent("a failing bulk endpoint changes nothing", async () => { - await using server = startRegistry({}, { bulkStatus: 500 }); - using dir = await setupVulnerableADep(server); - const lockBefore = await lock(dir); - - const { stdout, stderr, exitCode } = await auditFix(dir); - expect(stdout).not.toContain("fixing:"); - expect(stderr).toContain("audit request failed"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("a manifest that fails to download is reported, not fixed", async () => { - const denyManifests = new Set(); - await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }, { denyManifests }); - using dir = await setupVulnerableADep(server); - const lockBefore = await lock(dir); - denyManifests.add("a-dep"); - - const { stdout, stderr, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "no fix available: - a-dep@1.0.2 (failed to fetch the manifest) - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(stderr).toContain("a-dep"); - expect(stderr).not.toContain("Saved lockfile"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); - -test.concurrent("refuses to run without a lockfile", async () => { - await using server = startRegistry({}); - using dir = tempDir("audit-fix-", { - "package.json": JSON.stringify({ name: "foo" }), - "bunfig.toml": Bun.TOML.stringify({ install: { registry: server.url.href } }), - }); - - const { stderr, exitCode } = await auditFix(dir); - expect(stderr).toContain("Lockfile not found"); - expect(exitCode).toBe(1); - expect(await exists(join(dir, "bun.lock"))).toBeFalse(); -}); - -test.concurrent("refuses --no-save before contacting the registry", async () => { - await using server = startRegistry({}, { bulkStatus: 500 }); - using dir = await setupVulnerableADep(server); - const lockBefore = await lock(dir); - - const { stdout, stderr, exitCode } = await auditFix(dir, "--no-save"); - expect(stderr).toContain("error: bun audit fix needs to write bun.lock, but saving the lockfile is disabled"); - expect(stderr).not.toContain("audit request failed"); - expect(stdout).not.toContain("Fixed"); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); -}); diff --git a/test/cli/install/bun-audit.test.ts b/test/cli/install/bun-audit.test.ts index e9a49a72911e..5351b97d197f 100644 --- a/test/cli/install/bun-audit.test.ts +++ b/test/cli/install/bun-audit.test.ts @@ -1,6 +1,17 @@ -import { spawn } from "bun"; +import { file, spawn, write } from "bun"; import { afterAll, beforeAll, describe, expect, test } from "bun:test"; -import { bunEnv, bunExe, DirectoryTree, gunzipJsonRequest, lazyPromiseLike, tempDir } from "harness"; +import { exists, readlink } from "fs/promises"; +import { + DirectoryTree, + VerdaccioRegistry, + bunEnv, + bunExe, + gunzipJsonRequest, + lazyPromiseLike, + normalizeBunSnapshot, + runBunInstall, + tempDir, +} from "harness"; import { join } from "node:path"; import { resolveBulkAdvisoryFixture } from "./registry/fixtures/audit/audit-fixtures"; @@ -15,8 +26,9 @@ function fixture( } let server: Bun.Server; +const verdaccio = new VerdaccioRegistry(); -beforeAll(() => { +beforeAll(async () => { server = Bun.serve({ port: 0, fetch: async req => { @@ -32,10 +44,12 @@ beforeAll(() => { return Response.json(fixture); }, }); + await verdaccio.start(); }); afterAll(() => { server?.stop(); + verdaccio.stop(); }); function doAuditTest( @@ -88,6 +102,126 @@ function doAuditTest( }); } +type Advisory = { id: number; title: string; severity: string; url: string; vulnerable_versions: string }; + +function adv(range: string, id = 1): Advisory { + return { + id, + title: "test advisory", + severity: "high", + url: "https://example.invalid/advisory/" + id, + vulnerable_versions: range, + }; +} + +type RegistryOptions = { + // Serve the bulk response verbatim instead of filtering advisories by the submitted versions. + bulkResponse?: unknown; + bulkStatus?: number; + // Package names whose manifest requests answer 404; mutable so a test can break the registry after installing. + denyManifests?: Set; +}; + +// Answers the bulk-advisory endpoint itself and proxies everything else to verdaccio. +function startRegistry(advisories: Record, options: RegistryOptions = {}) { + return Bun.serve({ + port: 0, + async fetch(req) { + const url = new URL(req.url); + if (req.method === "POST" && url.pathname === "/-/npm/v1/security/advisories/bulk") { + if (options.bulkStatus) return new Response("registry exploded", { status: options.bulkStatus }); + if (options.bulkResponse !== undefined) return Response.json(options.bulkResponse); + const body: Record = await gunzipJsonRequest(req); + const out: Record = {}; + for (const [name, versions] of Object.entries(body)) { + const matching = (advisories[name] ?? []).filter(a => + versions.some(v => Bun.semver.satisfies(v, a.vulnerable_versions)), + ); + if (matching.length > 0) out[name] = matching; + } + return Response.json(out); + } + + if (options.denyManifests?.has(decodeURIComponent(url.pathname.slice(1)))) { + return new Response("not found", { status: 404 }); + } + + const up = await fetch(new URL(url.pathname + url.search, verdaccio.registryUrl()), { + method: req.method, + headers: { accept: req.headers.get("accept") ?? "*/*" }, + }); + return new Response(up.body, { + status: up.status, + headers: { "content-type": up.headers.get("content-type") ?? "application/octet-stream" }, + }); + }, + }); +} + +type Registry = ReturnType; + +function writeBunfig(dir: string, server: Registry) { + return write( + join(dir, "bunfig.toml"), + Bun.TOML.stringify({ + install: { cache: join(dir, ".bun-cache"), registry: server.url.href, saveTextLockfile: true }, + }), + ); +} + +// The CI runner exports one BUN_INSTALL_CACHE_DIR per file, which overrides the bunfig cache the concurrent cases rely on. +function installEnv(dir: string) { + return { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }; +} + +async function setup(server: Registry, pkgJson: object, extraFiles: Record = {}) { + const dir = tempDir("audit-fix-", { "package.json": JSON.stringify(pkgJson), ...extraFiles }); + await writeBunfig(dir, server); + await runBunInstall(installEnv(dir), dir); + return dir; +} + +async function reinstall(dir: string, pkgJson: object) { + await write(join(dir, "package.json"), JSON.stringify(pkgJson)); + await runBunInstall(installEnv(dir), dir); +} + +async function run(dir: string, args: string[], root: string = dir) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...args], + env: installEnv(root), + cwd: dir, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +function auditFix(dir: string, ...args: string[]) { + return run(dir, ["audit", "fix", ...args]); +} + +function audit(dir: string, ...args: string[]) { + return run(dir, ["audit", ...args]); +} + +function lock(dir: string) { + return file(join(dir, "bun.lock")).text(); +} + +async function installedVersion(dir: string, ...segments: string[]) { + return (await file(join(dir, "node_modules", ...segments, "package.json")).json()).version; +} + +// a-dep@1.0.2 stays installed after the range is widened because the still-satisfied edge is not re-resolved. +async function setupVulnerableADep(server: Registry) { + const dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2" } }); + await reinstall(dir, { name: "foo", dependencies: { "a-dep": "^1.0.2" } }); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + return dir; +} + describe("`bun audit`", () => { doAuditTest("should fail with no package.json", { exitCode: 1, @@ -399,3 +533,926 @@ describe("`bun audit`", () => { expect(exitCode).toBe(0); }); }); + +describe("`bun audit --prod`", () => { + // pnpm#13605: an optional peer that only a devDependency brought in is not a production dependency. + test.concurrent("bun audit --prod skips a dev-only optional peer of a production package", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "one-optional-peer-dep": "1.0.2" }, + devDependencies: { "no-deps": "1.0.0" }, + }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const all = await audit(dir); + expect(all.stdout).toContain("no-deps"); + expect(all.exitCode).toBe(1); + + const prod = await audit(dir, "--prod"); + expect(prod.stdout).toBe("No vulnerabilities found\n"); + expect(prod.exitCode).toBe(0); + }); + + // pnpm#13605: production status is per installed version, not per name. + test.concurrent( + "bun audit --prod skips a dev-only version of a name that is also a production dependency", + async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "one-dep": "1.0.0" }, + devDependencies: { "no-deps": "1.0.0" }, + }); + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + + const all = await audit(dir); + expect(all.stdout).toContain("no-deps"); + expect(all.exitCode).toBe(1); + + const prod = await audit(dir, "--prod"); + expect(prod.stdout).toBe("No vulnerabilities found\n"); + expect(prod.exitCode).toBe(0); + }, + ); + + test.concurrent( + "bun audit --prod still reports the production version of a name that also has a dev version", + async () => { + await using server = startRegistry({ "no-deps": [adv("1.0.1")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "one-dep": "1.0.0" }, + devDependencies: { "no-deps": "1.0.0" }, + }); + + const prod = await audit(dir, "--prod"); + expect(prod.stdout).toContain("no-deps"); + expect(prod.stdout).toContain("1 vulnerabilities (1 high)"); + expect(prod.exitCode).toBe(1); + }, + ); +}); + +describe("`bun audit fix`", () => { + test.concurrent("fixes a direct dependency to the lowest safe version, not the newest", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setupVulnerableADep(server); + const pkgJsonBefore = await file(join(dir, "package.json")).text(); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(stdout).not.toContain("remaining"); + expect(stderr).toContain("Saved lockfile"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).not.toContain('"a-dep@1.0.2"'); + expect(lockfile).not.toContain('"a-dep@1.0.10"'); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + expect(await file(join(dir, "package.json")).text()).toBe(pkgJsonBefore); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("fixes a transitive dependency and leaves its dependent alone", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "one-range-dep": "1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.1.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"one-range-dep@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("reports a fix that would violate a dependent's range and changes nothing", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-dep": "1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.1"'); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps@1.0.1 → 1.1.0 + one-dep@1.0.0 depends on no-deps@1.0.1 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("--dry-run prints the plan and writes nothing", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const { stdout, stderr, exitCode } = await auditFix(dir, "--dry-run"); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "fixing: + a-dep@1.0.2 → 1.0.4 + + Would fix 1 vulnerability in 1 package" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.2"); + }); + + test.concurrent("no fix available", async () => { + await using server = startRegistry({ "no-deps": [adv(">=2.0.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "^2.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@2.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "no fix available: + no-deps@2.0.0 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("no vulnerabilities", async () => { + await using server = startRegistry({}); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const lockBefore = await lock(dir); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).toBe("No vulnerabilities found\n"); + expect(stderr).toContain("bun audit fix v"); + expect(exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("peer dependency edges constrain the fix and are re-pointed", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0", "no-deps": "^1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"peer-deps-fixed@1.0.0"'); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stderr).not.toContain("incorrect peer dependency"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("instances of the same package are planned independently", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "^1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("fixing:"); + expect(stdout).toContain("no-deps@1.0.0 → 1.1.0"); + expect(stdout).toContain("requires a semver-major update:"); + expect(stdout).toContain("no-deps@1.0.1 → 1.1.0"); + expect(stdout).toContain("one-dep@1.0.0 depends on no-deps@1.0.1"); + // pnpm#10646: the advisory still applies to no-deps@1.0.1, so it is remaining, not fixed, and `bun audit` agrees. + expect(stdout).toContain("Fixed 0 vulnerabilities in 1 package"); + expect(stdout).toContain("1 vulnerability remaining"); + expect(exitCode).toBe(1); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.1.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(await installedVersion(dir, "no-deps")).toBe("1.1.0"); + + const recheck = await audit(dir); + expect(recheck.stdout).toContain("1 vulnerabilities (1 high)"); + expect(recheck.exitCode).toBe(1); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + // pnpm#10646: one advisory hitting two installed versions is one vulnerability, as `bun audit` counts it. + test.concurrent("one advisory across two fixable versions counts once", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); + const workspace = (name: string, range: string) => JSON.stringify({ name, dependencies: { "no-deps": range } }); + const root = { name: "root", workspaces: ["packages/*"] }; + using dir = await setup(server, root, { + "packages/a/package.json": workspace("a", "1.0.0"), + "packages/b/package.json": workspace("b", "1.0.1"), + }); + await write(join(dir, "packages", "a", "package.json"), workspace("a", "1.0.0 || >=1.1.0")); + await write(join(dir, "packages", "b", "package.json"), workspace("b", "^1.0.1")); + await runBunInstall(installEnv(dir), dir); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("fixing:\n no-deps@1.0.0 → 1.1.0\n no-deps@1.0.1 → 1.1.0\n"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(stdout).not.toContain("remaining"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.1.0"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.0.1"'); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + }); + + // pnpm#8943: a patch release on the current line wins over the next major that the range would also allow. + test.concurrent("prefers an in-line patch over a major that the range also allows", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "no-deps": ">=1.0.0" } }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@2.0.0"'); + }); + + // pnpm#12651 / #13824: an advisory with no released fix must not invent a version or leave bun.lock unusable. + test.concurrent("an advisory covering the newest release leaves a lockfile that still installs frozen", async () => { + await using server = startRegistry({ "a-dep": [adv("<=1.0.10")] }); + using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "^1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"a-dep@1.0.10"'); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "no fix available: + a-dep@1.0.10 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + // pnpm#11101: a workspace package sharing a name with an advised npm package is not audited. + test.concurrent("a workspace package is never matched against an advisory for its name", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup( + server, + { name: "root", workspaces: ["packages/*"] }, + { "packages/no-deps/package.json": JSON.stringify({ name: "no-deps", version: "1.0.0" }) }, + ); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@workspace:packages/no-deps"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toBe("No vulnerabilities found\n"); + expect(exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + }); + + // pnpm#10486 / #12487: a package kept alive only by a peer edge is still upgraded. + test.concurrent("fixes a package reachable only through a peer dependency edge", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "peer-deps-fixed": "1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.1.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("honours catalog ranges", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup( + server, + { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "1.0.0" } }, + { "packages/a/package.json": JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }) }, + ); + await reinstall(dir, { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "^1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + }); + + test.concurrent("--ignore and --audit-level filter what gets fixed", async () => { + await using server = startRegistry({ "a-dep": [{ ...adv("<1.0.4", 7), severity: "low" }] }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const ignored = await auditFix(dir, "--ignore", "7"); + expect(ignored.stdout).toBe("No vulnerabilities found\n"); + expect(ignored.exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + + const belowLevel = await auditFix(dir, "--audit-level", "high"); + expect(belowLevel.stdout).toBe("No vulnerabilities found\n"); + expect(belowLevel.exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + + const fixed = await auditFix(dir); + expect(fixed.stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(fixed.exitCode).toBe(0); + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); + }); + + test.concurrent("rejects --json and extra arguments", async () => { + await using server = startRegistry({}); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const lockBefore = await lock(dir); + + const json = await auditFix(dir, "--json"); + expect(json.stderr).toContain("error: --json is not supported by bun audit fix"); + expect(json.exitCode).toBe(1); + + const extra = await auditFix(dir, "extra"); + expect(extra.stderr).toContain("error: bun audit fix does not take arguments"); + expect(extra.exitCode).toBe(1); + + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("refuses to run against a frozen lockfile", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + for (const flag of ["--frozen-lockfile", "--production"]) { + const { stderr, exitCode } = await auditFix(dir, flag); + expect(stderr).toContain("error: bun audit fix needs to write bun.lock, but the lockfile is frozen"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + } + + const dryRun = await auditFix(dir, "--frozen-lockfile", "--dry-run"); + expect(dryRun.stdout).toContain("Would fix 1 vulnerability in 1 package"); + expect(dryRun.exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("an optional peer edge does not keep the vulnerable version alive", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "optional-peer-deps": "1.0.0", "no-deps": "1.0.0" }, + }); + await reinstall(dir, { name: "foo", dependencies: { "optional-peer-deps": "1.0.0", "no-deps": "^1.0.0" } }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + // The peer holder hoists before the dependent, so a slot still bound to the old version takes the root folder. + test.concurrent( + "an optional peer edge hoisted before the dependent does not keep the vulnerable version", + async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "one-optional-peer-dep": "1.0.2", "one-range-dep": "1.0.0", "no-deps": "1.0.0" }, + }); + await reinstall(dir, { + name: "foo", + dependencies: { "one-optional-peer-dep": "1.0.2", "one-range-dep": "1.0.0" }, + }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps": ["no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"one-range-dep/no-deps"'); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }, + ); + + test.concurrent("an advisory for an installed prerelease is matched and reported", async () => { + await using server = startRegistry({}, { bulkResponse: { "no-deps-backward-tags": [adv("<1.1.0")] } }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps-backward-tags": "1.0.0-rc.1" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps-backward-tags@1.0.0-rc.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps-backward-tags@1.0.0-rc.1 → 1.1.0 + foo depends on no-deps-backward-tags@1.0.0-rc.1 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("advisories that match no installed version are listed, not just counted", async () => { + await using server = startRegistry( + {}, + { bulkResponse: { "no-deps": [adv(">=5.0.0"), adv(">=5.0.0", 2), adv("not a range", 3)] } }, + ); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "not matched to an installed version: + no-deps@>=5.0.0 + no-deps@not a range + + No fixable vulnerabilities + 3 vulnerabilities remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("an unparsable advisory does not hide a real fix for the same package", async () => { + await using server = startRegistry({}, { bulkResponse: { "a-dep": [adv("<1.0.4"), adv("not a range", 2)] } }); + using dir = await setupVulnerableADep(server); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(stdout).toContain("a-dep@not a range"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(stdout).toContain("1 vulnerability remaining"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); + }); + + test.concurrent("a bundled dependency is never claimed as fixed", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "bundled-1": "1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.0"'); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps@1.0.0 → 1.0.1 + bundled-1@1.0.0 bundles no-deps@1.0.0 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + expect(await installedVersion(dir, "bundled-1", "node_modules", "no-deps")).toBe("1.0.0"); + }); + + test.concurrent("multiple advisories on one instance are cleared together", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4", 1), adv("<1.0.6", 2)] }); + using dir = await setupVulnerableADep(server); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.6"); + expect(stdout).toContain("Fixed 2 vulnerabilities in 1 package"); + expect(stdout).not.toContain("remaining"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.6"'); + expect(lockfile).not.toContain('"a-dep@1.0.4"'); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + }); + + // pnpm fixtures/update-multiple: two advisories for one name with disjoint ranges. + test.concurrent("disjoint advisory ranges for one package are all avoided", async () => { + await using server = startRegistry({ "no-deps": [adv(">=1.0.0 <1.0.1", 1), adv(">=1.1.0 <2.0.0", 2)] }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "no-deps": ">=1.0.0" } }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + expect(await lock(dir)).toContain('"no-deps@1.0.1"'); + }); + + // pnpm fixtures/update-single-depth-2/responses/unfixable-vulnerability.json: npm-style advisory objects carry + // fields (findings, patched_versions, ...) that must be ignored, and `>=0.0.0` is unfixable. + test.concurrent("ignores unknown advisory fields and treats >=0.0.0 as unfixable", async () => { + await using server = startRegistry( + {}, + { + bulkResponse: { + "no-deps": [ + { + ...adv(">=0.0.0", 1234), + findings: [{ version: "1.0.0", paths: ["no-deps"] }], + patched_versions: "<0.0.0", + recommendation: "None", + cwe: ["CWE-1"], + cvss: { score: 0, vectorString: null }, + }, + ], + }, + }, + ); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "^1.0.0" } }); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "no fix available: + no-deps@1.1.0 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + // Ported verbatim from pnpm's test/audit/utils/responses/all-vulnerabilities-response.json (51 packages, 111 advisories). + test.concurrent("parses a real bulk response and only plans installed packages", async () => { + const bulkResponse = await file( + join(import.meta.dir, "registry/fixtures/audit/pnpm-all-vulnerabilities-response.json"), + ).json(); + await using server = startRegistry({}, { bulkResponse }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).not.toContain("fixing:"); + expect(stdout).not.toContain("no-deps"); + expect(stdout).toContain("not matched to an installed version:"); + expect(stdout).toContain(" axios@<0.21.2\n"); + expect(stdout).toContain(" semver@>=2.0.0-alpha <5.7.2\n"); + expect(stdout).toContain("111 vulnerabilities remaining"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("fixes two packages in one run and leaves the rest of the lockfile alone", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")], "no-deps": [adv("<1.0.1", 2)] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "a-dep": "1.0.2", "one-range-dep": "1.0.0", "no-deps": "1.0.0", "@types/is-number": "1.0.0" }, + }); + await reinstall(dir, { + name: "foo", + dependencies: { "a-dep": "^1.0.2", "one-range-dep": "1.0.0", "@types/is-number": "1.0.0" }, + }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"a-dep@1.0.2"'); + expect(lockBefore).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("fixing:\n a-dep@1.0.2 → 1.0.4\n no-deps@1.0.0 → 1.0.1\n"); + expect(stdout).toContain("Fixed 2 vulnerabilities in 2 packages"); + expect(exitCode).toBe(0); + + const lockAfter = await lock(dir); + const packageRows = lockBefore.split("\n").filter(line => /^ "[^"]+": \["/.test(line)); + const untouched = packageRows.filter(line => !line.includes('"a-dep@') && !line.includes('"no-deps@')); + expect(untouched.map(line => line.split('"')[1]).sort()).toEqual(["@types/is-number", "one-range-dep"]); + for (const line of untouched) expect(lockAfter).toContain(line); + expect(lockAfter).toContain('"a-dep@1.0.4"'); + expect(lockAfter).toContain('"no-deps@1.0.1"'); + }); + + test.concurrent("fixes a scoped package", async () => { + await using server = startRegistry({ "@types/is-number": [adv("<2.0.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "@types/is-number": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "@types/is-number": ">=1.0.0" } }); + expect(await lock(dir)).toContain('"@types/is-number@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("@types/is-number@1.0.0 → 2.0.0"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"@types/is-number@2.0.0"'); + expect(lockfile).not.toContain('"@types/is-number@1.0.0"'); + expect(await installedVersion(dir, "@types", "is-number")).toBe("2.0.0"); + }); + + test.concurrent("fixes an npm: alias pointing at a vulnerable package", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setup(server, { name: "foo", dependencies: { nd: "npm:a-dep@1.0.2" } }); + await reinstall(dir, { name: "foo", dependencies: { nd: "npm:a-dep@^1.0.2" } }); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).not.toContain('"a-dep@1.0.2"'); + expect(await installedVersion(dir, "nd")).toBe("1.0.4"); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("an overrides pin is reported as the blocker", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "a-dep": "^1.0.2" }, + overrides: { "a-dep": "1.0.2" }, + }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + a-dep@1.0.2 → 1.0.4 + foo depends on a-dep@1.0.2 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + // pnpm fixtures/update-workspace-catalog-pinned: a pinned catalog entry blocks the fix. + test.concurrent("a pinned catalog entry is reported as the blocker", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup( + server, + { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "1.0.0" } }, + { "packages/a/package.json": JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }) }, + ); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps@1.0.0 → 1.0.1 + a depends on no-deps@1.0.0 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + // mismatched-peer-deps-lvl1 declares peer no-deps@<=1.0.1; its own dependency lvl2 declares peer no-deps@1.0.0, + // so the installs warn about an incorrect peer and cannot go through runBunInstall. + test.concurrent("a peer range is a blocker and is labelled with the peer dependent", async () => { + await using server = startRegistry({ "no-deps": [adv("<=1.0.1")] }); + const pkgJson = (noDeps: string) => + JSON.stringify({ name: "foo", dependencies: { "mismatched-peer-deps-lvl1": "1.0.0", "no-deps": noDeps } }); + using dir = tempDir("audit-fix-", { "package.json": pkgJson("1.0.1") }); + await writeBunfig(dir, server); + expect((await run(dir, ["install"])).exitCode).toBe(0); + await write(join(dir, "package.json"), pkgJson("^1.0.0")); + expect((await run(dir, ["install"])).exitCode).toBe(0); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.1 → 1.1.0"); + expect(stdout).toContain("mismatched-peer-deps-lvl1@1.0.0 depends on no-deps@<=1.0.1"); + expect(stdout).not.toContain("foo depends on"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("a depth-3 blocker names the immediate dependent", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.1.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-one-dep": "1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "requires a semver-major update: + no-deps@1.0.1 → 1.1.0 + one-dep@1.0.0 depends on no-deps@1.0.1 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + // Workspaces default to the isolated linker, so the linker is pinned to keep node_modules paths predictable. + test.concurrent("fixes a workspace member's dependency when run from the member directory", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + const member = (range: string) => JSON.stringify({ name: "a", dependencies: { "a-dep": range } }); + const rootPkgJson = JSON.stringify({ name: "root", workspaces: ["packages/*"] }); + using dir = tempDir("audit-fix-", { "package.json": rootPkgJson, "packages/a/package.json": member("1.0.2") }); + await writeBunfig(dir, server); + expect((await run(dir, ["install", "--linker", "hoisted"])).exitCode).toBe(0); + await write(join(dir, "packages", "a", "package.json"), member("^1.0.2")); + expect((await run(dir, ["install", "--linker", "hoisted"])).exitCode).toBe(0); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await run(join(dir, "packages", "a"), ["audit", "fix", "--linker", "hoisted"], dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).not.toContain('"a-dep@1.0.2"'); + expect(await exists(join(dir, "packages", "a", "bun.lock"))).toBeFalse(); + expect(await file(join(dir, "package.json")).text()).toBe(rootPkgJson); + expect(await file(join(dir, "packages", "a", "package.json")).text()).toBe(member("^1.0.2")); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + + const frozen = await run(dir, ["install", "--frozen-lockfile", "--linker", "hoisted"]); + expect(frozen.stderr).not.toContain("error:"); + expect(frozen.exitCode).toBe(0); + }); + + // `--linker isolated` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. + test.concurrent("isolated linker layout", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = tempDir("audit-fix-", { + "package.json": JSON.stringify({ name: "foo", dependencies: { "a-dep": "1.0.2" } }), + }); + await writeBunfig(dir, server); + expect((await run(dir, ["install", "--linker", "isolated"])).exitCode).toBe(0); + await write(join(dir, "package.json"), JSON.stringify({ name: "foo", dependencies: { "a-dep": "^1.0.2" } })); + expect((await run(dir, ["install", "--linker", "isolated"])).exitCode).toBe(0); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + expect(await readlink(join(dir, "node_modules", "a-dep"))).toContain("a-dep@1.0.2"); + + const { stdout, exitCode } = await auditFix(dir, "--linker", "isolated"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); + expect(await readlink(join(dir, "node_modules", "a-dep"))).toContain("a-dep@1.0.4"); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + + const frozen = await run(dir, ["install", "--frozen-lockfile", "--linker", "isolated"]); + expect(frozen.stderr).not.toContain("error:"); + expect(frozen.exitCode).toBe(0); + }); + + // Every a-dep release was published in 2023, so a 100-year minimum age gates all of them. + test.concurrent("a fix gated by --minimum-release-age is reported distinctly", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const gated = await auditFix(dir, "--minimum-release-age", "3153600000"); + expect(normalizeBunSnapshot(gated.stdout)).toMatchInlineSnapshot(` + "no fix available: + a-dep@1.0.2 (1.0.4 is newer than --minimum-release-age) + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(gated.exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + + const fixed = await auditFix(dir, "--minimum-release-age", "60"); + expect(fixed.stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(fixed.exitCode).toBe(0); + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); + }); + + test.concurrent("a failing bulk endpoint changes nothing", async () => { + await using server = startRegistry({}, { bulkStatus: 500 }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).not.toContain("fixing:"); + expect(stderr).toContain("audit request failed"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("a manifest that fails to download is reported, not fixed", async () => { + const denyManifests = new Set(); + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }, { denyManifests }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + denyManifests.add("a-dep"); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "no fix available: + a-dep@1.0.2 (failed to fetch the manifest) + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).toContain("a-dep"); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("refuses to run without a lockfile", async () => { + await using server = startRegistry({}); + using dir = tempDir("audit-fix-", { + "package.json": JSON.stringify({ name: "foo" }), + "bunfig.toml": Bun.TOML.stringify({ install: { registry: server.url.href } }), + }); + + const { stderr, exitCode } = await auditFix(dir); + expect(stderr).toContain("Lockfile not found"); + expect(exitCode).toBe(1); + expect(await exists(join(dir, "bun.lock"))).toBeFalse(); + }); + + test.concurrent("refuses --no-save before contacting the registry", async () => { + await using server = startRegistry({}, { bulkStatus: 500 }); + using dir = await setupVulnerableADep(server); + const lockBefore = await lock(dir); + + const { stdout, stderr, exitCode } = await auditFix(dir, "--no-save"); + expect(stderr).toContain("error: bun audit fix needs to write bun.lock, but saving the lockfile is disabled"); + expect(stderr).not.toContain("audit request failed"); + expect(stdout).not.toContain("Fixed"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); +}); diff --git a/test/cli/install/bun-dedupe.test.ts b/test/cli/install/bun-dedupe.test.ts index 9093cd2b8ea4..99e928439664 100644 --- a/test/cli/install/bun-dedupe.test.ts +++ b/test/cli/install/bun-dedupe.test.ts @@ -14,10 +14,13 @@ afterAll(() => { registry.stop(); }); +// CI exports BUN_INSTALL_CACHE_DIR, which overrides the harness bunfig's per-test `cache`; concurrent cases sharing one cache race on Windows. +const installEnv = (dir: string) => ({ ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }); + async function run(dir: string, ...cmd: string[]) { await using proc = Bun.spawn({ cmd: [bunExe(), ...cmd], - env: bunEnv, + env: installEnv(dir), cwd: dir, stdout: "pipe", stderr: "pipe", @@ -45,19 +48,19 @@ function nodeModulesVersion(packageDir: string, ...segments: string[]) { // A still-satisfied range edge is never re-resolved by a later install, so adding an exact pin afterwards leaves a duplicate. async function installTwice(packageDir: string, packageJson: string, first: object, second: object) { await write(packageJson, JSON.stringify(first)); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); await write(packageJson, JSON.stringify(second)); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); return lock(packageDir); } // one-range-dep@1.0.0 depends on no-deps@^1.0.0 (locked 1.1.0); root then pins no-deps@1.0.0. -async function setupRangeDuplicate(packageDir: string, packageJson: string, extra: Record = {}) { +async function setupRangeDuplicate(packageDir: string, packageJson: string) { const lockfile = await installTwice( packageDir, packageJson, - { name: "foo", ...extra, dependencies: { "one-range-dep": "1.0.0" } }, - { name: "foo", ...extra, dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" } }, + { name: "foo", dependencies: { "one-range-dep": "1.0.0" } }, + { name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" } }, ); expect(lockfile).toContain('"no-deps@1.0.0"'); expect(lockfile).toContain('"no-deps@1.1.0"'); @@ -71,9 +74,8 @@ test.concurrent("collapses a range onto the exact version that satisfies every e const pkgJsonBefore = await file(packageJson).text(); const { stdout, stderr, exitCode } = await dedupe(packageDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.1.0", ]); expect(stderr).toContain("Saved lockfile"); @@ -86,7 +88,7 @@ test.concurrent("collapses a range onto the exact version that satisfies every e expect(lockfile).not.toContain('"one-range-dep/no-deps"'); expect(await file(packageJson).text()).toBe(pkgJsonBefore); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); // pnpm/pnpm#6550: root's range moves up onto the version its dependents already use. @@ -98,15 +100,14 @@ test.concurrent("prefers the highest version when several satisfy every edge", a packageJson, JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "^1.0.0" } }), ); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); let lockfile = await lock(packageDir); expect(lockfile).toContain('"no-deps@1.0.0"'); expect(lockfile).toContain('"no-deps@1.1.0"'); const { stdout, stderr, exitCode } = await dedupe(packageDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.0.0", ]); expect(stderr).not.toContain("error:"); @@ -120,7 +121,7 @@ test.concurrent("prefers the highest version when several satisfy every edge", a version: "1.1.0", }); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); // pnpm/pnpm#13503: `--check` must not touch node_modules either. @@ -135,7 +136,6 @@ test.concurrent("--check reports and exits 1 without writing", async () => { const check = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` "bun dedupe () - 1 duplicate version can be removed: no-deps@1.1.0" `); expect(check.stderr).toContain("note: run 'bun dedupe' to remove them"); @@ -172,7 +172,6 @@ test.concurrent("already deduplicated", async () => { const check = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` "bun dedupe () - Already deduplicated." `); expect(check.exitCode).toBe(0); @@ -185,7 +184,7 @@ test.concurrent("keeps versions that no single version can replace", async () => packageJson, JSON.stringify({ name: "foo", dependencies: { "one-dep": "1.0.0", "one-fixed-dep": "1.0.0" } }), ); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); const lockBefore = await lock(packageDir); expect(lockBefore).toContain('"no-deps@1.0.0"'); expect(lockBefore).toContain('"no-deps@1.0.1"'); @@ -216,26 +215,25 @@ test.concurrent("honours catalog ranges", async () => { JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }), ), ]); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); let lockfile = await lock(packageDir); expect(lockfile).toContain('"no-deps@1.0.0"'); await write(packageJson, root("^1.0.0")); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); lockfile = await lock(packageDir); expect(lockfile).toContain('"no-deps@1.0.0"'); expect(lockfile).not.toContain('"no-deps@1.1.0"'); await write(packageJson, root("^1.0.0", { "no-deps": "1.1.0" })); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); lockfile = await lock(packageDir); expect(lockfile).toContain('"no-deps@1.0.0"'); expect(lockfile).toContain('"no-deps@1.1.0"'); const { stdout, stderr, exitCode } = await dedupe(packageDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.0.0", ]); expect(stderr).not.toContain("error:"); @@ -245,7 +243,7 @@ test.concurrent("honours catalog ranges", async () => { expect(lockfile).not.toContain('"no-deps@1.0.0"'); expect(lockfile).toContain('"no-deps@1.1.0"'); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); test.concurrent("override range wins over the edge's own range", async () => { @@ -261,7 +259,6 @@ test.concurrent("override range wins over the edge's own range", async () => { const { stdout, stderr, exitCode } = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` "bun dedupe () - 1 duplicate version can be removed: no-deps@1.0.0" `); expect(stderr).not.toContain("error:"); @@ -287,7 +284,7 @@ test.concurrent("errors without a lockfile", async () => { test.concurrent("rejects positional arguments", async () => { const { packageDir, packageJson } = await registry.createTestDir(); await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); const lockBefore = await lock(packageDir); const { stderr, exitCode } = await dedupe(packageDir, "no-deps"); @@ -332,9 +329,8 @@ test.concurrent("works with the isolated linker", async () => { expect(await nestedNoDeps()).toEqual({ name: "no-deps", version: "1.1.0" }); const { stdout, stderr, exitCode } = await dedupe(packageDir, "--linker", "isolated"); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.1.0", ]); expect(stderr).toContain("Saved lockfile"); @@ -376,16 +372,14 @@ test.concurrent("duplicate held only by an optional peer edge is deduplicated", const check = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` "bun dedupe () - 1 duplicate version can be removed: no-deps@1.1.0" `); expect(check.exitCode).toBe(1); expect(await lock(packageDir)).toBe(heldByPeer); const { stdout, stderr, exitCode } = await dedupe(packageDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.1.0", ]); expect(stderr).not.toContain("error:"); @@ -395,7 +389,7 @@ test.concurrent("duplicate held only by an optional peer edge is deduplicated", expect(after).toContain('"no-deps@1.0.0"'); expect(after).not.toContain('"no-deps@1.1.0"'); expect((await dedupe(packageDir, "--check")).exitCode).toBe(0); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); test.concurrent.each([ @@ -418,7 +412,7 @@ test.concurrent.each([ test.concurrent("--frozen-lockfile succeeds when already deduplicated", async () => { const { packageDir, packageJson } = await registry.createTestDir(); await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0" } })); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); const lockBefore = await lock(packageDir); const { stdout, stderr, exitCode } = await dedupe(packageDir, "--frozen-lockfile"); @@ -444,15 +438,13 @@ test.concurrent("cascading removal lists every unreachable duplicate in name ord const check = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` "bun dedupe () - 2 duplicate versions can be removed: no-deps@2.0.0, one-fixed-dep@2.0.0" `); expect(check.exitCode).toBe(1); const { stdout, stderr, exitCode } = await dedupe(packageDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 2 duplicate versions: no-deps@2.0.0, one-fixed-dep@2.0.0", ]); expect(stderr).not.toContain("error:"); @@ -465,7 +457,7 @@ test.concurrent("cascading removal lists every unreachable duplicate in name ord expect(after).not.toContain('"no-deps@2.0.0"'); expect(await nodeModulesVersion(packageDir, "one-fixed-dep")).toBe("1.0.0"); expect(await nodeModulesVersion(packageDir, "ofd")).toBe("1.0.0"); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); test.concurrent("multiple names removed in one run are sorted, scoped names first", async () => { @@ -483,15 +475,13 @@ test.concurrent("multiple names removed in one run are sorted, scoped names firs const check = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` "bun dedupe () - 2 duplicate versions can be removed: @types/is-number@2.0.0, no-deps@1.1.0" `); expect(check.exitCode).toBe(1); const { stdout, exitCode } = await dedupe(packageDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 2 duplicate versions: @types/is-number@2.0.0, no-deps@1.1.0", ]); expect(exitCode).toBe(0); @@ -499,7 +489,7 @@ test.concurrent("multiple names removed in one run are sorted, scoped names firs const after = await lock(packageDir); expect(after).not.toContain('"@types/is-number@2.0.0"'); expect(after).not.toContain('"no-deps@1.1.0"'); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); test.concurrent("workspace package edge is re-pointed when run from the workspace directory", async () => { @@ -519,9 +509,8 @@ test.concurrent("workspace package edge is re-pointed when run from the workspac expect(lockfile).toContain('"no-deps@1.1.0"'); const { stdout, stderr, exitCode } = await dedupe(workspaceDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.1.0", ]); expect(stderr).not.toContain("error:"); @@ -530,13 +519,13 @@ test.concurrent("workspace package edge is re-pointed when run from the workspac const after = await lock(packageDir); expect(after).not.toContain('"no-deps@1.1.0"'); expect(await exists(join(workspaceDir, "bun.lock"))).toBeFalse(); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); test.concurrent("corrupt bun.lock fails without rewriting it", async () => { const { packageDir, packageJson } = await registry.createTestDir(); await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); const corrupt = "{ not valid"; await write(join(packageDir, "bun.lock"), corrupt); @@ -561,7 +550,7 @@ test.concurrent("--check never creates node_modules", async () => { expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); expect(await lock(packageDir)).toBe(lockBefore); - await runBunInstall(bunEnv, packageDir, { savesLockfile: false }); + await runBunInstall(installEnv(packageDir), packageDir, { savesLockfile: false }); await dedupe(packageDir); const lockDeduped = await lock(packageDir); await rm(join(packageDir, "node_modules"), { recursive: true }); @@ -576,7 +565,7 @@ test.concurrent("--check never creates node_modules", async () => { test.concurrent("never upgrades past the locked version", async () => { const { packageDir, packageJson } = await registry.createTestDir(); await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); const pinned = await lock(packageDir); expect(pinned).toContain('"no-deps": "1.0.0"'); const widened = pinned.replace('"no-deps": "1.0.0"', '"no-deps": "^1.0.0"'); @@ -604,9 +593,11 @@ test.concurrent.each([ [undefined, true], ])("root lifecycle scripts with %s", async (flag, runs) => { const { packageDir, packageJson } = await registry.createTestDir(); - await setupRangeDuplicate(packageDir, packageJson, { scripts: { postinstall: "echo ran > postinstall.txt" } }); + await setupRangeDuplicate(packageDir, packageJson); + // Added after setup so only the dedupe run can create the marker. + const pkg = await file(packageJson).json(); + await write(packageJson, JSON.stringify({ ...pkg, scripts: { postinstall: "echo ran > postinstall.txt" } })); const marker = join(packageDir, "postinstall.txt"); - await rm(marker, { force: true }); const { stdout, stderr, exitCode } = await dedupe(packageDir, ...(flag ? [flag] : [])); expect(stdout).toContain("Removed 1 duplicate version: no-deps@1.1.0"); @@ -623,7 +614,7 @@ test.concurrent("root range collapses onto a transitive exact pin", async () => packageJson, JSON.stringify({ name: "foo", dependencies: { "a-dep": "^1.0.0", "uses-a-dep-3": "1.0.0" } }), ); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); const pkgJsonBefore = await file(packageJson).text(); const lockfile = await lock(packageDir); expect(lockfile).toContain('"a-dep@1.0.10"'); @@ -631,9 +622,8 @@ test.concurrent("root range collapses onto a transitive exact pin", async () => expect(await nodeModulesVersion(packageDir, "a-dep")).toBe("1.0.10"); const { stdout, exitCode } = await dedupe(packageDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: a-dep@1.0.10", ]); expect(exitCode).toBe(0); @@ -643,7 +633,7 @@ test.concurrent("root range collapses onto a transitive exact pin", async () => expect(after).toContain('"a-dep@1.0.3"'); expect(after).not.toContain('"a-dep@1.0.10"'); expect(await nodeModulesVersion(packageDir, "a-dep")).toBe("1.0.3"); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); test.concurrent("npm: alias edges are deduplicated by the aliased range", async () => { @@ -658,16 +648,15 @@ test.concurrent("npm: alias edges are deduplicated by the aliased range", async expect(lockfile).toContain('"no-deps@1.1.0"'); const { stdout, exitCode } = await dedupe(packageDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.1.0", ]); expect(exitCode).toBe(0); expect(await lock(packageDir)).not.toContain('"no-deps@1.1.0"'); expect(await nodeModulesVersion(packageDir, "nd")).toBe("1.0.0"); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); test.concurrent("dist-tag edges keep the version the tag resolved to", async () => { @@ -676,7 +665,7 @@ test.concurrent("dist-tag edges keep the version the tag resolved to", async () packageJson, JSON.stringify({ name: "foo", dependencies: { "dep-with-tags": "pre-1", "dwt": "npm:dep-with-tags@1.0.0" } }), ); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); const lockBefore = await lock(packageDir); expect(lockBefore).toContain('"dep-with-tags@1.0.1"'); expect(lockBefore).toContain('"dep-with-tags@1.0.0"'); @@ -706,7 +695,6 @@ test.concurrent("edges pointing at a patched version are never moved", async () const { stdout, stderr, exitCode } = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` "bun dedupe () - Already deduplicated." `); expect(stderr).not.toContain("error:"); @@ -721,9 +709,8 @@ test.concurrent("--lockfile-only rewrites bun.lock without installing", async () expect(await nested()).toBe("1.1.0"); const { stdout, stderr, exitCode } = await dedupe(packageDir, "--lockfile-only"); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.1.0", ]); expect(stderr).toContain("Saved lockfile"); @@ -732,7 +719,7 @@ test.concurrent("--lockfile-only rewrites bun.lock without installing", async () expect(await lock(packageDir)).not.toContain('"no-deps@1.1.0"'); expect(await nested()).toBe("1.1.0"); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); test.concurrent("--silent keeps the exit codes", async () => { @@ -771,7 +758,7 @@ test.concurrent("stale package.json is resolved after the dedupe", async () => { expect(after).not.toContain('"no-deps@1.1.0"'); expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("2.0.0"); expect((await dedupe(packageDir, "--check")).exitCode).toBe(0); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); // pnpm/pnpm#9213 (and #6619): one-fixed-dep@1.0.0 dies in this run, so its exact no-deps@1.0.0 edge must not drag the live "^1.0.0" edges down. @@ -781,7 +768,7 @@ test.concurrent("a version removed by the run does not vote for its own dependen packageJson, JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "^1.0.0" } }), ); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); expect(await lock(packageDir)).toContain('"no-deps@1.1.0"'); const lockfile = await installTwice( packageDir, @@ -811,15 +798,13 @@ test.concurrent("a version removed by the run does not vote for its own dependen const check = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` "bun dedupe () - 2 duplicate versions can be removed: no-deps@1.0.0, one-fixed-dep@1.0.0" `); expect(check.exitCode).toBe(1); const { stdout, stderr, exitCode } = await dedupe(packageDir); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 2 duplicate versions: no-deps@1.0.0, one-fixed-dep@1.0.0", ]); expect(stderr).not.toContain("error:"); @@ -838,7 +823,7 @@ test.concurrent("a version removed by the run does not vote for its own dependen expect(recheck.stdout).toContain("Already deduplicated."); expect(recheck.exitCode).toBe(0); expect(await lock(packageDir)).toBe(after); - await runBunInstall(bunEnv, packageDir, { frozenLockfile: true }); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); // A bundled edge is satisfied by the tarball's own copy, so it stays put and root's range collapses onto it instead. @@ -862,16 +847,14 @@ test.concurrent("bundled edges are never re-pointed", async () => { const check = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` "bun dedupe () - 1 duplicate version can be removed: no-deps@1.1.0" `); expect(check.exitCode).toBe(1); expect(await lock(packageDir)).toBe(widened); const { stdout, stderr, exitCode } = await dedupe(packageDir, "--lockfile-only"); - expect(firstLines(stdout, 3)).toEqual([ + expect(firstLines(stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.1.0", ]); expect(stderr).not.toContain("error:"); @@ -903,16 +886,14 @@ test.concurrent("does not contact the registry", async () => { const check = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` "bun dedupe () - 1 duplicate version can be removed: no-deps@1.1.0" `); expect(check.stderr).not.toContain("error:"); expect(check.exitCode).toBe(1); const apply = await dedupe(packageDir, "--lockfile-only"); - expect(firstLines(apply.stdout, 3)).toEqual([ + expect(firstLines(apply.stdout, 2)).toEqual([ "bun dedupe ()", - "", "Removed 1 duplicate version: no-deps@1.1.0", ]); expect(apply.stderr).toContain("Saved lockfile"); diff --git a/test/cli/install/bun-install-registry.test.ts b/test/cli/install/bun-install-registry.test.ts index e40b14dcaf7a..bf2495932341 100644 --- a/test/cli/install/bun-install-registry.test.ts +++ b/test/cli/install/bun-install-registry.test.ts @@ -5949,26 +5949,19 @@ describe("update", () => { version: "1.0.0", }); - // update package that doesn't exist to workspace, should add to package.json + // updating a package the workspace does not declare re-resolves the root's pin in place and never adds it to the workspace's package.json ({ out } = await runBunUpdate(env, join(packageDir, "packages", "pkg1"), ["no-deps"])); assertManifestsPopulated(join(packageDir, ".bun-cache"), registryUrl()); - expect(out).toEqual([ - expect.stringContaining("bun update v1."), - "", - "installed no-deps@2.0.0", - "", - "1 package installed", - ]); + expect(out[0]).toContain("bun update v1."); expect(await file(join(packageDir, "node_modules", "no-deps", "package.json")).json()).toMatchObject({ version: "1.0.0", }); - expect(await file(join(packageDir, "packages", "pkg1", "package.json")).json()).toMatchObject({ + expect(await file(join(packageDir, "packages", "pkg1", "package.json")).json()).toStrictEqual({ name: "pkg1", version: "1.0.0", dependencies: { "a-dep": "^1.0.0", - "no-deps": "^2.0.0", }, }); diff --git a/test/cli/install/bun-pm-licenses.test.ts b/test/cli/install/bun-pm-licenses.test.ts index 8146a909ce22..3fbcb7097fb0 100644 --- a/test/cli/install/bun-pm-licenses.test.ts +++ b/test/cli/install/bun-pm-licenses.test.ts @@ -18,6 +18,9 @@ afterAll(() => { registry.stop(); }); +// CI exports BUN_INSTALL_CACHE_DIR, which overrides the harness bunfig's per-test `cache`; concurrent cases sharing one cache race on Windows and share hardlinked manifests on Linux. +const installEnv = (dir: string) => ({ ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }); + const gitEnv = { ...bunEnv, GIT_CONFIG_NOSYSTEM: "1", @@ -64,7 +67,7 @@ const monorepoFiles: Files = { async function install(dir: string, linker: Linker, ...args: string[]) { await using proc = spawn({ cmd: [bunExe(), "install", "--linker", linker, ...args], - env: bunEnv, + env: installEnv(dir), cwd: dir, stdin: "ignore", stdout: "pipe", @@ -115,11 +118,18 @@ function names(parsed: Record) { .sort(); } +// On Linux/Windows installed files are hardlinks into the cache; unlink first so the cache copy is left alone. +function overwriteInstalledManifest(dir: string, pkg: string, contents: string) { + const path = join(dir, "node_modules", pkg, "package.json"); + rmSync(path); + writeFileSync(path, contents); +} + function patchInstalledManifest(dir: string, pkg: string, fields: Record) { const path = join(dir, "node_modules", pkg, "package.json"); const manifest = { ...JSON.parse(readFileSync(path, "utf8")), ...fields }; for (const [key, value] of Object.entries(fields)) if (value === undefined) delete manifest[key]; - writeFileSync(path, JSON.stringify(manifest)); + overwriteInstalledManifest(dir, pkg, JSON.stringify(manifest)); } const fullJson = { @@ -414,6 +424,20 @@ describe("bun pm licenses", () => { }); }); + test.concurrent("--prod omits a file: dependency's devDependencies, like bun install --production", async () => { + const dir = await setup("hoisted", { + "package.json": pkg({ dependencies: { "no-deps": "1.0.0", "sub-dep": "file:./sub-dep" } }), + "sub-dep/package.json": JSON.stringify({ + name: "sub-dep", + version: "2.5.0", + devDependencies: { "a-dep": "1.0.1" }, + }), + }); + + expect(names(await licensesJson(dir))).toEqual(["a-dep", "no-deps", "sub-dep"]); + expect(names(await licensesJson(dir, "--prod"))).toEqual(["no-deps", "sub-dep"]); + }); + test.concurrent.each(["hoisted", "isolated"] as Linker[])("link: dependency is not listed (%s)", async linker => { const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker }, @@ -422,7 +446,7 @@ describe("bun pm licenses", () => { "linked/package.json": JSON.stringify({ name: "linked", version: "1.0.0", license: "MIT" }), }, }); - const env = { ...bunEnv, BUN_INSTALL_GLOBAL_DIR: join(packageDir, ".global") }; + const env = { ...installEnv(packageDir), BUN_INSTALL_GLOBAL_DIR: join(packageDir, ".global") }; for (const [cmd, cwd] of [ [[bunExe(), "link"], join(packageDir, "linked")], [[bunExe(), "install", "--linker", linker], packageDir], @@ -522,7 +546,7 @@ describe("bun pm licenses", () => { const { packageDir } = await registry.createTestDir({ files: { "package.json": fixturePackageJson } }); await using install = spawn({ cmd: [bunExe(), "install", "--lockfile-only"], - env: bunEnv, + env: installEnv(packageDir), cwd: packageDir, stdout: "pipe", stderr: "pipe", @@ -540,7 +564,7 @@ describe("bun pm licenses", () => { "unparsable package.json is reported as Unknown, missing ones are omitted with a warning", async () => { const dir = await setup(); - writeFileSync(join(dir, "node_modules", "resolve", "package.json"), "{ not json"); + overwriteInstalledManifest(dir, "resolve", "{ not json"); rmSync(join(dir, "node_modules", "one-dep"), { recursive: true, force: true }); const [stdout, stderr, exitCode] = await licenses(dir, "--json"); diff --git a/test/cli/install/bun-prune.test.ts b/test/cli/install/bun-prune.test.ts index 0315de0d5c73..d0d95ed63c19 100644 --- a/test/cli/install/bun-prune.test.ts +++ b/test/cli/install/bun-prune.test.ts @@ -24,10 +24,18 @@ afterAll(() => { registry.stop(); }); +// CI exports BUN_INSTALL_CACHE_DIR, which overrides the harness bunfig's per-test `cache`; concurrent cases sharing one cache race on Windows. +const installEnv = (dir: string) => ({ ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }); + +const WARN = (expected: string, kept: string) => + `warn: ${expected} is not the version bun.lock installs there; keeping ${kept}`; +const NOTE = + "note: run 'bun install' with the same flags to install the versions bun.lock expects, then run 'bun prune' again"; + async function prune(dir: string, ...args: string[]) { await using proc = Bun.spawn({ cmd: [bunExe(), "prune", ...args], - env: bunEnv, + env: installEnv(dir), cwd: dir, stdout: "pipe", stderr: "pipe", @@ -78,14 +86,14 @@ type BunfigOpts = NonNullable[0]> async function setup(pkgJson: Record, bunfigOpts?: BunfigOpts) { const { packageDir, packageJson } = await registry.createTestDir({ bunfigOpts }); await write(packageJson, JSON.stringify(pkgJson)); - await runBunInstall(bunEnv, packageDir); + await runBunInstall(installEnv(packageDir), packageDir); return packageDir; } async function install(dir: string, ...args: string[]) { await using proc = Bun.spawn({ cmd: [bunExe(), "install", ...args], - env: bunEnv, + env: installEnv(dir), cwd: dir, stdout: "pipe", stderr: "pipe", @@ -210,7 +218,7 @@ test.concurrent.each([["--production"], ["--prod"], ["--omit=dev"]])( expectBinRemoved(nm, "what-bin"); expectBinInstalled(nm, "has-bin-entry"); - const { out: installOut } = await runBunInstall(bunEnv, dir, { production: true }); + const { out: installOut } = await runBunInstall(installEnv(dir), dir, { production: true }); expect(installOut).toContain("no changes"); expect(await lock(dir)).toBe(lockBefore); }, @@ -269,18 +277,7 @@ test.concurrent("--dry-run prints without deleting; --silent deletes without pri test.concurrent("nothing to prune when node_modules is missing or clean", async () => { const { packageDir, packageJson } = await registry.createTestDir(); await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); - { - await using proc = Bun.spawn({ - cmd: [bunExe(), "install", "--lockfile-only"], - env: bunEnv, - cwd: packageDir, - stdout: "pipe", - stderr: "pipe", - }); - const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); - expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); - } + await install(packageDir, "--lockfile-only"); const nm = join(packageDir, "node_modules"); rmSync(nm, { recursive: true, force: true }); expect(existsSync(join(packageDir, "bun.lock"))).toBeTrue(); @@ -369,7 +366,7 @@ test.concurrent("workspaces: prunes workspace folders, keeps workspace links, ru }), ), ]); - await runBunInstall(bunEnv, dir); + await runBunInstall(installEnv(dir), dir); const nm = join(dir, "node_modules"); const workspaceNoDeps = join(dir, "packages", "a", "node_modules", "no-deps"); expect(isSymlink(join(nm, "a"))).toBeTrue(); @@ -451,7 +448,7 @@ test.concurrent("isolated linker: removes unused store entries and their links", expect(() => lstatSync(join(hiddenHoist, "zzz"))).toThrow(); expect(await lock(dir)).toBe(lockBefore); - await runBunInstall(bunEnv, dir, { production: true }); + await runBunInstall(installEnv(dir), dir, { production: true }); }); test.concurrent("isolated linker + global store: unlinks the store link, never deletes the shared entry", async () => { @@ -461,7 +458,7 @@ test.concurrent("isolated linker + global store: unlinks the store link, never d ); const storeEntry = join(dir, "node_modules", ".bun", "one-dep@1.0.0"); expect(isSymlink(storeEntry)).toBeTrue(); - const linksDir = join(dir, ".bun-cache", "links"); + const linksDir = join(installEnv(dir).BUN_INSTALL_CACHE_DIR, "links"); const globalEntry = readdirSync(linksDir).find(name => name.startsWith("one-dep@1.0.0-")); expect(globalEntry).toBeDefined(); const globalPkgJson = join(linksDir, globalEntry!, "node_modules", "one-dep", "package.json"); @@ -648,7 +645,7 @@ test.concurrent( expect(existsSync(join(nm, "@scoped"))).toBeFalse(); expect(await file(join(nm, "no-deps", "package.json")).json()).toMatchObject({ version: "2.0.0" }); expectBinRemoved(nm, "has-bin-entry"); - const { out: installOut } = await runBunInstall(bunEnv, dir, { savesLockfile: false }); + const { out: installOut } = await runBunInstall(installEnv(dir), dir, { savesLockfile: false }); expect(installOut).toContain("no changes"); }, ); @@ -731,7 +728,7 @@ test.concurrent("hoisted: --production empties a workspace folder that only held expect(existsSync(nested)).toBeFalse(); expect(isSymlink(join(nm, "a"))).toBeTrue(); expect(await file(join(nm, "no-deps", "package.json")).json()).toMatchObject({ version: "2.0.0" }); - await runBunInstall(bunEnv, dir, { production: true }); + await runBunInstall(installEnv(dir), dir, { production: true }); }); test.concurrent( @@ -771,7 +768,7 @@ test.concurrent( // Diverges from pnpm on purpose: an alias that names a workspace is never removed. expect(existsSync(join(appNm, "tool", "package.json"))).toBeTrue(); expect(existsSync(join(dir, "packages", "tool", "package.json"))).toBeTrue(); - await runBunInstall(bunEnv, dir, { production: true }); + await runBunInstall(installEnv(dir), dir, { production: true }); }, ); @@ -1014,7 +1011,7 @@ test.concurrent("keeps dependencies bundled inside a file: dependency", async () JSON.stringify({ name: "inner", version: "1.0.0" }), ), ]); - await runBunInstall(bunEnv, dir); + await runBunInstall(installEnv(dir), dir); const inner = join(dir, "node_modules", "local", "node_modules", "inner", "package.json"); expect(existsSync(inner)).toBeTrue(); const junk = plant(dir, "node_modules/junk"); @@ -1031,30 +1028,197 @@ test.concurrent("keeps dependencies bundled inside a file: dependency", async () }); // pnpm#13676 -test.concurrent("hoisted: a nested copy left behind after its dependency started hoisting is removed", async () => { - const dir = await setup({ name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "2.0.0" } }); - const nm = join(dir, "node_modules"); - const nested = join(nm, "one-dep", "node_modules", "no-deps"); - expect(await file(join(nested, "package.json")).json()).toMatchObject({ version: "1.0.1" }); +test.concurrent( + "hoisted: a nested copy is kept while the root copy is still the old version and removed once bun install replaced it", + async () => { + const dir = await setup({ name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "2.0.0" } }); + const nm = join(dir, "node_modules"); + const nested = join(nm, "one-dep", "node_modules", "no-deps"); + const rootPkgJson = join(nm, "no-deps", "package.json"); + expect(await file(join(nested, "package.json")).json()).toMatchObject({ version: "1.0.1" }); - await write( - join(dir, "package.json"), - JSON.stringify({ name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "1.0.1" } }), - ); - await install(dir, "--lockfile-only"); - expect(existsSync(nested)).toBeTrue(); + await write( + join(dir, "package.json"), + JSON.stringify({ name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "1.0.1" } }), + ); + await install(dir, "--lockfile-only"); + expect(existsSync(nested)).toBeTrue(); + expect(await file(rootPkgJson).json()).toMatchObject({ version: "2.0.0" }); + + const stale = await prune(dir); + expect(out(stale.stdout)).toEndWith("Nothing to prune."); + expect(out(stale.stderr)).toContain(WARN("node_modules/no-deps", "node_modules/one-dep/node_modules/no-deps")); + expect(out(stale.stderr)).toContain(NOTE); + expect(stale.exitCode).toBe(0); + expect(existsSync(join(nested, "package.json"))).toBeTrue(); + expect(await file(rootPkgJson).json()).toMatchObject({ version: "2.0.0" }); + + await install(dir); + expect(await file(rootPkgJson).json()).toMatchObject({ version: "1.0.1" }); + expect(existsSync(nested)).toBeTrue(); + + const { stdout, stderr, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/one-dep/node_modules/no-deps + Removed 1 package" + `); + expect(stderr).not.toContain("warn:"); + expect(exitCode).toBe(0); + expect(existsSync(nested)).toBeFalse(); + expect(existsSync(join(nm, "one-dep", "package.json"))).toBeTrue(); + expect(existsSync(rootPkgJson)).toBeTrue(); + }, +); - const { stdout, exitCode } = await prune(dir); - expect(out(stdout)).toMatchInlineSnapshot(` - "bun prune () - - node_modules/one-dep/node_modules/no-deps - Removed 1 package" - `); - expect(exitCode).toBe(0); - expect(existsSync(nested)).toBeFalse(); - expect(existsSync(join(nm, "one-dep", "package.json"))).toBeTrue(); - expect(existsSync(join(nm, "no-deps", "package.json"))).toBeTrue(); -}); +test.concurrent( + "hoisted: --production keeps the nested copy a production package resolves to while the root holds the dev version", + async () => { + const pkg = { name: "foo", dependencies: { "one-fixed-dep": "1.0.0" }, devDependencies: { "no-deps": "2.0.0" } }; + const [dir, silentDir] = await Promise.all([setup(pkg), setup(pkg)]); + const rootPkgJson = join(dir, "node_modules", "no-deps", "package.json"); + const nestedPkgJson = join(dir, "node_modules", "one-fixed-dep", "node_modules", "no-deps", "package.json"); + expect(await file(rootPkgJson).json()).toMatchObject({ version: "2.0.0" }); + expect(await file(nestedPkgJson).json()).toMatchObject({ version: "1.0.0" }); + + const { stdout, stderr, exitCode } = await prune(dir, "--production"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + Nothing to prune." + `); + expect(out(stderr)).toContain(WARN("node_modules/no-deps", "node_modules/one-fixed-dep/node_modules/no-deps")); + expect(out(stderr)).toContain(NOTE); + expect(exitCode).toBe(0); + expect(await file(nestedPkgJson).json()).toMatchObject({ version: "1.0.0" }); + expect(await file(rootPkgJson).json()).toMatchObject({ version: "2.0.0" }); + await runBunInstall(installEnv(dir), dir, { production: true }); + + const silent = await prune(silentDir, "--production", "--silent"); + expect(silent.stdout).toBe(""); + expect(silent.stderr).not.toContain("warn:"); + expect(silent.stderr).not.toContain("note:"); + expect(silent.exitCode).toBe(0); + expect( + existsSync(join(silentDir, "node_modules", "one-fixed-dep", "node_modules", "no-deps", "package.json")), + ).toBeTrue(); + }, +); + +test.concurrent( + "hoisted: inside a tree folder, junk is removed but a copy shadowed by a mismatched root package is kept", + async () => { + const dir = await setup({ + name: "foo", + dependencies: { "one-dep": "1.0.0", "no-deps": "2.0.0", "a-dep": "1.0.2" }, + }); + const nm = join(dir, "node_modules"); + const nestedNoDeps = join(nm, "one-dep", "node_modules", "no-deps"); + expect(await file(join(nestedNoDeps, "package.json")).json()).toMatchObject({ version: "1.0.1" }); + expect(await file(join(nm, "a-dep", "package.json")).json()).toMatchObject({ version: "1.0.2" }); + + await write( + join(dir, "package.json"), + JSON.stringify({ name: "foo", dependencies: { "one-dep": "1.0.0", "no-deps": "2.0.0", "a-dep": "1.0.1" } }), + ); + await install(dir, "--lockfile-only"); + const shadowed = plant(dir, "node_modules/one-dep/node_modules/a-dep"); + const junk = plant(dir, "node_modules/one-dep/node_modules/junk"); + + const { stdout, stderr, exitCode } = await prune(dir); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/one-dep/node_modules/junk + Removed 1 package" + `); + expect(out(stderr)).toContain(WARN("node_modules/a-dep", "node_modules/one-dep/node_modules/a-dep")); + expect(out(stderr)).toContain(NOTE); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + expect(existsSync(shadowed)).toBeTrue(); + expect(await file(join(nestedNoDeps, "package.json")).json()).toMatchObject({ version: "1.0.1" }); + expect(await file(join(nm, "a-dep", "package.json")).json()).toMatchObject({ version: "1.0.2" }); + }, +); + +test.concurrent( + "hoisted + workspaces: --production keeps a workspace's copy while the root still holds the dev version", + async () => { + const { packageDir: dir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); + await Promise.all([ + write( + packageJson, + JSON.stringify({ name: "root", workspaces: ["packages/*"], devDependencies: { "no-deps": "2.0.0" } }), + ), + write( + join(dir, "packages", "a", "package.json"), + JSON.stringify({ name: "a", version: "1.0.0", dependencies: { "no-deps": "1.0.0" } }), + ), + ]); + await install(dir, "--linker", "hoisted"); + const nm = join(dir, "node_modules"); + const rootPkgJson = join(nm, "no-deps", "package.json"); + const workspacePkgJson = join(dir, "packages", "a", "node_modules", "no-deps", "package.json"); + expect(await file(rootPkgJson).json()).toMatchObject({ version: "2.0.0" }); + expect(await file(workspacePkgJson).json()).toMatchObject({ version: "1.0.0" }); + + const { stdout, stderr, exitCode } = await prune(dir, "--production", "--linker", "hoisted"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + Nothing to prune." + `); + expect(out(stderr)).toContain(WARN("node_modules/no-deps", "packages/a/node_modules/no-deps")); + expect(out(stderr)).toContain(NOTE); + expect(exitCode).toBe(0); + expect(await file(workspacePkgJson).json()).toMatchObject({ version: "1.0.0" }); + expect(isSymlink(join(nm, "a"))).toBeTrue(); + expect(await file(rootPkgJson).json()).toMatchObject({ version: "2.0.0" }); + }, +); + +test.concurrent( + "isolated: --production removes a dev-only link and its bin even though production still needs the store entry", + async () => { + // no-deps-bins' tarball lacks its bin file, and bun install skips bin links whose target is missing; what-bin ships one. + const dir = await setupWithLinker("isolated", { + name: "foo", + dependencies: { "uses-what-bin": "1.0.0" }, + devDependencies: { "what-bin": "1.0.0" }, + }); + const nm = join(dir, "node_modules"); + const store = join(nm, ".bun"); + expect(isSymlink(join(nm, "what-bin"))).toBeTrue(); + expect(existsSync(join(store, "what-bin@1.0.0"))).toBeTrue(); + expectBinInstalled(nm, "what-bin"); + + const dryRun = await prune(dir, "--production", "--dry-run", "--linker", "isolated"); + expect(out(dryRun.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/what-bin + Would remove 1 package" + `); + expect(dryRun.exitCode).toBe(0); + expect(isSymlink(join(nm, "what-bin"))).toBeTrue(); + expectBinInstalled(nm, "what-bin"); + + const { stdout, exitCode } = await prune(dir, "--production", "--linker", "isolated"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/what-bin + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(() => lstatSync(join(nm, "what-bin"))).toThrow(); + expect(existsSync(join(store, "what-bin@1.0.0"))).toBeTrue(); + expect(existsSync(join(store, "uses-what-bin@1.0.0", "node_modules", "what-bin", "package.json"))).toBeTrue(); + expect(existsSync(join(nm, "uses-what-bin", "package.json"))).toBeTrue(); + expectBinRemoved(nm, "what-bin"); + + const again = await prune(dir, "--production", "--linker", "isolated"); + expect(out(again.stdout)).toEndWith("Nothing to prune."); + expect(again.exitCode).toBe(0); + await runBunInstall(installEnv(dir), dir, { production: true }); + }, +); // pnpm#13676 test.concurrent("hoisted: nested node_modules of packages without a tree node are pruned", async () => { @@ -1156,7 +1320,7 @@ test.concurrent.each(["hoisted", "isolated"] as Linker[])( if (linker === "isolated") { expect(existsSync(join(nm, ".bun", "no-deps@1.0.0"))).toBeTrue(); } - await runBunInstall(bunEnv, dir, { production: true }); + await runBunInstall(installEnv(dir), dir, { production: true }); }, ); diff --git a/test/cli/install/bun-remove.test.ts b/test/cli/install/bun-remove.test.ts index 15abd2ea5056..9f54d9830848 100644 --- a/test/cli/install/bun-remove.test.ts +++ b/test/cli/install/bun-remove.test.ts @@ -1,7 +1,8 @@ import { file, spawn } from "bun"; import { afterAll, beforeAll, expect, it } from "bun:test"; +import { existsSync } from "fs"; import { mkdir, writeFile } from "fs/promises"; -import { bunExe, bunEnv as env, tmpdirSync } from "harness"; +import { bunExe, bunEnv as env, tempDir, tmpdirSync } from "harness"; import { join, relative } from "path"; import { createTestContext, destroyTestContext, dummyAfterAll, dummyBeforeAll } from "./dummy.registry"; @@ -340,3 +341,83 @@ it.concurrent("should remove peerDependencies", async () => { destroyTestContext(ctx); } }); + +const local = (name: string) => ({ [`${name}/package.json`]: JSON.stringify({ name, version: "1.0.0" }) }); + +async function remove(dir: string, ...names: string[]) { + await using proc = spawn({ + cmd: [bunExe(), "remove", ...names], + cwd: dir, + env, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +it.concurrent("bun remove drops every duplicate key of the removed package", async () => { + using dir = tempDir("bun-remove-dup", { + ...local("foo"), + ...local("bar"), + "package.json": `{ + "name": "dup", + "dependencies": { + "foo": "file:./foo", + "bar": "file:./bar", + "foo": "file:./foo" + } +} +`, + }); + + const { stderr, exitCode } = await remove(String(dir), "foo"); + + expect(stderr).not.toContain("error:"); + expect(await file(join(String(dir), "package.json")).json()).toEqual({ + name: "dup", + dependencies: { bar: "file:./bar" }, + }); + expect(existsSync(join(String(dir), "node_modules", "foo"))).toBe(false); + expect(existsSync(join(String(dir), "node_modules", "bar", "package.json"))).toBe(true); + expect(exitCode).toBe(0); +}); + +it.concurrent("bun remove drops the list when the removed package is its only (duplicated) entry", async () => { + using dir = tempDir("bun-remove-dup-only", { + ...local("foo"), + ...local("bar"), + "package.json": `{ + "name": "dup", + "dependencies": { "bar": "file:./bar" }, + "devDependencies": { + "foo": "file:./foo", + "foo": "file:./foo" + } +} +`, + }); + + const { stderr, exitCode } = await remove(String(dir), "foo"); + + expect(stderr).not.toContain("error:"); + expect(await file(join(String(dir), "package.json")).json()).toEqual({ + name: "dup", + dependencies: { bar: "file:./bar" }, + }); + expect(exitCode).toBe(0); +}); + +it.concurrent( + "bun remove rejects a dependency list that is not an object and leaves package.json untouched", + async () => { + const pkg = JSON.stringify({ name: "x", dependencies: ["foo"], devDependencies: { bar: "file:./bar" } }); + using dir = tempDir("bun-remove-malformed", { ...local("bar"), "package.json": pkg }); + + const { stderr, exitCode } = await remove(String(dir), "bar"); + + expect(stderr).toContain("dependencies expects a map of specifiers"); + expect(await file(join(String(dir), "package.json")).text()).toBe(pkg); + expect(exitCode).toBe(1); + }, +); diff --git a/test/cli/install/bun-security-scanner-matrix-runner.ts b/test/cli/install/bun-security-scanner-matrix-runner.ts index 399829cbab50..13d0e877e03b 100644 --- a/test/cli/install/bun-security-scanner-matrix-runner.ts +++ b/test/cli/install/bun-security-scanner-matrix-runner.ts @@ -140,6 +140,9 @@ async function runSecurityScannerTest(options: SecurityScannerTestOptions) { } : {}), + // `bun update ` only updates a declared dependency; it never adds one + ...(command === "update" ? Object.fromEntries(args.map(arg => [arg, SimpleRegistry.packages[arg][0]])) : {}), + // For npm scanner, add it to dependencies so it gets installed ...(scannerType === "npm" ? { diff --git a/test/cli/install/bun-update-lockfile-sync.test.ts b/test/cli/install/bun-update-lockfile-sync.test.ts new file mode 100644 index 000000000000..fa4cd07dc358 --- /dev/null +++ b/test/cli/install/bun-update-lockfile-sync.test.ts @@ -0,0 +1,559 @@ +import { Archive, file, write } from "bun"; +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { exists } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe, runBunInstall, runBunUpdate, tempDir } from "harness"; +import { join } from "path"; + +// Registry: no-deps 1.0.0/1.0.1/1.1.0/2.0.0, @types/no-deps 1.0.0/2.0.0, a-dep 1.0.1..1.0.10, one-range-dep@1.0.0 -> no-deps ^1.0.0. + +const verdaccio = new VerdaccioRegistry(); + +beforeAll(async () => { + await verdaccio.start(); +}); + +afterAll(() => { + verdaccio.stop(); +}); + +type Json = Record; +const GROUPS = ["dependencies", "devDependencies", "optionalDependencies", "peerDependencies"] as const; + +function json(contents: Json | string) { + return typeof contents === "string" ? contents : JSON.stringify(contents, null, 2) + "\n"; +} + +async function setup( + files: Record, + opts: { exact?: boolean; text?: boolean; install?: boolean; allowWarnings?: boolean } = {}, +): Promise { + const dir = String( + tempDir( + "lockfile-sync-", + Object.fromEntries(Object.entries(files).map(([path, contents]) => [path, json(contents)])), + ), + ); + await write( + join(dir, "bunfig.toml"), + Bun.TOML.stringify({ + install: { + cache: join(dir, ".bun-cache"), + registry: verdaccio.registryUrl(), + saveTextLockfile: opts.text ?? true, + linker: "hoisted", + exact: opts.exact, + }, + }), + ); + if (opts.install !== false) await runBunInstall(bunEnv, dir, { allowWarnings: opts.allowWarnings }); + return dir; +} + +async function run(cwd: string, ...args: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...args], + cwd, + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + stdin: "ignore", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + return { stdout, stderr, exitCode }; +} + +const pkg = (dir: string, rel = ""): Promise => file(join(dir, rel, "package.json")).json(); +const writePkg = (dir: string, contents: Json, rel = "") => write(join(dir, rel, "package.json"), json(contents)); +const installed = (dir: string, name: string): Promise => + file(join(dir, "node_modules", name, "package.json")).json(); +const lockText = (dir: string) => file(join(dir, "bun.lock")).text(); +const lock = async (dir: string): Promise => Bun.JSONC.parse(await lockText(dir)) as Json; + +function declaredLiteral(manifest: Json, name: string): string | undefined { + for (const group of GROUPS) { + const literal = manifest[group]?.[name]; + if (literal !== undefined) return literal; + } +} + +async function expectInSync(dir: string, workspaces: string[] = [""], allowWarnings = false) { + const lockfile = await lock(dir); + for (const key of workspaces) { + const manifest = await pkg(dir, key); + for (const group of GROUPS) { + if (manifest[group] === undefined) continue; + expect({ [key || "."]: { [group]: lockfile.workspaces[key]?.[group] } }).toEqual({ + [key || "."]: { [group]: manifest[group] }, + }); + } + if (key !== "") continue; + const overrides = manifest.overrides ?? manifest.resolutions; + if (overrides !== undefined) { + const expected = Object.fromEntries( + Object.entries(overrides).map(([name, value]) => [ + name, + typeof value === "string" && value.startsWith("$") ? declaredLiteral(manifest, value.slice(1)) : value, + ]), + ); + expect(lockfile.overrides).toEqual(expected); + } + const catalog = manifest.workspaces?.catalog ?? manifest.catalog; + if (catalog !== undefined) expect(lockfile.catalog).toEqual(catalog); + const catalogs = manifest.workspaces?.catalogs ?? manifest.catalogs; + if (catalogs !== undefined) expect(lockfile.catalogs).toEqual(catalogs); + } + await runBunInstall(bunEnv, dir, { frozenLockfile: true, allowWarnings }); + const before = await lockText(dir); + const { err } = await runBunInstall(bunEnv, dir, { savesLockfile: false, allowWarnings }); + expect(err).not.toContain("Saved lockfile"); + expect(await lockText(dir)).toBe(before); +} + +const root = (fields: Json): Json => ({ name: "foo", ...fields }); + +const MONOREPO = (pkg1: Json = {}, rootFields: Json = {}) => ({ + "package.json": { name: "root", workspaces: ["packages/*"], ...rootFields }, + "packages/pkg1/package.json": { name: "pkg1", version: "1.0.0", ...pkg1 }, +}); +const PKG1 = "packages/pkg1"; + +describe.concurrent("bun update rewrites bun.lock together with package.json", () => { + test("bun update", async () => { + const dir = await setup({ + "package.json": root({ dependencies: { "no-deps": "^1.0.0", aliased: "npm:no-deps@~1.0.0" } }), + }); + await run(dir, "update"); + const expected = { "no-deps": "^1.1.0", aliased: "npm:no-deps@~1.0.1" }; + expect((await pkg(dir)).dependencies).toEqual(expected); + expect((await lock(dir)).workspaces[""].dependencies).toEqual(expected); + await expectInSync(dir); + }); + + test("bun update --latest", async () => { + const dir = await setup({ + "package.json": root({ dependencies: { "no-deps": "~1.0.0", aliased: "npm:no-deps@~1.0.0" } }), + }); + await run(dir, "update", "--latest"); + const expected = { "no-deps": "~2.0.0", aliased: "npm:no-deps@~2.0.0" }; + expect((await pkg(dir)).dependencies).toEqual(expected); + const ws = (await lock(dir)).workspaces[""]; + expect(ws.dependencies).toEqual(expected); + expect(JSON.stringify(ws)).not.toContain("latest"); + await expectInSync(dir); + }); + + test("bun update keeps the pin style", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "~1.0.0" } }) }); + await run(dir, "update", "no-deps"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "~1.0.1" }); + expect((await lock(dir)).workspaces[""].dependencies).toEqual({ "no-deps": "~1.0.1" }); + await expectInSync(dir); + }); + + test("bun update on an exact literal", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "1.0.0" } }) }); + await run(dir, "update", "no-deps"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "1.0.0" }); + expect((await lock(dir)).workspaces[""].dependencies).toEqual({ "no-deps": "1.0.0" }); + await expectInSync(dir); + }); + + test("bun update keeps the alias", async () => { + const dir = await setup({ "package.json": root({ dependencies: { aliased: "npm:no-deps@~1.0.0" } }) }); + await run(dir, "update", "aliased"); + expect((await pkg(dir)).dependencies).toEqual({ aliased: "npm:no-deps@~1.0.1" }); + expect((await lock(dir)).workspaces[""].dependencies).toEqual({ aliased: "npm:no-deps@~1.0.1" }); + await expectInSync(dir); + }); + + test("bun update @", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "~1.0.0" } }) }); + await run(dir, "update", "no-deps@^1.0.0"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "~1.1.0" }); + expect((await lock(dir)).workspaces[""].dependencies).toEqual({ "no-deps": "~1.1.0" }); + await expectInSync(dir); + }); + + // `bun install` warns about a name declared in two groups, so these two allow warnings. + const inBothGroups = (version: string) => + root({ dependencies: { "no-deps": version }, devDependencies: { "no-deps": version } }); + + test("bun update with the name in dependencies and devDependencies", async () => { + const dir = await setup({ "package.json": inBothGroups("~1.0.0") }, { allowWarnings: true }); + await run(dir, "update", "no-deps"); + const manifest = await pkg(dir); + expect(manifest.dependencies).toEqual({ "no-deps": "~1.0.1" }); + expect(manifest.devDependencies).toEqual({ "no-deps": "~1.0.0" }); + await expectInSync(dir, [""], true); + }); + + test("bun update with the name in dependencies and devDependencies moves one group", async () => { + const dir = await setup({ "package.json": inBothGroups("1.0.0") }, { allowWarnings: true }); + await writePkg(dir, inBothGroups("~1.0.0")); + const { out } = await runBunUpdate(bunEnv, dir); + expect(out.join("\n")).toMatch(/no-deps 1\.0\.0 (→|->) 1\.0\.1/); + const manifest = await pkg(dir); + expect([manifest.dependencies["no-deps"], manifest.devDependencies["no-deps"]].sort()).toEqual([ + "~1.0.0", + "~1.0.1", + ]); + await expectInSync(dir, [""], true); + }); + + test("install.exact", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "^1.0.0" } }) }, { exact: true }); + await run(dir, "update"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "1.1.0" }); + expect((await lock(dir)).workspaces[""].dependencies).toEqual({ "no-deps": "1.1.0" }); + await expectInSync(dir); + }); + + test.each([[[]], [["--latest"]]])("bun update %j leaves folder, tarball and workspace literals alone", async args => { + const dependencies = { + "no-deps": "^1.0.0", + "folder-dep": "file:./folder-target", + "tgz-dep": "file:./tgz-dep-1.0.0.tgz", + pkg1: "workspace:*", + }; + const dir = await setup( + { + ...MONOREPO({}, { dependencies }), + "folder-target/package.json": { name: "folder-dep", version: "1.0.0" }, + }, + { install: false }, + ); + await Archive.write( + join(dir, "tgz-dep-1.0.0.tgz"), + { "package/package.json": JSON.stringify({ name: "tgz-dep", version: "1.0.0" }) }, + { compress: "gzip" }, + ); + await runBunInstall(bunEnv, dir); + await run(dir, "update", ...args); + const expected = { ...dependencies, "no-deps": args.length ? "^2.0.0" : "^1.1.0" }; + expect((await pkg(dir)).dependencies).toEqual(expected); + expect((await lock(dir)).workspaces[""].dependencies).toEqual(expected); + await expectInSync(dir, ["", PKG1]); + }); + + test("bun update -r", async () => { + const dir = await setup(MONOREPO({ dependencies: { "no-deps": "~1.0.0" } })); + await run(dir, "update", "-r"); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "~1.0.1" }); + expect((await lock(dir)).workspaces[PKG1].dependencies).toEqual({ "no-deps": "~1.0.1" }); + await expectInSync(dir, ["", PKG1]); + }); + + test("bun update from a workspace member", async () => { + const dir = await setup(MONOREPO({ dependencies: { "no-deps": "~1.0.0" } })); + await run(join(dir, PKG1), "update"); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "~1.0.1" }); + expect((await lock(dir)).workspaces[PKG1].dependencies).toEqual({ "no-deps": "~1.0.1" }); + await expectInSync(dir, ["", PKG1]); + }); +}); + +describe.concurrent("npm: aliases", () => { + test.each([ + { + before: "npm:no-deps@~1.0.0", + args: ["aliased", "--latest"], + after: "npm:no-deps@~2.0.0", + installs: ["no-deps", "2.0.0"], + }, + { before: "npm:no-deps", args: [], after: "npm:no-deps@^2.0.0", installs: ["no-deps", "2.0.0"] }, + { before: "npm:no-deps", args: ["aliased"], after: "npm:no-deps@^2.0.0", installs: ["no-deps", "2.0.0"] }, + { + before: "npm:@types/no-deps", + args: ["--latest"], + after: "npm:@types/no-deps@^2.0.0", + installs: ["@types/no-deps", "2.0.0"], + }, + { + before: "npm:no-deps@~1.0.0", + args: ["aliased@2.0.0"], + after: "npm:no-deps@~2.0.0", + installs: ["no-deps", "2.0.0"], + }, + { before: "npm:no-deps@^1.0.0", args: ["aliased"], after: "npm:no-deps@^1.1.0", installs: ["no-deps", "1.1.0"] }, + ])('"aliased": "$before" + bun update $args -> "$after"', async ({ before, args, after, installs }) => { + const dir = await setup({ "package.json": root({ dependencies: { aliased: before } }) }); + await run(dir, "update", ...args); + expect((await pkg(dir)).dependencies.aliased).toBe(after); + const [name, version] = installs; + expect(await installed(dir, "aliased")).toMatchObject({ name, version }); + await expectInSync(dir); + }); + + test("bun update @npm: retargets the alias", async () => { + const dir = await setup({ "package.json": root({ dependencies: { aliased: "npm:no-deps@~1.0.0" } }) }); + await run(dir, "update", "aliased@npm:a-dep"); + expect((await pkg(dir)).dependencies).toEqual({ aliased: "npm:a-dep@~1.0.10" }); + expect(await installed(dir, "aliased")).toMatchObject({ name: "a-dep", version: "1.0.10" }); + await expectInSync(dir); + }); + + test("bun update @npm:@ refuses to add; bun add keeps the target", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "a-dep": "1.0.1" } }) }); + const [pkgBefore, lockBefore] = await Promise.all([file(join(dir, "package.json")).text(), lockText(dir)]); + await using proc = Bun.spawn({ + cmd: [bunExe(), "update", "new-alias@npm:@types/no-deps@^1.0.0"], + cwd: dir, + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + stdin: "ignore", + }); + const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); + expect(stderr).toContain('error: "new-alias" is not in the lockfile, so there is nothing to update'); + expect(exitCode).toBe(1); + expect(await file(join(dir, "package.json")).text()).toBe(pkgBefore); + expect(await lockText(dir)).toBe(lockBefore); + + await run(dir, "add", "new-alias@npm:@types/no-deps@^1.0.0"); + const expected = { "a-dep": "1.0.1", "new-alias": "npm:@types/no-deps@^1.0.0" }; + expect((await pkg(dir)).dependencies).toEqual(expected); + expect((await lock(dir)).workspaces[""].dependencies).toEqual(expected); + expect(await installed(dir, "new-alias")).toMatchObject({ name: "@types/no-deps", version: "1.0.0" }); + await expectInSync(dir); + }); + + test("bun update @npm:@ retargets a scoped alias in the declared pin style", async () => { + const dir = await setup({ "package.json": root({ dependencies: { aliased: "npm:no-deps@~1.0.0" } }) }); + await run(dir, "update", "aliased@npm:@types/no-deps@^1.0.0"); + expect((await pkg(dir)).dependencies).toEqual({ aliased: "npm:@types/no-deps@~1.0.0" }); + expect(await installed(dir, "aliased")).toMatchObject({ name: "@types/no-deps", version: "1.0.0" }); + await expectInSync(dir); + }); +}); + +describe.concurrent("bun add", () => { + test.each([ + { args: ["no-deps"], expected: { "no-deps": "^2.0.0" } }, + { args: ["no-deps", "--exact"], expected: { "no-deps": "2.0.0" } }, + { args: ["no-deps@~1.0.0"], expected: { "no-deps": "~1.0.0" } }, + { args: ["no-deps@latest"], expected: { "no-deps": "^2.0.0" } }, + { args: ["x@npm:no-deps@~1.0.0"], expected: { x: "npm:no-deps@~1.0.0" } }, + { args: ["x@npm:no-deps@latest"], expected: { x: "npm:no-deps@^2.0.0" } }, + { args: ["x@npm:no-deps"], expected: { x: "npm:no-deps" } }, + ])("bun add $args", async ({ args, expected }) => { + const dir = await setup({ "package.json": root({}) }, { install: false }); + await run(dir, "add", ...args); + expect((await pkg(dir)).dependencies).toEqual(expected); + expect((await lock(dir)).workspaces[""].dependencies).toEqual(expected); + await expectInSync(dir); + }); + + test("bun add @workspace:*", async () => { + const dir = await setup(MONOREPO()); + await run(dir, "add", "pkg1@workspace:*"); + expect((await pkg(dir)).dependencies).toEqual({ pkg1: "workspace:*" }); + expect((await lock(dir)).workspaces[""].dependencies).toEqual({ pkg1: "workspace:*" }); + await expectInSync(dir, ["", PKG1]); + }); + + test("bun add --filter", async () => { + const dir = await setup(MONOREPO()); + await run(dir, "add", "x@npm:no-deps@latest", "--filter", "pkg1"); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ x: "npm:no-deps@^2.0.0" }); + expect((await lock(dir)).workspaces[PKG1].dependencies).toEqual({ x: "npm:no-deps@^2.0.0" }); + await expectInSync(dir, ["", PKG1]); + }); + + test("bun add --lockfile-only", async () => { + const dir = await setup({ "package.json": root({}) }, { install: false }); + await run(dir, "add", "no-deps", "--lockfile-only"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "^2.0.0" }); + expect(await exists(join(dir, "node_modules"))).toBe(false); + await expectInSync(dir); + }); + + test("bun add --trust", async () => { + const dir = await setup({ "package.json": root({}) }, { install: false }); + await run(dir, "add", "uses-what-bin@1.0.0", "--trust"); + expect(await pkg(dir)).toEqual({ + name: "foo", + dependencies: { "uses-what-bin": "1.0.0" }, + trustedDependencies: ["uses-what-bin"], + }); + expect(await exists(join(dir, "node_modules", "uses-what-bin", "what-bin.txt"))).toBe(true); + await expectInSync(dir); + }); + + test("bun add --trust of a package already listed in devDependencies", async () => { + const dir = await setup({ "package.json": root({ devDependencies: { "uses-what-bin": "1.0.0" } }) }); + await run(dir, "add", "uses-what-bin@1.0.0", "--trust"); + const manifest = await pkg(dir); + expect(manifest.dependencies).toBeUndefined(); + expect(manifest.devDependencies).toEqual({ "uses-what-bin": "1.0.0" }); + expect(manifest.trustedDependencies).toEqual(["uses-what-bin"]); + expect(await exists(join(dir, "node_modules", "uses-what-bin", "what-bin.txt"))).toBe(true); + await expectInSync(dir); + }); + + test("bun add --dry-run writes neither file", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "a-dep": "1.0.1" } }) }); + const [pkgBefore, lockBefore] = await Promise.all([file(join(dir, "package.json")).text(), lockText(dir)]); + await run(dir, "add", "no-deps", "--dry-run"); + expect(await file(join(dir, "package.json")).text()).toBe(pkgBefore); + expect(await lockText(dir)).toBe(lockBefore); + }); +}); + +describe.concurrent("catalogs", () => { + const CATALOG_REPO = (catalog: Json = { "no-deps": "^1.0.0", aliased: "npm:no-deps" }) => + MONOREPO( + { dependencies: { "no-deps": "catalog:", aliased: "catalog:" } }, + { workspaces: { packages: ["packages/*"], catalog } }, + ); + + test("bun update", async () => { + const dir = await setup(CATALOG_REPO()); + await run(dir, "update"); + const expected = { "no-deps": "^1.1.0", aliased: "npm:no-deps@^2.0.0" }; + expect((await pkg(dir)).workspaces.catalog).toEqual(expected); + expect((await lock(dir)).catalog).toEqual(expected); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "catalog:", aliased: "catalog:" }); + await expectInSync(dir, ["", PKG1]); + }); + + test.each([[""], [PKG1]])("bun update --latest from '%s' installs offline afterwards", async cwd => { + const dir = await setup(CATALOG_REPO()); + await run(join(dir, cwd), "update", "--latest"); + const expected = { "no-deps": "^2.0.0", aliased: "npm:no-deps@^2.0.0" }; + expect((await pkg(dir)).workspaces.catalog).toEqual(expected); + const lockfile = await lock(dir); + expect(lockfile.catalog).toEqual(expected); + expect(JSON.stringify(lockfile)).not.toContain('"latest"'); + await expectInSync(dir, ["", PKG1]); + + await write( + join(dir, "bunfig.toml"), + Bun.TOML.stringify({ + install: { cache: join(dir, ".bun-cache"), registry: "http://127.0.0.1:1/", saveTextLockfile: true }, + }), + ); + await run(join(dir, cwd), "install", "--frozen-lockfile"); + }); + + test("bun add --catalog --filter", async () => { + const dir = await setup(CATALOG_REPO()); + await run(dir, "add", "a-dep", "--catalog", "--filter", "pkg1"); + expect((await pkg(dir)).workspaces.catalog["a-dep"]).toBe("^1.0.10"); + expect((await lock(dir)).catalog["a-dep"]).toBe("^1.0.10"); + expect((await pkg(dir, PKG1)).dependencies["a-dep"]).toBe("catalog:"); + await expectInSync(dir, ["", PKG1]); + }); +}); + +describe.concurrent("$ref overrides", () => { + test("$name follows the rewritten dependency", async () => { + const overrides = { "no-deps": "$no-deps" }; + const dir = await setup({ + "package.json": root({ dependencies: { "no-deps": "1.0.0", "one-range-dep": "1.0.0" }, overrides }), + }); + await writePkg(dir, root({ dependencies: { "no-deps": "^1.0.0", "one-range-dep": "1.0.0" }, overrides })); + await run(dir, "update"); + const manifest = await pkg(dir); + expect(manifest.dependencies).toEqual({ "no-deps": "^1.1.0", "one-range-dep": "1.0.0" }); + expect(manifest.overrides).toEqual(overrides); + expect((await lock(dir)).overrides).toEqual({ "no-deps": "^1.1.0" }); + await expectInSync(dir); + }); + + test.each([["^1.0.1"], ["^1.0.0"]])("literal override %s stays as written", async override => { + const dir = await setup({ + "package.json": root({ dependencies: { "no-deps": "^1.0.0" }, overrides: { "no-deps": override } }), + }); + await run(dir, "update"); + const manifest = await pkg(dir); + expect(manifest.dependencies).toEqual({ "no-deps": "^1.1.0" }); + expect(manifest.overrides).toEqual({ "no-deps": override }); + expect((await lock(dir)).overrides).toEqual({ "no-deps": override }); + await expectInSync(dir); + }); + + test("$alias follows the rewritten alias", async () => { + const dir = await setup({ + "package.json": root({ dependencies: { a1: "npm:no-deps@^1.0.0" }, overrides: { a1: "$a1" } }), + }); + await run(dir, "update"); + expect((await pkg(dir)).dependencies).toEqual({ a1: "npm:no-deps@^1.1.0" }); + expect((await lock(dir)).overrides).toEqual({ a1: "npm:no-deps@^1.1.0" }); + await expectInSync(dir); + }); +}); + +describe.concurrent("bumping a direct dependency re-points its dependents", () => { + const nested = (dir: string) => exists(join(dir, "node_modules", "one-range-dep", "node_modules")); + const deps = (noDeps: string) => root({ dependencies: { "one-range-dep": "1.0.0", "no-deps": noDeps } }); + + test.each([ + ["text", true], + ["binary", false], + ])("bun install after editing package.json (%s lockfile)", async (_, text) => { + const dir = await setup({ "package.json": deps("1.0.0") }, { text }); + expect(await installed(dir, "no-deps")).toMatchObject({ version: "1.0.0" }); + expect(await nested(dir)).toBe(false); + + await writePkg(dir, deps("1.0.1")); + await runBunInstall(bunEnv, dir); + expect(await installed(dir, "no-deps")).toMatchObject({ version: "1.0.1" }); + expect(await nested(dir)).toBe(false); + if (text) { + const { packages } = await lock(dir); + expect(Object.keys(packages).sort()).toEqual(["no-deps", "one-range-dep"]); + expect(packages["no-deps"][0]).toBe("no-deps@1.0.1"); + } + + await writePkg(dir, deps("2.0.0")); + await runBunInstall(bunEnv, dir); + expect(await installed(dir, "no-deps")).toMatchObject({ version: "2.0.0" }); + expect(await installed(dir, "one-range-dep/node_modules/no-deps")).toMatchObject({ version: "1.0.1" }); + if (text) { + const { packages } = await lock(dir); + expect(packages["no-deps"][0]).toBe("no-deps@2.0.0"); + expect(packages["one-range-dep/no-deps"][0]).toBe("no-deps@1.0.1"); + } + }); + + test("declared by a workspace member", async () => { + const dir = await setup(MONOREPO({ dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" } })); + expect((await lock(dir)).packages["no-deps"][0]).toBe("no-deps@1.0.0"); + + await writePkg( + dir, + { name: "pkg1", version: "1.0.0", dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.1" } }, + PKG1, + ); + await runBunInstall(bunEnv, dir); + const { packages } = await lock(dir); + expect(packages["no-deps"][0]).toBe("no-deps@1.0.1"); + expect(packages["one-range-dep/no-deps"]).toBeUndefined(); + expect(await nested(dir)).toBe(false); + }); + + test("bun update", async () => { + const dir = await setup({ "package.json": deps("1.0.0") }); + await writePkg(dir, deps("^1.0.0")); + await run(dir, "update"); + const { packages } = await lock(dir); + expect(packages["no-deps"][0]).toBe("no-deps@1.1.0"); + expect(packages["one-range-dep/no-deps"]).toBeUndefined(); + expect(await nested(dir)).toBe(false); + await expectInSync(dir); + }); + + test("a dependency added later never drags dependents along", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "one-range-dep": "1.0.0" } }) }); + expect((await lock(dir)).packages["no-deps"][0]).toBe("no-deps@1.1.0"); + + await writePkg(dir, deps("1.0.0")); + await runBunInstall(bunEnv, dir); + const { packages } = await lock(dir); + expect(packages["no-deps"][0]).toBe("no-deps@1.0.0"); + expect(packages["one-range-dep/no-deps"][0]).toBe("no-deps@1.1.0"); + await expectInSync(dir); + }); +}); diff --git a/test/cli/install/bun-update-transitive.test.ts b/test/cli/install/bun-update-transitive.test.ts new file mode 100644 index 000000000000..50dba8f44495 --- /dev/null +++ b/test/cli/install/bun-update-transitive.test.ts @@ -0,0 +1,533 @@ +import { file, write } from "bun"; +import { afterAll, beforeAll, expect, test } from "bun:test"; +import { VerdaccioRegistry, bunEnv, bunExe, normalizeBunSnapshot, tempDir } from "harness"; +import { join } from "path"; + +// Registry: no-deps 1.0.0 / 1.0.1 / 1.1.0 / 2.0.0; one-range-dep@1.0.0 depends on `no-deps: ^1.0.0`; +// one-fixed-dep@1.0.0 depends on `no-deps: 1.0.0`; dep-with-tags has 3.0.1 published above its `latest` (3.0.0); +// prereleases-1 has 1.0.0-future.7 published above its `latest` (1.0.0-future.4). + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +type Json = Record; +type Linker = "hoisted" | "isolated"; +type Layout = { text?: boolean; linker?: Linker }; + +const pkgJson = (dependencies: Json, extra: Json = {}) => ({ name: "foo", dependencies, ...extra }); +const stringify = (json: Json) => JSON.stringify(json, null, 2) + "\n"; + +const linkerArgs = (layout: Layout) => ["--linker", layout.linker ?? "hoisted"]; + +async function run(dir: string, ...args: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...args], + cwd: dir, + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + stdin: "ignore", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +async function install(dir: string, ...args: string[]) { + const { stderr, exitCode } = await run(dir, "install", ...args); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + return stderr; +} + +async function setup(files: Record, layout: Layout = {}) { + const { packageDir } = await registry.createTestDir({ + bunfigOpts: { saveTextLockfile: layout.text ?? true, linker: layout.linker ?? "hoisted" }, + files: Object.fromEntries(Object.entries(files).map(([path, json]) => [path, stringify(json)])), + }); + await install(packageDir, ...linkerArgs(layout)); + return packageDir; +} + +async function reinstall(dir: string, packageJson: Json, layout: Layout = {}, rel = "") { + await write(join(dir, rel, "package.json"), stringify(packageJson)); + expect(await install(dir, ...linkerArgs(layout))).toContain("Saved lockfile"); +} + +const packageJsonOf = (dir: string, rel = ""): Promise => file(join(dir, rel, "package.json")).json(); +const packageJsonText = (dir: string, rel = "") => file(join(dir, rel, "package.json")).text(); +const lockText = (dir: string) => file(join(dir, "bun.lock")).text(); +const lock = async (dir: string): Promise => Bun.JSONC.parse(await lockText(dir)) as Json; + +// Every version of `name` resolved anywhere in bun.lock. +async function lockedVersions(dir: string, name: string) { + const { packages } = await lock(dir); + const versions = Object.entries(packages as Record) + .filter(([key]) => key === name || key.endsWith(`/${name}`)) + .map(([, [resolution]]) => resolution.slice(name.length + 1)); + return [...new Set(versions)].sort(); +} + +async function installedVersion(dir: string, ...segments: string[]) { + return (await file(join(dir, "node_modules", ...segments, "package.json")).json()).version; +} + +const noDepsPath = (linker: Linker = "hoisted") => + linker === "isolated" ? [".bun", "one-range-dep@1.0.0", "node_modules", "no-deps"] : ["no-deps"]; + +// no-deps@1.0.0 survives being dropped from package.json because one-range-dep's `^1.0.0` edge is still satisfied, +// leaving a transitive dependency that a plain `bun install` never moves. +async function stale(layout: Layout = {}) { + const dir = await setup({ "package.json": pkgJson({ "one-range-dep": "1.0.0", "no-deps": "1.0.0" }) }, layout); + const packageJson = pkgJson({ "one-range-dep": "1.0.0" }); + await reinstall(dir, packageJson, layout); + const noDeps = noDepsPath(layout.linker); + if (layout.text ?? true) { + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + } + expect(await installedVersion(dir, ...noDeps)).toBe("1.0.0"); + return { dir, packageJson, noDeps }; +} + +async function expectTransitiveBump( + { dir, packageJson, noDeps }: Awaited>, + layout: Layout, + ...args: string[] +) { + const { stdout, stderr, exitCode } = await run(dir, "update", ...args, ...linkerArgs(layout)); + expect(stderr).not.toContain("error:"); + expect(stderr).toContain("Saved lockfile"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect(await installedVersion(dir, ...noDeps)).toBe("1.1.0"); + if (layout.text ?? true) { + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + expect(await lockedVersions(dir, "one-range-dep")).toStrictEqual(["1.0.0"]); + } + await install(dir, "--frozen-lockfile", ...linkerArgs(layout)); + expect(exitCode).toBe(0); + return stdout; +} + +test.concurrent.each<[string, Layout]>([ + ["text lockfile", { text: true }], + ["binary lockfile", { text: false }], + ["text lockfile + isolated linker", { text: true, linker: "isolated" }], +])("`bun update` moves a transitive dependency within its dependent's range (%s)", async (_, layout) => { + const fixture = await stale(layout); + const stdout = await expectTransitiveBump(fixture, layout); + expect(normalizeBunSnapshot(stdout.split("\n").slice(0, 3).join("\n"))).toMatchInlineSnapshot(` + "bun update () + updating: + no-deps@1.0.0 → 1.1.0" + `); +}); + +test.concurrent("`bun update --latest` still moves transitive dependencies only within their ranges", async () => { + const fixture = await stale(); + const stdout = await expectTransitiveBump(fixture, {}, "--latest"); + expect(stdout).toContain(" no-deps@1.0.0 → 1.1.0\n"); +}); + +test.concurrent("`bun update ` reaches a package that is only a transitive dependency", async () => { + const fixture = await stale(); + const stdout = await expectTransitiveBump(fixture, {}, "no-deps"); + expect(stdout).not.toContain("updating:"); +}); + +test.concurrent("`bun update ` naming a package with nothing newer changes nothing", async () => { + const { dir, packageJson } = await stale(); + const before = await lock(dir); + const { stderr, exitCode } = await run(dir, "update", "one-range-dep"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect(await lock(dir)).toStrictEqual(before); + expect(exitCode).toBe(0); +}); + +test.concurrent("`bun update ` rejects a name that is not in the lockfile", async () => { + const { dir, packageJson } = await stale(); + const before = await lockText(dir); + const { stderr, exitCode } = await run(dir, "update", "does-not-exist"); + expect(stderr).toContain('error: "does-not-exist" is not in the lockfile, so there is nothing to update'); + expect(stderr).toContain("bun add"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect(await lockText(dir)).toBe(before); + expect(exitCode).toBe(1); +}); + +test.concurrent("`bun update --dry-run` prints the transitive plan and writes nothing", async () => { + const { dir, packageJson, noDeps } = await stale(); + const before = await lockText(dir); + const { stdout, stderr, exitCode } = await run(dir, "update", "--dry-run"); + expect(stdout).toContain(" no-deps@1.0.0 → 1.1.0\n"); + expect(stdout).toContain("Would update 1 package"); + expect(stderr).not.toContain("error:"); + expect(stderr).not.toContain("Saved lockfile"); + expect(await lockText(dir)).toBe(before); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect(await installedVersion(dir, ...noDeps)).toBe("1.0.0"); + expect(exitCode).toBe(0); +}); + +test.concurrent("a direct dependency's declared range is left alone when only its dependency moves", async () => { + const { dir } = await stale(); + await run(dir, "update"); + expect(await packageJsonOf(dir)).toStrictEqual(pkgJson({ "one-range-dep": "1.0.0" })); + expect((await lock(dir)).workspaces[""].dependencies).toStrictEqual({ "one-range-dep": "1.0.0" }); +}); + +test.concurrent("a transitive dependency pinned exactly by its dependent stays put", async () => { + const dir = await setup({ "package.json": pkgJson({ "one-fixed-dep": "1.0.0" }) }); + const before = await lock(dir); + const { stdout, stderr, exitCode } = await run(dir, "update"); + expect(stdout).not.toContain("updating:"); + expect(stderr).not.toContain("error:"); + expect(await lock(dir)).toStrictEqual(before); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + expect(exitCode).toBe(0); +}); + +// A root `no-deps@1.0.0` dedupes both dependents' edges onto 1.0.0 before it is dropped (a fresh install would already +// fork); `bun update` then moves only the `^1.0.0` edge, forking away from the sibling's exact pin. +test.concurrent("dependents with different ranges are resolved independently", async () => { + const dependents = { "one-fixed-dep": "1.0.0", "one-range-dep": "1.0.0" }; + const dir = await setup({ "package.json": pkgJson({ "no-deps": "1.0.0", ...dependents }) }); + const packageJson = pkgJson(dependents); + await reinstall(dir, packageJson); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + const { stdout, stderr, exitCode } = await run(dir, "update"); + expect(stdout).toContain(" no-deps@1.0.0 → 1.1.0\n"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0", "1.1.0"]); + const { packages } = await lock(dir); + expect([packages["no-deps"][0], packages["one-range-dep/no-deps"][0]]).toStrictEqual([ + "no-deps@1.0.0", + "no-deps@1.1.0", + ]); + await install(dir, "--frozen-lockfile"); + expect(exitCode).toBe(0); +}); + +test.concurrent("an override holds a transitive dependency back", async () => { + const { dir } = await stale(); + const packageJson = pkgJson({ "one-range-dep": "1.0.0" }, { overrides: { "no-deps": "1.0.0" } }); + await reinstall(dir, packageJson); + const before = await lock(dir); + const { stdout, stderr, exitCode } = await run(dir, "update"); + expect(stdout).not.toContain("updating:"); + expect(stderr).not.toContain("error:"); + expect(await lock(dir)).toStrictEqual(before); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + expect(exitCode).toBe(0); +}); + +test.concurrent("a lockfile that already resolves the newest allowed versions is left alone", async () => { + const dir = await setup({ "package.json": pkgJson({ "one-range-dep": "1.0.0" }) }); + const before = await lock(dir); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + const { stdout, stderr, exitCode } = await run(dir, "update"); + expect(stdout).not.toContain("updating:"); + expect(stderr).not.toContain("error:"); + expect(await lock(dir)).toStrictEqual(before); + expect(exitCode).toBe(0); +}); + +test.concurrent("without a lockfile `bun update` resolves everything fresh", async () => { + const { packageDir } = await registry.createTestDir({ + bunfigOpts: { saveTextLockfile: true, linker: "hoisted" }, + files: { "package.json": stringify(pkgJson({ "one-range-dep": "1.0.0" })) }, + }); + const { stdout, stderr, exitCode } = await run(packageDir, "update"); + expect(stdout).not.toContain("updating:"); + expect(stderr).not.toContain("error:"); + expect(stderr).toContain("Saved lockfile"); + expect(await lockedVersions(packageDir, "no-deps")).toStrictEqual(["1.1.0"]); + expect(exitCode).toBe(0); +}); + +test.concurrent("in a workspace, `bun update` from the root moves a member's transitive dependency", async () => { + const root = { name: "root", workspaces: ["packages/*"] }; + const member = (dependencies: Json) => ({ name: "pkg1", version: "1.0.0", dependencies }); + const dir = await setup({ + "package.json": root, + "packages/pkg1/package.json": member({ "one-range-dep": "1.0.0", "no-deps": "1.0.0" }), + }); + const pkg1 = member({ "one-range-dep": "1.0.0" }); + await reinstall(dir, pkg1, {}, "packages/pkg1"); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + + const { stdout, stderr, exitCode } = await run(dir, "update"); + expect(stdout).toContain(" no-deps@1.0.0 → 1.1.0\n"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(root); + expect(await packageJsonOf(dir, "packages/pkg1")).toStrictEqual(pkg1); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + expect(await installedVersion(dir, "no-deps")).toBe("1.1.0"); + await install(dir, "--frozen-lockfile"); + expect(exitCode).toBe(0); +}); + +test.concurrent.each([ + ["dep-with-tags", "3.0.1"], + ["prereleases-1", "1.0.0-future.7"], +])("`bun update --latest` does not downgrade %s from %s, which is ahead of `latest`", async (name, version) => { + const packageJson = pkgJson({ [name]: version }); + const dir = await setup({ "package.json": packageJson }); + expect(await installedVersion(dir, name)).toBe(version); + + const { stderr, exitCode } = await run(dir, "update", "--latest"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect(await lockedVersions(dir, name)).toStrictEqual([version]); + expect(await lockText(dir)).not.toContain('"latest"'); + expect(await installedVersion(dir, name)).toBe(version); + await install(dir, "--frozen-lockfile"); + expect(exitCode).toBe(0); +}); + +test.concurrent("`bun update --help` no longer offers a transitive flag", async () => { + const { packageDir } = await registry.createTestDir(); + const { stdout, exitCode } = await run(packageDir, "update", "--help"); + expect(stdout).not.toContain("--transitive"); + expect(exitCode).toBe(0); +}); + +test.concurrent("`bun update --silent` prints no plan but still moves the transitive dependency", async () => { + const { dir, packageJson, noDeps } = await stale(); + const { stdout, stderr, exitCode } = await run(dir, "update", "--silent"); + expect(stdout).toBe(""); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + expect(await installedVersion(dir, ...noDeps)).toBe("1.1.0"); + await install(dir, "--frozen-lockfile"); + expect(exitCode).toBe(0); +}); + +test.concurrent("`bun update --no-save` moves the transitive dependency in node_modules only", async () => { + const { dir, noDeps } = await stale(); + const packageJsonBefore = await packageJsonText(dir); + const lockBefore = await lockText(dir); + const { stdout, stderr, exitCode } = await run(dir, "update", "--no-save"); + expect(stdout).toContain(" no-deps@1.0.0 → 1.1.0\n"); + expect(stderr).not.toContain("error:"); + expect(stderr).not.toContain("Saved lockfile"); + expect(await packageJsonText(dir)).toBe(packageJsonBefore); + expect(await lockText(dir)).toBe(lockBefore); + expect(await installedVersion(dir, ...noDeps)).toBe("1.1.0"); + expect(exitCode).toBe(0); +}); + +async function expectNoop(dir: string, ...args: string[]) { + const packageJson = await packageJsonOf(dir); + const before = await lockText(dir); + const { stdout, stderr, exitCode } = await run(dir, "update", ...args); + expect(stdout).not.toContain("updating:"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect(await lockText(dir)).toBe(before); + await install(dir, "--frozen-lockfile"); + expect(exitCode).toBe(0); +} + +// hoist-lockfile-{1,2,3} depend on hoist-lockfile-shared (1.0.1 / 1.0.2 / 2.0.1 / 2.0.2) as `*` / `^1.0.1` / `>=1.0.1`. +const HOIST_DEPENDENTS = { "hoist-lockfile-1": "1.0.0", "hoist-lockfile-2": "1.0.0", "hoist-lockfile-3": "1.0.0" }; + +async function staleShared() { + const dir = await setup({ "package.json": pkgJson({ ...HOIST_DEPENDENTS, "hoist-lockfile-shared": "1.0.1" }) }); + const packageJson = pkgJson(HOIST_DEPENDENTS); + await reinstall(dir, packageJson); + expect(await lockedVersions(dir, "hoist-lockfile-shared")).toStrictEqual(["1.0.1"]); + return { dir, packageJson }; +} + +test.concurrent.each([ + ["bare", []], + ["named", ["hoist-lockfile-shared"]], + ["named with an ignored @version", ["hoist-lockfile-shared@1.0.1"]], +])("every dependent's range on a shared package is re-resolved on its own (%s)", async (_, args) => { + const { dir, packageJson } = await staleShared(); + const { stderr, exitCode } = await run(dir, "update", ...args); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect((await lock(dir)).workspaces[""].dependencies).toStrictEqual(HOIST_DEPENDENTS); + expect(await lockedVersions(dir, "hoist-lockfile-shared")).toStrictEqual(["1.0.2", "2.0.2"]); + await install(dir, "--frozen-lockfile"); + expect(exitCode).toBe(0); +}); + +test.concurrent("the plan counts packages, not the edges that move onto them", async () => { + const { dir } = await staleShared(); + const before = await lockText(dir); + const { stdout, stderr, exitCode } = await run(dir, "update", "--dry-run"); + expect(stdout).toContain(" hoist-lockfile-shared@1.0.1 → 1.0.2\n"); + expect(stdout).toContain(" hoist-lockfile-shared@1.0.1 → 2.0.2\n"); + expect(stdout.match(/^ hoist-lockfile-shared@/gm)).toHaveLength(2); + expect(stdout).toContain("Would update 2 packages"); + expect(stderr).not.toContain("error:"); + expect(await lockText(dir)).toBe(before); + expect(exitCode).toBe(0); +}); + +// peer-deps-fixed@1.0.0 declares peer `no-deps: ^1.0.0`; the root's exact no-deps@1.0.0 is its only provider. +test.concurrent.each([ + ["bare", []], + ["--latest peer-deps-fixed", ["--latest", "peer-deps-fixed"]], + ["no-deps", ["no-deps"]], +])("a peer edge keeps following the root's pinned provider instead of forking (%s)", async (_, args) => { + const dir = await setup({ "package.json": pkgJson({ "peer-deps-fixed": "1.0.0", "no-deps": "1.0.0" }) }); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + await expectNoop(dir, ...args); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.0"); +}); + +// peer-deps@1.0.0 has nothing but a `no-deps: *` peer, which the install auto-installs at latest. +test.concurrent("a package with only peer dependencies is a clean no-op", async () => { + const dir = await setup({ "package.json": pkgJson({ "peer-deps": "1.0.0" }) }); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["2.0.0"]); + await expectNoop(dir); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["2.0.0"]); +}); + +// dep-loop-entry@1.0.0 and dep-loop-exit@1.0.0 pin each other; bundled-1@1.0.0 ships its own no-deps@1.0.0. +test.concurrent.each([ + ["bare", []], + ["no-deps", ["no-deps"]], + ["dep-loop-exit", ["dep-loop-exit"]], +])("a dependency cycle and a bundled dependency are left alone (%s)", async (_, args) => { + const dir = await setup({ "package.json": pkgJson({ "dep-loop-entry": "1.0.0", "bundled-1": "1.0.0" }) }); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + await expectNoop(dir, ...args); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + expect(await lockedVersions(dir, "dep-loop-entry")).toStrictEqual(["1.0.0"]); + expect(await lockedVersions(dir, "dep-loop-exit")).toStrictEqual(["1.0.0"]); +}); + +test.concurrent.each([ + ["bare", []], + ["no-deps", ["no-deps"]], +])("a bundled edge is not re-resolved even when its range would allow it (%s)", async (_, args) => { + const dir = await setup({ "package.json": pkgJson({ "bundled-1": "1.0.0" }) }); + const pinned = await lockText(dir); + expect(pinned).toContain('"bundled-1/no-deps": ["no-deps@1.0.0"'); + expect(pinned.split('{ "dependencies": { "no-deps": "1.0.0" } }')).toHaveLength(2); + const widened = pinned.replace( + '{ "dependencies": { "no-deps": "1.0.0" } }', + '{ "dependencies": { "no-deps": "^1.0.0" } }', + ); + await write(join(dir, "bun.lock"), widened); + const { stdout, stderr, exitCode } = await run(dir, "update", ...args); + expect(stdout).not.toContain("updating:"); + expect(stderr).not.toContain("error:"); + expect(await lockText(dir)).toBe(widened); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + expect(exitCode).toBe(0); +}); + +// pkg1 and pkg2 declare the same range; only the workspace `bun update` runs in gets its package.json rewritten. +test.concurrent("in a workspace, `bun update` from one member also re-points a sibling's identical range", async () => { + const root = { name: "root", workspaces: ["packages/*"] }; + const member = (name: string, range: string) => ({ name, version: "1.0.0", dependencies: { "no-deps": range } }); + const dir = await setup({ + "package.json": root, + "packages/pkg1/package.json": member("pkg1", "1.0.0"), + "packages/pkg2/package.json": member("pkg2", "1.0.0"), + }); + await write(join(dir, "packages/pkg2/package.json"), stringify(member("pkg2", "~1.0.0"))); + await reinstall(dir, member("pkg1", "~1.0.0"), {}, "packages/pkg1"); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + + const { stderr, exitCode } = await run(join(dir, "packages/pkg1"), "update"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(root); + expect(await packageJsonOf(dir, "packages/pkg1")).toStrictEqual(member("pkg1", "~1.0.1")); + expect(await packageJsonOf(dir, "packages/pkg2")).toStrictEqual(member("pkg2", "~1.0.0")); + const { workspaces } = await lock(dir); + expect(workspaces["packages/pkg1"].dependencies).toStrictEqual({ "no-deps": "~1.0.1" }); + expect(workspaces["packages/pkg2"].dependencies).toStrictEqual({ "no-deps": "~1.0.0" }); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.1"]); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); + await install(dir, "--frozen-lockfile"); + expect(exitCode).toBe(0); +}); + +type Manifests = Record }>>; + +// Serves one manifest per name from memory; verdaccio has no parent whose newer version keeps a range on the same child. +async function serveRegistry(manifests: Manifests) { + const tarballs = new Map(); + for (const [name, versions] of Object.entries(manifests)) { + for (const [version, extra] of Object.entries(versions)) { + const archive = new Bun.Archive( + { "package/package.json": JSON.stringify({ name, version, ...extra }) }, + { compress: "gzip" }, + ); + tarballs.set(`/${name}-${version}.tgz`, await archive.bytes()); + } + } + return Bun.serve({ + port: 0, + fetch(request) { + const { origin, pathname } = new URL(request.url); + const tarball = tarballs.get(pathname); + if (tarball) return new Response(tarball); + const name = pathname.slice(1); + const entry = manifests[name]; + if (!entry) return new Response("not found", { status: 404 }); + const versions: Json = {}; + for (const [version, extra] of Object.entries(entry)) { + versions[version] = { name, version, dist: { tarball: `${origin}/${name}-${version}.tgz` }, ...extra }; + } + const latest = Object.keys(entry).sort(Bun.semver.order).at(-1); + return Response.json({ name, versions, "dist-tags": { latest } }); + }, + }); +} + +test.concurrent("`bun update ` leaves the named package's own dependencies where they are", async () => { + using server = await serveRegistry({ + parent: { "1.0.0": { dependencies: { leaf: "^1.0.0" } }, "1.1.0": { dependencies: { leaf: "^1.0.0" } } }, + leaf: { "1.0.0": {}, "1.1.0": {} }, + }); + using tmp = tempDir("update-named-children-", { + "package.json": stringify(pkgJson({ parent: "1.0.0", leaf: "1.0.0" })), + }); + const dir = String(tmp); + await write( + join(dir, "bunfig.toml"), + Bun.TOML.stringify({ + install: { cache: join(dir, ".bun-cache"), registry: server.url.href, saveTextLockfile: true, linker: "hoisted" }, + }), + ); + await install(dir); + await reinstall(dir, pkgJson({ parent: "^1.0.0" })); + expect(await lockedVersions(dir, "parent")).toStrictEqual(["1.0.0"]); + expect(await lockedVersions(dir, "leaf")).toStrictEqual(["1.0.0"]); + + const named = await run(dir, "update", "parent"); + expect(named.stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(pkgJson({ parent: "^1.1.0" })); + expect(await lockedVersions(dir, "parent")).toStrictEqual(["1.1.0"]); + expect(await lockedVersions(dir, "leaf")).toStrictEqual(["1.0.0"]); + expect(await installedVersion(dir, "parent")).toBe("1.1.0"); + expect(await installedVersion(dir, "leaf")).toBe("1.0.0"); + await install(dir, "--frozen-lockfile"); + expect(named.exitCode).toBe(0); + + const bare = await run(dir, "update"); + expect(bare.stdout).toContain(" leaf@1.0.0 → 1.1.0\n"); + expect(bare.stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(pkgJson({ parent: "^1.1.0" })); + expect(await lockedVersions(dir, "leaf")).toStrictEqual(["1.1.0"]); + expect(await installedVersion(dir, "leaf")).toBe("1.1.0"); + await install(dir, "--frozen-lockfile"); + expect(bare.exitCode).toBe(0); +}); diff --git a/test/cli/install/bun-update.test.ts b/test/cli/install/bun-update.test.ts index 27d53ba0cd7e..182b93ecf1d6 100644 --- a/test/cli/install/bun-update.test.ts +++ b/test/cli/install/bun-update.test.ts @@ -1,7 +1,7 @@ import { file, spawn } from "bun"; -import { afterAll, afterEach, beforeAll, beforeEach, expect, it } from "bun:test"; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from "bun:test"; import { access, mkdir, readFile, rm, writeFile } from "fs/promises"; -import { bunExe, bunEnv as env, pack, readdirSorted, toBeValidBin, toHaveBins } from "harness"; +import { VerdaccioRegistry, bunExe, bunEnv as env, pack, readdirSorted, toBeValidBin, toHaveBins } from "harness"; import { basename, join } from "path"; import { dummyAfterAll, @@ -689,52 +689,64 @@ it("--filter with multiple patterns selects the union of matching workspaces", a // `bun update ` re-resolves every `` entry in the lockfile, but // only the cwd package.json is rewritten. Named updates don't fan the // package.json edit out to members. -it("named update with --recursive rewrites only the cwd package.json", async () => { - const urls: string[] = []; - setHandler(dummyRegistry(urls, { "0.0.3": {}, "0.0.5": {}, latest: "0.0.5" })); +// `bun update ` already re-resolves in every workspace, so the workspace selectors are rejected rather than silently ignored. +for (const flags of [["--recursive"], ["--filter", "pkg-a"]]) { + it(`named update rejects ${flags[0]}`, async () => { + const urls: string[] = []; + setHandler(dummyRegistry(urls, { "0.0.3": {}, "0.0.5": {}, latest: "0.0.5" })); - await writeFile( - join(package_dir, "package.json"), - JSON.stringify({ - name: "root", - private: true, - workspaces: ["packages/*"], - dependencies: { baz: "~0.0.3" }, - }), - ); - await mkdir(join(package_dir, "packages", "pkg-a"), { recursive: true }); - await writeFile( - join(package_dir, "packages", "pkg-a", "package.json"), - JSON.stringify({ name: "pkg-a", dependencies: { baz: "~0.0.3" } }), - ); + await writeFile( + join(package_dir, "package.json"), + JSON.stringify({ + name: "root", + private: true, + workspaces: ["packages/*"], + dependencies: { baz: "~0.0.3" }, + }), + ); + await mkdir(join(package_dir, "packages", "pkg-a"), { recursive: true }); + await writeFile( + join(package_dir, "packages", "pkg-a", "package.json"), + JSON.stringify({ name: "pkg-a", dependencies: { baz: "~0.0.3" } }), + ); - { const { stderr, exited } = spawn({ - cmd: [bunExe(), "install", "--linker=hoisted"], + cmd: [bunExe(), "update", "baz", ...flags, "--linker=hoisted"], cwd: package_dir, stdout: "ignore", stderr: "pipe", env, }); - expect(await new Response(stderr).text()).not.toContain("error:"); - expect(await exited).toBe(0); - } + expect(await new Response(stderr).text()).toContain( + "error: --recursive and --filter cannot be combined with package names", + ); + expect(await exited).not.toBe(0); - const { stderr, exited } = spawn({ - cmd: [bunExe(), "update", "baz", "--recursive", "--linker=hoisted"], - cwd: package_dir, - stdout: "ignore", - stderr: "pipe", - env, + const root = await file(join(package_dir, "package.json")).json(); + const a = await file(join(package_dir, "packages", "pkg-a", "package.json")).json(); + expect(root.dependencies.baz).toBe("~0.0.3"); + expect(a.dependencies.baz).toBe("~0.0.3"); + expect(urls).toStrictEqual([]); }); - expect(await new Response(stderr).text()).not.toContain("error:"); - expect(await exited).toBe(0); +} - const root = await file(join(package_dir, "package.json")).json(); - const a = await file(join(package_dir, "packages", "pkg-a", "package.json")).json(); - expect(root.dependencies.baz).toBe("~0.0.5"); - // Named updates do not fan the package.json edit out to members. - expect(a.dependencies.baz).toBe("~0.0.3"); +it("--production is rejected", async () => { + const urls: string[] = []; + setHandler(dummyRegistry(urls, { "0.0.3": {}, latest: "0.0.3" })); + await writeFile(join(package_dir, "package.json"), JSON.stringify({ name: "foo", dependencies: { baz: "~0.0.3" } })); + + for (const flag of ["--production", "-p", "--prod", "-P"]) { + const { stderr, exited } = spawn({ + cmd: [bunExe(), "update", flag], + cwd: package_dir, + stdout: "ignore", + stderr: "pipe", + env, + }); + expect(await new Response(stderr).text()).toContain("error: --production cannot be used with bun update"); + expect(await exited).not.toBe(0); + } + expect(urls).toStrictEqual([]); }); // https://github.com/oven-sh/bun/issues/33176 @@ -1453,31 +1465,217 @@ it("should update transitive resolutions of a named package", async () => { ).toMatchObject({ version: "1.1.0" }); }); -// `` appears only transitively: nothing in any package.json depends on -// it directly. `bun update ` promotes it to a direct dependency of the -// invoking package.json, and the transitive slot must re-resolve too. -it("should update a resolution reachable only through a transitive dependency", async () => { +// `shared` appears only transitively (dep-x@1.0.0 depends on `shared@^1.0.0`); shared@1.1.0 is published after +// the lockfile pinned 1.0.0. +async function setupTransitiveOnlyShared() { const tgzDir = join(package_dir, ".tarballs"); const depX = { "dep-x": { versions: { "1.0.0": { dependencies: { shared: "^1.0.0" } } }, latest: "1.0.0" } }; - // Only shared@1.0.0 exists when the lockfile is created. setHandler(await perNameRegistry(tgzDir, { ...depX, shared: { versions: { "1.0.0": {} }, latest: "1.0.0" } })); await writePerNameBunfig(); - await writeFile( - join(package_dir, "package.json"), - JSON.stringify({ name: "root", dependencies: { "dep-x": "^1.0.0" } }), - ); + const packageJson = { name: "root", dependencies: { "dep-x": "^1.0.0" } }; + await writeFile(join(package_dir, "package.json"), JSON.stringify(packageJson)); await runInPackageDir("install"); - expect(await lockedSharedResolutions()).toEqual(['"shared@1.0.0"']); - - // shared@1.1.0 is published after the lockfile pinned 1.0.0. + expect(await lockedSharedResolutions()).toStrictEqual(['"shared@1.0.0"']); setHandler( await perNameRegistry(tgzDir, { ...depX, shared: { versions: { "1.0.0": {}, "1.1.0": {} }, latest: "1.1.0" } }), ); + return packageJson; +} - // Both the new root entry and dep-x's `^1.0.0` must land on 1.1.0. +it("bun update updates a package that is only a transitive dependency without adding it to package.json", async () => { + const packageJson = await setupTransitiveOnlyShared(); await runInPackageDir("update", "shared"); - expect(await file(join(package_dir, "package.json")).json()).toMatchObject({ - dependencies: { "dep-x": "^1.0.0", shared: "^1.1.0" }, + expect(await file(join(package_dir, "package.json")).json()).toStrictEqual(packageJson); + expect(await lockedSharedResolutions()).toStrictEqual(['"shared@1.1.0"']); + expect(await file(join(package_dir, "node_modules", "shared", "package.json")).json()).toMatchObject({ + version: "1.1.0", + }); +}); + +it("bun update updates transitive dependencies", async () => { + const packageJson = await setupTransitiveOnlyShared(); + const out = await runInPackageDir("update"); + expect(out).toContain("updating:\n shared@1.0.0 → 1.1.0\n"); + expect(await file(join(package_dir, "package.json")).json()).toStrictEqual(packageJson); + expect(await lockedSharedResolutions()).toStrictEqual(['"shared@1.1.0"']); + expect(await file(join(package_dir, "node_modules", "shared", "package.json")).json()).toMatchObject({ + version: "1.1.0", + }); +}); + +it("bun update rejects a name that is not in the lockfile", async () => { + const packageJson = await setupTransitiveOnlyShared(); + const lockBefore = await file(join(package_dir, "bun.lock")).text(); + const { stderr, exited } = spawn({ + cmd: [bunExe(), "update", "not-a-dep"], + cwd: package_dir, + stdout: "ignore", + stderr: "pipe", + env, + }); + expect(await stderr.text()).toContain('error: "not-a-dep" is not in the lockfile, so there is nothing to update'); + expect(await exited).toBe(1); + expect(await file(join(package_dir, "package.json")).json()).toStrictEqual(packageJson); + expect(await file(join(package_dir, "bun.lock")).text()).toBe(lockBefore); +}); + +// Registry: no-deps 1.0.0/1.0.1/1.1.0/2.0.0 (latest 2.0.0); a-dep 1.0.1..1.0.10; dep-with-tags latest=3.0.0, pre-2=2.0.1. +describe("bun update semantics", () => { + type Json = Record; + const verdaccio = new VerdaccioRegistry(); + + beforeAll(async () => { + await verdaccio.start(); + }); + + afterAll(() => { + verdaccio.stop(); + }); + + const manifest = (dependencies: Json): Json => ({ name: "foo", dependencies }); + const stringify = (json: Json) => JSON.stringify(json, null, 2) + "\n"; + const packageJsonOf = (dir: string): Promise => file(join(dir, "package.json")).json(); + const packageJsonText = (dir: string) => file(join(dir, "package.json")).text(); + const lockText = (dir: string) => file(join(dir, "bun.lock")).text(); + const lock = async (dir: string): Promise => Bun.JSONC.parse(await lockText(dir)) as Json; + const installedVersion = async (dir: string, name: string): Promise => + (await file(join(dir, "node_modules", name, "package.json")).json()).version; + + async function run(dir: string, ...args: string[]) { + await using proc = spawn({ + cmd: [bunExe(), ...args], + cwd: dir, + env, + stdout: "pipe", + stderr: "pipe", + stdin: "ignore", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; + } + + async function install(dir: string, ...args: string[]) { + const { stderr, exitCode } = await run(dir, "install", ...args); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + return stderr; + } + + async function update(dir: string, ...args: string[]) { + const result = await run(dir, "update", ...args); + expect(result.stderr).not.toContain("error:"); + expect(result.exitCode).toBe(0); + return result; + } + + async function setup(dependencies: Json) { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { saveTextLockfile: true, linker: "hoisted" }, + files: { "package.json": stringify(manifest(dependencies)) }, + }); + await install(packageDir); + return packageDir; + } + + // Pin exactly, then widen: the locked versions still satisfy the new ranges, so only `bun update` moves them. + async function stale(pinned: Json, widened: Json) { + const dir = await setup(pinned); + await writeFile(join(dir, "package.json"), stringify(manifest(widened))); + expect(await install(dir)).toContain("Saved lockfile"); + for (const [name, literal] of Object.entries(pinned as Record)) { + expect(await installedVersion(dir, name)).toBe(literal.replace(/^npm:[^@]+@/, "")); + } + return dir; + } + + // Every version of `name` resolved anywhere in bun.lock, including behind an alias. + async function lockedVersions(dir: string, name: string) { + const { packages } = await lock(dir); + const versions = Object.values(packages as Record) + .map(([resolution]) => resolution) + .filter(resolution => resolution.startsWith(`${name}@`)) + .map(resolution => resolution.slice(name.length + 1)); + return [...new Set(versions)].sort(); + } + + async function expectInSync(dir: string, dependencies: Json) { + expect((await packageJsonOf(dir)).dependencies).toEqual(dependencies); + expect((await lock(dir)).workspaces[""].dependencies).toEqual(dependencies); + await install(dir, "--frozen-lockfile"); + } + + const SIBLINGS_PINNED = { "no-deps": "1.0.0", "a-dep": "1.0.1" }; + const SIBLINGS_WIDENED = { "no-deps": "^1.0.0", "a-dep": "^1.0.1" }; + + for (const flags of [[], ["--latest"]]) { + it(`bun update ${["a-dep", ...flags].join(" ")} leaves a stale unnamed sibling alone`, async () => { + const dir = await stale(SIBLINGS_PINNED, SIBLINGS_WIDENED); + await update(dir, "a-dep", ...flags); + await expectInSync(dir, { "no-deps": "^1.0.0", "a-dep": "^1.0.10" }); + expect(await lockedVersions(dir, "a-dep")).toEqual(["1.0.10"]); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.0"]); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.10"); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.0"); + }); + } + + it("bun update keeps a dist-tag literal as written", async () => { + const dir = await setup({ "dep-with-tags": "pre-2" }); + expect(await installedVersion(dir, "dep-with-tags")).toBe("2.0.1"); + await update(dir, "dep-with-tags"); + await expectInSync(dir, { "dep-with-tags": "pre-2" }); + expect(await lockedVersions(dir, "dep-with-tags")).toEqual(["2.0.1"]); + }); + + it("bun update --latest replaces a dist-tag literal with a caret range on latest", async () => { + const dir = await setup({ "dep-with-tags": "pre-2" }); + await update(dir, "dep-with-tags", "--latest"); + await expectInSync(dir, { "dep-with-tags": "^3.0.0" }); + expect(await lockedVersions(dir, "dep-with-tags")).toEqual(["3.0.0"]); + expect(await installedVersion(dir, "dep-with-tags")).toBe("3.0.0"); + }); + + it("bun update reaches a dependency declared behind an npm: alias", async () => { + const dir = await stale({ aliased: "npm:no-deps@1.0.0" }, { aliased: "npm:no-deps@~1.0.0" }); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.0"]); + await update(dir, "no-deps"); + await expectInSync(dir, { aliased: "npm:no-deps@~1.0.1" }); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.1"]); + expect(await installedVersion(dir, "aliased")).toBe("1.0.1"); + }); + + it("bun update @ --latest is an error and writes nothing", async () => { + const dir = await setup({ "no-deps": "~1.0.0" }); + const packageJsonBefore = await packageJsonText(dir); + const lockBefore = await lockText(dir); + const { stderr, exitCode } = await run(dir, "update", "no-deps@1", "--latest"); + expect(stderr).toMatch(/error: .*--latest/); + expect(stderr).not.toContain("Saved lockfile"); + expect(await packageJsonText(dir)).toBe(packageJsonBefore); + expect(await lockText(dir)).toBe(lockBefore); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); + expect(exitCode).not.toBe(0); + }); + + it("bun update --dry-run writes nothing", async () => { + const dir = await stale({ "no-deps": "1.0.0" }, { "no-deps": "^1.0.0" }); + const packageJsonBefore = await packageJsonText(dir); + const lockBefore = await lockText(dir); + const { stderr } = await update(dir, "no-deps", "--dry-run"); + expect(stderr).not.toContain("Saved lockfile"); + expect(await packageJsonText(dir)).toBe(packageJsonBefore); + expect(await lockText(dir)).toBe(lockBefore); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.0"); + }); + + it("bun update leaves an =x.y.z pin untouched", async () => { + const dir = await setup({ "no-deps": "=1.0.0" }); + const packageJsonBefore = await packageJsonText(dir); + const lockBefore = await lockText(dir); + await update(dir); + expect(await packageJsonText(dir)).toBe(packageJsonBefore); + expect(await lockText(dir)).toBe(lockBefore); + await expectInSync(dir, { "no-deps": "=1.0.0" }); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.0"); }); - expect(await lockedSharedResolutions()).toEqual(['"shared@1.1.0"']); }); diff --git a/test/cli/install/catalog-peer-hoist.test.ts b/test/cli/install/catalog-peer-hoist.test.ts deleted file mode 100644 index c28aa9ea814f..000000000000 --- a/test/cli/install/catalog-peer-hoist.test.ts +++ /dev/null @@ -1,466 +0,0 @@ -import { readTarball } from "bun:internal-for-testing"; -import { afterAll, beforeAll, describe, expect, test } from "bun:test"; -import { existsSync, lstatSync, readlinkSync } from "fs"; -import { readdir, rm } from "fs/promises"; -import { VerdaccioRegistry, bunEnv, bunExe, pack } from "harness"; -import { join } from "path"; - -var registry = new VerdaccioRegistry(); - -beforeAll(async () => { - await registry.start(); -}); - -afterAll(() => { - registry.stop(); -}); - -type Linker = "hoisted" | "isolated"; - -// `--linker` in addition to bunfig: a user-level ~/.npmrc `install-strategy` would otherwise override bunfig's linker. -async function spawnInstall(dir: string, linker: Linker, ...args: string[]) { - await using proc = Bun.spawn({ - cmd: [bunExe(), "install", "--linker", linker, ...args], - cwd: dir, - env: bunEnv, - stdout: "pipe", - stderr: "pipe", - }); - const [out, err, code] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); - return { out, err, code }; -} - -async function install(dir: string, linker: Linker, ...args: string[]) { - const result = await spawnInstall(dir, linker, ...args); - expect(result.err).not.toContain("error:"); - expect(result.code).toBe(0); - return result; -} - -async function packageKeys(dir: string): Promise { - const lockfile = Bun.JSONC.parse(await Bun.file(join(dir, "bun.lock")).text()) as { - packages: Record; - }; - return Object.keys(lockfile.packages).sort(); -} - -async function layout(dir: string, peerName = "no-deps"): Promise { - const path = join(dir, "packages", "lib", "node_modules", peerName); - if (!existsSync(path)) return ""; - if (lstatSync(path).isSymbolicLink()) return readlinkSync(path); - const { version } = await Bun.file(join(path, "package.json")).json(); - return `nested:${version}`; -} - -function rootPackageJson(opts: { - catalog?: Record; - catalogs?: Record>; - rootDependencies?: Record; - overrides?: Record; -}) { - return { - name: "root", - workspaces: { - packages: ["packages/*"], - ...(opts.catalog ? { catalog: opts.catalog } : {}), - ...(opts.catalogs ? { catalogs: opts.catalogs } : {}), - }, - dependencies: opts.rootDependencies ?? { "one-fixed-dep": "2.0.0" }, - ...(opts.overrides ? { overrides: opts.overrides } : {}), - }; -} - -type RepoOpts = { - peerSpec: string; - peerName?: string; - catalog?: Record; - catalogs?: Record>; - rootDependencies?: Record; - overrides?: Record; - optionalPeer?: boolean; - appDependencies?: Record; - extraWorkspaces?: Record; - libVersion?: string; - linker: Linker; - saveTextLockfile?: boolean; -}; - -async function makeRepo(opts: RepoOpts): Promise { - const peerName = opts.peerName ?? "no-deps"; - const extraFiles: Record = {}; - for (const [name, pkg] of Object.entries(opts.extraWorkspaces ?? {})) { - extraFiles[`packages/${name}/package.json`] = JSON.stringify({ name, ...pkg }); - } - const { packageDir } = await registry.createTestDir({ - bunfigOpts: { linker: opts.linker, saveTextLockfile: opts.saveTextLockfile }, - files: { - "package.json": JSON.stringify(rootPackageJson(opts)), - "packages/app/package.json": JSON.stringify({ - name: "app", - dependencies: opts.appDependencies ?? { - "no-deps": "1.0.0", - lib: "workspace:*", - }, - }), - "packages/lib/package.json": JSON.stringify({ - name: "lib", - ...(opts.libVersion ? { version: opts.libVersion } : {}), - peerDependencies: { - [peerName]: opts.peerSpec, - }, - ...(opts.optionalPeer ? { peerDependenciesMeta: { [peerName]: { optional: true } } } : {}), - }), - ...extraFiles, - }, - }); - return packageDir; -} - -async function rewriteRootPackageJson(dir: string, opts: Parameters[0]) { - await Bun.write(join(dir, "package.json"), JSON.stringify(rootPackageJson(opts))); -} - -async function rmNodeModules(dir: string) { - const workspaces = await readdir(join(dir, "packages")); - await Promise.all( - [join(dir, "node_modules"), ...workspaces.map(ws => join(dir, "packages", ws, "node_modules"))].map(path => - rm(path, { recursive: true, force: true }), - ), - ); -} - -const dedupedKeys = ["app", "lib", "no-deps", "one-fixed-dep", "one-fixed-dep/no-deps"]; -const nestedKeys = ["app", "lib", "lib/no-deps", "no-deps", "one-fixed-dep", "one-fixed-dep/no-deps"]; -const isolatedNoDeps = (version: string) => - join("..", "..", "..", "node_modules", ".bun", `no-deps@${version}`, "node_modules", "no-deps"); -const isolatedNoDeps2 = isolatedNoDeps("2.0.0"); -const linkers = ["hoisted", "isolated"] as const; - -test.concurrent("default catalog peer dedupes onto the satisfying ancestor", async () => { - const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker: "hoisted" }); - await install(dir, "hoisted"); - expect(await packageKeys(dir)).toEqual(dedupedKeys); - expect(existsSync(join(dir, "packages", "lib", "node_modules", "no-deps"))).toBeFalse(); -}); - -test.concurrent("named catalog peer (catalog:peers) dedupes the same way", async () => { - const dir = await makeRepo({ - catalogs: { peers: { "no-deps": ">=1.0.0" } }, - peerSpec: "catalog:peers", - linker: "hoisted", - }); - await install(dir, "hoisted"); - expect(await packageKeys(dir)).toEqual(dedupedKeys); - expect(existsSync(join(dir, "packages", "lib", "node_modules", "no-deps"))).toBeFalse(); -}); - -test.concurrent("optional catalog peer dedupes too", async () => { - const dir = await makeRepo({ - catalog: { "no-deps": ">=1.0.0" }, - peerSpec: "catalog:", - optionalPeer: true, - linker: "hoisted", - }); - await install(dir, "hoisted"); - expect(await packageKeys(dir)).toEqual(dedupedKeys); - expect(existsSync(join(dir, "packages", "lib", "node_modules", "no-deps"))).toBeFalse(); -}); - -test.concurrent("scoped package name as a catalog peer", async () => { - const dir = await makeRepo({ - rootDependencies: {}, - catalog: { "@scoped/has-bin-entry": ">=1.0.0" }, - peerName: "@scoped/has-bin-entry", - peerSpec: "catalog:", - appDependencies: { "@scoped/has-bin-entry": "1.0.0", lib: "workspace:*" }, - linker: "hoisted", - }); - await install(dir, "hoisted"); - expect(await packageKeys(dir)).toEqual(["@scoped/has-bin-entry", "app", "lib"]); - expect(existsSync(join(dir, "packages", "lib", "node_modules", "@scoped"))).toBeFalse(); -}); - -async function record(dir: string, linker: Linker, peerName?: string) { - await install(dir, linker); - const keys = await packageKeys(dir); - const fresh = await layout(dir, peerName); - await rmNodeModules(dir); - const { err } = await install(dir, linker); - const keysAfterReload = await packageKeys(dir); - const reload = await layout(dir, peerName); - return { keys, fresh, keysAfterReload, reload, reloadSavedLockfile: err.includes("Saved lockfile") }; -} - -describe.each([ - [">=1.0.0", "dedupes"], - ["^2.0.0", "stays nested"], -] as const)("peer range %s (%s)", (range, outcome) => { - describe.each(linkers)("linker=%s", linker => { - test.concurrent("catalog: peer produces the same lockfile and layout as the inline range", async () => { - const [catalogDir, inlineDir] = await Promise.all([ - makeRepo({ catalog: { "no-deps": range }, peerSpec: "catalog:", linker }), - makeRepo({ peerSpec: range, linker }), - ]); - const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); - - const expectedKeys = outcome === "dedupes" ? dedupedKeys : nestedKeys; - expect(fromInline.keys).toEqual(expectedKeys); - expect(fromInline.keysAfterReload).toEqual(expectedKeys); - expect(fromInline.reloadSavedLockfile).toBeFalse(); - if (linker === "isolated") { - expect(fromInline.fresh).toEndWith(isolatedNoDeps2); - expect(fromInline.reload).toEndWith(isolatedNoDeps2); - } else { - const expectedLayout = outcome === "dedupes" ? "" : "nested:2.0.0"; - expect(fromInline.fresh).toBe(expectedLayout); - expect(fromInline.reload).toBe(expectedLayout); - } - - expect(fromCatalog).toEqual(fromInline); - }); - }); -}); - -describe.each(linkers)("linker=%s", linker => { - test.concurrent("catalog `*` peer behaves exactly like an inline `*` peer", async () => { - const [catalogDir, inlineDir] = await Promise.all([ - makeRepo({ catalog: { "no-deps": "*" }, peerSpec: "catalog:", linker }), - makeRepo({ peerSpec: "*", linker }), - ]); - const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); - expect(fromInline.keys).toEqual(dedupedKeys); - expect(fromInline.keysAfterReload).toEqual(dedupedKeys); - expect(fromInline.reloadSavedLockfile).toBeFalse(); - expect(fromCatalog).toEqual(fromInline); - }); - - test.concurrent("aliased catalog entry peer matches the inline alias", async () => { - const [catalogDir, inlineDir] = await Promise.all([ - makeRepo({ catalog: { "no-deps": "npm:no-deps@>=1.0.0" }, peerSpec: "catalog:", linker }), - makeRepo({ peerSpec: "npm:no-deps@>=1.0.0", linker }), - ]); - const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); - expect(fromInline.keysAfterReload).toEqual(fromInline.keys); - expect(fromInline.reloadSavedLockfile).toBeFalse(); - expect(fromCatalog).toEqual(fromInline); - }); - - test.concurrent("optional catalog peer matches the inline optional peer on reload", async () => { - const [catalogDir, inlineDir] = await Promise.all([ - makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", optionalPeer: true, linker }), - makeRepo({ peerSpec: ">=1.0.0", optionalPeer: true, linker }), - ]); - const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); - expect(fromInline.keys).toEqual(dedupedKeys); - expect(fromInline.keysAfterReload).toEqual(dedupedKeys); - expect(fromInline.reloadSavedLockfile).toBeFalse(); - expect(fromCatalog).toEqual(fromInline); - }); - - // pnpm: deps-installer/test/catalogs.ts "importer with different peers uses correct peer" - test.concurrent( - "two consumers providing different peer versions: catalog peer still equals inline peer", - async () => { - const twoConsumers = (peerSpec: string, catalog?: Record) => - makeRepo({ - peerSpec, - catalog, - rootDependencies: {}, - appDependencies: { "no-deps": "1.0.0", lib: "workspace:*" }, - extraWorkspaces: { app2: { dependencies: { "no-deps": "2.0.0", lib: "workspace:*" } } }, - linker, - }); - const [catalogDir, inlineDir] = await Promise.all([ - twoConsumers("catalog:", { "no-deps": ">=1.0.0" }), - twoConsumers(">=1.0.0"), - ]); - const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); - expect(fromInline.keys).toEqual(["app", "app2", "app2/no-deps", "lib", "no-deps"]); - expect(fromInline.keysAfterReload).toEqual(fromInline.keys); - expect(fromInline.reloadSavedLockfile).toBeFalse(); - expect(fromCatalog).toEqual(fromInline); - }, - ); - - // pnpm: deps-installer/test/catalogs.ts "catalog resolutions should be consistent with peer dependencies" - test.concurrent("warm install leaves bun.lock byte-identical and --frozen-lockfile passes", async () => { - const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker }); - await install(dir, linker); - const lockfile = await Bun.file(join(dir, "bun.lock")).text(); - await rmNodeModules(dir); - const warm = await install(dir, linker); - expect(warm.err).not.toContain("Saved lockfile"); - expect(await Bun.file(join(dir, "bun.lock")).text()).toBe(lockfile); - await rmNodeModules(dir); - const frozen = await install(dir, linker, "--frozen-lockfile"); - expect(frozen.err).not.toContain("lockfile had changes"); - if (linker === "hoisted") expect(await layout(dir)).toBe(""); - else expect(await layout(dir)).toEndWith(isolatedNoDeps2); - }); -}); - -// pnpm: deps-installer/test/catalogs.ts "lockfile is updated if catalog config changes" -test.concurrent("changing the catalog range of a peer re-hoists on the next install (both directions)", async () => { - const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker: "hoisted" }); - await install(dir, "hoisted"); - expect(await packageKeys(dir)).toEqual(dedupedKeys); - - await rewriteRootPackageJson(dir, { catalog: { "no-deps": "^2.0.0" } }); - let { err } = await install(dir, "hoisted"); - expect(err).toContain("Saved lockfile"); - expect(await packageKeys(dir)).toEqual(nestedKeys); - expect(await layout(dir)).toBe("nested:2.0.0"); - - await rewriteRootPackageJson(dir, { catalog: { "no-deps": ">=1.0.0" } }); - ({ err } = await install(dir, "hoisted")); - expect(err).toContain("Saved lockfile"); - expect(await packageKeys(dir)).toEqual(dedupedKeys); -}); - -// pnpm: deps-installer/test/catalogs.ts "frozen lockfile error is thrown if catalog config changes" -test.concurrent("--frozen-lockfile fails when only a peer's catalog range changed", async () => { - const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker: "hoisted" }); - await install(dir, "hoisted"); - const lockfile = await Bun.file(join(dir, "bun.lock")).text(); - - await rewriteRootPackageJson(dir, { catalog: { "no-deps": "^2.0.0" } }); - const { err, code } = await spawnInstall(dir, "hoisted", "--frozen-lockfile"); - expect(err).toContain("error: lockfile had changes, but lockfile is frozen"); - expect(code).not.toBe(0); - expect(await Bun.file(join(dir, "bun.lock")).text()).toBe(lockfile); -}); - -test.concurrent("catalog peer with bun.lockb dedupes and reloads identically", async () => { - const dir = await makeRepo({ - catalog: { "no-deps": ">=1.0.0" }, - peerSpec: "catalog:", - linker: "hoisted", - saveTextLockfile: false, - }); - await install(dir, "hoisted"); - expect(existsSync(join(dir, "bun.lock"))).toBeFalse(); - const lockb = await Bun.file(join(dir, "bun.lockb")).bytes(); - expect(await layout(dir)).toBe(""); - await rmNodeModules(dir); - await install(dir, "hoisted"); - expect(await layout(dir)).toBe(""); - expect(await Bun.file(join(dir, "bun.lockb")).bytes()).toEqual(lockb); -}); - -// pnpm applies overrides before catalogs, keyed by the peer's own name, even when the catalog entry aliases another package. -describe.each([ - ["plain", ">=2.0.0"], - ["aliased", "npm:a-dep@1.0.1"], -] as const)("override beats the %s catalog entry of a peer", (_, entry) => { - test.concurrent("fresh == reload", async () => { - const dir = await makeRepo({ - overrides: { "no-deps": "1.0.0" }, - rootDependencies: { "one-fixed-dep": "2.0.0", "a-dep": "1.0.1" }, - catalog: { "no-deps": entry }, - peerSpec: "catalog:", - linker: "isolated", - }); - const result = await record(dir, "isolated"); - expect(result.fresh).toEndWith(isolatedNoDeps("1.0.0")); - const keys = ["a-dep", "app", "lib", "no-deps", "one-fixed-dep"]; - expect(result).toEqual({ - keys, - fresh: result.fresh, - keysAfterReload: keys, - reload: result.fresh, - reloadSavedLockfile: false, - }); - }); -}); - -// Only the entry the spec names is `^2.0.0`; a `>=1.0.0` decoy or an unresolved peer would give dedupedKeys instead (pnpm: resolveFromCatalog.test.ts). -describe.each([ - ["catalog:peers", { catalog: { "no-deps": ">=1.0.0" }, catalogs: { peers: { "no-deps": "^2.0.0" } } }], - ["catalog:", { catalog: { "no-deps": "^2.0.0" }, catalogs: { peers: { "no-deps": ">=1.0.0" } } }], - ["catalog:default", { catalog: { "no-deps": "^2.0.0" } }], - ["catalog:", { catalogs: { default: { "no-deps": "^2.0.0" } } }], - ["catalog:default", { catalogs: { default: { "no-deps": "^2.0.0" } } }], - ["catalog:default", { catalog: { "no-deps": ">=1.0.0" }, catalogs: { default: { "no-deps": "^2.0.0" } } }], - ["catalog:", { catalog: { "no-deps": "^2.0.0" }, catalogs: { default: { "no-deps": ">=1.0.0" } } }], -] as const)("peer %s resolves through the entry named by its spec (%o)", (peerSpec, catalogFields) => { - test.concurrent("fresh and reload", async () => { - const dir = await makeRepo({ ...catalogFields, peerSpec, linker: "hoisted" }); - expect(await record(dir, "hoisted")).toEqual({ - keys: nestedKeys, - fresh: "nested:2.0.0", - keysAfterReload: nestedKeys, - reload: "nested:2.0.0", - reloadSavedLockfile: false, - }); - }); -}); - -// pnpm errors here; Bun never fails an install over an unresolved peer, so the peer must simply not get a nested copy. -describe.each([ - ["catalog:", { catalog: {} }], - ["catalog:peers", { catalogs: { other: { "no-deps": ">=1.0.0" } } }], - ["catalog:default", { catalogs: { other: { "no-deps": ">=1.0.0" } } }], - ["catalog:", { catalog: { "no-deps": "catalog:other" }, catalogs: { other: { "no-deps": ">=1.0.0" } } }], -] as const)("peer %s with no usable catalog entry (%o)", (peerSpec, catalogFields) => { - test.concurrent("installs without a nested copy and is stable on reload", async () => { - const dir = await makeRepo({ ...catalogFields, peerSpec, linker: "hoisted" }); - expect(await record(dir, "hoisted")).toEqual({ - keys: dedupedKeys, - fresh: "", - keysAfterReload: dedupedKeys, - reload: "", - reloadSavedLockfile: false, - }); - }); -}); - -// pnpm #12159 shape: an override whose value is a catalog reference wins over the peer's own range, fresh and on reload. -describe.each(linkers)("linker=%s", linker => { - describe.each([ - [">=1.0.0", "catalog:", { catalog: { "no-deps": "1.0.0" } }], - ["^2.0.0", "catalog:", { catalog: { "no-deps": "1.0.0" } }], - ["catalog:", "catalog:", { catalog: { "no-deps": "1.0.0" } }], - ["catalog:", "catalog:pins", { catalog: { "no-deps": ">=1.0.0" }, catalogs: { pins: { "no-deps": "1.0.0" } } }], - ] as const)("peer %s overridden to %s", (peerSpec, override, catalogFields) => { - test.concurrent("binds to the overriding catalog entry, fresh == reload", async () => { - const dir = await makeRepo({ ...catalogFields, overrides: { "no-deps": override }, peerSpec, linker }); - const result = await record(dir, linker); - const keys = ["app", "lib", "no-deps", "one-fixed-dep"]; - expect(result).toEqual({ - keys, - fresh: linker === "isolated" ? expect.stringContaining(isolatedNoDeps("1.0.0")) : "", - keysAfterReload: keys, - reload: result.fresh, - reloadSavedLockfile: false, - }); - const { packages } = Bun.JSONC.parse(await Bun.file(join(dir, "bun.lock")).text()) as { - packages: Record; - }; - expect(packages["no-deps"][0]).toBe("no-deps@1.0.0"); - }); - }); -}); - -// pnpm #8996 (`catalog:` peers survive `pack` unsubstituted) and #7072 (`catalog:` / `catalog:default` are one catalog). -describe.each([ - ["catalog:", { catalog: { "no-deps": ">=1.0.0" } }], - ["catalog:peers", { catalogs: { peers: { "no-deps": ">=1.0.0" } } }], - ["catalog:", { catalogs: { default: { "no-deps": ">=1.0.0" } } }], - ["catalog:default", { catalog: { "no-deps": ">=1.0.0" } }], -] as const)("bun pm pack substitutes the %s peer (%o)", (peerSpec, catalogFields) => { - test.concurrent("with the catalog's range", async () => { - const dir = await makeRepo({ ...catalogFields, peerSpec, libVersion: "1.2.3", linker: "hoisted" }); - await install(dir, "hoisted"); - const libDir = join(dir, "packages", "lib"); - await pack(libDir, bunEnv); - const tarball = readTarball(join(libDir, "lib-1.2.3.tgz")); - const packageJson = tarball.entries.find( - (entry: { pathname: string }) => entry.pathname === "package/package.json", - ); - expect(JSON.parse(packageJson.contents)).toEqual({ - name: "lib", - version: "1.2.3", - peerDependencies: { "no-deps": ">=1.0.0" }, - }); - }); -}); diff --git a/test/cli/install/catalogs.test.ts b/test/cli/install/catalogs.test.ts index 9dc234e2ad61..b93888673d93 100644 --- a/test/cli/install/catalogs.test.ts +++ b/test/cli/install/catalogs.test.ts @@ -1,7 +1,9 @@ import { file, spawn, write } from "bun"; +import { readTarball } from "bun:internal-for-testing"; import { afterAll, beforeAll, describe, expect, test } from "bun:test"; -import { exists } from "fs/promises"; -import { VerdaccioRegistry, bunEnv, bunExe, runBunInstall } from "harness"; +import { existsSync, lstatSync, readlinkSync } from "fs"; +import { exists, readdir, rm } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe, pack, runBunInstall } from "harness"; import { join } from "path"; var registry = new VerdaccioRegistry(); @@ -613,3 +615,455 @@ describe("errors", () => { expect(err).toContain("no-deps@catalog: failed to resolve"); }); }); + +describe("peer dependencies", () => { + type Linker = "hoisted" | "isolated"; + + // `--linker` in addition to bunfig: a user-level ~/.npmrc `install-strategy` would otherwise override bunfig's linker. + async function spawnInstall(dir: string, linker: Linker, ...args: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), "install", "--linker", linker, ...args], + cwd: dir, + env: { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }, + stdout: "pipe", + stderr: "pipe", + }); + const [out, err, code] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { out, err, code }; + } + + async function install(dir: string, linker: Linker, ...args: string[]) { + const result = await spawnInstall(dir, linker, ...args); + expect(result.err).not.toContain("error:"); + expect(result.code).toBe(0); + return result; + } + + async function packageKeys(dir: string): Promise { + const lockfile = Bun.JSONC.parse(await Bun.file(join(dir, "bun.lock")).text()) as { + packages: Record; + }; + return Object.keys(lockfile.packages).sort(); + } + + async function layout(dir: string, peerName = "no-deps"): Promise { + const path = join(dir, "packages", "lib", "node_modules", peerName); + if (!existsSync(path)) return ""; + if (lstatSync(path).isSymbolicLink()) return readlinkSync(path); + const { version } = await Bun.file(join(path, "package.json")).json(); + return `nested:${version}`; + } + + function rootPackageJson(opts: { + catalog?: Record; + catalogs?: Record>; + rootDependencies?: Record; + overrides?: Record; + }) { + return { + name: "root", + workspaces: { + packages: ["packages/*"], + ...(opts.catalog ? { catalog: opts.catalog } : {}), + ...(opts.catalogs ? { catalogs: opts.catalogs } : {}), + }, + dependencies: opts.rootDependencies ?? { "one-fixed-dep": "2.0.0" }, + ...(opts.overrides ? { overrides: opts.overrides } : {}), + }; + } + + type RepoOpts = { + peerSpec: string; + peerName?: string; + catalog?: Record; + catalogs?: Record>; + rootDependencies?: Record; + overrides?: Record; + optionalPeer?: boolean; + appDependencies?: Record; + extraWorkspaces?: Record; + libVersion?: string; + linker: Linker; + saveTextLockfile?: boolean; + }; + + async function makeRepo(opts: RepoOpts): Promise { + const peerName = opts.peerName ?? "no-deps"; + const extraFiles: Record = {}; + for (const [name, pkg] of Object.entries(opts.extraWorkspaces ?? {})) { + extraFiles[`packages/${name}/package.json`] = JSON.stringify({ name, ...pkg }); + } + const { packageDir } = await registry.createTestDir({ + bunfigOpts: { linker: opts.linker, saveTextLockfile: opts.saveTextLockfile }, + files: { + "package.json": JSON.stringify(rootPackageJson(opts)), + "packages/app/package.json": JSON.stringify({ + name: "app", + dependencies: opts.appDependencies ?? { + "no-deps": "1.0.0", + lib: "workspace:*", + }, + }), + "packages/lib/package.json": JSON.stringify({ + name: "lib", + ...(opts.libVersion ? { version: opts.libVersion } : {}), + peerDependencies: { + [peerName]: opts.peerSpec, + }, + ...(opts.optionalPeer ? { peerDependenciesMeta: { [peerName]: { optional: true } } } : {}), + }), + ...extraFiles, + }, + }); + return packageDir; + } + + async function rewriteRootPackageJson(dir: string, opts: Parameters[0]) { + await Bun.write(join(dir, "package.json"), JSON.stringify(rootPackageJson(opts))); + } + + async function rmNodeModules(dir: string) { + const workspaces = await readdir(join(dir, "packages")); + await Promise.all( + [join(dir, "node_modules"), ...workspaces.map(ws => join(dir, "packages", ws, "node_modules"))].map(path => + rm(path, { recursive: true, force: true }), + ), + ); + } + + const dedupedKeys = ["app", "lib", "no-deps", "one-fixed-dep", "one-fixed-dep/no-deps"]; + const nestedKeys = ["app", "lib", "lib/no-deps", "no-deps", "one-fixed-dep", "one-fixed-dep/no-deps"]; + const isolatedNoDeps = (version: string) => + join("..", "..", "..", "node_modules", ".bun", `no-deps@${version}`, "node_modules", "no-deps"); + const isolatedNoDeps2 = isolatedNoDeps("2.0.0"); + const linkers = ["hoisted", "isolated"] as const; + + test.concurrent("default catalog peer dedupes onto the satisfying ancestor", async () => { + const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker: "hoisted" }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); + expect(existsSync(join(dir, "packages", "lib", "node_modules", "no-deps"))).toBeFalse(); + }); + + test.concurrent("named catalog peer (catalog:peers) dedupes the same way", async () => { + const dir = await makeRepo({ + catalogs: { peers: { "no-deps": ">=1.0.0" } }, + peerSpec: "catalog:peers", + linker: "hoisted", + }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); + expect(existsSync(join(dir, "packages", "lib", "node_modules", "no-deps"))).toBeFalse(); + }); + + test.concurrent("optional catalog peer dedupes too", async () => { + const dir = await makeRepo({ + catalog: { "no-deps": ">=1.0.0" }, + peerSpec: "catalog:", + optionalPeer: true, + linker: "hoisted", + }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); + expect(existsSync(join(dir, "packages", "lib", "node_modules", "no-deps"))).toBeFalse(); + }); + + test.concurrent("scoped package name as a catalog peer", async () => { + const dir = await makeRepo({ + rootDependencies: {}, + catalog: { "@scoped/has-bin-entry": ">=1.0.0" }, + peerName: "@scoped/has-bin-entry", + peerSpec: "catalog:", + appDependencies: { "@scoped/has-bin-entry": "1.0.0", lib: "workspace:*" }, + linker: "hoisted", + }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(["@scoped/has-bin-entry", "app", "lib"]); + expect(existsSync(join(dir, "packages", "lib", "node_modules", "@scoped"))).toBeFalse(); + }); + + async function record(dir: string, linker: Linker, peerName?: string) { + await install(dir, linker); + const keys = await packageKeys(dir); + const fresh = await layout(dir, peerName); + await rmNodeModules(dir); + const { err } = await install(dir, linker); + const keysAfterReload = await packageKeys(dir); + const reload = await layout(dir, peerName); + return { keys, fresh, keysAfterReload, reload, reloadSavedLockfile: err.includes("Saved lockfile") }; + } + + describe.each([ + [">=1.0.0", "dedupes"], + ["^2.0.0", "stays nested"], + ] as const)("peer range %s (%s)", (range, outcome) => { + describe.each(linkers)("linker=%s", linker => { + test.concurrent("catalog: peer produces the same lockfile and layout as the inline range", async () => { + const [catalogDir, inlineDir] = await Promise.all([ + makeRepo({ catalog: { "no-deps": range }, peerSpec: "catalog:", linker }), + makeRepo({ peerSpec: range, linker }), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + + const expectedKeys = outcome === "dedupes" ? dedupedKeys : nestedKeys; + expect(fromInline.keys).toEqual(expectedKeys); + expect(fromInline.keysAfterReload).toEqual(expectedKeys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + if (linker === "isolated") { + expect(fromInline.fresh).toEndWith(isolatedNoDeps2); + expect(fromInline.reload).toEndWith(isolatedNoDeps2); + } else { + const expectedLayout = outcome === "dedupes" ? "" : "nested:2.0.0"; + expect(fromInline.fresh).toBe(expectedLayout); + expect(fromInline.reload).toBe(expectedLayout); + } + + expect(fromCatalog).toEqual(fromInline); + }); + }); + }); + + describe.each(linkers)("linker=%s", linker => { + test.concurrent("catalog `*` peer behaves exactly like an inline `*` peer", async () => { + const [catalogDir, inlineDir] = await Promise.all([ + makeRepo({ catalog: { "no-deps": "*" }, peerSpec: "catalog:", linker }), + makeRepo({ peerSpec: "*", linker }), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + expect(fromInline.keys).toEqual(dedupedKeys); + expect(fromInline.keysAfterReload).toEqual(dedupedKeys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + expect(fromCatalog).toEqual(fromInline); + }); + + test.concurrent("aliased catalog entry peer matches the inline alias", async () => { + const [catalogDir, inlineDir] = await Promise.all([ + makeRepo({ catalog: { "no-deps": "npm:no-deps@>=1.0.0" }, peerSpec: "catalog:", linker }), + makeRepo({ peerSpec: "npm:no-deps@>=1.0.0", linker }), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + expect(fromInline.keysAfterReload).toEqual(fromInline.keys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + expect(fromCatalog).toEqual(fromInline); + }); + + test.concurrent("optional catalog peer matches the inline optional peer on reload", async () => { + const [catalogDir, inlineDir] = await Promise.all([ + makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", optionalPeer: true, linker }), + makeRepo({ peerSpec: ">=1.0.0", optionalPeer: true, linker }), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + expect(fromInline.keys).toEqual(dedupedKeys); + expect(fromInline.keysAfterReload).toEqual(dedupedKeys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + expect(fromCatalog).toEqual(fromInline); + }); + + // pnpm: deps-installer/test/catalogs.ts "importer with different peers uses correct peer" + test.concurrent( + "two consumers providing different peer versions: catalog peer still equals inline peer", + async () => { + const twoConsumers = (peerSpec: string, catalog?: Record) => + makeRepo({ + peerSpec, + catalog, + rootDependencies: {}, + appDependencies: { "no-deps": "1.0.0", lib: "workspace:*" }, + extraWorkspaces: { app2: { dependencies: { "no-deps": "2.0.0", lib: "workspace:*" } } }, + linker, + }); + const [catalogDir, inlineDir] = await Promise.all([ + twoConsumers("catalog:", { "no-deps": ">=1.0.0" }), + twoConsumers(">=1.0.0"), + ]); + const [fromCatalog, fromInline] = await Promise.all([record(catalogDir, linker), record(inlineDir, linker)]); + expect(fromInline.keys).toEqual(["app", "app2", "app2/no-deps", "lib", "no-deps"]); + expect(fromInline.keysAfterReload).toEqual(fromInline.keys); + expect(fromInline.reloadSavedLockfile).toBeFalse(); + expect(fromCatalog).toEqual(fromInline); + }, + ); + + // pnpm: deps-installer/test/catalogs.ts "catalog resolutions should be consistent with peer dependencies" + test.concurrent("warm install leaves bun.lock byte-identical and --frozen-lockfile passes", async () => { + const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker }); + await install(dir, linker); + const lockfile = await Bun.file(join(dir, "bun.lock")).text(); + await rmNodeModules(dir); + const warm = await install(dir, linker); + expect(warm.err).not.toContain("Saved lockfile"); + expect(await Bun.file(join(dir, "bun.lock")).text()).toBe(lockfile); + await rmNodeModules(dir); + const frozen = await install(dir, linker, "--frozen-lockfile"); + expect(frozen.err).not.toContain("lockfile had changes"); + if (linker === "hoisted") expect(await layout(dir)).toBe(""); + else expect(await layout(dir)).toEndWith(isolatedNoDeps2); + }); + }); + + // pnpm: deps-installer/test/catalogs.ts "lockfile is updated if catalog config changes" + test.concurrent("changing the catalog range of a peer re-hoists on the next install (both directions)", async () => { + const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker: "hoisted" }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); + + await rewriteRootPackageJson(dir, { catalog: { "no-deps": "^2.0.0" } }); + let { err } = await install(dir, "hoisted"); + expect(err).toContain("Saved lockfile"); + expect(await packageKeys(dir)).toEqual(nestedKeys); + expect(await layout(dir)).toBe("nested:2.0.0"); + + await rewriteRootPackageJson(dir, { catalog: { "no-deps": ">=1.0.0" } }); + ({ err } = await install(dir, "hoisted")); + expect(err).toContain("Saved lockfile"); + expect(await packageKeys(dir)).toEqual(dedupedKeys); + }); + + // pnpm: deps-installer/test/catalogs.ts "frozen lockfile error is thrown if catalog config changes" + test.concurrent("--frozen-lockfile fails when only a peer's catalog range changed", async () => { + const dir = await makeRepo({ catalog: { "no-deps": ">=1.0.0" }, peerSpec: "catalog:", linker: "hoisted" }); + await install(dir, "hoisted"); + const lockfile = await Bun.file(join(dir, "bun.lock")).text(); + + await rewriteRootPackageJson(dir, { catalog: { "no-deps": "^2.0.0" } }); + const { err, code } = await spawnInstall(dir, "hoisted", "--frozen-lockfile"); + expect(err).toContain("error: lockfile had changes, but lockfile is frozen"); + expect(code).not.toBe(0); + expect(await Bun.file(join(dir, "bun.lock")).text()).toBe(lockfile); + }); + + test.concurrent("catalog peer with bun.lockb dedupes and reloads identically", async () => { + const dir = await makeRepo({ + catalog: { "no-deps": ">=1.0.0" }, + peerSpec: "catalog:", + linker: "hoisted", + saveTextLockfile: false, + }); + await install(dir, "hoisted"); + expect(existsSync(join(dir, "bun.lock"))).toBeFalse(); + const lockb = await Bun.file(join(dir, "bun.lockb")).bytes(); + expect(await layout(dir)).toBe(""); + await rmNodeModules(dir); + await install(dir, "hoisted"); + expect(await layout(dir)).toBe(""); + expect(await Bun.file(join(dir, "bun.lockb")).bytes()).toEqual(lockb); + }); + + // pnpm applies overrides before catalogs, keyed by the peer's own name, even when the catalog entry aliases another package. + describe.each([ + ["plain", ">=2.0.0"], + ["aliased", "npm:a-dep@1.0.1"], + ] as const)("override beats the %s catalog entry of a peer", (_, entry) => { + test.concurrent("fresh == reload", async () => { + const dir = await makeRepo({ + overrides: { "no-deps": "1.0.0" }, + rootDependencies: { "one-fixed-dep": "2.0.0", "a-dep": "1.0.1" }, + catalog: { "no-deps": entry }, + peerSpec: "catalog:", + linker: "isolated", + }); + const result = await record(dir, "isolated"); + expect(result.fresh).toEndWith(isolatedNoDeps("1.0.0")); + const keys = ["a-dep", "app", "lib", "no-deps", "one-fixed-dep"]; + expect(result).toEqual({ + keys, + fresh: result.fresh, + keysAfterReload: keys, + reload: result.fresh, + reloadSavedLockfile: false, + }); + }); + }); + + // Only the entry the spec names is `^2.0.0`; a `>=1.0.0` decoy or an unresolved peer would give dedupedKeys instead (pnpm: resolveFromCatalog.test.ts). + describe.each([ + ["catalog:peers", { catalog: { "no-deps": ">=1.0.0" }, catalogs: { peers: { "no-deps": "^2.0.0" } } }], + ["catalog:", { catalog: { "no-deps": "^2.0.0" }, catalogs: { peers: { "no-deps": ">=1.0.0" } } }], + ["catalog:default", { catalog: { "no-deps": "^2.0.0" } }], + ["catalog:", { catalogs: { default: { "no-deps": "^2.0.0" } } }], + ["catalog:default", { catalogs: { default: { "no-deps": "^2.0.0" } } }], + ["catalog:default", { catalog: { "no-deps": ">=1.0.0" }, catalogs: { default: { "no-deps": "^2.0.0" } } }], + ["catalog:", { catalog: { "no-deps": "^2.0.0" }, catalogs: { default: { "no-deps": ">=1.0.0" } } }], + ] as const)("peer %s resolves through the entry named by its spec (%o)", (peerSpec, catalogFields) => { + test.concurrent("fresh and reload", async () => { + const dir = await makeRepo({ ...catalogFields, peerSpec, linker: "hoisted" }); + expect(await record(dir, "hoisted")).toEqual({ + keys: nestedKeys, + fresh: "nested:2.0.0", + keysAfterReload: nestedKeys, + reload: "nested:2.0.0", + reloadSavedLockfile: false, + }); + }); + }); + + // pnpm errors here; Bun never fails an install over an unresolved peer, so the peer must simply not get a nested copy. + describe.each([ + ["catalog:", { catalog: {} }], + ["catalog:peers", { catalogs: { other: { "no-deps": ">=1.0.0" } } }], + ["catalog:default", { catalogs: { other: { "no-deps": ">=1.0.0" } } }], + ["catalog:", { catalog: { "no-deps": "catalog:other" }, catalogs: { other: { "no-deps": ">=1.0.0" } } }], + ] as const)("peer %s with no usable catalog entry (%o)", (peerSpec, catalogFields) => { + test.concurrent("installs without a nested copy and is stable on reload", async () => { + const dir = await makeRepo({ ...catalogFields, peerSpec, linker: "hoisted" }); + expect(await record(dir, "hoisted")).toEqual({ + keys: dedupedKeys, + fresh: "", + keysAfterReload: dedupedKeys, + reload: "", + reloadSavedLockfile: false, + }); + }); + }); + + // pnpm #12159 shape: an override whose value is a catalog reference wins over the peer's own range, fresh and on reload. + describe.each(linkers)("linker=%s", linker => { + describe.each([ + [">=1.0.0", "catalog:", { catalog: { "no-deps": "1.0.0" } }], + ["^2.0.0", "catalog:", { catalog: { "no-deps": "1.0.0" } }], + ["catalog:", "catalog:", { catalog: { "no-deps": "1.0.0" } }], + ["catalog:", "catalog:pins", { catalog: { "no-deps": ">=1.0.0" }, catalogs: { pins: { "no-deps": "1.0.0" } } }], + ] as const)("peer %s overridden to %s", (peerSpec, override, catalogFields) => { + test.concurrent("binds to the overriding catalog entry, fresh == reload", async () => { + const dir = await makeRepo({ ...catalogFields, overrides: { "no-deps": override }, peerSpec, linker }); + const result = await record(dir, linker); + const keys = ["app", "lib", "no-deps", "one-fixed-dep"]; + expect(result).toEqual({ + keys, + fresh: linker === "isolated" ? expect.stringContaining(isolatedNoDeps("1.0.0")) : "", + keysAfterReload: keys, + reload: result.fresh, + reloadSavedLockfile: false, + }); + const { packages } = Bun.JSONC.parse(await Bun.file(join(dir, "bun.lock")).text()) as { + packages: Record; + }; + expect(packages["no-deps"][0]).toBe("no-deps@1.0.0"); + }); + }); + }); + + // pnpm #8996 (`catalog:` peers survive `pack` unsubstituted) and #7072 (`catalog:` / `catalog:default` are one catalog). + describe.each([ + ["catalog:", { catalog: { "no-deps": ">=1.0.0" } }], + ["catalog:peers", { catalogs: { peers: { "no-deps": ">=1.0.0" } } }], + ["catalog:", { catalogs: { default: { "no-deps": ">=1.0.0" } } }], + ["catalog:default", { catalog: { "no-deps": ">=1.0.0" } }], + ] as const)("bun pm pack substitutes the %s peer (%o)", (peerSpec, catalogFields) => { + test.concurrent("with the catalog's range", async () => { + const dir = await makeRepo({ ...catalogFields, peerSpec, libVersion: "1.2.3", linker: "hoisted" }); + await install(dir, "hoisted"); + const libDir = join(dir, "packages", "lib"); + await pack(libDir, bunEnv); + const tarball = readTarball(join(libDir, "lib-1.2.3.tgz")); + const packageJson = tarball.entries.find( + (entry: { pathname: string }) => entry.pathname === "package/package.json", + ); + expect(JSON.parse(packageJson.contents)).toEqual({ + name: "lib", + version: "1.2.3", + peerDependencies: { "no-deps": ">=1.0.0" }, + }); + }); + }); +}); diff --git a/test/cli/install/config-precedence.test.ts b/test/cli/install/config-precedence.test.ts index fa083177bf0d..383830dd961f 100644 --- a/test/cli/install/config-precedence.test.ts +++ b/test/cli/install/config-precedence.test.ts @@ -62,6 +62,8 @@ async function install(root: string, args: string[] = []) { USERPROFILE: home, XDG_CONFIG_HOME: home, BUN_INSTALL_CACHE_DIR: join(root, "cache"), + // Authenticated requests leak their header buffer into the HTTP client (NetworkTask.rs); LSan would abort the install. + ASAN_OPTIONS: [bunEnv.ASAN_OPTIONS, "detect_leaks=0"].filter(Boolean).join(":"), }, stdout: "pipe", stderr: "pipe", @@ -183,6 +185,39 @@ describe.concurrent("bun install config precedence", () => { expect(existsSync(join(String(dir), "project", "node_modules", "@types", "no-deps", "package.json"))).toBe(true); }); + test("bunfig registry credentials survive a same-URL registry= line in project .npmrc", async () => { + using dir = tempDir("config-precedence", { + "project/.npmrc": `registry=${registry.registryUrl()}\n`, + "project/bunfig.toml": bunfig({ registry: { url: registry.registryUrl(), token: authToken } }), + "project/package.json": packageJson({ "@needs-auth/test-pkg": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(existsSync(join(String(dir), "project", "node_modules", "@needs-auth", "test-pkg", "package.json"))).toBe( + true, + ); + }); + + test("bunfig scoped registry credentials survive the same scope URL in project .npmrc", async () => { + using dead = deadRegistry(); + using dir = tempDir("config-precedence", { + "project/.npmrc": `@needs-auth:registry=${registry.registryUrl()}\n`, + "project/bunfig.toml": bunfig({ + registry: dead.url, + scopes: { "needs-auth": { url: registry.registryUrl(), token: authToken } }, + }), + "project/package.json": packageJson({ "@needs-auth/test-pkg": "1.0.0" }), + }); + const { stderr, exitCode } = await install(String(dir)); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(dead.hits).toBe(0); + expect(existsSync(join(String(dir), "project", "node_modules", "@needs-auth", "test-pkg", "package.json"))).toBe( + true, + ); + }); + test("--linker beats ~/.npmrc, project .npmrc and bunfig", async () => { using dir = tempDir("config-precedence", { "home/.npmrc": "install-strategy=hoisted\n", diff --git a/test/cli/install/frozen-lockfile-missing-workspace.test.ts b/test/cli/install/frozen-lockfile-missing-workspace.test.ts new file mode 100644 index 000000000000..bd263a35aebd --- /dev/null +++ b/test/cli/install/frozen-lockfile-missing-workspace.test.ts @@ -0,0 +1,112 @@ +import { file, write } from "bun"; +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { exists, rm } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe } from "harness"; +import { join } from "path"; + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +const app = { name: "app", version: "1.0.0", dependencies: { "no-deps": "1.0.0" } }; +const shared = { name: "shared", version: "1.0.0" }; + +const notFound = 'Workspace not found "packages/api"'; + +// `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. +async function bun(dir: string, args: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...args, "--linker", "hoisted"], + cwd: dir, + env: bunEnv, + stdin: "ignore", + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; +} + +// bun.lock ends up describing `onDisk`; package.json lists `listed`; `remove` folders and node_modules are gone. +async function tree(onDisk: Record, listed: string[], remove: string[] = []) { + const { packageDir: dir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); + await Promise.all([ + write(join(dir, "package.json"), JSON.stringify({ name: "mono", workspaces: Object.keys(onDisk) })), + ...Object.entries(onDisk).map(([path, pkg]) => write(join(dir, path, "package.json"), JSON.stringify(pkg))), + ]); + + const { stderr, exitCode } = await bun(dir, ["install"]); + expect(stderr).toContain("Saved lockfile"); + expect(exitCode).toBe(0); + const lock = await file(join(dir, "bun.lock")).text(); + + await Promise.all([ + write(join(dir, "package.json"), JSON.stringify({ name: "mono", workspaces: listed })), + rm(join(dir, "node_modules"), { recursive: true }), + ...remove.map(path => rm(join(dir, path), { recursive: true })), + ]); + return { dir, lock }; +} + +async function expectRejected(dir: string, lock: string, args: string[]) { + const { stderr, exitCode } = await bun(dir, args); + expect(stderr).toContain(notFound); + expect(stderr).not.toContain("lockfile had changes"); + expect(await file(join(dir, "bun.lock")).text()).toBe(lock); + expect(await exists(join(dir, "node_modules"))).toBeFalse(); + expect(exitCode).toBe(1); + return { stderr }; +} + +const notFoundLine = (stderr: string) => stderr.split("\n").find(line => line.includes("Workspace not found")); + +describe("a listed workspace that is on neither disk nor in bun.lock", () => { + test.concurrent("--frozen-lockfile rejects it like a plain install", async () => { + const { dir, lock } = await tree({ "packages/app": app }, ["packages/app", "packages/api"]); + + const plain = await bun(dir, ["install"]); + expect(plain.stderr).toContain(notFound); + expect(plain.exitCode).toBe(1); + expect(await exists(join(dir, "node_modules"))).toBeFalse(); + + const { stderr } = await expectRejected(dir, lock, ["install", "--frozen-lockfile"]); + + expect(notFoundLine(stderr)).toBeDefined(); + expect(notFoundLine(stderr)).toBe(notFoundLine(plain.stderr)); + }); + + test.concurrent("--production rejects it too", async () => { + const { dir, lock } = await tree({ "packages/app": app }, ["packages/app", "packages/api"]); + + await expectRejected(dir, lock, ["install", "--production"]); + }); + + test.concurrent("bun ci rejects it too", async () => { + const { dir, lock } = await tree({ "packages/app": app }, ["packages/app", "packages/api"]); + + await expectRejected(dir, lock, ["ci"]); + }); + + test.concurrent( + "a workspace pruned from disk but still in bun.lock is tolerated while an unknown one next to it is still rejected", + async () => { + const { dir, lock } = await tree( + { "packages/app": app, "packages/shared": shared }, + ["packages/app", "packages/shared", "packages/api"], + ["packages/shared"], + ); + expect(lock).toContain('"packages/shared"'); + expect(lock).not.toContain('"packages/api"'); + + const { stderr } = await expectRejected(dir, lock, ["install", "--frozen-lockfile"]); + + expect(stderr).not.toContain('Workspace not found "packages/shared"'); + }, + ); +}); diff --git a/test/cli/install/frozen-lockfile-pruned.test.ts b/test/cli/install/frozen-lockfile-pruned.test.ts index 94c3a30a78c7..ecf42de44872 100644 --- a/test/cli/install/frozen-lockfile-pruned.test.ts +++ b/test/cli/install/frozen-lockfile-pruned.test.ts @@ -18,10 +18,11 @@ afterAll(() => { registry.stop(); }); -const rootPackageJson: PackageJson = { name: "mono", workspaces: ["packages/*"] }; +const rootPackageJson: PackageJson = { name: "mono", workspaces: ["packages/*"], trustedDependencies: ["a-dep"] }; const appPackageJson: PackageJson = { name: "app", version: "1.0.0", + bin: { "app-cli": "cli.js" }, dependencies: { shared: "workspace:*", "a-dep": "1.0.1" }, }; const sharedPackageJson: PackageJson = { @@ -46,6 +47,8 @@ async function writeTree(dir: string, tree: Tree, workspaces: string[] = Object. ]); } +const survivors = ["packages/app", "packages/shared"]; + const monorepo: Tree = { root: rootPackageJson, packages: { @@ -53,17 +56,15 @@ const monorepo: Tree = { "packages/shared": sharedPackageJson, "packages/other": otherPackageJson, }, + files: { "packages/app/cli.js": "" }, }; -const survivors = ["packages/app", "packages/shared"]; - -async function writeMonorepo(dir: string, { withOther }: { withOther: boolean }) { - await writeTree(dir, monorepo, withOther ? undefined : survivors); -} +// turbo prune rewrites the root workspaces list to the survivors and copies everything else, trustedDependencies included. +const turboOutput: Tree = { ...monorepo, root: { ...rootPackageJson, workspaces: survivors } }; const explicitMonorepo: Tree = { ...monorepo, - root: { name: "mono", workspaces: ["packages/app", "packages/shared", "packages/other"] }, + root: { ...rootPackageJson, workspaces: ["packages/app", "packages/shared", "packages/other"] }, }; // `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. @@ -71,7 +72,8 @@ async function raw(dir: string, linker: Linker, args: string[], cwd = dir) { await using proc = Bun.spawn({ cmd: [bunExe(), ...args, "--linker", linker], cwd, - env: bunEnv, + // CI exports BUN_INSTALL_CACHE_DIR, which overrides the bunfig's per-test `cache`; concurrent cases sharing one cache race on Windows. + env: { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }, stdin: "ignore", stdout: "pipe", stderr: "pipe", @@ -124,6 +126,8 @@ function fullLockfile(linker: Linker): Promise { expect(full).toContain('"no-deps"'); expect(full).toContain('"left-pad"'); expect(full).toContain('"packages/other"'); + expect(full).toContain('"trustedDependencies"'); + expect(full).toContain('"app-cli"'); return full; }); fullLockfiles.set(linker, lock); @@ -131,20 +135,35 @@ function fullLockfile(linker: Linker): Promise { return lock; } -// Same shape `turbo prune` emits: the workspace and its exclusive packages go, the peer-reachable no-deps entry stays. -function turboPrune(lock: string): string { +const trustedDependenciesSection = /\n "trustedDependencies": \[\n(?: [^\n]*\n)* \],/; +const workspaceBinField = /,\n "bin": \{\n(?: [^\n]*\n)* \}/; + +// The three edits turbo prune makes to Bun's own bun.lock: workspace + exclusive packages removed, trustedDependencies emitted empty (i.e. omitted), workspace bin dropped. +function stripTurboFields(lock: string): string { + expect(lock).toContain('"trustedDependencies"'); + expect(lock).toContain('"app-cli"'); const pruned = lock .replace(/ "packages\/other": \{\n(?: .*\n)* \},\n/, "") - .replace(/\n "(?:other|left-pad)": \[[^\n]*\],\n\n?/g, "\n"); - expect(pruned).toContain('"no-deps": ["no-deps@1.0.0"'); + .replace(/\n "(?:other|left-pad)": \[[^\n]*\],\n\n?/g, "\n") + .replace(trustedDependenciesSection, "") + .replace(workspaceBinField, ""); + expect(pruned).not.toContain('"trustedDependencies"'); + expect(pruned).not.toContain('"bin"'); + expect(pruned).toContain('"packages/app"'); expect(pruned).not.toContain('"other"'); expect(pruned).not.toContain('"left-pad"'); return pruned; } +function turboPrune(lock: string): string { + const pruned = stripTurboFields(lock); + expect(pruned).toContain('"no-deps": ["no-deps@1.0.0"'); + return pruned; +} + async function prunedTree(linker: Linker) { const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker } }); - await writeMonorepo(packageDir, { withOther: false }); + await writeTree(packageDir, turboOutput, survivors); const pruned = turboPrune(await fullLockfile(linker)); await write(join(packageDir, "bun.lock"), pruned); return { packageDir, pruned }; @@ -152,7 +171,7 @@ async function prunedTree(linker: Linker) { async function verbatimTree(linker: Linker) { const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker } }); - await writeMonorepo(packageDir, { withOther: false }); + await writeTree(packageDir, monorepo, survivors); const full = await fullLockfile(linker); await write(join(packageDir, "bun.lock"), full); return { packageDir, full }; @@ -175,27 +194,42 @@ function installedPath(dir: string, linker: Linker, name: string, version: strin describe.each(["hoisted", "isolated"] as Linker[])("linker: %s", linker => { // Also pins that the optional-peer-bound no-deps is installed even though only the pruned workspace needed it (pnpm#6264). - test.concurrent("turbo-pruned lockfile: package held only by an optional peer passes --frozen-lockfile", async () => { - const { packageDir, pruned } = await prunedTree(linker); + test.concurrent( + "turbo output: peer-held package survives --frozen-lockfile although turbo dropped trustedDependencies and the workspace bin from bun.lock", + async () => { + const { packageDir, pruned } = await prunedTree(linker); - await frozen(packageDir, linker, 0); + await frozen(packageDir, linker, 0); - expect(await lockText(packageDir)).toBe(pruned); - const noDeps = - linker === "hoisted" - ? join(packageDir, "node_modules", "no-deps", "package.json") - : join(packageDir, "packages", "shared", "node_modules", "no-deps", "package.json"); - expect(await file(noDeps).json()).toEqual({ name: "no-deps", version: "1.0.0" }); - }); + expect(await lockText(packageDir)).toBe(pruned); + const noDeps = + linker === "hoisted" + ? join(packageDir, "node_modules", "no-deps", "package.json") + : join(packageDir, "packages", "shared", "node_modules", "no-deps", "package.json"); + expect(await file(noDeps).json()).toEqual({ name: "no-deps", version: "1.0.0" }); + }, + ); - test.concurrent("turbo-pruned lockfile: plain bun install does not rewrite it", async () => { - const { packageDir, pruned } = await prunedTree(linker); + test.concurrent( + "turbo output: plain bun install writes only trustedDependencies back and keeps the peer-held package", + async () => { + const { packageDir } = await prunedTree(linker); - const { stderr } = await install(packageDir, linker); + const { stderr } = await install(packageDir, linker); - expect(stderr).not.toContain("Saved lockfile"); - expect(await lockText(packageDir)).toBe(pruned); - }); + expect(stderr).toContain("Saved lockfile"); + const lock = await lockText(packageDir); + expect(lock).toContain('"trustedDependencies": [\n "a-dep",\n ],'); + expect(lock).toContain('"no-deps": ["no-deps@1.0.0"'); + expect(lock).not.toContain('"other"'); + expect(lock).not.toContain('"left-pad"'); + + const second = await frozen(packageDir, linker, 0); + + expect(second.stderr).not.toContain("Saved lockfile"); + expect(await lockText(packageDir)).toBe(lock); + }, + ); test.concurrent("verbatim full lockfile with a workspace folder missing passes --frozen-lockfile", async () => { const { packageDir, full } = await verbatimTree(linker); @@ -319,7 +353,7 @@ describe.each(["hoisted", "isolated"] as Linker[])("linker: %s", linker => { expect(await exists(installedPath(packageDir, linker, "a-dep", "1.0.1"))).toBeTrue(); }); - // pnpm#11364: the root lists workspaces by path instead of a glob (what `turbo prune` copies verbatim). + // pnpm#11364: explicit workspaces list plus a hand-copied full bun.lock; turbo instead rewrites the list (see turboOutput). test.concurrent("explicitly listed workspace whose folder is missing passes --frozen-lockfile", async () => { const { packageDir, full } = await verbatimScenario(linker, explicitMonorepo, survivors); @@ -366,10 +400,33 @@ describe.each(["hoisted", "isolated"] as Linker[])("linker: %s", linker => { expect(await lockText(packageDir)).toBe(canary); expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); }); + + // Guard that passes on main too: the dropped trustedDependencies/bin alone are not a frozen failure. + test.concurrent("turbo output without a peer-held package passes --frozen-lockfile too", async () => { + const tree: Tree = { + root: rootPackageJson, + packages: { + "packages/app": appPackageJson, + "packages/shared": { name: "shared", version: "1.0.0" }, + "packages/other": otherPackageJson, + }, + files: monorepo.files, + }; + const { full } = await fullInstall(linker, tree); + const pruned = stripTurboFields(full).replace(/\n "no-deps": \[[^\n]*\],\n\n?/, "\n"); + expect(pruned).not.toContain('"no-deps"'); + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker } }); + await writeTree(packageDir, { ...tree, root: { ...rootPackageJson, workspaces: survivors } }, survivors); + await write(join(packageDir, "bun.lock"), pruned); + + await frozen(packageDir, linker, 0); + + expect(await lockText(packageDir)).toBe(pruned); + }); }); describe("hoisted", () => { - test.concurrent("bun ci behaves the same on the pruned turbo lockfile", async () => { + test.concurrent("bun ci behaves the same on turbo output", async () => { const { packageDir, pruned } = await prunedTree("hoisted"); await frozen(packageDir, "hoisted", 0, ["ci"]); @@ -381,24 +438,25 @@ describe("hoisted", () => { }); }); - // pnpm#11364, turbo shape: bun.lock no longer lists the workspace but the copied root package.json still does. - test.concurrent("explicitly listed workspace missing from disk and from a turbo-pruned bun.lock", async () => { - const { full } = await fullInstall("hoisted", explicitMonorepo); - const pruned = turboPrune(full); - const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); - await writeTree(packageDir, explicitMonorepo, survivors); - await write(join(packageDir, "bun.lock"), pruned); + // pnpm#11364: not on disk and not in bun.lock is a broken workspaces list, not a pruned checkout. + test.concurrent( + "explicitly listed workspace missing from disk and from a turbo-pruned bun.lock fails --frozen-lockfile", + async () => { + const { full } = await fullInstall("hoisted", explicitMonorepo); + const pruned = turboPrune(full); + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); + await writeTree(packageDir, explicitMonorepo, survivors); + await write(join(packageDir, "bun.lock"), pruned); - const { stderr } = await frozen(packageDir, "hoisted", 0); + const { stderr, exitCode } = await raw(packageDir, "hoisted", ["install", "--frozen-lockfile"]); - expect(stderr).not.toContain("Workspace not found"); - expect(stderr).not.toContain("not on disk"); - expect(await lockText(packageDir)).toBe(pruned); - expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); - expect(await file(join(packageDir, "node_modules", "no-deps", "package.json")).json()).toMatchObject({ - version: "1.0.0", - }); - }); + expect(stderr).toContain('Workspace not found "packages/other"'); + expect(stderr).not.toContain("not on disk"); + expect(await lockText(packageDir)).toBe(pruned); + expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); + expect(exitCode).toBe(1); + }, + ); test.concurrent( "explicitly listed workspace missing: --frozen-lockfile from inside a surviving workspace", @@ -541,7 +599,7 @@ describe("hoisted", () => { test.concurrent("pruned bun.lock written with the hoisted linker passes frozen under other settings", async () => { const pruned = turboPrune(await fullLockfile("hoisted")); const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); - await writeMonorepo(packageDir, { withOther: false }); + await writeTree(packageDir, turboOutput, survivors); await write(join(packageDir, "bun.lock"), pruned); await frozen(packageDir, "isolated", 0); @@ -764,6 +822,10 @@ describe("hoisted", () => { join(packageDir, "packages", "newpkg", "package.json"), JSON.stringify({ name: "newpkg", version: "1.0.0" }), ); + await write( + join(packageDir, "package.json"), + JSON.stringify({ ...rootPackageJson, workspaces: [...survivors, "packages/newpkg"] }), + ); await frozen(packageDir, "hoisted", 1); @@ -785,7 +847,7 @@ describe("hoisted", () => { test.concurrent("a workspace that is on disk but no longer globbed is not treated as pruned", async () => { const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); - await writeMonorepo(packageDir, { withOther: true }); + await writeTree(packageDir, monorepo); const full = await fullLockfile("hoisted"); await write(join(packageDir, "bun.lock"), full); await write( @@ -798,6 +860,44 @@ describe("hoisted", () => { expect(await lockText(packageDir)).toBe(full); }); + test.concurrent( + "single-package project: trustedDependencies stripped from bun.lock does not drop a peer-held package under --frozen-lockfile", + async () => { + const single: Tree = { + root: { + name: "single", + dependencies: { "optional-peer-deps": "1.0.0", "no-deps": "1.0.0" }, + trustedDependencies: ["optional-peer-deps"], + }, + packages: {}, + }; + const { full } = await fullInstall("hoisted", single); + expect(full).toContain('"trustedDependencies"'); + // The 8-space row is the root's declared dependency; the package entry stays, held only by optional-peer-deps' peer slot. + const pruned = full.replace(/\n "no-deps": "1\.0\.0",/, "").replace(trustedDependenciesSection, ""); + expect(pruned).not.toBe(full); + expect(pruned).toContain('"no-deps": ["no-deps@1.0.0"'); + const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); + await write( + join(packageDir, "package.json"), + JSON.stringify({ + name: "single", + dependencies: { "optional-peer-deps": "1.0.0" }, + trustedDependencies: ["optional-peer-deps"], + }), + ); + await write(join(packageDir, "bun.lock"), pruned); + + await frozen(packageDir, "hoisted", 0); + + expect(await lockText(packageDir)).toBe(pruned); + expect(await file(join(packageDir, "node_modules", "no-deps", "package.json")).json()).toEqual({ + name: "no-deps", + version: "1.0.0", + }); + }, + ); + // An invalid prune (turbo always keeps transitive workspace deps): the survivor's `workspace:` edge is reported. test.concurrent("a survivor depending on a pruned workspace fails naming the missing workspace", async () => { const tree: Tree = { diff --git a/test/cli/install/isolated-relink.test.ts b/test/cli/install/isolated-relink.test.ts new file mode 100644 index 000000000000..39e12f1d4766 --- /dev/null +++ b/test/cli/install/isolated-relink.test.ts @@ -0,0 +1,75 @@ +import { file, write } from "bun"; +import { afterAll, beforeAll, expect, test } from "bun:test"; +import { lstat, realpath } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe, normalizeBunSnapshot } from "harness"; +import { dirname, join } from "path"; + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +// `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. +async function install(dir: string, ...args: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), "install", ...args, "--linker", "isolated"], + env: bunEnv, + cwd: dir, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + return stdout; +} + +// Store entry names may carry hash suffixes, so reach one-range-dep's store node_modules through the top-level link. +async function nestedNoDeps(packageDir: string) { + const storeEntry = await realpath(join(packageDir, "node_modules", "one-range-dep")); + return join(dirname(storeEntry), "no-deps"); +} + +function nestedNoDepsPackageJson(link: string) { + return file(join(link, "package.json")).json(); +} + +test.concurrent("an existing store entry is re-linked when an override re-resolves its dependency", async () => { + const { packageDir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); + const storeHashPath = join(packageDir, "node_modules", ".bun", ".store-hash"); + const storeHash = file(storeHashPath); + + await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0" } })); + await install(packageDir); + const link = await nestedNoDeps(packageDir); + expect(await nestedNoDepsPackageJson(link)).toEqual({ name: "no-deps", version: "1.1.0" }); + const storeHashAfterFirstInstall = await storeHash.text(); + + await write( + packageJson, + JSON.stringify({ + name: "foo", + dependencies: { "one-range-dep": "1.0.0" }, + overrides: { "no-deps": "1.0.0" }, + }), + ); + const out = await install(packageDir); + expect(await nestedNoDepsPackageJson(link)).toEqual({ name: "no-deps", version: "1.0.0" }); + expect(out).toMatch(/\d+ packages? installed/); + expect(out).not.toContain("(no changes)"); + expect(await storeHash.text()).not.toBe(storeHashAfterFirstInstall); + + const linkMtime = (await lstat(link)).mtimeMs; + const storeHashMtime = (await lstat(storeHashPath)).mtimeMs; + const again = await install(packageDir); + expect(normalizeBunSnapshot(again)).toContain("(no changes)"); + expect((await lstat(link)).mtimeMs).toBe(linkMtime); + expect(await nestedNoDepsPackageJson(link)).toEqual({ name: "no-deps", version: "1.0.0" }); + expect(await storeHash.exists()).toBe(true); + expect((await lstat(storeHashPath)).mtimeMs).toBe(storeHashMtime); +}); diff --git a/test/cli/install/lockfile-only.test.ts b/test/cli/install/lockfile-only.test.ts index cdc9f1477ef3..4e52ddd6f180 100644 --- a/test/cli/install/lockfile-only.test.ts +++ b/test/cli/install/lockfile-only.test.ts @@ -1,7 +1,8 @@ import { spawn } from "bun"; -import { afterAll, afterEach, beforeAll, beforeEach, expect, it } from "bun:test"; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from "bun:test"; +import { existsSync, readFileSync, writeFileSync } from "fs"; import { access, writeFile } from "fs/promises"; -import { bunExe, bunEnv as env } from "harness"; +import { bunExe, bunEnv as env, tempDir } from "harness"; import { join } from "path"; import { dummyAfterAll, @@ -82,3 +83,135 @@ it.each(["bun.lockb", "bun.lock"])("should not download tarballs with --lockfile await access(join(package_dir, lockfile)); }); + +describe("--lockfile-only under --frozen-lockfile", () => { + const project = { + "foo/package.json": JSON.stringify({ name: "foo", version: "1.0.0" }), + "package.json": JSON.stringify({ name: "mig", dependencies: { foo: "file:./foo" } }), + }; + const npmLock = JSON.stringify({ + name: "mig", + lockfileVersion: 3, + packages: { + "": { name: "mig", dependencies: { foo: "file:./foo" } }, + foo: { name: "foo", version: "1.0.0" }, + "node_modules/foo": { resolved: "foo", link: true }, + }, + }); + const pnpmLock = [ + "lockfileVersion: '9.0'", + "importers:", + " .:", + " dependencies:", + " foo:", + " specifier: file:./foo", + " version: file:foo", + "packages:", + " foo@file:foo:", + " resolution: {directory: foo, type: directory}", + "snapshots:", + " foo@file:foo: {}", + "", + ].join("\n"); + const migrations: [string, string][] = [ + ["package-lock.json", npmLock], + ["pnpm-lock.yaml", pnpmLock], + ]; + + async function run(dir: string, ...args: string[]) { + await using proc = spawn({ + cmd: [bunExe(), "install", ...args], + cwd: dir, + env, + stdout: "pipe", + stderr: "pipe", + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; + } + const lock = (dir: string) => join(dir, "bun.lock"); + + it.concurrent.each(["--frozen-lockfile", "--production"])( + "%s --lockfile-only leaves an up-to-date bun.lock byte-identical", + async flag => { + using tmp = tempDir("lockfile-only-frozen", project); + const dir = String(tmp); + expect((await run(dir, "--lockfile-only")).exitCode).toBe(0); + const canary = readFileSync(lock(dir), "utf8") + "\n"; + writeFileSync(lock(dir), canary); + + const { stdout, stderr, exitCode } = await run(dir, flag, "--lockfile-only"); + expect(stdout + stderr).not.toContain("Saved"); + expect(readFileSync(lock(dir), "utf8")).toBe(canary); + expect(existsSync(join(dir, "node_modules"))).toBe(false); + expect(exitCode).toBe(0); + }, + ); + + it.concurrent.each(["--frozen-lockfile", "--production"])( + "%s --lockfile-only does not create a missing bun.lock", + async flag => { + using tmp = tempDir("lockfile-only-frozen-missing", project); + const dir = String(tmp); + + const { stdout, stderr, exitCode } = await run(dir, flag, "--lockfile-only"); + expect(stdout + stderr).not.toContain("Saved"); + expect(existsSync(lock(dir))).toBe(false); + expect(existsSync(join(dir, "node_modules"))).toBe(false); + expect(exitCode).toBe(0); + }, + ); + + it.concurrent("--lockfile-only rewrites an up-to-date bun.lock", async () => { + using tmp = tempDir("lockfile-only-rewrite", project); + const dir = String(tmp); + expect((await run(dir, "--lockfile-only")).exitCode).toBe(0); + const original = readFileSync(lock(dir), "utf8"); + writeFileSync(lock(dir), original + "\n"); + + const { stdout, exitCode } = await run(dir, "--lockfile-only"); + expect(stdout).toContain("Saved bun.lock (2 packages)"); + expect(readFileSync(lock(dir), "utf8")).toBe(original); + expect(existsSync(join(dir, "node_modules"))).toBe(false); + expect(exitCode).toBe(0); + }); + + it.concurrent.each(migrations)("--lockfile-only migrates %s", async (name, contents) => { + using tmp = tempDir("lockfile-only-migrate", { ...project, [name]: contents }); + const dir = String(tmp); + + const { stdout, stderr, exitCode } = await run(dir, "--lockfile-only"); + expect(stderr).toContain(`migrated lockfile from ${name}`); + expect(stdout).toContain("Saved bun.lock (2 packages)"); + expect(readFileSync(lock(dir), "utf8")).toContain('"foo": ["foo@file:foo", {}]'); + expect(existsSync(join(dir, "node_modules"))).toBe(false); + expect(exitCode).toBe(0); + }); + + it.concurrent.each(migrations)( + "--frozen-lockfile --lockfile-only still writes bun.lock when migrating from %s", + async (name, contents) => { + using tmp = tempDir("lockfile-only-frozen-migrate", { ...project, [name]: contents }); + const dir = String(tmp); + + const { stdout, stderr, exitCode } = await run(dir, "--frozen-lockfile", "--lockfile-only"); + expect(stderr).toContain(`migrated lockfile from ${name}`); + expect(stdout).toContain("Saved bun.lock (2 packages)"); + expect(readFileSync(lock(dir), "utf8")).toContain('"foo": ["foo@file:foo", {}]'); + expect(existsSync(join(dir, "node_modules"))).toBe(false); + expect(exitCode).toBe(0); + }, + ); + + it.concurrent.each(migrations)("--frozen-lockfile writes bun.lock when migrating from %s", async (name, contents) => { + using tmp = tempDir("frozen-migrate", { ...project, [name]: contents }); + const dir = String(tmp); + + const { stderr, exitCode } = await run(dir, "--frozen-lockfile"); + expect(stderr).toContain(`migrated lockfile from ${name}`); + expect(existsSync(lock(dir))).toBe(true); + expect(readFileSync(lock(dir), "utf8")).toContain('"foo": ["foo@file:foo", {}]'); + expect(existsSync(join(dir, "node_modules", "foo", "package.json"))).toBe(true); + expect(exitCode).toBe(0); + }); +}); diff --git a/test/cli/install/migration/__snapshots__/yarn-lock-migration.test.ts.snap b/test/cli/install/migration/__snapshots__/yarn-lock-migration.test.ts.snap index d27dcfed3132..189ad8e4655e 100644 --- a/test/cli/install/migration/__snapshots__/yarn-lock-migration.test.ts.snap +++ b/test/cli/install/migration/__snapshots__/yarn-lock-migration.test.ts.snap @@ -195,7 +195,7 @@ exports[`yarn.lock migration basic yarn.lock with npm aliases: aliases-yarn-migr exports[`yarn.lock migration basic yarn.lock with resolutions: resolutions-yarn-migration 1`] = ` "{ - "lockfileVersion": 2, + "lockfileVersion": 3, "configVersion": 1, "workspaces": { "": { @@ -207,6 +207,9 @@ exports[`yarn.lock migration basic yarn.lock with resolutions: resolutions-yarn- }, "overrides": { "acorn": "8.11.3", + "webpack": { + "acorn": "8.11.2", + }, }, "packages": { "acorn": ["acorn@8.11.3", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-Y9rRfJG5jcKOE0CLisYbojUjIrIEE7AGMzA/Sm4BslANhbS+cDMpgBdcPT91oJ7OuJ9hYJBx59RjbhxVnrF8Xg=="], diff --git a/test/cli/install/migration/pnpm-lock-v9.test.ts b/test/cli/install/migration/pnpm-lock-v9.test.ts index 182c399b4bcc..6c61d2a24de7 100644 --- a/test/cli/install/migration/pnpm-lock-v9.test.ts +++ b/test/cli/install/migration/pnpm-lock-v9.test.ts @@ -681,8 +681,16 @@ importers: `; } - // pnpm/pnpm#5928 (`-` removes the dependency) and pnpm/pnpm#6774 (`name@range` / `parent>child` keys) - test.concurrent("unsupported removal and selector keys warn but migrate", async () => { + function overridesSection(bunLock: string) { + const start = bunLock.indexOf(` "overrides": {`); + expect(start).not.toBe(-1); + const end = bunLock.indexOf("\n },", start); + expect(end).not.toBe(-1); + return bunLock.slice(start, end + "\n },".length); + } + + // pnpm/pnpm#5928 (`-` removes the dependency) warns; pnpm/pnpm#6774 (`name@range` keys) now migrates as ranged rules + test.concurrent("removal values warn; name@range keys migrate as ranged rules", async () => { using dir = tempDir("pnpm-v9-overrides-unsupported", { "package.json": JSON.stringify({ name: "overrides-unsupported" }), "pnpm-lock.yaml": overridesLockfile(` left-pad: '-' @@ -696,23 +704,70 @@ importers: const { stderr, exitCode } = await migrate(String(dir)); expect(stderr).toContain("pnpm-lock.yaml override 'left-pad' removes the dependency ('-')"); - expect(stderr).toContain( - "pnpm-lock.yaml override 'semver@<7.5.2' is scoped to a version range or parent package", - ); - expect(stderr).toContain("pnpm-lock.yaml override 'foo>bar' is scoped to a version range or parent package"); - expect(stderr).toContain( - "pnpm-lock.yaml override '@scope/pkg@^1' is scoped to a version range or parent package", - ); + expect(stderr).not.toContain("override 'semver@<7.5.2'"); + expect(stderr).not.toContain("override '@scope/pkg@^1'"); + expect(stderr).not.toContain("override 'foo>bar'"); expect(stderr).not.toContain("override '@scope/plain'"); expect(stderr).not.toContain("override 'plain'"); expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); expect(exitCode).toBe(0); const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"lockfileVersion": 3`); + expect(bunLock).toContain(`"semver@<7.5.2": {`); + expect(bunLock).toContain(`".": "7.5.2"`); + expect(bunLock).toContain(`"@scope/pkg@^1": {`); + expect(bunLock).toContain(`".": "1.9.0"`); + expect(bunLock).toContain(`"foo": {`); + expect(bunLock).toContain(`"bar": "2.0.0"`); expect(bunLock).toContain(`"plain": "1.0.0"`); expect(bunLock).toContain(`"@scope/plain": "3.0.0"`); }); + test.concurrent("parent selectors become nested rules", async () => { + using dir = tempDir("pnpm-v9-overrides-nested", { + "package.json": JSON.stringify({ name: "overrides-nested" }), + "pnpm-lock.yaml": overridesLockfile(` foo>bar: 2.0.0 + foo@^1>baz: 1.0.0 + '@s/a>@t/b': 3.0.0`), + }); + using tooDeep = tempDir("pnpm-v9-overrides-too-deep", { + "package.json": JSON.stringify({ name: "overrides-too-deep" }), + "pnpm-lock.yaml": overridesLockfile(" a>b>c: 1.0.0"), + }); + + const nested = await migrate(String(dir)); + + expect(nested.stderr).not.toContain("does not support"); + expect(nested.stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(nested.exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`"lockfileVersion": 3`); + expect(overridesSection(bunLock)).toMatchInlineSnapshot(` + " "overrides": { + "@s/a": { + "@t/b": "3.0.0", + }, + "foo": { + "bar": "2.0.0", + }, + "foo@^1": { + "baz": "1.0.0", + }, + }," + `); + + const deep = await migrate(String(tooDeep)); + + expect(deep.stderr).toContain( + "pnpm-lock.yaml override 'a>b>c' uses a selector bun does not support; it will not apply", + ); + expect(deep.stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(deep.exitCode).toBe(0); + expect(await bunLockOf(String(tooDeep))).not.toContain("a>b"); + }); + test.concurrent("an unparsable value names the override", async () => { using dir = tempDir("pnpm-v9-overrides-invalid", { "package.json": JSON.stringify({ name: "overrides-invalid" }), diff --git a/test/cli/install/nested-overrides.test.ts b/test/cli/install/nested-overrides.test.ts new file mode 100644 index 000000000000..58340e740804 --- /dev/null +++ b/test/cli/install/nested-overrides.test.ts @@ -0,0 +1,1016 @@ +import { file, write } from "bun"; +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { existsSync, realpathSync } from "fs"; +import { rm } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe } from "harness"; +import { join } from "path"; + +const registry = new VerdaccioRegistry(); + +beforeAll(async () => { + await registry.start(); +}); + +afterAll(() => { + registry.stop(); +}); + +type Linker = "hoisted" | "isolated"; + +// CI exports BUN_INSTALL_CACHE_DIR, which overrides the harness bunfig's per-test `cache`; concurrent cases sharing one cache race on Windows. +const installEnv = (dir: string) => ({ ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }); + +async function run(dir: string, ...cmd: string[]) { + await using proc = Bun.spawn({ + cmd: [bunExe(), ...cmd], + cwd: dir, + env: installEnv(dir), + stdout: "pipe", + stderr: "pipe", + }); + const [out, err, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { out, err, exitCode }; +} + +const install = (dir: string, ...args: string[]) => run(dir, "install", ...args); +const migrate = (dir: string) => run(dir, "pm", "migrate", "-f"); + +async function installOk(dir: string, ...args: string[]) { + const result = await install(dir, ...args); + expect(result.err).not.toContain("error:"); + expect(result.exitCode).toBe(0); + return result; +} + +async function versionSeenBy(packageDir: string, from: string | undefined, name: string): Promise { + const cwd = + from === undefined + ? packageDir + : from.includes("/") + ? join(packageDir, from) + : realpathSync(join(packageDir, "node_modules", from)); + await using proc = Bun.spawn({ + cmd: [bunExe(), "-e", `console.log(require(${JSON.stringify(name + "/package.json")}).version)`], + cwd, + env: bunEnv, + stdout: "pipe", + stderr: "pipe", + }); + const [out, err, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + expect(err).toBe(""); + expect(exitCode).toBe(0); + return out.trim(); +} + +const lock = (dir: string) => file(join(dir, "bun.lock")).text(); + +function overridesSection(text: string) { + const start = text.indexOf(' "overrides": {'); + expect(start).not.toBe(-1); + const end = text.indexOf("\n },\n", start); + expect(end).not.toBe(-1); + return text + .slice(start, end + "\n },".length) + .split("\n") + .map(line => line.slice(2)) + .join("\n"); +} + +async function integrityOf(name: string, version: string): Promise { + const manifest = await file(join(registry.packagesPath, name, "package.json")).json(); + return manifest.versions[version].dist.integrity; +} + +function project(pkg: Record, linker: Linker = "hoisted", extraFiles: Record = {}) { + return registry + .createTestDir({ + bunfigOpts: { linker }, + files: { + "package.json": JSON.stringify({ name: "nested-overrides", ...pkg }), + ...extraFiles, + }, + }) + .then(({ packageDir }) => packageDir); +} + +const twoParents = { ofd1: "npm:one-fixed-dep@1.0.0", ofd2: "npm:one-fixed-dep@2.0.0" }; + +const npmObjectProject = { + dependencies: { "one-dep": "1.0.0", "one-range-dep": "1.0.0" }, + overrides: { "one-dep": { "no-deps": "2.0.0" } }, +}; + +const precedenceProject = { + dependencies: { ...twoParents, "one-range-dep": "1.0.0" }, + overrides: { + "no-deps": "1.0.0", + "one-fixed-dep": { "no-deps": "1.0.1" }, + "one-fixed-dep@2": { "no-deps": "1.1.0" }, + }, +}; + +// one-range-dep -> no-deps 1.0.0, one-dep -> no-deps 2.0.0 +const rangedProject = { + dependencies: { "one-range-dep": "1.0.0", "one-dep": "1.0.0" }, + overrides: { "no-deps@1": "1.0.0", "one-dep": { "no-deps@1": "2.0.0" } }, +}; + +describe.concurrent("syntax", () => { + test("npm object scopes the rule to the parent's edge", async () => { + const dir = await project(npmObjectProject); + const { err } = await installOk(dir); + expect(err).not.toContain("does not support"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + const text = await lock(dir); + expect(overridesSection(text)).toMatchInlineSnapshot(` + ""overrides": { + "one-dep": { + "no-deps": "2.0.0", + }, + }," + `); + expect(text).toContain('"lockfileVersion": 3'); + }); + + test('"." overrides the parent itself next to its children', async () => { + const dir = await project({ + dependencies: { "one-fixed-dep": "^2.0.0" }, + overrides: { "one-fixed-dep": { ".": "1.0.0", "no-deps": "1.1.0" } }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, undefined, "one-fixed-dep")).toBe("1.0.0"); + expect(await versionSeenBy(dir, "one-fixed-dep", "no-deps")).toBe("1.1.0"); + expect(overridesSection(await lock(dir))).toMatchInlineSnapshot(` + ""overrides": { + "one-fixed-dep": { + ".": "1.0.0", + "no-deps": "1.1.0", + }, + }," + `); + }); + + test("parent range is matched against the parent's resolved version, through an alias", async () => { + const dir = await project({ + dependencies: twoParents, + overrides: { "one-fixed-dep@1": { "no-deps": "1.1.0" } }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "ofd1", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("2.0.0"); + expect(overridesSection(await lock(dir))).toMatchInlineSnapshot(` + ""overrides": { + "one-fixed-dep@1": { + "no-deps": "1.1.0", + }, + }," + `); + }); + + test("$ref inside a nested value resolves against the root's dependencies", async () => { + const dir = await project({ + dependencies: { "no-deps": "1.1.0", "one-dep": "1.0.0" }, + overrides: { "one-dep": { "no-deps": "$no-deps" } }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.1.0"); + expect(overridesSection(await lock(dir))).toMatchInlineSnapshot(` + ""overrides": { + "one-dep": { + "no-deps": "1.1.0", + }, + }," + `); + }); + + test("$ref inside a nested value that names nothing warns and is skipped", async () => { + const dir = await project({ + dependencies: { "one-dep": "1.0.0" }, + overrides: { "one-dep": { "no-deps": "$nope" } }, + }); + const { err, exitCode } = await install(dir); + expect(err).toContain('Could not resolve override "$nope" (you need "nope" in your dependencies)'); + expect(exitCode).toBe(0); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); + }); + + describe.concurrent.each([ + "one-dep/no-deps", + "**/one-dep/no-deps", + "**/one-dep/**/no-deps", + "one-dep@npm:1.0.0/no-deps", + ])("yarn resolutions path %s", key => { + test("applies to one-dep's edge only", async () => { + const dir = await project({ + dependencies: { "one-dep": "1.0.0", "one-range-dep": "1.0.0" }, + resolutions: { [key]: "2.0.0" }, + }); + const { err } = await installOk(dir); + expect(err).not.toContain("warn:"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + }); + }); + + test("yarn resolutions path with scoped parent and child", async () => { + const dir = await project({ + dependencies: { "@scoped/has-bin-entry": "1.0.0" }, + resolutions: { "@scoped/has-bin-entry/@types/no-deps": "1.0.0" }, + }); + const { err, exitCode } = await install(dir); + expect(err).not.toContain("warn:"); + expect(exitCode).toBe(0); + const text = await lock(dir); + expect(text).toContain('"@scoped/has-bin-entry": {'); + expect(text).toContain('"@types/no-deps": "1.0.0"'); + }); + + test("pnpm parent>child selector", async () => { + const dir = await project({ + dependencies: { "one-dep": "1.0.0" }, + overrides: { "one-dep>no-deps": "2.0.0" }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + }); + + test("pnpm parent@range>child selectors, including a range containing >", async () => { + const dir = await project({ + dependencies: twoParents, + overrides: { "one-fixed-dep@1>no-deps": "1.1.0", "one-fixed-dep@>=2 <3>no-deps": "1.0.1" }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "ofd1", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("1.0.1"); + expect(overridesSection(await lock(dir))).toMatchInlineSnapshot(` + ""overrides": { + "one-fixed-dep@1": { + "no-deps": "1.1.0", + }, + "one-fixed-dep@>=2 <3": { + "no-deps": "1.0.1", + }, + }," + `); + }); + + test("precedence: ranged parent > unranged parent > flat", async () => { + const dir = await project(precedenceProject); + await installOk(dir); + expect(await versionSeenBy(dir, "ofd1", "no-deps")).toBe("1.0.1"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + expect(overridesSection(await lock(dir))).toMatchInlineSnapshot(` + ""overrides": { + "no-deps": "1.0.0", + "one-fixed-dep": { + "no-deps": "1.0.1", + }, + "one-fixed-dep@2": { + "no-deps": "1.1.0", + }, + }," + `); + }); + + test("a flat rule and an unranged group for the same name share one lockfile object", async () => { + const dir = await project({ + dependencies: { "one-dep": "^1.0.0" }, + overrides: { "one-dep": "1.0.0", "one-dep>no-deps": "2.0.0" }, + }); + await installOk(dir); + const first = await lock(dir); + expect(overridesSection(first)).toMatchInlineSnapshot(` + ""overrides": { + "one-dep": { + ".": "1.0.0", + "no-deps": "2.0.0", + }, + }," + `); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + const { err } = await installOk(dir); + expect(err).not.toContain("Saved lockfile"); + expect(await lock(dir)).toBe(first); + }); + + describe.concurrent("rejected keys warn and are ignored", () => { + test("two levels of objects", async () => { + const dir = await project({ + dependencies: { "one-one-dep": "1.0.0" }, + overrides: { "one-one-dep": { "one-dep": { "no-deps": "2.0.0" } } }, + }); + const { err, exitCode } = await install(dir); + expect(err).toContain('Bun currently only supports one level of nested "overrides"'); + expect(exitCode).toBe(0); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); + expect(await lock(dir)).not.toContain('"lockfileVersion": 3'); + }); + + test("three-segment resolutions path", async () => { + const dir = await project({ + dependencies: { "one-one-dep": "1.0.0" }, + resolutions: { "one-one-dep/one-dep/no-deps": "2.0.0" }, + }); + const { err, exitCode } = await install(dir); + expect(err).toContain('Bun currently only supports one level of nested "resolutions"'); + expect(exitCode).toBe(0); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); + expect(await lock(dir)).not.toContain('"lockfileVersion": 3'); + }); + + test("unparsable parent range", async () => { + const dir = await project({ + dependencies: { "one-dep": "1.0.0" }, + overrides: { "one-dep@banana": { "no-deps": "2.0.0" } }, + }); + const { err, exitCode } = await install(dir); + expect(err).toContain('Invalid version range "banana" for "one-dep"'); + expect(exitCode).toBe(0); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); + expect(await lock(dir)).not.toContain('"lockfileVersion": 3'); + }); + }); +}); + +// The selector range is matched against the range the dependent declares; a rule applies when the two intersect. +describe.concurrent("version-scoped targets", () => { + test("a flat name@range rule applies to edges whose declared range intersects it", async () => { + const dir = await project({ + dependencies: { "one-range-dep": "1.0.0", ...twoParents }, + overrides: { "no-deps@1": "1.0.0" }, + }); + const { err } = await installOk(dir); + expect(err).not.toContain("does not support"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("2.0.0"); + expect(await versionSeenBy(dir, "ofd1", "no-deps")).toBe("1.0.0"); + const text = await lock(dir); + expect(overridesSection(text)).toMatchInlineSnapshot(` + ""overrides": { + "no-deps@1": { + ".": "1.0.0", + }, + }," + `); + expect(text).toContain('"lockfileVersion": 3'); + }); + + test("pnpm audit --fix shaped key with a compound range", async () => { + const dir = await project({ + dependencies: { "one-dep": "1.0.0", ...twoParents }, + overrides: { "no-deps@>=1.0.0 <1.1.0": "1.1.0" }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd1", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("2.0.0"); + expect(overridesSection(await lock(dir))).toContain('"no-deps@>=1.0.0 <1.1.0": {'); + }); + + test("|| alternatives are matched independently", async () => { + const dir = await project({ + dependencies: { "one-dep": "1.0.0", ...twoParents }, + overrides: { "no-deps@1.0.0 || 2.0.0": "1.1.0" }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "ofd1", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); + }); + + test("a non-intersecting range leaves the edge alone and is still recorded", async () => { + const dir = await project({ + dependencies: { "one-range-dep": "1.0.0", "one-dep": "1.0.0" }, + overrides: { "no-deps@2": "1.0.0", "no-deps@<1.0.1": "2.0.0" }, + }); + const { err } = await installOk(dir); + expect(err).not.toContain("warn"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("2.0.0"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); + const section = overridesSection(await lock(dir)); + expect(section).toContain('"no-deps@2": {'); + expect(section).toContain('"no-deps@<1.0.1": {'); + await installOk(dir, "--frozen-lockfile"); + }); + + test("edges declared with a dist-tag never match", async () => { + const dir = await project({ + dependencies: { "no-deps": "latest", "one-range-dep": "1.0.0" }, + overrides: { "no-deps@>=1": "1.0.0" }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, undefined, "no-deps")).toBe("2.0.0"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + }); + + describe.concurrent.each([ + { overrides: { "one-dep>no-deps@1": "2.0.0" } }, + { overrides: { "one-dep": { "no-deps@1": "2.0.0" } } }, + { resolutions: { "one-dep/no-deps@1": "2.0.0" } }, + ])("nested rules accept a target range in every spelling %j", rules => { + test("applies to one-dep's edge only", async () => { + const dir = await project({ dependencies: { "one-dep": "1.0.0", "one-range-dep": "1.0.0" }, ...rules }); + const { err } = await installOk(dir); + expect(err).not.toContain("does not support"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + expect(overridesSection(await lock(dir))).toMatchInlineSnapshot(` + ""overrides": { + "one-dep": { + "no-deps@1": "2.0.0", + }, + }," + `); + }); + }); + + test("a nested rule whose target range does not intersect is inert", async () => { + const dir = await project({ + dependencies: { "one-dep": "1.0.0" }, + overrides: { "one-dep>no-deps@2": "2.0.0", "one-dep>no-deps@1.0.x": "1.1.0" }, + }); + const { err } = await installOk(dir); + expect(err).not.toContain("warn:"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.1.0"); + }); + + test('"." inside a ranged group overrides the parent itself only where the declared range intersects', async () => { + const dir = await project({ + dependencies: { "one-fixed-dep": "^2.0.0" }, + overrides: { "one-fixed-dep@^2": { ".": "1.0.0", "no-deps": "1.1.0" } }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, undefined, "one-fixed-dep")).toBe("1.0.0"); + // The parented rule needs a resolved parent in ^2, and "." just moved the parent to 1.0.0. + expect(await versionSeenBy(dir, "one-fixed-dep", "no-deps")).toBe("1.0.0"); + const first = await lock(dir); + expect(overridesSection(first)).toMatchInlineSnapshot(` + ""overrides": { + "one-fixed-dep@^2": { + ".": "1.0.0", + "no-deps": "1.1.0", + }, + }," + `); + const { err } = await installOk(dir); + expect(err).not.toContain("Saved lockfile"); + expect(await lock(dir)).toBe(first); + }); + + test("precedence: parent tiers first, then a matching target range, then flat", async () => { + const dir = await project({ + dependencies: { "one-range-dep": "1.0.0", "one-dep": "1.0.0", ...twoParents }, + overrides: { + "no-deps": "1.0.0", + "no-deps@1": "1.1.0", + "one-range-dep": { "no-deps": "1.0.1", "no-deps@1": "2.0.0" }, + "one-dep": { "no-deps": "2.0.0" }, + }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("2.0.0"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + expect(await versionSeenBy(dir, "ofd1", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("1.0.0"); + expect(overridesSection(await lock(dir))).toMatchInlineSnapshot(` + ""overrides": { + "no-deps": "1.0.0", + "no-deps@1": { + ".": "1.1.0", + }, + "one-dep": { + "no-deps": "2.0.0", + }, + "one-range-dep": { + "no-deps": "1.0.1", + "no-deps@1": "2.0.0", + }, + }," + `); + }); + + test("two matching ranged rules: the range text that sorts first wins", async () => { + const dir = await project({ + dependencies: { "one-range-dep": "1.0.0" }, + overrides: { "no-deps@>=1": "1.0.0", "no-deps@1": "1.0.1" }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.1"); + }); + + test("an empty selector warns and is skipped", async () => { + const dir = await project({ dependencies: { "one-range-dep": "1.0.0" }, overrides: { "no-deps@": "1.0.0" } }); + const { err, exitCode } = await install(dir); + expect(err).toContain("does not support an empty version selector"); + expect(err).toContain('override "no-deps@" will not apply'); + expect(exitCode).toBe(0); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + expect(await lock(dir)).not.toContain('"overrides"'); + }); + + test("an unparsable target range warns with the same message as a parent range", async () => { + const dir = await project({ + dependencies: { "one-range-dep": "1.0.0" }, + overrides: { "no-deps@banana": "1.0.0" }, + }); + const { err, exitCode } = await install(dir); + expect(err).toContain('Invalid version range "banana" for "no-deps"'); + expect(exitCode).toBe(0); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + expect(await lock(dir)).not.toContain('"lockfileVersion": 3'); + }); +}); + +describe.concurrent.each(["hoisted", "isolated"] as const)("linker=%s", linker => { + test("two dependents of one name see different versions", async () => { + const dir = await project(npmObjectProject, linker); + await installOk(dir); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + }); + + test("workspace packages can be parents", async () => { + const dir = await project( + { + workspaces: ["packages/*"], + overrides: { "app-a": { "no-deps": "1.0.0" }, "app-b": { "no-deps": "2.0.0" } }, + }, + linker, + { + "packages/app-a/package.json": JSON.stringify({ name: "app-a", dependencies: { "no-deps": "*" } }), + "packages/app-b/package.json": JSON.stringify({ name: "app-b", dependencies: { "no-deps": "*" } }), + }, + ); + await installOk(dir); + expect(await versionSeenBy(dir, "packages/app-a", "no-deps")).toBe("1.0.0"); + expect(await versionSeenBy(dir, "packages/app-b", "no-deps")).toBe("2.0.0"); + await installOk(dir, "--frozen-lockfile"); + }); +}); + +describe.concurrent("lockfile", () => { + test("round trip: stable across installs, value changes are frozen-lockfile changes", async () => { + const dir = await project(precedenceProject); + await installOk(dir); + const first = await lock(dir); + await installOk(dir, "--frozen-lockfile"); + const { err } = await installOk(dir); + expect(err).not.toContain("Saved lockfile"); + expect(await lock(dir)).toBe(first); + + // 2.0.0 keeps ofd2's edge nested; the hoisted installer leaves a nested folder behind when an edge moves up to the root's copy. + const pkg = structuredClone(precedenceProject); + pkg.overrides["one-fixed-dep@2"]["no-deps"] = "2.0.0"; + await write(join(dir, "package.json"), JSON.stringify({ name: "nested-overrides", ...pkg })); + const frozen = await install(dir, "--frozen-lockfile"); + expect(frozen.err).toContain("lockfile had changes, but lockfile is frozen"); + expect(frozen.exitCode).toBe(1); + await installOk(dir); + const second = await lock(dir); + expect(overridesSection(second)).toContain('"no-deps": "2.0.0"'); + expect(second).not.toContain("no-deps@1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("2.0.0"); + }); + + test("adding a nested rule while the flat rules stay the same is a change", async () => { + const dir = await project({ dependencies: { "one-dep": "1.0.0" }, overrides: { "no-deps": "1.0.0" } }); + await installOk(dir); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.0"); + await write( + join(dir, "package.json"), + JSON.stringify({ + name: "nested-overrides", + dependencies: { "one-dep": "1.0.0" }, + overrides: { "no-deps": "1.0.0", "one-dep": { "no-deps": "1.1.0" } }, + }), + ); + const frozen = await install(dir, "--frozen-lockfile"); + expect(frozen.err).toContain("lockfile had changes, but lockfile is frozen"); + expect(frozen.exitCode).toBe(1); + await installOk(dir); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.1.0"); + }); + + // An older Bun reading a v3 file takes bun-lock.test.ts's `lockfile version newer than this build supports` path. + test("lockfileVersion 3 is stamped only while a scoped rule exists", async () => { + const flatOnly = { dependencies: { "one-dep": "1.0.0" }, overrides: { "no-deps": "1.0.0" } }; + const [x, y, z] = await Promise.all([ + project(flatOnly), + project({ + dependencies: flatOnly.dependencies, + overrides: { ...flatOnly.overrides, "one-dep": { "no-deps": "1.0.0" } }, + }), + project({ dependencies: flatOnly.dependencies, overrides: { "no-deps@1": "1.0.0" } }), + ]); + await Promise.all([installOk(x), installOk(y), installOk(z)]); + const xLock = await lock(x); + expect(xLock).toContain('"lockfileVersion": 2'); + expect(xLock).toContain('\n "no-deps": "1.0.0",\n'); + expect(await lock(y)).toContain('"lockfileVersion": 3'); + expect(await lock(z)).toContain('"lockfileVersion": 3'); + + const flatOnlyJson = JSON.stringify({ name: "nested-overrides", ...flatOnly }); + await Promise.all([write(join(y, "package.json"), flatOnlyJson), write(join(z, "package.json"), flatOnlyJson)]); + await Promise.all([installOk(y), installOk(z)]); + expect(await lock(y)).toBe(xLock); + expect(await lock(z)).toBe(xLock); + }); + + test("ranged rules round-trip: stable, frozen-clean", async () => { + const dir = await project(rangedProject); + await installOk(dir); + const first = await lock(dir); + expect(overridesSection(first)).toMatchInlineSnapshot(` + ""overrides": { + "no-deps@1": { + ".": "1.0.0", + }, + "one-dep": { + "no-deps@1": "2.0.0", + }, + }," + `); + await installOk(dir, "--frozen-lockfile"); + const { err } = await installOk(dir); + expect(err).not.toContain("Saved lockfile"); + expect(await lock(dir)).toBe(first); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + }); + + // `1` and `^1.0.1` differ as ranges even though Version::eql treats `1` and `^1.0.0` as equal. + test("changing a flat target range is a frozen-lockfile change", async () => { + const dir = await project(rangedProject); + await installOk(dir); + await write( + join(dir, "package.json"), + JSON.stringify({ + name: "nested-overrides", + dependencies: rangedProject.dependencies, + overrides: { "no-deps@^1.0.1": "1.0.0", "one-dep": rangedProject.overrides["one-dep"] }, + }), + ); + const frozen = await install(dir, "--frozen-lockfile"); + expect(frozen.err).toContain("lockfile had changes, but lockfile is frozen"); + expect(frozen.exitCode).toBe(1); + await installOk(dir); + const section = overridesSection(await lock(dir)); + expect(section).toContain('"no-deps@^1.0.1": {'); + expect(section).not.toContain('"no-deps@1": {'); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + }); + + test("changing a nested target range is a frozen-lockfile change", async () => { + const dir = await project(rangedProject); + await installOk(dir); + await write( + join(dir, "package.json"), + JSON.stringify({ + name: "nested-overrides", + dependencies: rangedProject.dependencies, + overrides: { "no-deps@1": "1.0.0", "one-dep": { "no-deps@^1.0.1": "2.0.0" } }, + }), + ); + const frozen = await install(dir, "--frozen-lockfile"); + expect(frozen.err).toContain("lockfile had changes, but lockfile is frozen"); + expect(frozen.exitCode).toBe(1); + await installOk(dir); + expect(overridesSection(await lock(dir))).toContain('"no-deps@^1.0.1": "2.0.0"'); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + }); + + // Released Bun wrote a `name@range` key as a top-level string row that never applied; that row is not a ranged rule. + test("a v2 lockfile carrying a legacy dead name@range string row is re-resolved", async () => { + const dir = await project({ dependencies: { "one-range-dep": "1.0.0" } }); + await installOk(dir); + const text = await lock(dir); + expect(text).toContain('\n "packages": {'); + await write( + join(dir, "bun.lock"), + text.replace('\n "packages": {', '\n "overrides": {\n "no-deps@1": "1.0.0",\n },\n "packages": {'), + ); + await write( + join(dir, "package.json"), + JSON.stringify({ + name: "nested-overrides", + dependencies: { "one-range-dep": "1.0.0" }, + overrides: { "no-deps@1": "1.0.0" }, + }), + ); + const frozen = await install(dir, "--frozen-lockfile"); + expect(frozen.err).toContain("lockfile had changes, but lockfile is frozen"); + expect(frozen.exitCode).toBe(1); + await installOk(dir); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + const after = await lock(dir); + expect(after).toContain('"lockfileVersion": 3'); + expect(after).toContain('"no-deps@1": {'); + await installOk(dir, "--frozen-lockfile"); + }); + + test("objects in the overrides section are read at any lockfileVersion", async () => { + const dir = await project(npmObjectProject); + await installOk(dir); + const text = await lock(dir); + expect(text).toContain('"lockfileVersion": 3'); + await write(join(dir, "bun.lock"), text.replace('"lockfileVersion": 3', '"lockfileVersion": 2')); + await installOk(dir, "--frozen-lockfile"); + }); + + test("bun.lockb round-trips nested rules", async () => { + const { packageDir: dir } = await registry.createTestDir({ + bunfigOpts: { linker: "hoisted", saveTextLockfile: false }, + files: { "package.json": JSON.stringify({ name: "nested-overrides", ...npmObjectProject }) }, + }); + await installOk(dir); + expect(existsSync(join(dir, "bun.lockb"))).toBe(true); + expect(existsSync(join(dir, "bun.lock"))).toBe(false); + await rm(join(dir, "node_modules"), { recursive: true, force: true }); + await installOk(dir, "--frozen-lockfile"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + }); + + test("bun.lockb round-trips ranged rules", async () => { + const { packageDir: dir } = await registry.createTestDir({ + bunfigOpts: { linker: "hoisted", saveTextLockfile: false }, + files: { "package.json": JSON.stringify({ name: "nested-overrides", ...rangedProject }) }, + }); + await installOk(dir); + expect(existsSync(join(dir, "bun.lockb"))).toBe(true); + expect(existsSync(join(dir, "bun.lock"))).toBe(false); + await rm(join(dir, "node_modules"), { recursive: true, force: true }); + await installOk(dir, "--frozen-lockfile"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + }); +}); + +describe.concurrent("migration", () => { + test("yarn.lock resolutions paths become nested rules", async () => { + const url = registry.registryUrl(); + const [oneDep, noDeps] = await Promise.all([integrityOf("one-dep", "1.0.0"), integrityOf("no-deps", "2.0.0")]); + const dir = await project( + { dependencies: { "one-dep": "1.0.0" }, resolutions: { "one-dep/no-deps": "2.0.0" } }, + "hoisted", + { + "yarn.lock": `# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY. +# yarn lockfile v1 + + +no-deps@1.0.1: + version "2.0.0" + resolved "${url}no-deps/-/no-deps-2.0.0.tgz" + integrity ${noDeps} + +one-dep@1.0.0: + version "1.0.0" + resolved "${url}one-dep/-/one-dep-1.0.0.tgz" + integrity ${oneDep} + dependencies: + no-deps "1.0.1" +`, + }, + ); + const migrated = await migrate(dir); + expect(migrated.err).not.toContain("error:"); + expect(migrated.exitCode).toBe(0); + const text = await lock(dir); + expect(text).toContain('"lockfileVersion": 3'); + expect(text).toContain('"one-dep": {'); + expect(text).toContain('"no-deps": "2.0.0"'); + await installOk(dir); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + }); + + function pnpmLock(overrides: string, oneDep: string, noDeps: string) { + return `lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +overrides: + ${overrides} + +importers: + + .: + dependencies: + one-dep: + specifier: 1.0.0 + version: 1.0.0 + +packages: + + no-deps@2.0.0: + resolution: {integrity: ${noDeps}} + + one-dep@1.0.0: + resolution: {integrity: ${oneDep}} + +snapshots: + + no-deps@2.0.0: {} + + one-dep@1.0.0: + dependencies: + no-deps: 2.0.0 +`; + } + + test("pnpm-lock.yaml parent>child overrides become nested rules that package.json agrees with", async () => { + const [oneDep, noDeps] = await Promise.all([integrityOf("one-dep", "1.0.0"), integrityOf("no-deps", "2.0.0")]); + const dir = await project( + { dependencies: { "one-dep": "1.0.0" }, pnpm: { overrides: { "one-dep>no-deps": "2.0.0" } } }, + "hoisted", + { "pnpm-lock.yaml": pnpmLock("one-dep>no-deps: 2.0.0", oneDep, noDeps) }, + ); + const migrated = await migrate(dir); + expect(migrated.err).not.toContain("does not support"); + expect(migrated.err).not.toContain("error:"); + expect(migrated.exitCode).toBe(0); + const text = await lock(dir); + expect(text).toContain('"one-dep": {'); + expect(text).toContain('"no-deps": "2.0.0"'); + expect((await file(join(dir, "package.json")).json()).overrides).toEqual({ "one-dep>no-deps": "2.0.0" }); + await installOk(dir, "--frozen-lockfile"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + }); + + test("pnpm-lock.yaml parent>child@range override becomes a ranged nested rule", async () => { + const [oneDep, noDeps] = await Promise.all([integrityOf("one-dep", "1.0.0"), integrityOf("no-deps", "2.0.0")]); + const pnpmOverrides = { "one-dep>no-deps@1": "2.0.0" }; + const dir = await project({ dependencies: { "one-dep": "1.0.0" }, pnpm: { overrides: pnpmOverrides } }, "hoisted", { + "pnpm-lock.yaml": pnpmLock("one-dep>no-deps@1: 2.0.0", oneDep, noDeps), + }); + const migrated = await migrate(dir); + expect(migrated.err).not.toContain("does not support"); + expect(migrated.err).not.toContain("error:"); + expect(migrated.exitCode).toBe(0); + const text = await lock(dir); + expect(text).toContain('"one-dep": {'); + expect(text).toContain('"no-deps@1": "2.0.0"'); + expect((await file(join(dir, "package.json")).json()).overrides).toEqual(pnpmOverrides); + await installOk(dir, "--frozen-lockfile"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + }); + + test("pnpm-lock.yaml name@range override becomes a ranged flat rule", async () => { + const [oneDep, noDeps] = await Promise.all([integrityOf("one-dep", "1.0.0"), integrityOf("no-deps", "2.0.0")]); + const pnpmOverrides = { "no-deps@1": "2.0.0" }; + const dir = await project({ dependencies: { "one-dep": "1.0.0" }, pnpm: { overrides: pnpmOverrides } }, "hoisted", { + "pnpm-lock.yaml": pnpmLock("no-deps@1: 2.0.0", oneDep, noDeps), + }); + const migrated = await migrate(dir); + expect(migrated.err).not.toContain("does not support"); + expect(migrated.err).not.toContain("error:"); + expect(migrated.exitCode).toBe(0); + const text = await lock(dir); + expect(text).toContain('"lockfileVersion": 3'); + expect(text).toContain('"no-deps@1": {'); + expect(text).toContain('".": "2.0.0"'); + expect((await file(join(dir, "package.json")).json()).overrides).toEqual(pnpmOverrides); + await installOk(dir, "--frozen-lockfile"); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + }); +}); + +describe.concurrent("flat override fixes", () => { + // pnpm/pnpm#8223 + test("$ref to another package applies that spec under the overridden name", async () => { + const dir = await project({ + dependencies: { "one-range-dep": "1.0.0", "one-fixed-dep": "1.0.0" }, + overrides: { "no-deps": "$one-fixed-dep" }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + const first = await lock(dir); + const section = overridesSection(first); + expect(section).toContain('"no-deps": "1.0.0"'); + expect(section).not.toContain('"one-fixed-dep"'); + await installOk(dir, "--frozen-lockfile"); + const { err } = await installOk(dir); + expect(err).not.toContain("Saved lockfile"); + expect(await lock(dir)).toBe(first); + }); + + // pnpm/pnpm#9295 + describe.concurrent("$ref resolves against workspace members", () => { + const root = (overrides: Record, rootDeps: Record = {}) => ({ + workspaces: ["packages/*"], + dependencies: { "one-range-dep": "1.0.0", ...rootDeps }, + overrides, + }); + const member = (name: string, version: string) => JSON.stringify({ name, dependencies: { "no-deps": version } }); + + test("one member declares it", async () => { + const dir = await project(root({ "no-deps": "$no-deps" }), "hoisted", { + "packages/app/package.json": member("app", "1.0.0"), + }); + const { err } = await installOk(dir); + expect(err).not.toContain("Could not resolve"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + expect(overridesSection(await lock(dir))).toContain('"no-deps": "1.0.0"'); + await installOk(dir, "--frozen-lockfile"); + }); + + // one-dep pins no-deps@1.0.1, so its view is unaffected by whichever member's version happens to be resolved first. + test("members disagree", async () => { + const dir = await project(root({ "no-deps": "$no-deps" }, { "one-dep": "1.0.0" }), "hoisted", { + "packages/app/package.json": member("app", "1.0.0"), + "packages/b/package.json": member("b", "1.1.0"), + }); + const { err, exitCode } = await install(dir); + expect(err).toContain( + 'Could not resolve override "$no-deps": workspaces declare different versions of "no-deps"', + ); + expect(exitCode).toBe(0); + expect(await lock(dir)).not.toContain('"overrides"'); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); + }); + + test("the root's own declaration wins over members", async () => { + const dir = await project(root({ "no-deps": "$no-deps" }, { "no-deps": "1.0.1" }), "hoisted", { + "packages/app/package.json": member("app", "1.0.0"), + }); + const { err } = await installOk(dir); + expect(err).not.toContain("Could not resolve"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.1"); + }); + }); + + // pnpm/pnpm#12159 + describe.concurrent("catalog-valued rules follow catalog edits", () => { + const withCatalog = (version: string, deps: Record, overrides: Record) => + JSON.stringify({ + name: "nested-overrides", + workspaces: { packages: [], catalog: { "no-deps": version } }, + dependencies: deps, + overrides, + }); + + test("flat rule", async () => { + const deps = { "one-range-dep": "1.0.0" }; + const overrides = { "no-deps": "catalog:" }; + const { packageDir: dir } = await registry.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { "package.json": withCatalog("1.0.0", deps, overrides) }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + + await write(join(dir, "package.json"), withCatalog("1.0.1", deps, overrides)); + await installOk(dir); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.1"); + expect(await lock(dir)).not.toContain("no-deps@1.0.0"); + }); + + test("nested rule", async () => { + const deps = { "one-dep": "1.0.0", "one-range-dep": "1.0.0" }; + const overrides = { "one-dep": { "no-deps": "catalog:" } }; + const { packageDir: dir } = await registry.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { "package.json": withCatalog("1.0.0", deps, overrides) }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.0"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + + await write(join(dir, "package.json"), withCatalog("1.0.1", deps, overrides)); + await installOk(dir); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + }); + }); +}); + +describe.concurrent("per-edge effective range", () => { + test("bun dedupe does not collapse edges whose scoped rules pin different versions", async () => { + const dir = await project({ + dependencies: twoParents, + overrides: { "one-fixed-dep@1": { "no-deps": "1.1.0" }, "one-fixed-dep@2": { "no-deps": "1.0.0" } }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "ofd1", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("1.0.0"); + const { out, err, exitCode } = await run(dir, "dedupe", "--check"); + expect(out).toContain("Already deduplicated."); + expect(err).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + + test("bun dedupe moves an edge whose scoped rule is a range onto the version its sibling's rule pins", async () => { + const dir = await project({ + dependencies: twoParents, + overrides: { "one-fixed-dep@1": { "no-deps": "^1.0.0" }, "one-fixed-dep@2": { "no-deps": "1.0.0" } }, + }); + await installOk(dir); + expect(await versionSeenBy(dir, "ofd1", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("1.0.0"); + const { out, exitCode } = await run(dir, "dedupe", "--check"); + expect(out).toContain("1 duplicate version can be removed: no-deps@1.1.0"); + expect(exitCode).toBe(1); + }); +}); diff --git a/test/cli/install/registry/packages/@scoped/pkg-1/package.json b/test/cli/install/registry/packages/@scoped/pkg-1/package.json new file mode 100644 index 000000000000..73ad985dca84 --- /dev/null +++ b/test/cli/install/registry/packages/@scoped/pkg-1/package.json @@ -0,0 +1,44 @@ +{ + "name": "@scoped/pkg-1", + "versions": { + "1.1.1": { + "name": "@scoped/pkg-1", + "version": "1.1.1", + "dependencies": { + "@scoped/pkg-1": "1.1.1" + }, + "_id": "@scoped/pkg-1@1.1.1", + "_integrity": "sha512-ZukE1diuSo8MITnpi03IfVcMXL9cUr4oxvGuuVnIga3ju9ctyntuWqJSi6BKdzyvvV1qpeRkduDDNZ6ZDtly2A==", + "_nodeVersion": "26.3.0", + "_npmVersion": "10.8.3", + "integrity": "sha512-ZukE1diuSo8MITnpi03IfVcMXL9cUr4oxvGuuVnIga3ju9ctyntuWqJSi6BKdzyvvV1qpeRkduDDNZ6ZDtly2A==", + "shasum": "096b75abed5582b7c73a93578f891e588b82987a", + "dist": { + "integrity": "sha512-ZukE1diuSo8MITnpi03IfVcMXL9cUr4oxvGuuVnIga3ju9ctyntuWqJSi6BKdzyvvV1qpeRkduDDNZ6ZDtly2A==", + "shasum": "096b75abed5582b7c73a93578f891e588b82987a", + "tarball": "http://localhost:45907/@scoped/pkg-1/-/@scoped/pkg-1-1.1.1.tgz" + }, + "contributors": [] + } + }, + "time": { + "modified": "2026-08-14T04:24:23.926Z", + "created": "2026-08-14T04:24:23.926Z", + "1.1.1": "2026-08-14T04:24:23.926Z" + }, + "users": {}, + "dist-tags": { + "latest": "1.1.1" + }, + "_uplinks": {}, + "_distfiles": {}, + "_attachments": { + "pkg-1-1.1.1.tgz": { + "shasum": "096b75abed5582b7c73a93578f891e588b82987a", + "version": "1.1.1" + } + }, + "_rev": "", + "_id": "@scoped/pkg-1", + "readme": "" +} \ No newline at end of file diff --git a/test/cli/install/registry/packages/@scoped/pkg-1/pkg-1-1.1.1.tgz b/test/cli/install/registry/packages/@scoped/pkg-1/pkg-1-1.1.1.tgz new file mode 100644 index 0000000000000000000000000000000000000000..7bbd312e95a0117c1b509a649d07452ae44e1ed1 GIT binary patch literal 165 zcmV;W09yYaiwFP!00002|LxDg3d0}_24K%Vg?QfDNY%qmdlk`8x-ylz?lSuBOBoEh za~T`>dH5kQkYwja*JGPs*<4DBY-a#qwH9lV&je7N5zrY6N?8k9P(guV0@xSK<>n?@ zWrrg-$Nh;1m(g8Oo`gIXpE8CvkNr`}1DAK4%8=sPQw_bf7BFKB82x~S8Z^Egd;jGN TUszRDRrRZ$E(fv~00;m8E7L~n literal 0 HcmV?d00001 From 399aa5bbb958d6581ed56bb84a90faaba1891b56 Mon Sep 17 00:00:00 2001 From: Jarred Sumner Date: Thu, 13 Aug 2026 22:27:00 -0700 Subject: [PATCH 03/25] install: close the remaining pnpm behavior differences add --catalog reuses existing entries, keeps a range an explicit version fits, catalogs the declared range instead of re-resolving, decides the catalog per target, and refuses workspace names and local paths; plain bun add uses a default-catalog entry when one exists. --filter gains pnpm's relation (foo..., ...foo, foo^..., ...^foo) and {dir} selectors through one shared selection engine, add/remove no longer select the root implicitly, and named updates fan out across -r/--filter. audit fix rewrites exact pins and catalog entries, moves dependents independently, reports from the written lockfile, supports --json and non-default registries, and lets security fixes through the release-age gate with an annotation. Catalogs apply only to workspace importers' peers; dedupe no longer downgrades a direct dependency unless that is the only way to drop a version and refuses to run on a lockfile that is behind package.json. Frozen installs detect a survivor depending on a pruned workspace, tolerate a catalog subset, and fail on overrides/catalog/ patchedDependencies changes; prune checks package.json first, drops dev-only workspace links under --production in isolated layouts, and takes --filter. pnpm-lock.yaml migration handles multi-document files, runtime: entries, named registries, peer-suffixed keys, injected workspaces and manifest-only importer deps. bun pm licenses gains --dev, --long, --filter, a (dev) marker and license/description JSON fields. bun update preserves non-caret ranges and dist-tags, accepts patterns, --dev/--prod/--no-optional, -L and the up alias. Also: id-indexed sets use DynamicBitSet; the authenticated-request header buffer in NetworkTask is owned by the task and freed; ~230 cases ported from pnpm's suites plus test-quality fixes across the new files. No-Verification-Needed: user asked to skip --- completions/bun.fish | 9 + completions/bun.zsh | 5 + docs/pm/catalogs.mdx | 10 +- docs/pm/cli/add.mdx | 18 +- docs/pm/cli/audit.mdx | 44 +- docs/pm/cli/dedupe.mdx | 11 +- docs/pm/cli/install.mdx | 23 +- docs/pm/cli/pm.mdx | 18 +- docs/pm/cli/prune.mdx | 23 +- docs/pm/cli/remove.mdx | 2 +- docs/pm/cli/update.mdx | 32 +- docs/pm/filter.mdx | 31 +- docs/snippets/cli/update.mdx | 18 +- src/install/NetworkTask.rs | 97 +- src/install/PackageManager.rs | 181 +-- .../PackageManager/CommandLineArguments.rs | 81 +- .../PackageManager/PackageJSONEditor.rs | 160 +- .../PackageManager/PackageManagerEnqueue.rs | 23 +- src/install/PackageManager/add_catalog.rs | 524 ++++-- .../PackageManager/add_remove_with_filter.rs | 264 +++- .../PackageManager/install_with_manager.rs | 191 ++- .../PackageManager/package_json_write_back.rs | 45 +- .../updatePackageJSONAndInstall.rs | 29 +- .../PackageManager/workspace_selection.rs | 367 +++++ src/install/audit_fix.rs | 675 ++++++-- src/install/audit_fix/json.rs | 149 ++ src/install/audit_fix/package_json_edits.rs | 181 +++ src/install/dedupe.rs | 379 +++-- src/install/dependency.rs | 9 +- src/install/isolated_install.rs | 48 +- src/install/lib.rs | 1 + src/install/lockfile/CatalogMap.rs | 17 +- src/install/lockfile/Package.rs | 120 +- src/install/lockfile/Tree.rs | 48 +- src/install/lockfile/bun.lock.rs | 9 +- src/install/lockfile/pruned_workspaces.rs | 168 +- src/install/lockfile/reachable.rs | 196 ++- src/install/pnpm.rs | 827 ++++++---- src/install/prune.rs | 282 +++- src/install/update_scope.rs | 361 +++++ src/install/update_transitive.rs | 40 +- src/runtime/cli/audit_command.rs | 250 ++- src/runtime/cli/mod.rs | 2 +- src/runtime/cli/package_manager_command.rs | 27 +- src/runtime/cli/pm_licenses_command.rs | 173 +- src/runtime/cli/prune_command.rs | 4 +- .../__snapshots__/bun-audit.test.ts.snap | 9 + test/cli/install/bun-add-catalog.test.ts | 836 +++++++++- test/cli/install/bun-add-filter.test.ts | 648 ++++++-- test/cli/install/bun-add.test.ts | 77 + test/cli/install/bun-audit.test.ts | 1052 ++++++++++-- test/cli/install/bun-dedupe.test.ts | 569 ++++++- test/cli/install/bun-install-registry.test.ts | 85 +- test/cli/install/bun-pm-licenses.test.ts | 636 ++++++-- test/cli/install/bun-prune.test.ts | 664 +++++++- .../install/bun-update-lockfile-sync.test.ts | 488 ++++-- .../cli/install/bun-update-transitive.test.ts | 418 ++++- test/cli/install/bun-update.test.ts | 978 +++++++++--- test/cli/install/bun-workspaces.test.ts | 51 + test/cli/install/catalogs.test.ts | 342 +++- test/cli/install/config-precedence.test.ts | 36 +- .../frozen-lockfile-missing-workspace.test.ts | 121 +- .../install/frozen-lockfile-pruned.test.ts | 355 ++++- test/cli/install/isolated-relink.test.ts | 37 +- test/cli/install/lockfile-only.test.ts | 8 +- .../pnpm-comprehensive.test.ts.snap | 32 +- .../pnpm-migration-complete.test.ts.snap | 11 +- .../install/migration/pnpm-lock-v9.test.ts | 1408 +++++++++++++++-- .../pnpm/v9-importer-peers/package.json | 22 + .../v9-importer-peers/packages/a/package.json | 6 + .../pnpm/v9-importer-peers/pnpm-lock.yaml | 40 + .../pnpm/v9-local-tarballs/package.json | 4 +- .../pnpm/v9-local-tarballs/pnpm-lock.yaml | 18 + .../pnpm/v9-peer-range-dedupe/package.json | 8 + .../pnpm/v9-peer-range-dedupe/pnpm-lock.yaml | 54 + .../package.json | 10 + .../packages/dir/package.json | 4 + .../pnpm-lock.yaml | 36 + .../vendor/local/package.json | 7 + test/cli/install/nested-overrides.test.ts | 128 +- .../packages/@scoped/pkg-1/package.json | 8 +- .../catalog-dep/catalog-dep-1.0.0.tgz | Bin 0 -> 192 bytes .../packages/catalog-dep/package.json | 22 + .../catalog-peer/catalog-peer-1.0.0.tgz | Bin 0 -> 197 bytes .../catalog-peer/catalog-peer-2.0.0.tgz | Bin 0 -> 201 bytes .../packages/catalog-peer/package.json | 35 + .../packages/create-catalog-packages.ts | 57 + .../publish-version-update/package.json | 47 + .../publish-version-update-9.9.9.tgz | Bin 0 -> 367 bytes .../packages/republish-test-1/package.json | 41 + .../republish-test-1-1.0.0.tgz | Bin 0 -> 147 bytes .../packages/republish-test-2/package.json | 41 + .../republish-test-2-1.0.0.tgz | Bin 0 -> 147 bytes .../packages/republish-test-3/package.json | 41 + .../republish-test-3-1.0.0.tgz | Bin 0 -> 147 bytes 95 files changed, 12769 insertions(+), 2893 deletions(-) create mode 100644 src/install/PackageManager/workspace_selection.rs create mode 100644 src/install/audit_fix/json.rs create mode 100644 src/install/audit_fix/package_json_edits.rs create mode 100644 src/install/update_scope.rs create mode 100644 test/cli/install/migration/pnpm/v9-importer-peers/package.json create mode 100644 test/cli/install/migration/pnpm/v9-importer-peers/packages/a/package.json create mode 100644 test/cli/install/migration/pnpm/v9-importer-peers/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-peer-range-dedupe/package.json create mode 100644 test/cli/install/migration/pnpm/v9-peer-range-dedupe/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/package.json create mode 100644 test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/packages/dir/package.json create mode 100644 test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/vendor/local/package.json create mode 100644 test/cli/install/registry/packages/catalog-dep/catalog-dep-1.0.0.tgz create mode 100644 test/cli/install/registry/packages/catalog-dep/package.json create mode 100644 test/cli/install/registry/packages/catalog-peer/catalog-peer-1.0.0.tgz create mode 100644 test/cli/install/registry/packages/catalog-peer/catalog-peer-2.0.0.tgz create mode 100644 test/cli/install/registry/packages/catalog-peer/package.json create mode 100644 test/cli/install/registry/packages/create-catalog-packages.ts create mode 100644 test/cli/install/registry/packages/publish-version-update/package.json create mode 100644 test/cli/install/registry/packages/publish-version-update/publish-version-update-9.9.9.tgz create mode 100644 test/cli/install/registry/packages/republish-test-1/package.json create mode 100644 test/cli/install/registry/packages/republish-test-1/republish-test-1-1.0.0.tgz create mode 100644 test/cli/install/registry/packages/republish-test-2/package.json create mode 100644 test/cli/install/registry/packages/republish-test-2/republish-test-2-1.0.0.tgz create mode 100644 test/cli/install/registry/packages/republish-test-3/package.json create mode 100644 test/cli/install/registry/packages/republish-test-3/republish-test-3-1.0.0.tgz diff --git a/completions/bun.fish b/completions/bun.fish index 1b12cdcf794b..f7b6c424d0ba 100644 --- a/completions/bun.fish +++ b/completions/bun.fish @@ -187,6 +187,15 @@ complete -c bun \ complete -c bun \ -n "__fish_seen_subcommand_from pm; and __fish_seen_subcommand_from licenses" -l 'production' -d 'Omit devDependencies' -f +complete -c bun \ + -n "__fish_seen_subcommand_from pm; and __fish_seen_subcommand_from licenses" -l 'dev' -s 'D' -d 'List only what devDependencies pull in' -f + +complete -c bun \ + -n "__fish_seen_subcommand_from pm; and __fish_seen_subcommand_from licenses" -l 'long' -d 'Also print author, description and homepage' -f + +complete -c bun \ + -n "__fish_seen_subcommand_from pm; and __fish_seen_subcommand_from licenses" -l 'filter' -s 'F' -d 'List only the matching workspaces' -r + # Add built-in subcommands with descriptions. complete -c bun -n "__fish_use_subcommand" -a "create" -f -d "Create a new project from a template" complete -c bun -n "__fish_use_subcommand" -a "build bun" --require-parameter -F -d "Transpile and bundle one or more files" diff --git a/completions/bun.zsh b/completions/bun.zsh index 26adc17a72b1..1d077349c4fd 100644 --- a/completions/bun.zsh +++ b/completions/bun.zsh @@ -311,6 +311,11 @@ _bun_pm_completion() { "--json[output as JSON]" "--prod[omit devDependencies]" "--production[omit devDependencies]" + "--dev[list only what devDependencies pull in]" + "-D[list only what devDependencies pull in]" + "--long[also print author, description and homepage]" + "*--filter[list only the matching workspaces' dependencies]:workspace pattern" + "*-F[list only the matching workspaces' dependencies]:workspace pattern" ) _arguments -s -C \ diff --git a/docs/pm/catalogs.mdx b/docs/pm/catalogs.mdx index bdeba48201f4..bed797b5ee2f 100644 --- a/docs/pm/catalogs.mdx +++ b/docs/pm/catalogs.mdx @@ -91,7 +91,7 @@ In your workspace packages, use the `catalog:` protocol to reference versions: } ``` -`catalog:` and `catalog:` are accepted in `dependencies`, `devDependencies`, `optionalDependencies` and `peerDependencies`, and as the value of a root [`overrides` / `resolutions`](/pm/overrides) rule. A catalog peer is resolved and hoisted exactly as if the catalog's range were written inline: if an ancestor already provides a version that satisfies the catalog range, the workspace package reuses that copy instead of getting its own. +`catalog:` and `catalog:` are accepted in `dependencies`, `devDependencies`, `optionalDependencies` and `peerDependencies`, and as the value of a root [`overrides` / `resolutions`](/pm/overrides) rule. A catalog peer is resolved and hoisted exactly as if the catalog's range were written inline: if an ancestor already provides a version that satisfies the catalog range, the workspace package reuses that copy instead of getting its own. If no ancestor provides a satisfying version, the workspace package gets its own copy of whatever the catalog range resolves to, just as it would with the range written inline. `bun.lock` stores the catalog ranges themselves, not a resolved version per reference, so a catalog reference binds to whichever version in the lockfile satisfies the range, again like an inline range. (pnpm additionally pins each catalog entry to the version it first resolved to; Bun does not.) ### 3. Run Bun Install @@ -238,15 +238,15 @@ To update versions across all packages, change the version in the root package.j Then run `bun install` to update all packages. -To add a new dependency to the catalog (or refresh an existing entry), run `bun add` with `--catalog` (or `--catalog=` for a named catalog) from the workspace package or from the root: +To add a dependency to the catalog, run `bun add` with `--catalog` (or `--catalog=`) from the workspace package or from the root; an entry that already exists is reused unless you give a version: ```bash terminal icon="terminal" bun add react --catalog ``` -Bun writes the resolved version to the catalog in the root `package.json` and `"catalog:"` to the `package.json` you ran the command in. See [`bun add --catalog`](/pm/cli/add#--catalog). +Bun writes the entry to the catalog in the root `package.json` (the range the current `package.json` already declares, otherwise the resolved version) and `"catalog:"` to the `package.json` you ran the command in. See [`bun add --catalog`](/pm/cli/add#--catalog). -The catalog is only consulted when `--catalog` is passed: `bun add react` writes a concrete range to the current `package.json` even if `react` is already in the catalog (Bun has no equivalent of pnpm's `catalogMode`). +`bun add react` without the flag also uses the default catalog when it lists `react`: the package gets `"catalog:"`, and an existing `"catalog:"` reference is kept. `bun add react@latest` (or any explicit version) writes a concrete range instead. Named catalogs are only used through `--catalog=` or an existing reference, and a version you type is never checked against the catalog (there is no equivalent of pnpm's `catalogMode` setting). ## Lockfile Integration @@ -296,7 +296,7 @@ Bun's lockfile tracks catalog versions, so installs are consistent across enviro - Empty strings and whitespace in catalog names are ignored (treated as default catalog) - `catalog:default` is the same as `catalog:`; both read the `catalog` field, or `catalogs.default` if that is where the entry is defined (as pnpm names it) - Invalid dependency versions in catalogs fail to resolve during `bun install` -- Catalogs are only available within workspaces; they cannot be used outside the monorepo +- `catalog:` is only understood in the root `package.json` and in workspace packages. A `catalog:` specifier inside an installed package (from the registry, git, a tarball or a `file:` folder) never reads your catalogs: as a regular dependency it fails to resolve (`x@catalog: failed to resolve`), and as a peer dependency it is treated as unsatisfiable and no copy is installed for it. Publish with `bun publish` or `bun pm pack`, which substitute the ranges (see [Publishing](#publishing)) ## Publishing diff --git a/docs/pm/cli/add.mdx b/docs/pm/cli/add.mdx index f66afdfb45b2..341462852ff5 100644 --- a/docs/pm/cli/add.mdx +++ b/docs/pm/cli/add.mdx @@ -83,7 +83,7 @@ bun add --help ## `--catalog` -In a workspace, `bun add react --catalog` resolves the version as usual, but writes it to the root `package.json` [catalog](/pm/catalogs) (`workspaces.catalog`, or `workspaces.catalogs.` with `--catalog=`) and adds `"react": "catalog:"` (or `"catalog:"`) to the current package. An existing catalog entry is updated. Combine with `--exact`, `--dev`, etc. as usual. +In a workspace, `bun add react --catalog` resolves the version as usual, but writes it to the root `package.json` [catalog](/pm/catalogs) (`workspaces.catalog`, or `workspaces.catalogs.` with `--catalog=`) and adds `"react": "catalog:"` (or `"catalog:"`) to the current package. An entry that is already in the catalog is reused unless you give a version. Combine with `--exact`, `--dev`, etc. as usual. ```bash terminal icon="terminal" bun add react --catalog @@ -112,20 +112,23 @@ bun add vitest --catalog=testing A few things to know, some of which differ from pnpm's `--save-catalog`: -- If the catalog already has an entry for the package, it is replaced with the newly resolved version (`bun add react --catalog` moves every workspace package that depends on `"catalog:"` to the new version). pnpm leaves an existing entry alone. -- The flag decides which catalog is used. If the current package already depends on `"react": "catalog:legacy"`, `bun add react --catalog` moves it to the default catalog; the old `legacy` entry is left in place. +- If the catalog already has an entry for the package, `bun add react --catalog` leaves it alone and only writes `"react": "catalog:"` to the current package. `bun add react@19.1.0 --catalog` keeps an entry such as `^19.0.0` that the version satisfies and only changes which version is installed; any other version or range replaces the entry, which moves every package that depends on `"catalog:"` along with it. +- Without a version, a range the current `package.json` already has (`"react": "^18.2.0"`) is what gets cataloged. The package is only resolved to its latest version when neither the catalog nor the `package.json` mentions it. +- A package that already depends on `"react": "catalog:legacy"` stays in `legacy`, whatever name the flag carries: the `legacy` entry is reused, or replaced if you gave a version. With `--filter`, this is decided for each selected package. - `--catalog` and `--catalog=default` are the same catalog: the entry is written to whichever of `catalog` or `catalogs.default` the root `package.json` already defines (`catalog` is created when there is neither), so a repository migrated from pnpm never ends up with the package in both. - The recorded version is the one that was installed, so an entry in [`overrides`](/pm/overrides) for the package ends up in the catalog too. - The name must be attached with `=` (`--catalog=testing`); `--catalog testing` adds a package called `testing`. - pnpm's `--save-catalog` is spelled `--catalog` and `--save-catalog-name testing` is `--catalog=testing`; Bun silently skips flags it does not know, so the pnpm spellings add the package as an ordinary dependency (and `--save-catalog-name testing` also tries to add a package called `testing`) instead of failing. -- Packages must be added by name (`react`, `react@^18`, `alias@npm:react`). Bare URLs, paths and `workspace:` versions are rejected. +- Packages must be added by name (`react`, `react@^18`, `alias@npm:react`, `alias@https://example.com/react.tgz`). Bare URLs and paths, relative `file:`/`link:`/folder specs, and workspace packages (by name or via `workspace:`) are refused before anything is written. - The root `package.json` must have a `workspaces` field. +Catalogs are also used without the flag: in a workspace whose default catalog lists `react`, `bun add react` with no version writes `"react": "catalog:"`, and a package that already depends on `"catalog:"` keeps that reference. Pass a version (`bun add react@latest`) to write a concrete range instead. + ## `--filter` **Alias** — `-F` -In a monorepo, `--filter` adds the package to the matching workspace package(s) instead of the package in the current directory. Patterns match workspace names or paths (`./packages/*`), `*` matches every workspace including the root, and `!pattern` excludes. Repeat the flag to combine patterns. Every matching package is edited; if nothing matches, the command fails without touching anything, and a pattern that matches nothing while others do prints a warning. `bun remove` and `bun install ` accept the same flag. See [filtering](/pm/filter). +In a monorepo, `--filter` adds the package to the matching workspace package(s) instead of the package in the current directory. Patterns match workspace names or paths (`./packages/*`), `*` matches every workspace package (not the root), and `!pattern` excludes; the `{dir}` and `...` relation selectors from the filtering page work here too (`--filter '{packages}'`, `--filter 'api...'`). Repeat the flag to combine patterns. Every matching package is edited; if nothing matches, the command fails without touching anything, and a pattern that matches nothing while others do prints a warning. `bun remove` and `bun install ` accept the same flag. See [filtering](/pm/filter). ```bash terminal icon="terminal" bun add zod --filter api @@ -136,10 +139,11 @@ bun remove zod --filter '*' --filter '!api' A few things to know, some of which differ from pnpm: -- The workspace root is selected by `*` and by filter sets made only of `!` patterns; exclude it explicitly with `--filter '!'`. To target only the root, use its name or `--filter .` from the root directory. +- The workspace root is only edited when a pattern names it: its name, `--filter .` from the root directory, `{.}`, or a relation that reaches it. `*` and filter sets made only of `!` patterns leave it alone (`--filter '*' --filter ''` edits everything). A `package.json` without workspaces is still selected by `*`. +- With a `...` relation, `bun add` and `bun remove` read the workspace links from `bun.lock`, so run `bun install` first in a fresh clone. The closure includes the named package itself unless you write `^` (`api^...`, `...^api`). - A `!` pattern always wins, whatever order the flags are given in: `--filter '!./packages/*' --filter api` selects nothing. - Name patterns are globs, and `*` does not cross `/`: `*-utils` does not match `@acme/date-utils`; use `*/*-utils` or `@acme/*`. -- Path patterns must match a workspace directory exactly, relative to the current directory: `--filter ./packages` selects nothing, `--filter ./packages/*` selects every package in it, and `--filter .` only works from a workspace's own directory. +- Path patterns must match a workspace directory exactly, relative to the current directory: `--filter ./packages` selects nothing, `--filter ./packages/*` or `--filter '{packages}'` selects every package in it, and `--filter .` only works from a workspace's own directory. - Local paths (`./vendor/logger`, `file:../logger`, `./logger.tgz`) are relative to the current directory and are re-spelled relative to each selected package (`"logger": "../../vendor/logger"` in `packages/api/package.json`). - The flag chooses which `package.json` files are edited, not what is installed: a single install of the whole workspace runs afterwards. The `package.json` files are written as soon as the lockfile is saved, so they agree with `bun.lock` even if a root `postinstall` script then fails. - The install summary is printed from the point of view of one selected workspace (the first one that received every requested package); the other selected `package.json` files get the same edits even though they are not listed in the output. diff --git a/docs/pm/cli/audit.mdx b/docs/pm/cli/audit.mdx index f9de0126f42d..13499b669a0a 100644 --- a/docs/pm/cli/audit.mdx +++ b/docs/pm/cli/audit.mdx @@ -9,9 +9,9 @@ Run the command in a project with a `bun.lock` file: bun audit ``` -Bun sends the list of installed packages and versions to npm, and prints a report of any vulnerabilities found. Packages installed from registries other than the default registry are skipped. +Bun sends the list of installed packages and versions to npm, and prints a report of any vulnerabilities found. Packages that come from a registry configured for their scope are sent to that registry's advisory endpoint instead, and its answer is merged into the same report; a registry that does not answer is listed after the report (`Skipped @acme/a, @acme/b because https://npm.acme.dev could not be audited`) and does not affect the exit code. -The package list comes from `bun.lock` alone, so `node_modules` does not need to exist, and `bun audit` writes nothing: `package.json`, `bun.lock` and `node_modules` are left as they are (only `bun audit fix`, below, changes `bun.lock` and `node_modules`). +The package list comes from `bun.lock` alone, so `node_modules` does not need to exist, and `bun audit` writes nothing: `package.json`, `bun.lock` and `node_modules` are left as they are (only `bun audit fix`, below, changes `bun.lock` and `node_modules`, and `package.json` only when it rewrites an exact pin). If no vulnerabilities are found, the command prints: @@ -19,12 +19,17 @@ If no vulnerabilities are found, the command prints: No vulnerabilities found ``` -When vulnerabilities are detected, Bun lists each affected package with the severity, a short description, and a link to the advisory. At the end of the report it prints a summary and hints for updating: +When vulnerabilities are detected, Bun lists each affected package with the severity, a short description, and a link to the advisory. At the end of the report it prints a summary (`1 vulnerability (1 high)` when there is only one) and hints for updating: ``` 3 vulnerabilities (1 high, 2 moderate) + +To upgrade only the vulnerable packages, within their declared ranges: + bun audit fix + To update all dependencies to the latest compatible versions: bun update + To update all dependencies to the latest versions (including breaking changes): bun update --latest ``` @@ -43,6 +48,12 @@ bun audit --audit-level=high bun audit --prod ``` +**`--omit=`** - Leave out packages that are only reached through edges of the given kind, using the same reachability rule as `--prod` (`--omit=dev` is the same as `--prod`). Repeat the flag to leave out several kinds: + +```bash terminal icon="terminal" +bun audit --omit=optional --omit=peer +``` + **`--ignore `** - Ignore specific advisories by GHSA ID or numeric advisory ID (repeat the flag to ignore several). The registry's advisory data does not include CVE IDs, so a CVE ID does not match anything: ```bash terminal icon="terminal" @@ -59,11 +70,11 @@ Use the `--json` flag to print the raw JSON response from the registry instead o bun audit --json ``` -The response is printed as received: `--prod` still limits which packages are sent, and `--audit-level` and `--ignore` do not remove anything from the printed JSON. They do decide the exit code, which is `1` only if an advisory is left after both filters are applied (and `0` otherwise), so `bun audit --json --audit-level=high` prints every advisory but only fails the run for high or critical ones. If the response cannot be parsed, it is still printed and the command exits with `1`. +The response is printed as received: `--prod` and `--omit` still limit which packages are sent, and `--audit-level` and `--ignore` do not remove anything from the printed JSON. They do decide the exit code, which is `1` only if an advisory is left after both filters are applied (and `0` otherwise), so `bun audit --json --audit-level=high` prints every advisory but only fails the run for high or critical ones. If the response cannot be parsed, it is still printed and the command exits with `1`. ### `bun audit fix` -`bun audit fix` runs the audit and then upgrades each vulnerable package in `bun.lock` to the lowest version that is not affected by any advisory and still satisfies every range that depends on it (including `overrides` and catalog entries), then installs. `package.json` is never modified (Bun does not add `overrides` on your behalf; see `requires a semver-major update:` below) and unrelated packages are not touched. `fix` is a subcommand, not a flag: `bun audit --fix` is rejected as an unknown flag. +`bun audit fix` runs the audit and then moves each vulnerable version installed in `bun.lock` to the lowest release that no advisory covers and that the ranges of its dependents accept (`overrides` and catalog entries count as ranges), then installs. A dependency that the root `package.json`, a workspace `package.json` or a catalog entry pins to an exact version is planned as if it were written `^version`; when that finds a fix, that `package.json` entry (or the catalog entry in the root; workspace files that say `catalog:` are left as they are) is rewritten to the new version. This is the only time the command edits `package.json`: `overrides` are never added or changed, and unrelated packages are not touched. `fix` is a subcommand, not a flag: `bun audit --fix` is rejected as an unknown flag. ```bash terminal icon="terminal" bun audit fix @@ -72,26 +83,29 @@ bun audit fix ``` fixing: ms@0.7.0 → 0.7.1 + lodash@4.17.20 → 4.17.21 + package.json: 4.17.20 → 4.17.21 -requires a semver-major update: +blocked by a dependent's range: minimatch@0.3.0 → 3.0.2 express@3.21.2 depends on minimatch@0.3.0 -Fixed 1 vulnerability in 1 package +Fixed 2 vulnerabilities in 2 packages 1 vulnerability remaining ``` -- Packages whose only safe version falls outside a dependent's range are listed under `requires a semver-major update:` and left alone. Update the dependent, or add an `overrides` entry, and run the command again. A dependency bundled inside another package (`express@3.21.2 bundles minimatch@0.3.0`) can only be fixed by updating the package that bundles it. -- Packages are only ever upgraded, never downgraded, and only stable releases are candidates. `no fix available:` lists packages with no safe newer release, and says so when a safe release exists but is newer than `--minimum-release-age` or the package's manifest could not be fetched. +- `blocked by a dependent's range:` lists the dependents whose range accepts no safe release; they stay on the version they have. Other dependents of the same version whose ranges do accept the fix are still moved, in which case the package appears under both `fixing:` and this heading. Update the dependent, or add an `overrides` entry, and run the command again. A dependency bundled inside another package (`express@3.21.2 bundles minimatch@0.3.0`) can only be fixed by updating the package that bundles it. +- Only stable releases are candidates. When no newer release is safe, the highest safe older release the ranges accept is installed instead and marked `(downgrade)`. `no fix available:` lists versions for which no safe release exists in either direction; `manifest could not be fetched:` lists versions whose registry request failed, and nothing is installed for them. +- A safe release published more recently than `--minimum-release-age` is still installed, marked `(newer than --minimum-release-age)`: the age gate is lifted for that package during this run only. - Advisories that match none of the installed versions are listed under `not matched to an installed version:` and count as remaining. -- Counts are advisories, as in `bun audit`. A package installed as several versions is upgraded per version, and an advisory only counts as fixed once every version it affects has been upgraded; otherwise it counts as remaining. -- The fix lives in `bun.lock` only. A later dependency that pins the vulnerable version, or regenerating the lockfile, can bring it back; re-run `bun audit`. -- `--dry-run` prints the plan without changing anything. +- Counts are advisories, as in `bun audit`. After installing, Bun checks the `bun.lock` it wrote against the advisories again, so the `Fixed`/`remaining` lines, the exit code and the `vulnerable after install:` list (versions that survived, or that the fix itself pulled in) describe the lockfile on disk rather than the plan; an advisory counts as fixed only when no version it covers is left. `--dry-run` prints the plan without changing anything, and its counts are the plan's. +- A fix that only moved `bun.lock` can be undone by a later dependency that pins the vulnerable version or by regenerating the lockfile; a rewritten `package.json` pin stays. Re-run `bun audit` after such changes. +- `--json` prints one JSON object instead of the text report, with the fields `fixes`, `blocked`, `unfixable`, `manifestUnavailable`, `unmatched`, `unaudited`, `vulnerableAfterInstall`, `fixed`, `remaining` and `dryRun`. It is printed once the install has finished (with `--dry-run`, as soon as the plan is made); if a root lifecycle script writes to stdout, pass `--ignore-scripts` to keep the output parseable. - `--registry` applies to both the advisory request and the install of the fixed versions; there is no way to query one registry for advisories and install from another. -- `--audit-level`, `--ignore` and `--prod` behave as for `bun audit`, except that `--prod` (like `--frozen-lockfile` and `--no-save`) stops `bun.lock` from being written, so `bun audit fix` rejects it before contacting the registry. +- `--audit-level`, `--ignore` and `--prod` behave as for `bun audit`, except that `--prod` (like `--frozen-lockfile` and `--no-save`) stops `bun.lock` from being written, so `bun audit fix` rejects it before contacting the registry. `--omit` does not narrow what `bun audit fix` audits: every package in `bun.lock` is checked and fixed. ### Exit code -`bun audit` exits with code `0` if no vulnerabilities are found and `1` if any are left after `--audit-level` and `--ignore` are applied; `--json` uses the same rule even though it prints the unfiltered response. `bun audit fix` exits with `0` when nothing vulnerable remains after the fix (or, with `--dry-run`, would remain) and `1` otherwise. +`bun audit` exits with code `0` if no vulnerabilities are found and `1` if any are left after `--audit-level` and `--ignore` are applied; `--json` uses the same rule even though it prints the unfiltered response. `bun audit fix` exits with `0` when its check of the written `bun.lock` finds nothing remaining (or, with `--dry-run`, when the plan leaves nothing remaining) and `1` otherwise. -When the registry cannot be reached, or answers with a 4xx or 5xx status, both commands print `audit request failed` to stderr and also exit with `1`; there is no flag to turn a registry failure into a passing run, so a CI step that wants to distinguish an outage from a vulnerable tree has to look at stderr. +When the default registry cannot be reached, or answers with a 4xx or 5xx status, both commands print `audit request failed` to stderr and also exit with `1`; there is no flag to turn a registry failure into a passing run, so a CI step that wants to distinguish an outage from a vulnerable tree has to look at stderr. A scoped registry that fails is only reported as skipped, as described above. diff --git a/docs/pm/cli/dedupe.mdx b/docs/pm/cli/dedupe.mdx index cbb7a88fd94c..7d42930f3872 100644 --- a/docs/pm/cli/dedupe.mdx +++ b/docs/pm/cli/dedupe.mdx @@ -3,7 +3,7 @@ title: "bun dedupe" description: "Remove duplicate versions of packages from bun.lock" --- -After bumping or adding a dependency, `bun.lock` can end up with several versions of the same package even though one of them satisfies every range that requests it — for example `esbuild@0.15.10` and `esbuild@0.15.11` when the ranges are `^0.15.7` and `^0.15.8`. `bun dedupe` re-points every dependency range onto the locked version that satisfies the most ranges (the highest version on ties), drops the versions nothing needs any more, saves `bun.lock`, and installs. +After bumping or adding a dependency, `bun.lock` can end up with several versions of the same package even though one of them satisfies every range that requests it — for example `esbuild@0.15.10` and `esbuild@0.15.11` when the ranges are `^0.15.7` and `^0.15.8`. `bun dedupe` keeps the smallest set of already-locked versions that still satisfies every range (preferring newer versions), moves only the ranges whose version is being dropped, removes those versions, saves `bun.lock`, and installs. It only uses versions that are already in the lockfile. It never moves a dependency outside its range and never contacts the registry to resolve anything; use `bun update` for that. @@ -23,7 +23,7 @@ Already deduplicated. ### `--check` -`bun dedupe --check` prints the versions that would be removed and exits with code `1` without installing packages or modifying `bun.lock`. It exits with code `0` when the lockfile is already deduplicated, which makes it usable in CI. `--dry-run` is an alias. +`bun dedupe --check` prints the versions that would be removed and exits with code `1` without installing packages or modifying `bun.lock`. It exits with code `0` when the lockfile is already deduplicated, which makes it usable in CI. Like `bun dedupe`, it exits with code `1` and `error: the lockfile is out of date with package.json, nothing was deduplicated` when `package.json` changed since the last install. `--dry-run` is an alias. ```bash terminal icon="terminal" bun dedupe --check @@ -36,10 +36,11 @@ bun dedupe --check ### Notes - Overrides and catalogs are honoured: a dependency is re-pointed using its effective range. -- Because it keeps the fewest versions, a direct dependency can be moved to an older version that still satisfies its range when a transitive dependency pins that older version. -- Dependencies pointing at a version listed in `patchedDependencies` are never moved, bundled dependencies stay on the copy inside their tarball, and dependencies specified as a dist-tag (such as `latest`), a git URL, or a tarball keep the version they resolved to. +- A dependency listed directly in the root or a workspace `package.json` is only moved to an older version when that is the only way to remove a version: if a transitive dependency pins an older version exactly, the direct range is collapsed onto it (pnpm keeps both versions in that case). Use `bun update` or an [override](/pm/overrides) if you want the newer version to win. +- A range whose version survives is never moved. +- Dependencies pointing at a version listed in `patchedDependencies` are never moved, bundled dependencies stay on the copy inside their tarball, and dependencies specified as a dist-tag (such as `latest`), a git URL, or a tarball keep the version they resolved to. A dist-tag keeps its version because only the registry knows what the tag points at now; other ranges are still collapsed onto that version. - Only the ranges of packages that remain installed are counted; a version that the same run removes does not influence where its own dependencies end up, so running the command twice never changes anything the second time. -- It works from the ranges recorded in `bun.lock`; changes made to `package.json` since the last install are applied by the install that follows. +- It works from the ranges recorded in `bun.lock`. If the dependencies, overrides, or catalogs in any `package.json` changed since the last install, it stops with `error: the lockfile is out of date with package.json, nothing was deduplicated` and exits with code `1` (also with `--check`); run `bun install` first. Edits that do not affect resolution (`scripts`, `trustedDependencies`) do not block it. - The command needs an existing `bun.lock` to work from: when there is none, `bun dedupe` (and `bun dedupe --check`) reports `missing lockfile, nothing to dedupe` and exits with code `1`, so run `bun install` first. - `bun.lock` and `node_modules` change; `package.json` never does. The result only lives in `bun.lock`: deleting the lockfile and reinstalling can bring the duplicates back. Add an [override](/pm/overrides) to pin a version permanently. - `--lockfile-only` rewrites `bun.lock` without installing. diff --git a/docs/pm/cli/install.mdx b/docs/pm/cli/install.mdx index e201ff47ea66..9925ebd572a6 100644 --- a/docs/pm/cli/install.mdx +++ b/docs/pm/cli/install.mdx @@ -174,7 +174,7 @@ bun install --frozen-lockfile Bun does not enable `--frozen-lockfile` on its own when it detects a CI environment (`CI=1` only turns off the progress bar); pass the flag or run `bun ci` if a stale `bun.lock` should fail the build. When there is no `bun.lock` at all, `bun install --frozen-lockfile` (and `bun ci`) resolves and installs from `package.json` without writing a lockfile; the error is only raised when a lockfile exists and does not match `package.json`. -`--frozen-lockfile` also works on a pruned copy of a monorepo (for example the output of `turbo prune`, or a Docker context that copies `bun.lock` with only some workspace folders): workspaces listed in `bun.lock` whose `package.json` is not on disk are skipped rather than treated as a lockfile change, and packages the pruned lockfile still lists are installed as written. This works whether the root `workspaces` field uses globs or lists each folder explicitly, and Bun prints a `note:` with the number of skipped workspaces (`--verbose` names them), so a `bun.lock` that is stale because a workspace was deleted without re-running `bun install` is still visible in CI logs. Only workspaces that `bun.lock` knows about are skipped: a `workspaces` entry whose folder is missing and which `bun.lock` does not list (for example a typo) still fails with `Workspace not found`, exactly as it does without `--frozen-lockfile`. A workspace is only skipped when its `package.json` is missing — removing it from the `workspaces` globs while the folder is still present, changing any `package.json` that is on disk, or trimming `overrides` or `catalog` entries from `bun.lock` still fails the install. The skipped workspaces' exclusive dependencies are not downloaded or installed, but they remain in `bun.lock`, so `bun pm ls` and `bun audit` still report them. A package that a surviving workspace lists as an optional peer dependency is installed if the pruned `bun.lock` still contains it, even when only a skipped workspace depended on it. This also holds when only `trustedDependencies` (which `turbo prune` leaves out of the pruned `bun.lock`) or `patchedDependencies` differ between `package.json` and `bun.lock`; a package is only dropped from the lockfile when a dependency, `overrides` or `catalog` entry actually changed, or when `bun audit fix` upgrades it. +`--frozen-lockfile` also works on a pruned copy of a monorepo (for example the output of `turbo prune`, or a Docker context that copies `bun.lock` with only some workspace folders): workspaces listed in `bun.lock` whose `package.json` is not on disk are skipped rather than treated as a lockfile change, and packages the pruned lockfile still lists are installed as written. This works whether the root `workspaces` field uses globs or lists each folder explicitly, and Bun prints a `note:` with the number of skipped workspaces (`--verbose` names them), so a `bun.lock` that is stale because a workspace was deleted without re-running `bun install` is still visible in CI logs. Only workspaces that `bun.lock` knows about are skipped: a `workspaces` entry whose folder is missing and which `bun.lock` does not list (for example a typo) still fails with `Workspace not found`, exactly as it does without `--frozen-lockfile`. If a remaining workspace (or the root `package.json`) depends on a skipped workspace, the install fails with `workspace "app" depends on workspace "other" (packages/other), which is listed in bun.lock but not on disk`: whatever prunes the checkout has to keep the workspaces the survivors depend on (`turbo prune` does). This is stricter than pnpm, whose `--frozen-lockfile` also ignores a workspace that is still on disk but was removed from the `workspaces` list — Bun only skips a workspace whose `package.json` is gone; un-listing one that is still present is a lockfile change. Changing any `package.json` that is on disk or trimming `overrides` from `bun.lock` still fails the install; `catalog`/`catalogs` entries, on the other hand, may be missing from a pruned `bun.lock` as long as only the skipped workspaces used them — every entry `bun.lock` does keep must still match `package.json`, and an entry that a remaining workspace or an `overrides` rule refers to must be present. The skipped workspaces' exclusive dependencies are not downloaded or installed, but they remain in `bun.lock`, so `bun pm ls` and `bun audit` still report them. A package that a surviving workspace lists as an optional peer dependency is installed if the pruned `bun.lock` still contains it, even when only a skipped workspace depended on it. This also holds when only `trustedDependencies` (which `turbo prune` leaves out of the pruned `bun.lock`) or `patchedDependencies` differ between `package.json` and `bun.lock`; a package is only dropped from the lockfile when a dependency, `overrides` or `catalog` entry actually changed, or when `bun audit fix` upgrades it. [`bun prune`](/pm/cli/prune) accepts such a checkout as well. `--frozen-lockfile` never writes `bun.lock`, including with `--lockfile-only`. The one exception is a lockfile format migration: converting `bun.lockb` with `--save-text-lockfile`, or migrating from `package-lock.json`, `yarn.lock` or `pnpm-lock.yaml`, still writes `bun.lock`. To check a lockfile without installing anything, use `bun install --frozen-lockfile --dry-run`. @@ -520,11 +520,17 @@ The migration process handles: ### Lockfile Migration -- Converts `pnpm-lock.yaml` to `bun.lock` format -- Preserves package versions and resolution information -- Maintains dependency relationships and peer dependencies -- Handles patched dependencies with integrity hashes -- Migrates git (`git+ssh://`, `git+https://`), GitHub, tarball URL and `file:` dependencies, including transitive ones and npm aliases (`npm:`) recorded in the lockfile +- Converts `pnpm-lock.yaml` versions 7 through 9 to `bun.lock`; a newer `lockfileVersion` is read with the 9.0 rules and produces a warning +- When the file holds several YAML documents (pnpm 11 writes one for its own tooling dependencies ahead of the project's), the last document is the one migrated +- Keeps every package's resolved version and integrity; registry packages are downloaded from the registry Bun is configured to use for that package, and a tarball URL recorded by pnpm is only kept when it lives under that registry +- `name@registry:version` entries written for pnpm's named registries use the URL from `namedRegistries` in `pnpm-workspace.yaml` (`gh` and `npmjs` are built in); an unknown name falls back to Bun's registry with a warning +- Peer dependencies keep the ranges each package declares, together with `peerDependenciesMeta`, and the `peerDependencies` of the root and of every workspace are read from their `package.json`, so the migrated `bun.lock` is the file `bun install` would have written and the next install leaves it unchanged +- A peer dependency that pnpm did not record is reported and left for the next `bun install` to resolve +- When a package appears in `snapshots:` under several peer suffixes, the first one in the file is used +- A `file:` directory whose `packages:` entry was removed by a pruning tool such as `turbo prune` is rebuilt from its snapshot, and injected workspace packages (`dependenciesMeta.*.injected`) become ordinary workspace dependencies +- Migrates git (`git+ssh://`, `git+https://`), GitHub, tarball URL and `file:` dependencies, including transitive ones and npm aliases (`npm:`) recorded in the lockfile; a local tarball is recognised by a `.tgz`, `.tar.gz` or `.tar` extension in any case +- Handles patched dependencies with integrity hashes; pnpm's current hash-only `patchedDependencies` entries are matched to the patch files listed in `package.json` or `pnpm-workspace.yaml` +- `runtime:` entries (Node.js versions managed by pnpm) are skipped with a warning, and `packages:` entries without a snapshot are ignored ### Workspace Configuration @@ -586,12 +592,15 @@ Bun migrates the following pnpm configuration from both `pnpm-lock.yaml` and `pn - **Patched Dependencies**: Moved from `pnpm.patchedDependencies` to root-level `patchedDependencies` in `package.json` - **Workspace Overrides**: Applied from `pnpm-workspace.yaml` to root `package.json` -### Requirements +### Requirements and limitations - Requires pnpm lockfile version 7 or higher - Workspace packages must have a `name` field in their `package.json` - All catalog entries referenced by dependencies must exist in the catalogs definition - Every workspace listed in `pnpm-lock.yaml` must have its `package.json` on disk (when building in Docker, copy each workspace's `package.json` before running `bun install`); otherwise migration reports the missing importer and `bun install` falls back to a fresh resolution +- `link:` dependencies that point at a relative path are not migrated, because Bun's `link:` protocol refers to packages registered with `bun link`: when pnpm recorded them the migration stops with a message, and when pnpm left them out through `excludeLinksFromLockfile` each omitted dependency is reported and the rest of the lockfile is migrated +- Git dependencies that select a sub-directory of the repository (`resolution.path`) stop the migration with a message naming the package +- When the migration stops, `bun install` resolves everything from scratch and prints why After migration, you can safely remove `pnpm-lock.yaml` and `pnpm-workspace.yaml` files. diff --git a/docs/pm/cli/pm.mdx b/docs/pm/cli/pm.mdx index 47a3af013ea2..5f3e0daf62d9 100644 --- a/docs/pm/cli/pm.mdx +++ b/docs/pm/cli/pm.mdx @@ -181,15 +181,15 @@ MIT (2) └── resolve@1.9.0 Unknown (4) -├── a-dep@1.0.1 +├── a-dep@1.0.1 (dev) ├── no-deps@1.0.0 ├── no-deps@1.0.1 └── one-dep@1.0.0 ``` -License groups are printed in name order with `Unknown` always last, and packages within a group are sorted by name and then version; if nothing is installed the text output is empty and `--json` prints `{}`. +License groups are printed in name order with `Unknown` always last, and packages within a group are sorted by name and then version. Packages that are only reached through `devDependencies` are marked `(dev)`. When nothing is listed, the text output is the single line `No packages found` and `--json` prints `{}`. -Pass `--prod` (or `-P`, `-p`, `--production`) to skip `devDependencies`, and `--json` to get a machine-readable object keyed by license, where each entry has `name`, `versions` (in semver order) and, when present, the `homepage` and `author` of the newest listed version: +Pass `--prod` (or `-P`, `-p`, `--production`) to skip `devDependencies`, and `--json` to get a machine-readable object keyed by license, where each entry has `name`, `versions` (in semver order), `license` (the same string as the key, `Unknown` included) and, when the newest listed version declares them, its `homepage`, `author` and `description`: ```bash terminal icon="terminal" bun pm licenses --json --prod @@ -201,24 +201,30 @@ bun pm licenses --json --prod { "name": "path-parse", "versions": ["1.0.6"], + "license": "MIT", "homepage": "https://github.com/jbgutierrez/path-parse#readme", - "author": "Javier Blanco " + "author": "Javier Blanco ", + "description": "Node.js path.parse() ponyfill" } ] } ``` -There is no `--long` flag; the text output only shows names and versions, so use `--json` when you need `author` or `homepage`. +`--long` prints each package's `author`, `description` and `homepage` (whichever it declares) on indented lines under its entry in the text output; it does not change `--json`. `--prod` drops every `devDependencies` edge — the same packages `bun install --production` leaves out, including the `devDependencies` of `file:` dependencies; `optionalDependencies`, `peerDependencies` and everything a production dependency pulls in are still listed, and from a workspace root every workspace's production dependencies are included. Run it inside a workspace package to list only that package's dependencies, and use [`bun why`](/pm/cli/why) to see which dependency pulls in an unexpected package. +`--dev` (or `-D`) lists only the packages that the `devDependencies` of the root (or of every workspace, from a workspace root) pull in, including their transitive dependencies. It is an error to combine it with `--prod`, `--omit=dev` or `install.production`. + +`--filter ` (or `-F`, repeatable) lists the dependencies of the matching workspaces only, from any directory of the monorepo. It takes the same patterns as [`bun outdated --filter`](/pm/cli/outdated): workspace names, `./relative/paths` resolved from the current directory, `!` to exclude, and `*` for every workspace. With a filter each selected workspace contributes only its own dependencies, so selecting the root does not pull in the workspace members, and a filter that matches nothing is an error. Without a filter, running from the workspace root lists every workspace's dependencies and running inside a workspace package lists only that package's. + `--omit=dev` is the same as `--prod` (as is [`install.production`](/runtime/bunfig#install-production) in `bunfig.toml`), and `--omit=optional` / `--omit=peer` skip `optionalDependencies` / `peerDependencies` as well, so the listing follows the same dependency types `bun install` would with those flags. `bun pm licenses` only reads `bun.lock` and `node_modules`; it never writes to either of them or to `package.json`, and it prints an error and exits with code `1` when the project has no lockfile or no `node_modules` directory. Packages that are in the lockfile but not in `node_modules` (for example after `bun install --production`) are omitted, and a `warn:` line with the number of omitted packages is printed to stderr. Packages that don't apply to the current platform (`os`/`cpu`) are omitted silently. -This is the equivalent of `pnpm licenses list`. +This is the equivalent of `pnpm licenses list`, including its `--json`, `--prod`, `--dev`, `--long` and `--filter` flags. ## whoami diff --git a/docs/pm/cli/prune.mdx b/docs/pm/cli/prune.mdx index 8e9fb734a7d1..3ed3de9068b7 100644 --- a/docs/pm/cli/prune.mdx +++ b/docs/pm/cli/prune.mdx @@ -3,7 +3,7 @@ title: "bun prune" description: "Remove packages that are not in bun.lock from node_modules" --- -`bun prune` deletes everything in `node_modules` that the current `bun.lock` does not install there — packages left behind after switching branches, editing `bun.lock`, or installing with another package manager. It only reads `bun.lock`; it never contacts the registry and never changes `bun.lock` or `package.json`. It takes no package names: passing one is an error, because it always removes every package `bun.lock` does not account for. +`bun prune` deletes everything in `node_modules` that the current `bun.lock` does not install there — packages left behind after switching branches, editing `bun.lock`, or installing with another package manager. It reads `bun.lock` and first checks that it still matches your `package.json` files; if you changed dependencies, `overrides` or a catalog since `bun.lock` was written it stops with `bun.lock does not match package.json` — run `bun install` first. It never contacts the registry and never changes `bun.lock` or `package.json`. It takes no package names: passing one is an error, because it always removes every package `bun.lock` does not account for. ```bash terminal icon="terminal" bun prune @@ -48,14 +48,29 @@ bun prune --production --dry-run Would remove 1 package ``` +### `--filter` + +`bun prune --filter ` (alias `-F`, repeatable; the same workspace name and `./path` globs as [`bun install --filter`](/pm/filter)) prunes only the `node_modules` folders of the selected workspaces. The part of the install that every workspace shares — the root `node_modules` with the hoisted linker, `node_modules/.bun` with the isolated linker — is cleaned in the same run, but anything a workspace you did not select depends on stays there, so `bun prune --production --filter app` removes only what `app` alone needed. A filter that matches no workspace is an error. + +```bash terminal icon="terminal" +bun prune --production --filter app +``` + +``` +- node_modules/.bun/typescript@5.4.0 +Removed 1 package +``` + ### Notes - Works with both linkers. With the hoisted linker it checks every `node_modules` folder `bun install` would install into — including workspace folders and the nested `node_modules` folders of installed packages, so copies left behind by an earlier install are removed too (subject to the version check below); with the isolated linker it removes unused entries in `node_modules/.bun` and the symlinks that pointed at them. With `--production`, the symlink of a dev-only dependency is removed and listed even when its `node_modules/.bun` entry stays because a production dependency also uses it. - The linker is chosen the same way `bun install` chooses it. If `node_modules` was installed with the other linker (for example `bun install --linker hoisted` in a project that defaults to isolated), `bun prune` refuses to run instead of removing packages the other layout needs; pass the same `--linker` you installed with, or run `bun install` to switch layouts. - Always runs against the workspace root, even when invoked inside a workspace package, and prunes the `node_modules` folder of every workspace package in that one run; with `--production`, a package that only a workspace's `devDependencies` need is removed too. -- Workspace symlinks (even `--production` keeps a workspace linked from another workspace's `devDependencies`), `.bin` entries that are still in use, dot-entries such as `.cache`, plain files, and dependencies bundled inside a package's tarball are never removed. `.bin` entries whose package was removed are cleaned up on every platform. +- Workspace folders themselves are never touched. With the isolated linker, `--production`/`--omit` also remove a workspace's link to another workspace when only its `devDependencies` listed it (a fresh `bun install --production` would not create that link); with the hoisted linker the workspace links live in the root folder and are always kept. `.bin` entries that are still in use, dot-entries such as `.cache`, plain files, and dependencies bundled inside a package's tarball are never removed. `.bin` entries whose package was removed are cleaned up on every platform. - Nothing outside `node_modules` is ever deleted. A package folder that you replaced with a symlink is left alone, and the `node_modules` folder inside it is not pruned. With `install.globalStore`, only the project's links into the store are removed. - Packages disabled for the current `os`/`cpu` are removed, just as `bun install` would skip them. Pass `--os` / `--cpu` to prune for another platform. -- Only `bun.lock` is consulted, and packages are matched by name: a package that is installed under the right name at the wrong version is left for `bun install` to replace, and a dependency removed from `package.json` is pruned only after the next `bun install` updates `bun.lock`. A copy nested inside another package (`node_modules/a/node_modules/b`) is only removed once the copy that replaces it higher up is installed at the version `bun.lock` expects (checked from its `package.json`, or `.bun-tag` for git dependencies); until then it is kept and `bun prune` prints a `warn:` line naming both folders — typically after `bun prune --production` when a `devDependency` pinned a different version of `b` at the root. Run `bun install` (with the same flags), then `bun prune` again. +- What stays is decided by `bun.lock` alone, and packages are matched by name: a package that is installed under the right name at the wrong version is left for `bun install` to replace, and a dependency you removed from `package.json` is pruned after the `bun install` that updates `bun.lock` (until then `bun prune` refuses to run). A copy nested inside another package (`node_modules/a/node_modules/b`) is only removed once the copy that replaces it higher up is installed at the version `bun.lock` expects (checked from its `package.json`, or `.bun-tag` for git dependencies); until then it is kept and `bun prune` prints a `warn:` line naming both folders — typically after `bun prune --production` when a `devDependency` pinned a different version of `b` at the root. Run `bun install` (with the same flags), then `bun prune` again. +- A `bun.lock` that still lists workspaces whose folders were removed (a pruned checkout that `bun install --frozen-lockfile` accepts) is accepted too. +- If an entry cannot be deleted, the others are still removed, `error: failed to remove ...` is printed for each failure and the command exits with code 1 (`pnpm prune` only warns) — in a Dockerfile, a package that could not be removed should fail the build. - Lifecycle scripts are never run. -- Equivalent to `pnpm prune` and `npm prune`. It does not touch the global cache; the counterpart of `pnpm store prune` is [`bun pm cache rm`](/pm/cli/pm#cache). Not related to `turbo prune`, which copies a subset of a monorepo's workspaces, `package.json` files and `bun.lock` into an output directory: `bun prune` only deletes from `node_modules`, so a Dockerfile can use `turbo prune`, then `bun install`, then `bun prune --production` after building. +- Equivalent to `pnpm prune` and `npm prune`. Unlike `pnpm prune`, it works in monorepos, and `--filter` narrows it to some workspaces. It does not touch the global cache; the counterpart of `pnpm store prune` is [`bun pm cache rm`](/pm/cli/pm#cache). Not related to `turbo prune`, which copies a subset of a monorepo's workspaces, `package.json` files and `bun.lock` into an output directory: `bun prune` only deletes from `node_modules`, so a Dockerfile can use `turbo prune`, then `bun install`, then `bun prune --production` after building. diff --git a/docs/pm/cli/remove.mdx b/docs/pm/cli/remove.mdx index d0c68bcce22b..b3f8ac0f155e 100644 --- a/docs/pm/cli/remove.mdx +++ b/docs/pm/cli/remove.mdx @@ -19,7 +19,7 @@ The package is deleted from every one of `dependencies`, `devDependencies`, `opt **Alias** — `-F` -In a monorepo, `--filter` removes the package from the matching workspace package(s) instead of the package in the current directory, using the same patterns as [`bun add --filter`](/pm/cli/add#--filter). To remove a package from every workspace, use `--filter '*'`; there is no `--recursive`/`-r` flag. Selected workspaces that don't list the package are left untouched, and if none of them list it no `package.json` is edited and the command exits 0. +In a monorepo, `--filter` removes the package from the matching workspace package(s) instead of the package in the current directory, using the same patterns as [`bun add --filter`](/pm/cli/add#--filter). To remove a package from every workspace, use `--filter '*'` (the root `package.json` is only included when a pattern names it, e.g. `--filter '*' --filter ''`); there is no `--recursive`/`-r` flag. Selected workspaces that don't list the package are left untouched, and if none of them list it no `package.json` is edited and the command exits 0. ```bash terminal icon="terminal" bun remove zod --filter api diff --git a/docs/pm/cli/update.mdx b/docs/pm/cli/update.mdx index fd5ae21936a9..349411a167c3 100644 --- a/docs/pm/cli/update.mdx +++ b/docs/pm/cli/update.mdx @@ -7,26 +7,29 @@ import Update from "/snippets/cli/update.mdx"; To upgrade your Bun CLI version, see [`bun upgrade`](/installation#upgrading). -`bun update` updates dependencies to the newest versions allowed by the ranges in `package.json`; [`bun update --latest`](#--latest) ignores those ranges. +`bun update` (also spelled `bun up`) updates dependencies to the newest versions allowed by the ranges in `package.json`; [`bun update --latest`](#--latest) ignores those ranges. ```sh terminal icon="terminal" bun update ``` -To update specific packages, pass their names. A `@range` suffix applies to the entry in your `package.json`: `bun update jquery@3` moves it to the newest `3.x` even if its current range does not allow that. A suffix cannot be combined with `--latest`; `bun update jquery@3 --latest` is an error. Glob patterns such as `@types/*` are not accepted here; use [`bun outdated`](/pm/cli/outdated) to list candidates. +To update specific packages, pass their names. A `@range` suffix applies to the entry in your `package.json`: `bun update jquery@3` moves it to the newest `3.x` even if its current range does not allow that. A suffix cannot be combined with `--latest`; `bun update jquery@3 --latest` is an error. + +A name can also be a pattern: `bun update '@types/*'` updates every package whose name matches, `bun update '!webpack'` updates everything except the excluded names, and the two combine (`bun update '@babel/*' '!@babel/core'`). Quote patterns so your shell does not expand them. Patterns are matched against the packages `bun.lock` records for the workspace you are in, so they reach nested packages too, and they need an existing `bun.lock`. A pattern that matches nothing is an error, and a pattern cannot carry a `@range`. ```sh terminal icon="terminal" bun update [package] bun update zod jquery@3 +bun update '@types/*' ``` A plain `bun update` updates the whole tree, like `pnpm update` and `npm update`: the dependencies declared in `package.json` move within their declared ranges and have their `package.json` entries rewritten, and every package that other packages in `bun.lock` depend on is re-resolved to the newest version its dependent's range allows. Each dependent is resolved against its own range, so two packages declaring different ranges on the same dependency can end up on different versions. This also applies when one of the dependents is your own `package.json`: an exact pin there stays put, and a package depending on a range of the same name moves to the newest version in that range, so `bun.lock` ends up with two copies. Ranges are never widened: a package depending on `^1.0.0` never picks up `2.x`; update the package that declares the range instead. Transitive updates only change `bun.lock` and `node_modules`, never `package.json`, and are listed before the install as `name@old → new`. -`bun update ` updates `` everywhere it occurs, at any depth, and nothing else: dependencies you did not name keep their locked versions, and so do the dependencies of `` itself as long as its new version still allows them. `` is matched by the name of the package that gets installed, so a dependency declared as `"foo": "npm:@^1"` is updated as well. If `` is declared in your `package.json`, that entry is rewritten too; if it is only a dependency of your dependencies, only `bun.lock` changes, which is how to pick up a fix in a nested package (`bun update caniuse-lite`) without adding it to your own dependencies or using `overrides`. A name that is not in `bun.lock` at all is an error, not a no-op; use `bun add` to add a dependency. Package names cannot be combined with `--recursive` or `--filter`, and `--production` is rejected because it would freeze the lockfile `bun update` needs to write. +`bun update ` re-resolves `` wherever the workspace you run it in depends on it, directly or through other packages, and nothing else: dependencies you did not name keep their locked versions, and so do the dependencies of `` itself as long as its new version still allows them. `` is matched by the name of the package that gets installed, so a dependency declared as `"foo": "npm:@^1"` is updated as well. If `` is declared in your `package.json`, that entry is rewritten too; if it is only a dependency of your dependencies, only `bun.lock` changes, which is how to pick up a fix in a nested package (`bun update caniuse-lite`) without adding it to your own dependencies or using `overrides`. A name that is not in `bun.lock` at all is an error, not a no-op; use `bun add` to add a dependency. In a monorepo, another workspace's own entry for `` is not re-resolved and its `package.json` is not touched; it only moves along when the version picked here also satisfies its range, so `bun.lock` does not gain a second copy. Run `bun update ` inside that workspace, or use `-r`/`--filter` ([below](#--recursive-and---filter)), to update it there; if only other workspaces depend on ``, `bun update ` is an error. ## What an update writes -`bun update` rewrites the version in `package.json` but keeps the operator you declared: `^1.1.0` becomes `^1.2.0`, `~` stays `~`, and an exact pin is left alone unless you pass `--latest`. With [`install.exact`](/runtime/bunfig#install-exact) (or `--exact`) the rewritten entry is an exact version even if it was declared with `^` or `~`. A `catalog:` reference is never rewritten; a plain `bun update` updates the catalog entry in the root `package.json` instead. A dist-tag such as `"foo": "latest"` or `"foo": "next"` stays as written, whether you run `bun update` or `bun update foo`, and only `bun.lock` moves; with `--latest` it is replaced by a `^` range on the version `latest` points at. `--dry-run` prints what would change without writing anything, and `--no-save` updates `node_modules` but leaves both `package.json` and `bun.lock` untouched, so it never changes what the next `bun install` produces. +`bun update` rewrites a `package.json` entry only when it is written as `^x.y.z`, `~x.y.z` or an exact version, and keeps the operator you declared: `^1.1.0` becomes `^1.2.0`, `~` stays `~`, and an exact pin is left alone unless you pass `--latest`. With [`install.exact`](/runtime/bunfig#install-exact) (or `--exact`) the rewritten entry is an exact version even if it was declared with `^` or `~`. Any other range (`*`, `1`, `1.x`, `>=1.0.0`, `1.0.0 - 1.5.0`, or an alias without a version such as `npm:foo`) is left exactly as written and only `bun.lock` and `node_modules` move; `--latest` replaces such a range with a `^` range on the latest version. A `catalog:` reference is never rewritten; a plain `bun update` updates the catalog entry in the root `package.json` instead. A dist-tag such as `"foo": "latest"` or `"foo": "next"` (also inside a catalog, or behind `npm:foo@next`) is always kept as written, with or without `--latest`, and `bun.lock` follows whatever the tag points at now. `--dry-run` prints what would change without writing anything, and `--no-save` updates `node_modules` but leaves both `package.json` and `bun.lock` untouched, so it never changes what the next `bun install` produces. After any `bun update` or `bun add`, `bun.lock` records the same ranges that were written to `package.json`, so a following `bun install --frozen-lockfile` passes. @@ -112,12 +115,26 @@ Within each section, individual packages may have a suffix (` dev`, ` peer`, ` o ## `--recursive` and `--filter` -In a monorepo, a plain `bun update` rewrites only the `package.json` of the workspace you run it in (transitive packages are shared, so they update either way). `bun update --recursive` (`-r`) also updates the direct dependencies of every workspace and rewrites each workspace's `package.json`; `bun update --filter ` limits that to the matching workspaces (globs and `!` negation are accepted). Both need an existing `bun.lock`, and neither can be combined with package names. They also work with `--interactive`, which then adds a "Workspace" column showing which workspace each dependency belongs to: +In a monorepo, a plain `bun update` rewrites only the `package.json` of the workspace you run it in (transitive packages are shared, so they update either way). `bun update --recursive` (`-r`) also updates the direct dependencies of every workspace and rewrites each workspace's `package.json`; `bun update --filter ` limits that to the matching workspaces; the pattern syntax, including `{dir}` and `...` relations, is described on the [filtering](/pm/filter) page. Both need an existing `bun.lock`. They also work with `--interactive`, which then adds a "Workspace" column showing which workspace each dependency belongs to. + +`bun update -r` and `bun update --filter ` rewrite the entry for `` in every selected workspace whose `package.json` declares it, each keeping its own `^`/`~`/exact style; the `package.json` of a workspace that was not selected is left alone, and nested packages that depend on `` still move in `bun.lock`. A name that none of the selected workspaces uses, directly or through their dependencies, is an error. `--latest` and `--dry-run` apply the same way: ```sh terminal icon="terminal" bun update --recursive -bun update --filter 'packages/*' +bun update --filter './packages/*' bun update -i -r +bun update zod -r +bun update zod --filter '...^ui' +``` + +## `--dev`, `--prod`, `--no-optional` + +`bun update --dev` (`-D`) only updates the entries in `devDependencies`; `--prod` (`-P`, also `--production`) only those in `dependencies` and `optionalDependencies`; `--no-optional` skips `optionalDependencies`. These select which entries of the current workspace's `package.json` (or of the workspaces chosen with `-r`/`--filter`) are updated, and combine with names, patterns and `--latest`. They do not change what gets installed: `bun update --production` still installs `devDependencies`, unlike `bun install --production`. They need an existing `bun.lock`, and print `No packages to update` when no entry qualifies: + +```sh terminal icon="terminal" +bun update --dev +bun update --prod --latest +bun update -D '@types/*' ``` ## `--global` @@ -133,10 +150,11 @@ bun update -g typescript By default, `bun update` updates each dependency to the latest version that satisfies the version range in your `package.json`. -To update the dependencies declared in `package.json` to the latest version regardless of whether it satisfies that range, use the `--latest` flag. Transitive packages still move within the ranges their dependents declare, and a dependency that is already ahead of the `latest` tag (for example a prerelease) is left where it is rather than downgraded: +To update the dependencies declared in `package.json` to the latest version regardless of whether it satisfies that range, use the `--latest` flag (`-L`). Transitive packages still move within the ranges their dependents declare, and a dependency that is already ahead of the `latest` tag (for example a prerelease) is left where it is rather than downgraded: ```sh terminal icon="terminal" bun update --latest +bun update -L ``` In interactive mode, press **l** to toggle a package between its target version (respecting semver) and the latest version. diff --git a/docs/pm/filter.mdx b/docs/pm/filter.mdx index 4e1448fe628b..e326502aa80c 100644 --- a/docs/pm/filter.mdx +++ b/docs/pm/filter.mdx @@ -3,9 +3,9 @@ title: "bun --filter" description: "Select packages by pattern in a monorepo using the --filter flag" --- -The `--filter` (or `-F`) flag selects packages in a monorepo by pattern. Patterns match package names or package paths, with full glob syntax. +The `--filter` (or `-F`) flag selects packages in a monorepo by pattern. A pattern is a package name glob, a `./path` glob, a `{dir}` directory selector, or a `...` dependency relation. -`bun install`, `bun add`, `bun remove`, `bun update` and `bun outdated` support `--filter` (pass it after the subcommand, or as `--filter=` before it); for `bun add`/`bun remove` it selects which workspace `package.json` files are edited. You can also use it to run scripts in multiple packages at once. +`bun install`, `bun add`, `bun remove`, `bun update` and `bun outdated` support `--filter` (pass it after the subcommand, or as `--filter=` before it); for `bun add`/`bun remove` it selects which workspace `package.json` files are edited. You can also use it to run scripts in multiple packages at once. The `{dir}` and `...` forms are only understood by these package-manager commands, not by `bun run --filter`. --- @@ -21,12 +21,39 @@ Path patterns start with `./` and select all packages in directories matching th Patterns are matched against the full package name -- `--filter core` does not select `@acme/core` (use `@acme/core` or `@acme/*`), and `*` does not cross `/`. Path patterns must match a workspace's directory itself: `--filter ./packages` selects nothing, `--filter './packages/*'` selects every workspace directly inside it, and `--filter '!./lib'` only excludes a workspace located at `./lib`. +### Directory `--filter '{}'` + +A directory in braces selects every workspace located in that directory or anywhere below it. The directory is resolved from the current directory: `--filter '{packages}'` and `--filter '{./packages}'` select every workspace under `packages`, and `--filter '{.}'` selects the workspace in the current directory and everything under it. Unlike a `./path` pattern, which has to match a workspace's own directory, the directory in braces can be any parent of the workspaces you want. A pattern that starts with `{` is always a directory selector. + +### Dependency relations `--filter 'foo...'` + +Adding `...` to a pattern also selects the workspaces related to it through workspace dependencies: + +| Pattern | Selects | +| ---------- | --------------------------------------------------------------------------- | +| `foo...` | `foo` and the workspaces it depends on, directly or transitively | +| `foo^...` | only the workspaces `foo` depends on, not `foo` itself | +| `...foo` | `foo` and the workspaces that depend on it, directly or transitively | +| `...^foo` | only the workspaces that depend on `foo`, not `foo` itself | + +The middle part is a name glob or a directory selector (`...{./packages/api}`), and `!` still goes first: `--filter '!...foo'` drops `foo` and everything that depends on it. + +Relations follow the workspace links recorded in `bun.lock`, from any dependency group. `bun install --filter 'web...'` works on a fresh clone, but `bun add`, `bun remove` and `bun update ` with a relation need an existing `bun.lock` and ask you to run `bun install` first when it is missing. + +```bash terminal icon="terminal" +bun install --filter 'web...' +bun add zod --filter '...^ui' +bun outdated --filter '{./packages/apps}' +``` + --- ## `bun install` and `bun outdated` By default, `bun install` installs dependencies for every package in the monorepo. To install dependencies for specific packages, use `--filter`. +A set of patterns selects everything matched by a positive pattern, minus everything matched by a `!` pattern, so `bun install --filter api --filter '!web'` installs only `api`. If nothing matches, `bun install` installs nothing and exits 0. + Given a monorepo with workspaces `pkg-a`, `pkg-b`, and `pkg-c` under `./packages`: ```bash terminal icon="terminal" diff --git a/docs/snippets/cli/update.mdx b/docs/snippets/cli/update.mdx index 9fe4013e019f..f840a6e3d6f4 100644 --- a/docs/snippets/cli/update.mdx +++ b/docs/snippets/cli/update.mdx @@ -1,7 +1,8 @@ ## CLI Usage ```bash terminal icon="terminal" -bun update @ +bun update [[@] | ]... +bun up ``` ### Update Strategy @@ -11,11 +12,24 @@ bun update @ - Update packages to their latest versions + Update packages to their latest versions. Alias: -L ### Dependency Scope + + Only update devDependencies. Alias: -D + + + + Only update dependencies and optionalDependencies. Aliases: -P,{" "} + --production + + + + Don't update optionalDependencies + + Install globally. Alias: -g diff --git a/src/install/NetworkTask.rs b/src/install/NetworkTask.rs index e4d507603b30..615177ac1646 100644 --- a/src/install/NetworkTask.rs +++ b/src/install/NetworkTask.rs @@ -1,11 +1,11 @@ -use core::mem::{ManuallyDrop, MaybeUninit}; +use core::mem::MaybeUninit; use core::ptr::{self, NonNull}; use core::sync::atomic::Ordering; use crate::bun_fs::{FileSystem, FilenameStore}; use bun_collections::HashMap; use bun_core::{self, fmt::quote}; -use bun_core::{MutableString, StringBuilder, strings}; +use bun_core::{MutableString, strings}; use bun_http::{ self as http, AsyncHTTP, HTTPClientResult, HTTPClientResultCallback, HTTPVerboseLevel, HeaderBuilder, async_http::Options as AsyncHTTPOptions, @@ -38,7 +38,7 @@ pub(crate) fn filename_store_appender() -> FilenameStoreAppender<'static> { } pub struct NetworkTask { - // Self-referential: borrows `url_buf` / leaked header content owned by + // Self-referential: borrows `url_buf` / `header_buf` owned by // sibling fields, so the lifetime is erased to `'static`. // `MaybeUninit` because the slot comes from `HiveArrayFallback` // as *uninitialized* memory (often zero-page on first mmap, but not @@ -60,6 +60,7 @@ pub struct NetworkTask { // `tarball.url` in the latter would be a self-reference // into `callback`; owning avoids that at the cost of one copy per tarball download. pub(crate) url_buf: Box<[u8]>, + pub(crate) header_buf: Box<[u8]>, pub(crate) retried: u16, pub(crate) response_buffer: MutableString, // BACKREF: PackageManager owns this task via `preallocated_network_tasks`. @@ -576,20 +577,19 @@ impl NetworkTask { if !etag.is_empty() { header_builder.count("If-None-Match", etag); - } - - if !last_modified.is_empty() { + } else if !last_modified.is_empty() { header_builder.count("If-Modified-Since", last_modified); } - if header_builder.header_count > 0 { + let headers_buf: &'static [u8] = if header_builder.header_count > 0 { let accept_header = if needs_extended { ACCEPT_HEADER_VALUE_EXTENDED } else { ACCEPT_HEADER_VALUE }; header_builder.count("Accept", accept_header); - if !last_modified.is_empty() && !etag.is_empty() { + let trailing_last_modified = !last_modified.is_empty() && !etag.is_empty(); + if trailing_last_modified { header_builder.content.count(last_modified); } header_builder.allocate()?; @@ -604,15 +604,19 @@ impl NetworkTask { header_builder.append("Accept", accept_header); - if !last_modified.is_empty() && !etag.is_empty() { - let appended = header_builder.content.append(last_modified); - // SAFETY: lifetime extension — the appended slice points into - // `header_builder.content`'s heap buffer, which is moved into - // `self.unsafe_http_client.client.header_buf` below and - // outlives the request. Detach the borrow so - // `header_builder.content` can be read again for `headers_buf`. - last_modified = unsafe { bun_ptr::detach_lifetime(appended) }; + let last_modified_start = header_builder.content.len; + if trailing_last_modified { + let _ = header_builder.content.append(last_modified); } + debug_assert_eq!(header_builder.content.len, header_builder.content.cap); + self.header_buf = header_builder.content.move_to_slice(); + if trailing_last_modified { + // SAFETY: `self.header_buf` outlives the request; it is freed when the slot returns to the pool. + last_modified = + unsafe { bun_ptr::detach_lifetime(&self.header_buf[last_modified_start..]) }; + } + // SAFETY: same invariant as `last_modified` above. + unsafe { bun_ptr::detach_lifetime(&*self.header_buf) } } else { let header_buf: &'static str = if needs_extended { EXTENDED_HEADERS_BUF @@ -630,34 +634,15 @@ impl NetworkTask { }, })?; header_builder.header_count = 1; - // SAFETY: header_buf is &'static str; StringBuilder borrows it - // mutably in type but is never written to on this path. - header_builder.content = StringBuilder { - ptr: NonNull::new(header_buf.as_ptr().cast_mut()), - len: header_buf.len(), - cap: header_buf.len(), - }; - } + self.header_buf = Box::default(); + header_buf.as_bytes() + }; self.response_buffer = MutableString::init(0)?; - // SAFETY: lifetime extension — `url_buf` and the header content buffer - // are heap allocations owned by / leaked into `*self`, which outlives - // the HTTP request. `AsyncHTTP::init` demands `'static` borrows - // because the HTTP thread reads them concurrently. See the - // identical pattern in `s3/simple_request.rs`. + // SAFETY: `self.url_buf` outlives the request, same as `header_buf` above (see `s3/simple_request.rs`). let url = URL::parse(unsafe { bun_ptr::detach_lifetime(&self.url_buf) }); let http_proxy = pm.http_proxy(&url); - // SAFETY: `written_slice()` is the safe (ptr,len) accessor; only the - // `'static` erasure remains unsafe — the buffer is leaked into the - // HTTP client below (`ManuallyDrop`), so it genuinely outlives this frame. - let headers_buf: &'static [u8] = - unsafe { bun_ptr::detach_lifetime(header_builder.content.written_slice()) }; - // `header_builder.content` is intentionally leaked (ownership - // transfers to the HTTP client). Forget it so - // `StringBuilder::drop` doesn't free the buffer that `headers_buf` / - // `last_modified` now alias. - let _ = ManuallyDrop::new(core::mem::take(&mut header_builder.content)); let completion_callback = self.get_completion_callback(); // MaybeUninit overwrite — see field doc; old slot value is // either uninitialized (fresh hive slot) or a stale bitwise copy from @@ -701,11 +686,7 @@ impl NetworkTask { .unwrap_or(false) { self.http_mut().client.flags.force_last_modified = true; - // SAFETY: lifetime extension — `last_modified` either points into - // the leaked `header_builder.content` buffer (reassigned above) or - // into the manifest's `string_buf`, which is the same allocation - // referenced by the `PackageManifest` we just cloned into - // `self.callback`. Both outlive the HTTP request. + // SAFETY: `last_modified` points into `self.header_buf` or into the manifest `string_buf` cloned into `self.callback`; both outlive the request. self.http_mut().client.if_modified_since = unsafe { bun_ptr::detach_lifetime(last_modified) }; } @@ -829,29 +810,22 @@ impl NetworkTask { self.response_buffer = MutableString::init_empty(); let mut header_builder = HeaderBuilder::default(); - let mut header_buf: &'static [u8] = b""; if send_auth { count_auth(&mut header_builder, scope); } - if header_builder.header_count > 0 { + let header_buf: &'static [u8] = if header_builder.header_count > 0 { header_builder.allocate()?; - - if send_auth { - append_auth(&mut header_builder, scope); - } - - // SAFETY: `written_slice()` is the safe (ptr,len) accessor; only the - // `'static` erasure remains unsafe — buffer is leaked below. - header_buf = - unsafe { bun_ptr::detach_lifetime(header_builder.content.written_slice()) }; - } - // `header_builder.content` is intentionally leaked (ownership - // transfers to the HTTP client). Forget it so - // `StringBuilder::drop` doesn't free the buffer that `header_buf` now - // aliases. - let _ = ManuallyDrop::new(core::mem::take(&mut header_builder.content)); + append_auth(&mut header_builder, scope); + debug_assert_eq!(header_builder.content.len, header_builder.content.cap); + self.header_buf = header_builder.content.move_to_slice(); + // SAFETY: `self.header_buf` outlives the request; it is freed when the slot returns to the pool. + unsafe { bun_ptr::detach_lifetime(&*self.header_buf) } + } else { + self.header_buf = Box::default(); + b"" + }; // SAFETY: lifetime extension — `url_buf` is a heap allocation owned by // `*self`, which outlives the HTTP request. `AsyncHTTP::init` demands a @@ -992,6 +966,7 @@ impl NetworkTask { // Struct-default fields. addr_of_mut!((*slot).response).write(HTTPClientResult::default()); addr_of_mut!((*slot).url_buf).write(Box::default()); + addr_of_mut!((*slot).header_buf).write(Box::default()); addr_of_mut!((*slot).retried).write(0); addr_of_mut!((*slot).next).write(bun_threading::Link::new()); addr_of_mut!((*slot).tarball_stream).write(None); diff --git a/src/install/PackageManager.rs b/src/install/PackageManager.rs index dc75bf2ab592..75acb5560103 100644 --- a/src/install/PackageManager.rs +++ b/src/install/PackageManager.rs @@ -8,7 +8,6 @@ use crate::bun_fs as fs; use crate::bun_fs::FileSystem; use crate::bun_progress::{Node as ProgressNode, Progress}; use crate::bun_schema::api as Api; -use bun_alloc::AllocError; use bun_collections::linear_fifo::{DynamicBuffer, StaticBuffer}; use bun_collections::{ArrayHashMap, HashMap, HiveArrayFallback, LinearFifo, StringArrayHashMap}; use bun_core::ZBox; @@ -89,6 +88,8 @@ pub mod update_package_json_and_install; pub mod update_request; #[path = "PackageManager/WorkspacePackageJSONCache.rs"] pub mod workspace_package_json_cache; +#[path = "PackageManager/workspace_selection.rs"] +pub mod workspace_selection; /// Lower-case path alias so `package_manager::options::Options` (used by the /// retired stub surface) keeps resolving. @@ -129,13 +130,16 @@ impl PackageManagerCommand { └ --quiet only output the tarball filename bun pm bin print the path to bin folder └ -g print the global path to bin folder - bun list list the dependency tree according to the current lockfile + bun pm ls list the dependency tree according to the current lockfile ├ --all list the entire dependency tree according to the current lockfile └ --trusted list only trusted dependencies bun pm why \ show dependency tree explaining why a package is installed bun pm licenses list installed packages grouped by license ├ --json output as JSON - └ --prod omit devDependencies + ├ --prod omit devDependencies + ├ --dev list only what devDependencies pull in + ├ --long also print author, description and homepage + └ --filter \ list only the matching workspaces' dependencies bun pm whoami print the current npm username bun pm view name[@version] view package metadata from the registry (use `bun info` instead) bun pm version [increment] bump the version in package.json and create a git tag @@ -427,6 +431,9 @@ pub struct PackageManager { // package.json cache entries that differ from disk; written by package_json_write_back::flush. pub(crate) edited_package_jsons: Vec, + // bun add: catalog references decided per target and the root entries they need; see add_catalog.rs + pub(crate) catalog_add: add_catalog::State, + pub(crate) patched_dependencies_to_remove: ArrayHashMap, @@ -498,7 +505,13 @@ impl Subcommand { pub(crate) fn supports_workspace_filtering(self) -> bool { matches!( self, - Self::Outdated | Self::Install | Self::Update | Self::Add | Self::Remove + Self::Outdated + | Self::Install + | Self::Update + | Self::Add + | Self::Remove + | Self::Prune + | Self::Pm ) } @@ -512,93 +525,25 @@ impl Subcommand { } } -pub enum WorkspaceFilter { - All, - Name(Box<[u8]>), - Path(Box<[u8]>), +/// The resolved outcome of `--filter` for one install: the importer ids whose dependencies get installed. +pub struct WorkspaceFilter { + pub(crate) workspace_ids: Box<[PackageID]>, } impl WorkspaceFilter { - pub fn init( - input: &[u8], - cwd: &[u8], - path_buf: &mut [u8], - ) -> Result { - if (input.len() == 1 && input[0] == b'*') || input == b"**" { - return Ok(WorkspaceFilter::All); - } - - let mut remain = input; - - let mut prepend_negate = false; - while !remain.is_empty() && remain[0] == b'!' { - prepend_negate = !prepend_negate; - remain = &remain[1..]; - } - - let is_path = !remain.is_empty() && remain[0] == b'.'; - - let filter: &[u8] = - if is_path { - strings::without_trailing_slash( - resolve_path::join_abs_string_buf::(cwd, path_buf, &[remain]), - ) - } else { - remain - }; - - if filter.is_empty() { - // won't match anything - return Ok(WorkspaceFilter::Path(Box::default())); - } - let copy_start = prepend_negate as usize; - let copy_end = copy_start + filter.len(); - - let mut buf = vec![0u8; copy_end].into_boxed_slice(); - buf[copy_start..copy_end].copy_from_slice(filter); - - if prepend_negate { - buf[0] = b'!'; + pub(crate) fn from_ids(mut ids: Vec) -> WorkspaceFilter { + ids.sort_unstable(); + ids.dedup(); + WorkspaceFilter { + workspace_ids: ids.into_boxed_slice(), } + } - // pattern = buf[0..copy_end] == buf (since buf.len() == copy_end) - Ok(if is_path { - WorkspaceFilter::Path(buf) - } else { - WorkspaceFilter::Name(buf) - }) - } - - pub fn matches_any(filters: &[WorkspaceFilter], name: &[u8], abs_posix_path: &[u8]) -> bool { - let has_positive = filters.iter().any(|f| match f { - WorkspaceFilter::All => true, - WorkspaceFilter::Path(p) | WorkspaceFilter::Name(p) => p.first() != Some(&b'!'), - }); - - let mut matched = !has_positive; - for filter in filters { - let (pattern, subject): (&[u8], &[u8]) = match filter { - WorkspaceFilter::All => { - matched = true; - continue; - } - WorkspaceFilter::Path(pattern) => { - if pattern.is_empty() { - continue; - } - (pattern, abs_posix_path) - } - WorkspaceFilter::Name(pattern) => (pattern, name), - }; - if pattern.first() == Some(&b'!') { - if bun_glob::r#match(&pattern[1..], subject).matches() { - return false; - } - } else if bun_glob::r#match(pattern, subject).matches() { - matched = true; - } - } - matched + #[inline] + pub(crate) fn is_selected(filters: &[WorkspaceFilter], pkg_id: PackageID) -> bool { + filters + .iter() + .all(|f| f.workspace_ids.binary_search(&pkg_id).is_ok()) } /// Every workspace (root included), filtered by `filter_patterns` (empty = all). @@ -607,65 +552,15 @@ impl WorkspaceFilter { filter_patterns: &[&[u8]], original_cwd: &[u8], ) -> Vec { - use crate::lockfile::package::PackageColumns as _; - - let packages = lockfile.packages.slice(); - let pkg_names = packages.items_name(); - let pkg_resolutions = packages.items_resolution(); - let string_buf = lockfile.buffers.string_bytes.as_slice(); - - let mut ids: Vec = Vec::new(); - for (pkg_id, res) in pkg_resolutions.iter().enumerate() { - if res.tag == crate::resolution::Tag::Workspace - || res.tag == crate::resolution::Tag::Root - { - ids.push(pkg_id as PackageID); - } - } - - if filter_patterns.is_empty() { - return ids; - } - - let mut path_buf = PathBuffer::uninit(); - let converted_filters: Vec = filter_patterns - .iter() - .map(|filter| { - bun_core::handle_oom(WorkspaceFilter::init(filter, original_cwd, &mut path_buf.0)) - }) - .collect(); - - let top_level_dir = FileSystem::instance().top_level_dir(); - - let mut i = 0; - while i < ids.len() { - let pkg_id = ids[i] as usize; - let name = pkg_names[pkg_id].slice(string_buf); - let res = &pkg_resolutions[pkg_id]; - let res_path: &[u8] = match res.tag { - crate::resolution::Tag::Workspace => res.workspace().slice(string_buf), - crate::resolution::Tag::Root => top_level_dir, - _ => unreachable!(), - }; - let abs = resolve_path::join_abs_string_buf::( - top_level_dir, - &mut path_buf.0, - &[res_path], - ); - let abs = strings::without_trailing_slash(abs); - if WorkspaceFilter::matches_any(&converted_filters, name, abs) { - i += 1; - } else { - ids.swap_remove(i); - } - } - - ids + workspace_selection::select_lockfile_workspaces( + lockfile, + filter_patterns, + original_cwd, + workspace_selection::RootSelection::Implicit, + ) } } -// deinit → Drop is automatic for Box<[u8]> variants; no explicit impl needed. - #[derive(Default)] pub struct PackageUpdateInfo { pub(crate) original_version_literal: Box<[u8]>, @@ -2218,6 +2113,7 @@ pub fn init( wr!(update_target_workspaces, None); wr!(pending_filtered_write, None); wr!(edited_package_jsons, Vec::new()); + wr!(catalog_add, add_catalog::State::default()); wr!(patched_dependencies_to_remove, ArrayHashMap::default()); wr!(last_reported_slow_lifecycle_script_at, 0); wr!(cached_tick_for_slow_lifecycle_script_logging, 0); @@ -2660,6 +2556,7 @@ fn init_with_runtime_once( wr!(update_target_workspaces, None); wr!(pending_filtered_write, None); wr!(edited_package_jsons, Vec::new()); + wr!(catalog_add, add_catalog::State::default()); wr!(patched_dependencies_to_remove, ArrayHashMap::default()); wr!(last_reported_slow_lifecycle_script_at, 0); wr!(cached_tick_for_slow_lifecycle_script_logging, 0); diff --git a/src/install/PackageManager/CommandLineArguments.rs b/src/install/PackageManager/CommandLineArguments.rs index 51e8cc45f16f..94759e3934f3 100644 --- a/src/install/PackageManager/CommandLineArguments.rs +++ b/src/install/PackageManager/CommandLineArguments.rs @@ -152,7 +152,7 @@ pub(crate) static UPDATE_PARAMS: &[ParamType] = concat_params![ SHARED_PARAMS, &[ clap::param!( - "--latest Update packages to their latest versions" + "-L, --latest Update packages to their latest versions, ignoring the ranges in package.json" ), clap::param!( "-i, --interactive Show an interactive list of outdated packages to select for update" @@ -161,7 +161,12 @@ pub(crate) static UPDATE_PARAMS: &[ParamType] = concat_params![ "--filter ... Update packages for the matching workspaces" ), clap::param!("-r, --recursive Update packages in all workspaces"), - clap::param!(" ... \"name\" of packages to update"), + clap::param!("-d, --dev Only update devDependencies"), + clap::param!("-D, --development"), + clap::param!("--no-optional Don't update optionalDependencies"), + clap::param!( + " ... \"name\" or pattern (\"@scope/*\", \"!name\") of packages to update" + ), ] ]; @@ -171,7 +176,15 @@ pub(crate) static PM_PARAMS: &[ParamType] = concat_params![ clap::param!("-a, --all"), clap::param!("--trusted"), clap::param!("--json Output in JSON format"), - // clap::param!("--filter ... Pack each matching workspace"), + clap::param!( + "-F, --filter ... List only the matching workspaces' dependencies (bun pm licenses)" + ), + clap::param!( + "-D, --dev List only the packages pulled in by devDependencies (bun pm licenses)" + ), + clap::param!( + "--long Also print author, description and homepage (bun pm licenses)" + ), clap::param!( "--destination The directory the tarball will be saved in" ), @@ -373,7 +386,12 @@ static DEDUPE_PARAMS: &[ParamType] = concat_params![ static PRUNE_PARAMS: &[ParamType] = concat_params![ SHARED_PARAMS, - &[clap::param!(" ... "),] + &[ + clap::param!( + "-F, --filter ... Only prune the node_modules folders of the matching workspaces" + ), + clap::param!(" ... "), + ] ]; const PRUNE_HELP_PARAMS: &[ParamType] = &[ @@ -395,6 +413,9 @@ const PRUNE_HELP_PARAMS: &[ParamType] = &[ clap::param!( "--linker Prune a node_modules installed with the given linker (one of \"isolated\" or \"hoisted\")" ), + clap::param!( + "-F, --filter ... Only prune the node_modules folders of the matching workspaces; the shared store / hoisted root folder keeps everything other workspaces use" + ), clap::param!("--silent Don't log anything"), clap::param!("--cwd Set a specific cwd"), clap::param!("-h, --help Print this help menu"), @@ -440,6 +461,7 @@ pub struct CommandLineArguments { pub json_output: bool, pub(crate) recursive: bool, pub(crate) filters: &'static [&'static [u8]], + pub update_groups: UpdateGroups, pub(crate) pack_destination: &'static [u8], pub(crate) pack_filename: &'static [u8], @@ -482,6 +504,10 @@ pub struct CommandLineArguments { pub top_only: bool, pub(crate) depth: Option, + // `bun pm licenses` options + pub dev_only: bool, + pub long: bool, + // `bun audit` options pub audit_level: Option, pub audit_ignore_list: &'static [&'static [u8]], @@ -524,6 +550,7 @@ impl Default for CommandLineArguments { json_output: false, recursive: false, filters: &[], + update_groups: UpdateGroups::default(), pack_destination: b"", pack_filename: b"", @@ -564,6 +591,9 @@ impl Default for CommandLineArguments { top_only: false, depth: None, + dev_only: false, + long: false, + audit_level: None, audit_ignore_list: &[], @@ -619,6 +649,19 @@ pub struct Omit { pub(crate) peer: bool, } +#[derive(Default, Copy, Clone, PartialEq, Eq)] +pub struct UpdateGroups { + pub dev: bool, + pub prod: bool, + pub no_optional: bool, +} + +impl UpdateGroups { + pub fn is_default(self) -> bool { + self == UpdateGroups::default() + } +} + impl CommandLineArguments { pub fn print_help(subcommand: Subcommand) { // the output of --help uses the following syntax highlighting @@ -655,6 +698,7 @@ Full documentation is available at https://bun.com/docs/cli/install. Subcommand::Update => { let intro_text = r" Usage: bun update [flags] \@\ +Alias: bun up Update dependencies to their most recent versions within the version range in package.json. @@ -674,6 +718,13 @@ Full documentation is available at https://bun.com/docs/cli/install. Update specific packages: bun update zod jquery@3 + Update every @types package, or everything except webpack: + bun update '@types/*' + bun update '!webpack' + + Only update devDependencies: + bun update --dev + Full documentation is available at https://bun.com/docs/cli/update. "; pretty_help(intro_text); @@ -1052,6 +1103,9 @@ Full documentation is available at https://bun.com/docs/pm/cli/dedupeShow what would be removed without deleting anything bun prune --dry-run + Only prune what the app workspace no longer needs + bun prune --production --filter app + Full documentation is available at https://bun.com/docs/pm/cli/prune. "; @@ -1468,17 +1522,12 @@ Full documentation is available at https://bun.com/docs/pm/cli/prune cli.latest = args.flag(b"--latest"); cli.interactive = args.flag(b"--interactive"); cli.recursive = args.flag(b"--recursive"); - if cli.production { - Output::err_generic("--production cannot be used with bun update\n", ()); - Global::crash(); - } - if cli.positionals.len() > 1 && (cli.recursive || !cli.filters.is_empty()) { - Output::err_generic( - "--recursive and --filter cannot be combined with package names\n", - (), - ); - Global::crash(); - } + cli.update_groups = UpdateGroups { + dev: args.flag(b"--dev") || args.flag(b"--development"), + prod: cli.production, + no_optional: args.flag(b"--no-optional"), + }; + cli.production = false; } let specified_backend: Option = 'brk: { @@ -1587,6 +1636,8 @@ Full documentation is available at https://bun.com/docs/pm/cli/prune if let Some(message) = args.option(b"--message") { cli.message = Some(message); } + cli.dev_only = args.flag(b"--dev"); + cli.long = args.flag(b"--long"); } // `bun pm why` and `bun why` options diff --git a/src/install/PackageManager/PackageJSONEditor.rs b/src/install/PackageManager/PackageJSONEditor.rs index 06d357f6152c..6b67c5120f35 100644 --- a/src/install/PackageManager/PackageJSONEditor.rs +++ b/src/install/PackageManager/PackageJSONEditor.rs @@ -78,13 +78,52 @@ fn with_alias_of<'a>( } } -/// `resolved` in the pin style of `original_version_literal`, behind its `npm:@` if any. +fn skip_ascii_digits(s: &[u8]) -> Option<&[u8]> { + let n = s.iter().take_while(|b| b.is_ascii_digit()).count(); + (n > 0).then(|| &s[n..]) +} + +/// A plain `bun update` only rewrites `^x…`, `~x…` and exact versions; every other range is kept as written. +fn keeps_declared_range(version_literal: &[u8]) -> bool { + let mut rest = strings::trim(version_literal, &strings::WHITESPACE_CHARS); + while let [ + b'=' | b'v' | b' ' | b'\t' | b'\n' | b'\r' | 0x0B | 0x0C, + tail @ .., + ] = rest + { + rest = tail; + } + if let [b'^' | b'~', tail @ ..] = rest { + return !strings::trim(tail, &strings::WHITESPACE_CHARS) + .first() + .is_some_and(u8::is_ascii_digit); + } + let Some(rest) = skip_ascii_digits(rest) + .and_then(|r| r.strip_prefix(b".")) + .and_then(skip_ascii_digits) + .and_then(|r| r.strip_prefix(b".")) + .and_then(skip_ascii_digits) + else { + return true; + }; + match rest { + [] => false, + [b'-' | b'+', tail @ ..] => strings::index_of_any(tail, b" \t|<>").is_some(), + _ => true, + } +} + +/// `resolved` in the pin style of the declared literal (None = the literal is kept as written). fn updated_version_literal( original_version_literal: &[u8], resolved: semver::Version, resolved_buf: &[u8], exact_versions: bool, -) -> Vec { + update_to_latest: bool, +) -> Option> { + if dependency::Tag::infer(original_version_literal) == dependency::Tag::DistTag { + return None; + } let mut v = Vec::new(); let version_literal = match split_npm_alias(original_version_literal) { Some((alias, version_literal)) => { @@ -93,6 +132,9 @@ fn updated_version_literal( } None => original_version_literal, }; + if !update_to_latest && keeps_declared_range(version_literal) { + return None; + } // `=1.0.0` round-trips as `=2.0.0`; `which_version_is_pinned` skips the `=` and reports Patch. let exact_prefix = @@ -112,7 +154,7 @@ fn updated_version_literal( }; write!(&mut v, "{}{}", range_prefix, resolved.fmt(resolved_buf)) .expect("infallible: in-memory write"); - v + Some(v) } /// Shallow-copy a `G::Property` for the JSON-editing path. Only `key`/`value` @@ -341,6 +383,52 @@ pub(crate) fn edit_update_no_args_in( update_to_latest: bool, current_package_json: &mut Expr, options: EditOptions, +) -> Result<(), bun_alloc::AllocError> { + edit_update_entries( + lockfile, + arena, + updating_packages, + workspace_name_hash, + update_to_latest, + current_package_json, + options, + &|_, _| true, + ) +} + +/// `bun update `: entries declared as `: npm:@…` move like every entry of a bare update does. +fn edit_update_aliases_of_requests( + manager: &mut PackageManager, + updates: &[UpdateRequest], + current_package_json: &mut Expr, + options: EditOptions, +) -> Result<(), bun_alloc::AllocError> { + let is_request = |name: &[u8]| updates.iter().any(|r| r.is_aliased && r.name == name); + let is_alias_of_request = |key: &[u8], literal: &[u8]| { + split_npm_alias(literal).is_some_and(|(alias, _)| is_request(&alias[b"npm:".len()..])) + && !is_request(key) + }; + edit_update_entries( + &manager.lockfile, + &manager.ast_arena, + &mut manager.updating_packages, + manager.workspace_name_hash, + manager.options.do_.contains(Do::UPDATE_TO_LATEST), + current_package_json, + options, + &is_alias_of_request, + ) +} + +fn edit_update_entries( + lockfile: &crate::Lockfile, + arena: &bun_alloc::Arena, + updating_packages: &mut StringArrayHashMap, + workspace_name_hash: Option, + update_to_latest: bool, + current_package_json: &mut Expr, + options: EditOptions, + selected: &dyn Fn(&[u8], &[u8]) -> bool, ) -> Result<(), bun_alloc::AllocError> { // using data store is going to result in undefined memory issues as // the store is cleared in some workspace situations. the solution @@ -376,14 +464,15 @@ pub(crate) fn edit_update_no_args_in( .unwrap_or_else(|| bun_core::out_of_memory()); let tag = dependency::Tag::infer(version_literal); - // npm versions only (and dist-tags with --latest); `catalog:` is handled by edit_catalogs_*. - if tag != dependency::Tag::Npm - && (tag != dependency::Tag::DistTag || !update_to_latest) - { + // npm ranges only: dist-tags stay as written even with --latest; `catalog:` is handled by edit_catalogs_*. + if tag != dependency::Tag::Npm { continue; } let key_str = key.as_utf8_string_literal().expect("unreachable"); + if !selected(key_str, version_literal) { + continue; + } // Capture the literal as an owned // copy before borrowing `updating_packages` mutably. let version_literal_owned = Box::<[u8]>::from(version_literal); @@ -454,6 +543,9 @@ pub(crate) fn edit_update_no_args_in( let key_str = key .as_utf8_string_literal() .unwrap_or_else(|| bun_core::out_of_memory()); + if !selected(key_str, value_literal) { + continue; + } 'updated: { // Only the first dependency group naming the package is rewritten. @@ -493,12 +585,15 @@ pub(crate) fn edit_update_no_args_in( } } - let new_version = updated_version_literal( + let Some(new_version) = updated_version_literal( &entry.original_version_literal, resolution.npm().version, string_buf, options.exact_versions, - ); + update_to_latest, + ) else { + break 'updated; + }; dep.value = Some(Expr::allocate( arena, E::EString::init(arena_str(arena, &new_version)), @@ -518,7 +613,7 @@ pub(crate) fn edit_update_no_args_in( /// Calls `f(catalog_name, entries_object)` for each catalog in the root /// package.json, matching the precedence `CatalogMap::parse_append` uses. -fn for_each_catalog_object( +pub(crate) fn for_each_catalog_object( root_package_json: &Expr, mut f: impl FnMut(&[u8], Expr) -> Result<(), bun_alloc::AllocError>, ) -> Result<(), bun_alloc::AllocError> { @@ -597,8 +692,7 @@ pub(crate) fn edit_catalogs_before_update( let tag = dependency::Tag::infer(version_literal); // same tag rule as direct dependencies - if tag != dependency::Tag::Npm && (tag != dependency::Tag::DistTag || !update_to_latest) - { + if tag != dependency::Tag::Npm { continue; } @@ -795,13 +889,15 @@ fn resolve_catalog_literals( } } - let new_literal = updated_version_literal( + if let Some(new_literal) = updated_version_literal( &infos[index].original_version_literal, resolution.npm().version, string_buf, exact_versions, - ); - infos[index].new_version_literal = Some(new_literal.into_boxed_slice()); + update_to_latest, + ) { + infos[index].new_version_literal = Some(new_literal.into_boxed_slice()); + } } } @@ -821,9 +917,6 @@ pub(crate) fn edit( // Process-lifetime arena for AST // nodes that must outlive `Expr.Data.Store.reset()`. See `PackageManager.ast_arena`. - // `arena` is a disjoint-field borrow held across the `&mut manager.updating_packages` accesses below. - let arena = &manager.ast_arena; - let update_to_latest = manager.subcommand == Subcommand::Update && manager.options.do_.contains(Do::UPDATE_TO_LATEST); if update_to_latest && options.before_install { @@ -841,6 +934,12 @@ pub(crate) fn edit( Global::crash(); } } + if manager.subcommand == Subcommand::Update { + edit_update_aliases_of_requests(manager, &**updates, current_package_json, options)?; + } + + // `arena` is a disjoint-field borrow held across the `&mut manager.updating_packages` accesses below. + let arena = &manager.ast_arena; let mut remaining = updates.len(); let mut replacing: usize = 0; @@ -1226,7 +1325,10 @@ pub(crate) fn edit( if let Some(existing) = existing { version_literal = with_alias_of(arena, existing, version_literal); } - if update_to_latest { + // a declared dist-tag (`next`, `npm:bar@next`) keeps resolving through that tag under --latest + let existing_is_dist_tag = + existing.is_some_and(|e| dependency::Tag::infer(e) == dependency::Tag::DistTag); + if update_to_latest && !existing_is_dist_tag { version_literal = with_alias_of(arena, version_literal, b"latest"); } e_string.data = arena_dup(arena, version_literal).into(); @@ -1239,7 +1341,7 @@ pub(crate) fn edit( let installed = request.version.literal.slice(request.version_buf()); let resolved = resolutions[request.package_id as usize].npm().version; let string_buf = manager.lockfile.buffers.string_bytes.as_slice(); - // `bun update ` keeps a dist-tag literal as written unless --latest, like the bare path. + // under --latest the row literal is `latest` for rewritable entries; a declared dist-tag is kept below via updated_version_literal. if manager.subcommand == Subcommand::Update && request.version.tag == dependency::Tag::DistTag && !update_to_latest @@ -1263,13 +1365,25 @@ pub(crate) fn edit( ), None => original, }; - let new_version = updated_version_literal( + match updated_version_literal( original, resolved, string_buf, options.exact_versions, - ); - break 'npm arena_str(arena, &new_version); + update_to_latest, + ) { + Some(new_version) => break 'npm arena_str(arena, &new_version), + // no explicit `@range`: the row still spells the declared literal, which stays as written + None => { + if strings::eql_long( + installed, + &entry.original_version_literal, + true, + ) { + break 'npm arena_dup(arena, installed); + } + } + } } } // `foo@npm:bar` (no version part) is saved like `foo` would be: `npm:bar@^`. diff --git a/src/install/PackageManager/PackageManagerEnqueue.rs b/src/install/PackageManager/PackageManagerEnqueue.rs index 45c3d5fe89ae..4ae968a637b8 100644 --- a/src/install/PackageManager/PackageManagerEnqueue.rs +++ b/src/install/PackageManager/PackageManagerEnqueue.rs @@ -2039,14 +2039,20 @@ fn get_or_put_resolved_package_with_find_result( // borrows `this.lockfile` and `this` at once. Split via raw root. let should_update = this.to_update && if !this.update_requests.is_empty() { - // `bun update `: every `` slot, else other resolutions stay pinned. - UpdateRequest::contains_name( + // bun update : every in-scope row (declared or `npm:@…` aliased, see update_scope); other resolutions stay pinned. + let string_buf = this.lockfile.buffers.string_bytes.as_slice(); + (UpdateRequest::contains_name( &this.update_requests, dependency.name_hash, - dependency - .name - .slice(this.lockfile.buffers.string_bytes.as_slice()), - ) + dependency.name.slice(string_buf), + ) || (name_hash != dependency.name_hash + && UpdateRequest::contains_name( + &this.update_requests, + name_hash, + name.slice(string_buf), + ))) + && crate::update_scope::UpdateScope::of(&*this) + .contains_dependency(&this.lockfile, dependency_id) } else if let Some(targets) = this.update_target_workspaces.as_deref() { // `bun update -r`/`--filter`: direct deps of the selected workspaces; catalogs are root-scoped. dependency.version.tag == dependency::version::Tag::Catalog @@ -2435,10 +2441,7 @@ fn get_or_put_resolved_package( // `bun update -r/--filter --latest`: resolve targeted workspaces' npm deps by dist-tag `latest`. let latest_for_target = !version_was_replaced - && matches!( - version.tag, - dependency::version::Tag::Npm | dependency::version::Tag::DistTag - ) + && version.tag == dependency::version::Tag::Npm && this.to_update && this.update_requests.is_empty() && this diff --git a/src/install/PackageManager/add_catalog.rs b/src/install/PackageManager/add_catalog.rs index 671d1bcf9cf7..ea1549fbcddb 100644 --- a/src/install/PackageManager/add_catalog.rs +++ b/src/install/PackageManager/add_catalog.rs @@ -4,24 +4,61 @@ use bstr::BStr; use bun_alloc::AllocError; use bun_ast::{E, Expr, ExprData, Loc, StoreStr}; use bun_collections::VecExt as _; -use bun_core::{Global, Output}; +use bun_core::{Global, Output, strings}; +use bun_semver::{self as Semver, SlicedString}; -use bun_install::dependency; +use bun_install::dependency::{self, TagExt as _}; use bun_install::lockfile::CatalogMap; use bun_install::lockfile::package::PackageColumns as _; use bun_install::{INVALID_PACKAGE_ID, Lockfile, PackageID, PackageNameHash, resolution}; +use bun_install_types::DependencyGroup; +use super::add_remove_with_filter::{ + WorkspaceTarget, fetch_entry_root, load_workspace_members, local_relative_path, + root_package_json_path, +}; +use super::package_json_editor::{self as PackageJSONEditor, EditOptions}; use super::update_package_json_and_install::print_package_json_into_cache_entry; use super::workspace_package_json_cache::MapEntry; -use super::{PackageManager, UpdateRequest}; +use super::{PackageManager, Subcommand, UpdateRequest}; type ExprDisabler = bun_ast::expr::Disabler; -fn catalog_name(manager: &PackageManager) -> &'static [u8] { - manager - .options - .add_catalog - .expect("add_catalog callers are gated on is_some") +#[derive(Default)] +pub(crate) struct State { + enabled: bool, + auto_use: Vec, + adds: Vec, +} + +struct Add { + name: &'static [u8], + name_hash: PackageNameHash, + group: Box<[u8]>, + candidate: Candidate, + outcome: Outcome, +} + +enum Candidate { + Explicit(Box<[u8]>), + Existing(Box<[u8]>), + Latest, +} + +impl Candidate { + fn literal(&self) -> &[u8] { + match self { + Candidate::Explicit(literal) | Candidate::Existing(literal) => &literal[..], + Candidate::Latest => b"latest".as_slice(), + } + } +} + +enum Outcome { + Pending, + Reused, + Seeded, + Moved { entry: Box<[u8]> }, } fn estring(arena: &bun_alloc::Arena, bytes: &[u8]) -> Expr { @@ -42,13 +79,6 @@ fn reference_literal<'a>(arena: &'a bun_alloc::Arena, name: &[u8]) -> &'a [u8] { arena.alloc_slice_copy(&literal) } -fn seed_literal(request: &UpdateRequest) -> &[u8] { - if request.version.tag == dependency::Tag::Uninitialized { - return b"latest"; - } - request.version.literal.slice(request.version_buf()) -} - fn set_property(mut object: Expr, arena: &bun_alloc::Arena, key: &[u8], value: Expr) { let obj = object .data @@ -81,6 +111,17 @@ fn object_property( Some(created) } +fn has_catalogs(expr: &Expr) -> bool { + expr.get(b"catalog").is_some() || expr.get(b"catalogs").is_some() +} + +fn root_defines_catalogs(root: &Expr) -> bool { + let Some(workspaces) = root.get(b"workspaces") else { + return false; + }; + has_catalogs(&workspaces) || has_catalogs(root) +} + fn entries_object(root: &Expr, name: &[u8], create: Option<&bun_alloc::Arena>) -> Option { let Some(workspaces) = root.get(b"workspaces") else { if create.is_some() { @@ -93,8 +134,6 @@ fn entries_object(root: &Expr, name: &[u8], create: Option<&bun_alloc::Arena>) - return None; }; - let has_catalogs = - |expr: &Expr| expr.get(b"catalog").is_some() || expr.get(b"catalogs").is_some(); let workspaces_is_object = matches!(workspaces.data, ExprData::EObject(_)); let container = if workspaces_is_object && has_catalogs(&workspaces) { workspaces @@ -124,71 +163,298 @@ fn entries_object(root: &Expr, name: &[u8], create: Option<&bun_alloc::Arena>) - existing.or_else(|| object_property(container, b"catalog", create)) } -/// Call right after `PackageJSONEditor::edit` on the same AST: rewrites only the slot `edit` bound per request. -pub(crate) fn rewrite_references(manager: &PackageManager, updates: &[UpdateRequest]) { - if updates.is_empty() { +fn request_literal(request: &UpdateRequest) -> &[u8] { + request.version.literal.slice(request.version_buf()) +} + +pub(crate) fn prepare(manager: &mut PackageManager, updates: &[UpdateRequest]) { + if manager.subcommand != Subcommand::Add || manager.options.global || updates.is_empty() { return; } + debug_assert!( + !manager.catalog_add.enabled + && manager.catalog_add.auto_use.is_empty() + && manager.catalog_add.adds.is_empty() + ); - for request in updates { - if !request.is_aliased { - Output::err_generic( - "--catalog can only add packages by name, but got \"{s}\"", - (BStr::new( - request.version.literal.slice(request.version_buf()), - ),), - ); - Global::crash(); + if manager.options.add_catalog.is_some() { + for request in updates { + if !request.is_aliased { + Output::err_generic( + "--catalog can only add packages by name, but got \"{s}\"", + (BStr::new(request_literal(request)),), + ); + Global::crash(); + } + if request.version.tag == dependency::Tag::Workspace { + Output::err_generic( + "--catalog cannot add a workspace package, but got \"{s}@{s}\"", + (BStr::new(request.name), BStr::new(request_literal(request))), + ); + Global::crash(); + } + if local_relative_path(request).is_some() { + Output::err_generic( + "--catalog cannot add \"{s}@{s}\": a local path in the catalog would resolve from the workspace root, not from the package that added it", + (BStr::new(request.name), BStr::new(request_literal(request))), + ); + Global::crash(); + } } - if request.version.tag == dependency::Tag::Workspace { - Output::err_generic( - "--catalog cannot add a workspace package, but got \"{s}@{s}\"", - ( - BStr::new(request.name), - BStr::new(request.version.literal.slice(request.version_buf())), - ), - ); - Global::crash(); + let ws = load_workspace_members(manager); + for request in updates { + if *ws.root_name == *request.name { + Output::err_generic( + "--catalog cannot add a workspace package, but \"{s}\" is the workspace root", + (BStr::new(request.name),), + ); + Global::crash(); + } + if let Some((rel, _)) = ws + .members + .keys() + .iter() + .zip(ws.members.values()) + .find(|(_, entry)| *entry.name == *request.name) + { + Output::err_generic( + "--catalog cannot add a workspace package, but \"{s}\" is the workspace at {s}", + (BStr::new(request.name), BStr::new(rel)), + ); + Global::crash(); + } } + manager.catalog_add.enabled = true; + return; } - let literal = StoreStr::new(reference_literal(&manager.ast_arena, catalog_name(manager))); - for request in updates { + let root = fetch_entry_root( + manager, + &WorkspaceTarget { + name: Box::default(), + name_hash: None, + package_json_path: root_package_json_path(), + }, + ); + if !root_defines_catalogs(&root) { + return; + } + manager.catalog_add.enabled = true; + let Some(entries) = entries_object(&root, b"", None) else { + return; + }; + for request in updates.iter().filter(|request| request.is_aliased) { + let Some(q) = entries.as_property(request.name) else { + continue; + }; + let Some(entry_text) = q.expr.as_utf8_string_literal() else { + continue; + }; + if request.version.tag == dependency::Tag::Uninitialized + || request_literal(request) == entry_text + { + manager.catalog_add.auto_use.push(request.name_hash); + } + } +} + +fn existing_slot(package_json: &Expr, name: &[u8]) -> Option { + for group in DependencyGroup::FOUR { + let Some(list) = package_json.get(group.prop) else { + continue; + }; + if !matches!(list.data, ExprData::EObject(_)) { + continue; + } + let Some(q) = list.as_property(name) else { + continue; + }; + return q.expr.data.e_string().map(|s| s.data); + } + None +} + +fn catalog_group_of(reference: &[u8]) -> &[u8] { + strings::trim(&reference[b"catalog:".len()..], &strings::WHITESPACE_CHARS) +} + +fn record_add(state: &mut State, request: &UpdateRequest, group: &[u8], existing: Option<&[u8]>) { + let candidate = if request.version.tag != dependency::Tag::Uninitialized { + Candidate::Explicit(request_literal(request).into()) + } else { + match existing.map(|literal| (dependency::Tag::infer(literal), literal)) { + Some((dependency::Tag::Npm | dependency::Tag::DistTag, literal)) => { + Candidate::Existing(literal.into()) + } + _ => Candidate::Latest, + } + }; + match state + .adds + .iter_mut() + .find(|add| add.name_hash == request.name_hash && CatalogMap::same_name(&add.group, group)) + { + None => state.adds.push(Add { + name: request.name, + name_hash: request.name_hash, + group: group.into(), + candidate, + outcome: Outcome::Pending, + }), + Some(add) => { + if matches!(add.candidate, Candidate::Latest) + && matches!(candidate, Candidate::Existing(_)) + { + add.candidate = candidate; + } + } + } +} + +pub(crate) fn edit_target( + manager: &mut PackageManager, + updates: &mut &mut [UpdateRequest], + package_json: &mut Expr, + dependency_list: &[u8], + options: EditOptions, +) -> Result<(), AllocError> { + if !manager.catalog_add.enabled { + return PackageJSONEditor::edit(manager, updates, package_json, dependency_list, options); + } + + let captured: Vec<(PackageNameHash, StoreStr)> = updates + .iter() + .filter(|request| request.is_aliased) + .filter_map(|request| { + existing_slot(package_json, request.name).map(|slot| (request.name_hash, slot)) + }) + .collect(); + + PackageJSONEditor::edit(manager, updates, package_json, dependency_list, options)?; + + let flag = manager.options.add_catalog; + let arena = &manager.ast_arena; + let state = &mut manager.catalog_add; + let flag_literal = flag.map(|name| StoreStr::new(reference_literal(arena, name))); + + for request in updates.iter() { + if !request.is_aliased { + continue; + } let Some(e_string) = request.e_string else { continue; }; + let existing = captured + .iter() + .find(|&&(name_hash, _)| name_hash == request.name_hash) + .map(|&(_, slot)| slot); + let existing_ref = existing + .filter(|slot| dependency::Tag::infer(slot.slice()) == dependency::Tag::Catalog); + + let literal = if !options.before_install { + let Some(reference) = existing_ref else { + continue; + }; + reference + } else { + match (existing_ref, flag) { + (Some(reference), Some(_)) => { + record_add(state, request, catalog_group_of(reference.slice()), None); + reference + } + (Some(reference), None) + if request.version.tag == dependency::Tag::Uninitialized => + { + reference + } + (None, Some(name)) => { + record_add(state, request, name, existing.map(|slot| slot.slice())); + flag_literal.expect("infallible: flag is Some") + } + (None, None) if state.auto_use.contains(&request.name_hash) => { + StoreStr::new(b"catalog:") + } + _ => continue, + } + }; // SAFETY: same slot `edit` just wrote through (PackageJSONEditor.rs `request.e_string` loop); the tree it points into is still live and no other borrow of it exists here. unsafe { (*e_string).data = literal }; } + Ok(()) +} + +fn exact_within(wanted: &[u8], entry: &[u8]) -> bool { + let Some(version) = Semver::query::parse(wanted, SlicedString::init(wanted, wanted)) + .ok() + .and_then(|group| group.get_exact_version()) + else { + return false; + }; + match Semver::query::parse(entry, SlicedString::init(entry, entry)) { + Ok(group) => !group.is_empty() && group.satisfies(version, entry, wanted), + Err(_) => false, + } } pub(crate) fn edit_root_before_install( - manager: &PackageManager, + manager: &mut PackageManager, root_package_json: &Expr, - updates: &[UpdateRequest], ) -> Result<(), AllocError> { - if updates.is_empty() { - return Ok(()); - } let _guard = ExprDisabler::scope(); - let arena = &manager.ast_arena; - let mut entries = entries_object(root_package_json, catalog_name(manager), Some(arena)) - .expect("infallible: created on demand"); - for request in updates { - set_property( - entries, - arena, - request.name, - estring(arena, seed_literal(request)), - ); + let adds = &mut manager.catalog_add.adds; + + let mut appended: Vec = Vec::new(); + for (i, add) in adds.iter_mut().enumerate() { + if !matches!(add.outcome, Outcome::Pending) { + continue; + } + let entries = entries_object(root_package_json, &add.group, Some(arena)) + .expect("infallible: created on demand"); + let existing: Option> = entries + .as_property(add.name) + .and_then(|q| q.expr.as_utf8_string_literal().map(Box::from)); + add.outcome = match (existing, &add.candidate) { + (Some(entry), Candidate::Explicit(wanted)) if *entry == **wanted => Outcome::Reused, + (Some(entry), Candidate::Explicit(wanted)) => { + set_property(entries, arena, add.name, estring(arena, wanted)); + if exact_within(wanted, &entry) { + Outcome::Moved { entry } + } else { + Outcome::Seeded + } + } + (Some(_), Candidate::Existing(_) | Candidate::Latest) => Outcome::Reused, + (None, candidate) => { + set_property( + entries, + arena, + add.name, + estring(arena, candidate.literal()), + ); + appended.push(i); + Outcome::Seeded + } + }; } - let obj = entries - .data - .e_object_mut() - .expect("infallible: entries_object returns objects"); - if obj.properties.len_u32() > 1 { - obj.alphabetize_properties(); + + for (n, &i) in appended.iter().enumerate() { + let group = &adds[i].group; + if appended[..n] + .iter() + .any(|&j| CatalogMap::same_name(&adds[j].group, group)) + { + continue; + } + let mut entries = entries_object(root_package_json, group, None) + .expect("infallible: created by the loop above"); + let obj = entries + .data + .e_object_mut() + .expect("infallible: entries_object returns objects"); + if obj.properties.len_u32() > 1 { + obj.alphabetize_properties(); + } } Ok(()) } @@ -197,11 +463,11 @@ pub(crate) fn edit_root_entry_before_install( manager: &mut PackageManager, root_package_json: &mut MapEntry, ) -> Result<(), crate::Error> { - if manager.update_requests.is_empty() { + if manager.catalog_add.adds.is_empty() { return Ok(()); } let root = root_package_json.root; - edit_root_before_install(&*manager, &root, &manager.update_requests)?; + edit_root_before_install(manager, &root)?; print_package_json_into_cache_entry(root_package_json, root); if let Err(err) = root_package_json.reparse_root(manager.log_mut()) { bun_core::pretty_errorln!("package.json failed to parse due to error {}", err.name()); @@ -210,20 +476,16 @@ pub(crate) fn edit_root_entry_before_install( Ok(()) } -/// Runs after `clean_with_logger`: replaces the dist-tag seeds in the root's catalog entries with the resolved range; the lockfile catalog is re-derived from the file by `package_json_write_back`. +/// Runs after `clean_with_logger`: puts moved entries back and replaces dist-tag seeds with the resolved range; the lockfile catalog is re-derived from the file by `package_json_write_back`. pub(crate) fn edit_root_after_install( manager: &PackageManager, root_package_json: &Expr, - updates: &[UpdateRequest], ) -> Result { - if updates.is_empty() { + let adds = &manager.catalog_add.adds; + if adds.is_empty() { return Ok(false); } let _guard = ExprDisabler::scope(); - let name = catalog_name(manager); - let Some(mut entries) = entries_object(root_package_json, name, None) else { - return Ok(false); - }; let arena = &manager.ast_arena; let exact = manager.options.enable.exact_versions(); @@ -231,73 +493,87 @@ pub(crate) fn edit_root_after_install( let buf = lockfile.buffers.string_bytes.as_slice(); let resolutions = lockfile.packages.items_resolution(); - let mut resolved: Vec<(PackageNameHash, PackageID)> = Vec::with_capacity(updates.len()); - for (dep, &pkg_id) in lockfile - .buffers - .dependencies + let mut resolved: Vec = vec![INVALID_PACKAGE_ID; adds.len()]; + let mut missing = adds .iter() - .zip(lockfile.buffers.resolutions.iter()) - { - if dep.version.tag != dependency::Tag::Catalog - || pkg_id == INVALID_PACKAGE_ID - || (pkg_id as usize) >= resolutions.len() - || resolutions[pkg_id as usize].tag != resolution::Tag::Npm - || !updates - .iter() - .any(|request| request.name_hash == dep.name_hash) - || resolved - .iter() - .any(|&(name_hash, _)| name_hash == dep.name_hash) - || !CatalogMap::same_name(dep.version.catalog().slice(buf), name) + .filter(|add| matches!(add.outcome, Outcome::Seeded)) + .count(); + if missing != 0 { + for (dep, &pkg_id) in lockfile + .buffers + .dependencies + .iter() + .zip(lockfile.buffers.resolutions.iter()) { - continue; - } - resolved.push((dep.name_hash, pkg_id)); - if resolved.len() == updates.len() { - break; + if dep.version.tag != dependency::Tag::Catalog + || pkg_id == INVALID_PACKAGE_ID + || (pkg_id as usize) >= resolutions.len() + || resolutions[pkg_id as usize].tag != resolution::Tag::Npm + { + continue; + } + let Some(i) = adds.iter().position(|add| { + matches!(add.outcome, Outcome::Seeded) + && add.name_hash == dep.name_hash + && CatalogMap::same_name(dep.version.catalog().slice(buf), &add.group) + }) else { + continue; + }; + if resolved[i] != INVALID_PACKAGE_ID { + continue; + } + resolved[i] = pkg_id; + missing -= 1; + if missing == 0 { + break; + } } } - if resolved.is_empty() { - return Ok(false); - } let mut changed = false; - for request in updates { - let Some(&(_, pkg_id)) = resolved - .iter() - .find(|&&(name_hash, _)| name_hash == request.name_hash) - else { - continue; + for (add, &pkg_id) in adds.iter().zip(resolved.iter()) { + let literal: Vec = match &add.outcome { + Outcome::Pending | Outcome::Reused => continue, + Outcome::Moved { entry } => entry.to_vec(), + Outcome::Seeded => { + if pkg_id == INVALID_PACKAGE_ID { + continue; + } + // The seed is read back from the lockfile catalog: the requests were rebound to the `catalog:` rows by `bind_update_requests`. + let Some(seed) = lockfile.catalogs.find(buf, &add.group, add.name) else { + continue; + }; + if seed.version.tag != dependency::Tag::DistTag { + continue; + } + let mut literal = Vec::new(); + if seed.version.literal.slice(buf).starts_with(b"npm:") { + write!( + &mut literal, + "npm:{}@", + BStr::new(seed.version.dist_tag().name.slice(buf)) + ) + .expect("infallible: in-memory write"); + } + write!( + &mut literal, + "{}{}", + if exact { "" } else { "^" }, + resolutions[pkg_id as usize].npm().version.fmt(buf) + ) + .expect("infallible: in-memory write"); + literal + } }; - // `request.version` was rebound to the `catalog:` row by `bind_update_requests`, so the dist-tag seed is read back from the lockfile catalog. - let Some(seed) = lockfile.catalogs.find(buf, name, request.name) else { + + let Some(mut entries) = entries_object(root_package_json, &add.group, None) else { continue; }; - if seed.version.tag != dependency::Tag::DistTag { - continue; - } - let mut literal = Vec::new(); - if seed.version.literal.slice(buf).starts_with(b"npm:") { - write!( - &mut literal, - "npm:{}@", - BStr::new(seed.version.dist_tag().name.slice(buf)) - ) - .expect("infallible: in-memory write"); - } - write!( - &mut literal, - "{}{}", - if exact { "" } else { "^" }, - resolutions[pkg_id as usize].npm().version.fmt(buf) - ) - .expect("infallible: in-memory write"); - let obj = entries .data .e_object_mut() .expect("infallible: entries_object returns objects"); - let Some(q) = obj.as_property(request.name) else { + let Some(q) = obj.as_property(add.name) else { continue; }; if q.expr.as_utf8_string_literal() == Some(&literal[..]) { diff --git a/src/install/PackageManager/add_remove_with_filter.rs b/src/install/PackageManager/add_remove_with_filter.rs index 73f4ec958dea..ddce7e020dee 100644 --- a/src/install/PackageManager/add_remove_with_filter.rs +++ b/src/install/PackageManager/add_remove_with_filter.rs @@ -4,6 +4,7 @@ use crate::Error; use crate::bun_fs::FileSystem; use crate::lockfile_real::package::value_loc_of; use crate::lockfile_real::package::workspace_map::{MissingWorkspace, NamesArray, WorkspaceMap}; +use bun_collections::StringArrayHashMap; use bun_core::{Global, Output, strings}; use bun_install::dependency; use bun_install::{Lockfile, PackageID, PackageNameHash}; @@ -14,14 +15,17 @@ use bun_sys::{Fd, File}; use super::add_catalog; use super::install_with_manager::install_with_manager; use super::options::Do; -use super::package_json_editor::{self as PackageJSONEditor, EditOptions}; +use super::package_json_editor::EditOptions; use super::package_json_write_back; use super::update_package_json_and_install::{ print_package_json_into_cache_entry, remove_dependencies_from_package_json, remove_leftover_node_modules, }; use super::workspace_package_json_cache::{GetJSONOptions, GetResult, MapEntry}; -use super::{Command, PackageManager, Subcommand, UpdateRequest, WorkspaceFilter}; +use super::workspace_selection::{self, Candidate, RootSelection, WorkspaceGraph}; +use super::{ + Command, PackageManager, PackageUpdateInfo, Subcommand, UpdateRequest, UpdateTargetWorkspace, +}; #[derive(Clone)] pub(crate) struct WorkspaceTarget { @@ -51,12 +55,13 @@ fn print_log_and_crash( Global::crash(); } -pub(crate) fn select_targets( - manager: &mut PackageManager, - original_cwd: &[u8], -) -> Result, Error> { - debug_assert!(!manager.options.filter_patterns.is_empty()); - let top_level = strings::without_trailing_slash(FileSystem::instance().top_level_dir()); +pub(crate) struct WorkspaceMembers { + pub(crate) root_path: Box<[u8]>, + pub(crate) root_name: Box<[u8]>, + pub(crate) members: WorkspaceMap, +} + +pub(crate) fn load_workspace_members(manager: &mut PackageManager) -> WorkspaceMembers { let root_path = root_package_json_path(); let (root_expr, root_source, root_name): (bun_ast::Expr, bun_ast::Source, Box<[u8]>) = { @@ -135,18 +140,26 @@ pub(crate) fn select_targets( } } + WorkspaceMembers { + root_path, + root_name, + members, + } +} + +pub(crate) fn select_targets( + manager: &mut PackageManager, + original_cwd: &[u8], +) -> Result, Error> { + let top_level = strings::without_trailing_slash(FileSystem::instance().top_level_dir()); + let WorkspaceMembers { + root_path, + root_name, + members, + } = load_workspace_members(manager); + let mut path_buf = path_buffer_pool::get(); let patterns = manager.options.filter_patterns; - let filters: Vec = patterns - .iter() - .map(|pattern| { - bun_core::handle_oom(WorkspaceFilter::init( - pattern, - original_cwd, - &mut path_buf.0, - )) - }) - .collect(); let root_subject: Box<[u8]> = strings::without_trailing_slash(join_abs_string_buf::( @@ -190,30 +203,35 @@ pub(crate) fn select_targets( )); } - let unmatched: Vec<&[u8]> = filters + let root_rule = if matches!(manager.subcommand, Subcommand::Add | Subcommand::Remove) { + RootSelection::ExplicitOnly + } else { + RootSelection::Implicit + }; + let graph: Option = workspace_selection::first_relational(patterns) + .map(|pattern| load_workspace_graph(manager, &candidates, pattern)); + let selection = { + let subjects: Vec> = candidates + .iter() + .map(|(target, subject)| Candidate { + name: &target.name, + abs_posix_dir: subject, + is_root: target.name_hash.is_none(), + }) + .collect(); + workspace_selection::select(patterns, original_cwd, &subjects, graph.as_ref(), root_rule) + }; + let unmatched: Vec<&[u8]> = selection + .unmatched_patterns .iter() - .zip(patterns) - .filter(|(filter, _)| { - let negated = match filter { - WorkspaceFilter::All => return false, - WorkspaceFilter::Name(p) | WorkspaceFilter::Path(p) => p.first() == Some(&b'!'), - }; - !negated - && !candidates.iter().any(|(target, subject)| { - WorkspaceFilter::matches_any( - core::slice::from_ref(filter), - &target.name, - subject, - ) - }) - }) - .map(|(_, pattern)| *pattern) + .map(|&i| patterns[i]) .collect(); let targets: Vec = candidates .into_iter() - .filter(|(target, subject)| WorkspaceFilter::matches_any(&filters, &target.name, subject)) - .map(|(target, _)| target) + .zip(selection.selected) + .filter(|(_, selected)| *selected) + .map(|((target, _), _)| target) .collect(); if targets.is_empty() { @@ -233,6 +251,43 @@ pub(crate) fn select_targets( Ok(targets) } +fn load_workspace_graph( + manager: &mut PackageManager, + candidates: &[(WorkspaceTarget, Box<[u8]>)], + pattern: &[u8], +) -> WorkspaceGraph { + use crate::lockfile::LoadResult; + let outcome: Result<(), Option> = if !manager.options.do_.load_lockfile() { + Err(None) + } else { + match manager.load_lockfile_from_cwd::() { + LoadResult::Ok(_) => Ok(()), + LoadResult::NotFound => Err(None), + LoadResult::Err(cause) => Err(Some(cause.value)), + } + }; + match outcome { + Ok(()) => {} + Err(None) => { + Output::err_generic( + "--filter \"{}\" selects workspaces by their dependency graph, which needs a bun.lock; run bun install first", + (BStr::new(pattern),), + ); + Global::crash(); + } + Err(Some(err)) => { + Output::err_generic( + "failed to load the lockfile needed by --filter \"{}\": {}", + (BStr::new(pattern), err.name()), + ); + Global::crash(); + } + } + let hashes: Vec> = + candidates.iter().map(|(t, _)| t.name_hash).collect(); + WorkspaceGraph::from_lockfile(&manager.lockfile, &hashes) +} + fn quote_patterns(patterns: &[&[u8]]) -> Vec { let mut out = Vec::new(); for (i, pattern) in patterns.iter().enumerate() { @@ -324,7 +379,7 @@ fn move_to_front(updates: &mut [UpdateRequest], wanted: &[PackageNameHash]) -> u } /// The `(prefix, path)` of a positional naming a local path, which is relative to the invoking cwd. -fn local_relative_path(request: &UpdateRequest) -> Option<(&'static [u8], &[u8])> { +pub(crate) fn local_relative_path(request: &UpdateRequest) -> Option<(&'static [u8], &[u8])> { let literal = request.version.literal.slice(request.version_buf()); let (prefix, path): (&'static [u8], &[u8]) = match request.version.tag { dependency::Tag::Folder | dependency::Tag::Tarball => { @@ -444,9 +499,15 @@ fn assign_requests( (requests, assigned) } -/// The `add`/`link --filter` targets and the requests each one received; edited again once resolved. +struct PendingTarget { + target: WorkspaceTarget, + received: Box<[PackageNameHash]>, + updating: StringArrayHashMap, +} + +/// The add/update --filter targets and the requests each one received; edited again once resolved. pub(crate) struct PendingWrite { - targets: Vec<(WorkspaceTarget, Box<[PackageNameHash]>)>, + targets: Vec, catalog_mode: bool, root_target: WorkspaceTarget, } @@ -460,17 +521,17 @@ impl PendingWrite { ) -> Vec { self.targets .iter() - .filter(|(_, received)| received.contains(&request)) - .filter_map(|(target, _)| { - let id = lockfile.get_workspace_package_id(target.name_hash); - (target.name_hash.is_none() || id != 0).then_some(id) + .filter(|pending| pending.received.contains(&request)) + .filter_map(|pending| { + let id = lockfile.get_workspace_package_id(pending.target.name_hash); + (pending.target.name_hash.is_none() || id != 0).then_some(id) }) .collect() } /// Writes the resolved versions into every target's cache entry; `flush` puts them on disk. pub(crate) fn edit_entries( - &self, + &mut self, manager: &mut PackageManager, updates: &mut [UpdateRequest], ) -> Result<(), Error> { @@ -478,12 +539,20 @@ impl PendingWrite { let exact_versions = manager.options.enable.exact_versions(); let summary_order: Vec = updates.iter().map(|r| r.name_hash).collect(); - for (target, received) in &self.targets { - let kept = move_to_front(updates, received); - let mut root = fetch_entry_root(manager, target); - reset_e_strings(updates); + for pending in &mut self.targets { + let kept = move_to_front(updates, &pending.received); + manager.lockfile.bind_update_requests( + None, + pending.target.name_hash, + &mut updates[..kept], + ); + let outer = core::mem::replace( + &mut manager.updating_packages, + core::mem::take(&mut pending.updating), + ); + let mut root = fetch_entry_root(manager, &pending.target); let mut slice: &mut [UpdateRequest] = &mut updates[..kept]; - PackageJSONEditor::edit( + let result = add_catalog::edit_target( manager, &mut slice, &mut root, @@ -492,17 +561,16 @@ impl PendingWrite { exact_versions, ..Default::default() }, - )?; - if self.catalog_mode { - add_catalog::rewrite_references(manager, &updates[..kept]); - } - store_entry(manager, target, root); + ); + pending.updating = core::mem::replace(&mut manager.updating_packages, outer); + result?; + store_entry(manager, &pending.target, root); } updates.sort_by_key(|r| summary_order.iter().position(|&h| h == r.name_hash)); if self.catalog_mode { let root = fetch_entry_root(manager, &self.root_target); - if add_catalog::edit_root_after_install(manager, &root, updates)? { + if add_catalog::edit_root_after_install(manager, &root)? { store_entry(manager, &self.root_target, root); } } @@ -510,6 +578,7 @@ impl PendingWrite { } } +/// bun add/remove --filter and bun update -r/--filter: edits every selected package.json, then runs one install. pub(super) fn update_filtered_workspaces_and_install( manager: &mut PackageManager, ctx: Command::Context, @@ -517,12 +586,29 @@ pub(super) fn update_filtered_workspaces_and_install( updates: Vec, ) -> Result<(), Error> { if manager.options.global { - Output::err_generic("--filter cannot be used with --global", ()); + let flag = if manager.options.filter_patterns.is_empty() { + "--recursive" + } else { + "--filter" + }; + Output::err_generic("{} cannot be used with --global", (flag,)); Global::crash(); } let targets = select_targets(manager, original_cwd)?; + add_catalog::prepare(manager, &updates); let subcommand = manager.subcommand; + let is_update = subcommand == Subcommand::Update; + let update_targets: Option> = is_update.then(|| { + targets + .iter() + .map(|t| UpdateTargetWorkspace { + is_root: t.name_hash.is_none(), + name_hash: t.name_hash.unwrap_or(0), + name: t.name.clone(), + }) + .collect() + }); let dependency_list: &'static [u8] = manager.options.update.prop; let exact_versions = manager.options.enable.exact_versions(); let catalog_mode = subcommand == Subcommand::Add && manager.options.add_catalog.is_some(); @@ -541,20 +627,20 @@ pub(super) fn update_filtered_workspaces_and_install( assign_requests(manager, original_cwd, updates, &targets) }; - let mut changed: Vec<(WorkspaceTarget, Box<[PackageNameHash]>)> = - Vec::with_capacity(targets.len()); + let mut changed: Vec = Vec::with_capacity(targets.len()); for (target, wanted) in targets.into_iter().zip(assigned) { let mut root = fetch_entry_root(manager, &target); - let received: Box<[PackageNameHash]> = if subcommand == Subcommand::Remove { + let (received, updating) = if subcommand == Subcommand::Remove { if !remove_dependencies_from_package_json(&mut root, &updates) { continue; } - Box::default() + (Box::default(), StringArrayHashMap::default()) } else { let wanted_len = move_to_front(&mut updates, &wanted); reset_e_strings(&mut updates); + let outer = core::mem::take(&mut manager.updating_packages); let mut slice: &mut [UpdateRequest] = &mut updates[..wanted_len]; - PackageJSONEditor::edit( + let result = add_catalog::edit_target( manager, &mut slice, &mut root, @@ -564,34 +650,55 @@ pub(super) fn update_filtered_workspaces_and_install( before_install: true, ..Default::default() }, - )?; + ); let kept = slice.len(); - if kept == 0 { - continue; + let mine = core::mem::replace(&mut manager.updating_packages, outer); + result?; + for (name, info) in mine.iter() { + let entry = manager.updating_packages.get_or_put(name)?; + if !entry.found_existing { + *entry.value_ptr = PackageUpdateInfo { + original_version_literal: info.original_version_literal.clone(), + ..Default::default() + }; + } } - if catalog_mode { - add_catalog::rewrite_references(manager, &updates[..kept]); + let received: Box<[PackageNameHash]> = updates[..kept] + .iter() + .filter(|r| r.e_string.is_some()) + .map(|r| r.name_hash) + .collect(); + if received.is_empty() { + continue; } - updates[..kept].iter().map(|r| r.name_hash).collect() + (received, mine) }; store_entry(manager, &target, root); - changed.push((target, received)); + changed.push(PendingTarget { + target, + received, + updating, + }); } - for (target, _) in &changed { - package_json_write_back::record(manager, target.clone(), subcommand != Subcommand::Remove); + for pending in &changed { + package_json_write_back::record( + manager, + pending.target.clone(), + subcommand != Subcommand::Remove, + ); } let any_changed = !changed.is_empty(); - if subcommand != Subcommand::Remove { + if subcommand == Subcommand::Add { updates.retain(|r| { changed .iter() - .any(|(_, received)| received.contains(&r.name_hash)) + .any(|pending| pending.received.contains(&r.name_hash)) }); } if catalog_mode && !updates.is_empty() { let root = fetch_entry_root(manager, &root_target); - add_catalog::edit_root_before_install(manager, &root, &updates)?; + add_catalog::edit_root_before_install(manager, &root)?; store_entry(manager, &root_target, root); package_json_write_back::record(manager, root_target.clone(), false); } @@ -599,11 +706,14 @@ pub(super) fn update_filtered_workspaces_and_install( // The install summary is printed from this workspace's point of view. let summary_target = changed .iter() - .find(|(_, received)| received.len() == updates.len()) + .find(|pending| pending.received.len() == updates.len()) .or_else(|| changed.first()); - manager.workspace_name_hash = summary_target.and_then(|(target, _)| target.name_hash); - manager.to_update = false; + manager.workspace_name_hash = summary_target.and_then(|pending| pending.target.name_hash); + manager.to_update = is_update; manager.update_requests = updates.into_boxed_slice(); + if let Some(update_targets) = update_targets { + manager.update_target_workspaces = Some(update_targets); + } if subcommand != Subcommand::Remove { manager.pending_filtered_write = Some(Box::new(PendingWrite { targets: changed, diff --git a/src/install/PackageManager/install_with_manager.rs b/src/install/PackageManager/install_with_manager.rs index ecf4319f319a..25812d9066a0 100644 --- a/src/install/PackageManager/install_with_manager.rs +++ b/src/install/PackageManager/install_with_manager.rs @@ -3,9 +3,7 @@ use core::sync::atomic::Ordering; use bun_core::time::nano_timestamp; use bun_core::{Global, Output}; -use crate::bun_fs::FileSystem; use bun_core::{ZStr, strings}; -use bun_glob as glob; use bun_semver::String as SemverString; use crate::GetJsonResult as WorkspacePackageJsonCacheResult; @@ -267,6 +265,9 @@ pub fn install_with_manager( }; had_any_diffs = manager.summary.has_diffs(); + if manager.subcommand == Subcommand::Dedupe { + crate::dedupe::dedupe_after_differ(manager); + } if manager.summary.changes_resolutions() { direct_deps_before = DirectDependencies::snapshot(&manager.lockfile); } @@ -298,6 +299,22 @@ pub fn install_with_manager( let new_dependencies = maybe_root.dependencies.get(&lockfile.buffers.dependencies); + let kept_pruned: Vec<(Dependency, PackageID)> = + if summary.pruned_workspaces.is_empty() { + Vec::new() + } else { + root.dependencies + .get(&lf.dependencies[..]) + .iter() + .zip(root.resolutions.get(&lf.resolutions[..])) + .filter(|(dep, _)| { + dep.behavior.is_workspace() + && summary.pruned_workspaces.contains(&dep.name_hash) + }) + .map(|(dep, &res)| (dep.clone(), res)) + .collect() + }; + for new_dep in new_dependencies { new_dep.count(&lockfile.buffers.string_bytes, builder); } @@ -323,7 +340,7 @@ pub fn install_with_manager( .count(&lockfile.buffers.string_bytes, builder); let off = lf.dependencies.len() as u32; - let len = new_dependencies.len() as u32; + let len = (new_dependencies.len() + kept_pruned.len()) as u32; let old_resolutions_list = lf.packages.items_resolutions()[0]; lf.packages.items_dependencies_mut()[0] = lockfile::DependencySlice::new(off, len); @@ -395,6 +412,11 @@ pub fn install_with_manager( lf.resolutions[off as usize + i] = old_resolutions[mapping[i] as usize]; } } + for (k, (dep, res)) in kept_pruned.into_iter().enumerate() { + let slot = off as usize + new_dependencies.len() + k; + lf.dependencies[slot] = dep; + lf.resolutions[slot] = res; + } lf.packages.items_scripts_mut()[0] = maybe_root .scripts @@ -594,7 +616,11 @@ pub fn install_with_manager( if needs_new_lockfile { if named_update { - reject_unknown_update_requests(manager, |_, request| request.e_string.is_none()); + reject_unknown_update_requests( + manager, + |_, request| request.e_string.is_none(), + |_| false, + ); } root = create_new_lockfile_and_enqueue( manager, @@ -666,11 +692,11 @@ pub fn install_with_manager( manager.verify_resolutions(log_level); + super::package_json_write_back::edit_after_resolve(manager)?; + if manager.options.security_scanner.is_some() { run_security_scanner(manager, ctx, original_cwd); } - - super::package_json_write_back::edit_after_resolve(manager)?; } // append scripts to lockfile before generating new metahash @@ -739,16 +765,17 @@ pub fn install_with_manager( && !matches!(load_result, lockfile::LoadResult::NotFound) { 'frozen_lockfile: { - if load_result.loaded_from_text_lockfile() { - if bun_core::handle_oom(Lockfile::eql( - &manager.lockfile, - &lockfile_before_clean, - packages_len_before_install, - )) { - break 'frozen_lockfile; - } - } else { - if !(manager + let changed_section = frozen_changed_section(manager); + if changed_section.is_none() { + if load_result.loaded_from_text_lockfile() { + if bun_core::handle_oom(Lockfile::eql( + &manager.lockfile, + &lockfile_before_clean, + packages_len_before_install, + )) { + break 'frozen_lockfile; + } + } else if !(manager .lockfile .has_meta_hash_changed( PackageManager::verbose_install() @@ -765,6 +792,12 @@ pub fn install_with_manager( bun_core::pretty_errorln!( "error: lockfile had changes, but lockfile is frozen" ); + if let Some(section) = changed_section { + bun_core::note!( + "\"{}\" in package.json changed since the lockfile was saved", + section + ); + } bun_core::note!( "try re-running without --frozen-lockfile and commit the updated lockfile" ); @@ -1285,67 +1318,31 @@ pub(crate) fn get_workspace_filters( manager: &mut PackageManager, original_cwd: &[u8], ) -> crate::Result<(Vec, bool)> { - let mut path_buf = bun_paths::path_buffer_pool::get(); - // RAII: guard puts the buffer back on Drop. - - let mut workspace_filters: Vec = Vec::new(); - // only populated when subcommand is `.install` - if manager.subcommand == Subcommand::Install && !manager.options.filter_patterns.is_empty() { - workspace_filters.reserve(manager.options.filter_patterns.len()); - for pattern in manager.options.filter_patterns { - workspace_filters.push(WorkspaceFilter::init(pattern, original_cwd, &mut path_buf)?); - } + if manager.subcommand != Subcommand::Install || manager.options.filter_patterns.is_empty() { + return Ok((Vec::new(), true)); } + let ids = super::workspace_selection::select_lockfile_workspaces( + &manager.lockfile, + manager.options.filter_patterns, + original_cwd, + super::workspace_selection::RootSelection::Implicit, + ); + let filters = vec![WorkspaceFilter::from_ids(ids)]; + let install_root_dependencies = WorkspaceFilter::is_selected(&filters, 0); + Ok((filters, install_root_dependencies)) +} - let mut install_root_dependencies = workspace_filters.is_empty(); - if !install_root_dependencies { - let pkg_names = manager.lockfile.packages.items_name(); - - let abs_root_path: &[u8] = 'abs_root_path: { - #[cfg(not(windows))] - { - break 'abs_root_path strings::without_trailing_slash( - FileSystem::instance().top_level_dir(), - ); - } - - #[cfg(windows)] - { - let abs_path = bun_paths::path_to_posix_buf::( - FileSystem::instance().top_level_dir, - &mut path_buf.0, - ); - break 'abs_root_path strings::without_trailing_slash( - &abs_path[bun_paths::windows_volume_name_len(abs_path).0..], - ); - } - }; - - for filter in &workspace_filters { - let (pattern, path_or_name): (&[u8], &[u8]) = match filter { - WorkspaceFilter::Name(pattern) => ( - pattern, - pkg_names[0].slice(&manager.lockfile.buffers.string_bytes), - ), - WorkspaceFilter::Path(pattern) => (pattern, abs_root_path), - WorkspaceFilter::All => { - install_root_dependencies = true; - continue; - } - }; - - let result = glob::r#match(pattern, path_or_name); - if result.matches() { - install_root_dependencies = true; - } else if result.is_negated() { - // always skip if a pattern specifically says "!" - install_root_dependencies = false; - break; - } - } +fn frozen_changed_section(manager: &PackageManager) -> Option<&'static str> { + let summary = &manager.summary; + if summary.overrides_changed { + Some("overrides") + } else if summary.catalogs_changed { + Some("catalogs") + } else if summary.patched_dependencies_changed { + Some("patchedDependencies") + } else { + None } - - Ok((workspace_filters, install_root_dependencies)) } /// Adds a contextual error for a dependency resolution failure. @@ -1447,11 +1444,13 @@ fn report_lockfile_load_error( Ok(()) } -/// `bun update …`: every row resolving to `` (by alias or real package name), at any depth, re-resolves within its own range; rows the differ already re-enqueued are left to it, and peer/bundled rows only follow the moved package via `redirect_moved_edges`, as in the bare update. +/// bun update …: every in-scope row resolving to (see update_scope) re-resolves within its own range; rows the differ re-enqueued are left to it, peers/bundled rows only follow via redirect_moved_edges. #[cold] #[inline(never)] fn enqueue_named_updates(manager: &mut PackageManager) -> Vec<(DependencyID, PackageID)> { + let walkable = crate::update_scope::UpdateScope::of(&*manager).walkable_rows(&manager.lockfile); let mut matched = vec![false; manager.update_requests.len()]; + let mut matched_elsewhere = vec![false; manager.update_requests.len()]; let mut moved: Vec<(DependencyID, PackageID)> = Vec::new(); let dependencies_len = manager.lockfile.buffers.dependencies.len(); for dependency_i in 0..dependencies_len { @@ -1460,6 +1459,10 @@ fn enqueue_named_updates(manager: &mut PackageManager) -> Vec<(DependencyID, Pac let Some(request) = index_of_named_update(manager, &dependency, package_id) else { continue; }; + if !walkable[dependency_i] { + matched_elsewhere[request] = true; + continue; + } matched[request] = true; if package_id == invalid_package_id || dependency.behavior.is_peer() @@ -1480,7 +1483,11 @@ fn enqueue_named_updates(manager: &mut PackageManager) -> Vec<(DependencyID, Pac } } - reject_unknown_update_requests(manager, |i, _| !matched[i]); + reject_unknown_update_requests( + manager, + |i, _| !matched[i] && !matched_elsewhere[i], + |i| !matched[i] && matched_elsewhere[i], + ); moved } @@ -1521,23 +1528,45 @@ fn index_of_named_update( fn reject_unknown_update_requests( manager: &PackageManager, is_unknown: impl Fn(usize, &UpdateRequest) -> bool, + is_out_of_scope: impl Fn(usize) -> bool, ) { - let unknown: Vec<&UpdateRequest> = manager - .update_requests + let requests = &manager.update_requests; + let out_of_scope: Vec<&UpdateRequest> = requests + .iter() + .enumerate() + .filter_map(|(i, request)| is_out_of_scope(i).then_some(request)) + .collect(); + let unknown: Vec<&UpdateRequest> = requests .iter() .enumerate() .filter_map(|(i, request)| is_unknown(i, request).then_some(request)) .collect(); - if unknown.is_empty() { + if out_of_scope.is_empty() && unknown.is_empty() { return; } - for request in unknown { + for request in &out_of_scope { + Output::err_generic( + "\"{}\" is only a dependency of other workspaces, so there is nothing to update here", + (bstr::BStr::new(request.get_name()),), + ); + } + for request in &unknown { Output::err_generic( "\"{}\" is not in the lockfile, so there is nothing to update", (bstr::BStr::new(request.get_name()),), ); } - bun_core::note!("to add a dependency, use bun add"); + if let Some(request) = out_of_scope.first() { + let name = bstr::BStr::new(request.get_name()); + bun_core::note!( + "run bun update -r {} to update it in every workspace, or run bun update {} inside a workspace that depends on it", + name, + name + ); + } + if !unknown.is_empty() { + bun_core::note!("to add a dependency, use bun add"); + } Output::flush(); Global::exit(1); } diff --git a/src/install/PackageManager/package_json_write_back.rs b/src/install/PackageManager/package_json_write_back.rs index 610a726ec073..84ad11f78c5c 100644 --- a/src/install/PackageManager/package_json_write_back.rs +++ b/src/install/PackageManager/package_json_write_back.rs @@ -1,3 +1,5 @@ +use bun_collections::DynamicBitSet; +use bun_collections::bit_set::Range as BitRange; use bun_core::{Global, strings}; use bun_paths::path_buffer_pool; use bun_paths::resolve_path::{join_abs_string_buf, platform}; @@ -77,7 +79,7 @@ fn edit_after_resolve_slow(manager: &mut PackageManager) -> crate::Result<()> { let exact = manager.options.enable.exact_versions(); let cwd = edited.iter().position(|e| e.received_requests); - let result = if let Some(pending) = manager.pending_filtered_write.take() { + let result = if let Some(mut pending) = manager.pending_filtered_write.take() { let result = pending.edit_entries(manager, &mut updates); manager.pending_filtered_write = Some(pending); result @@ -104,7 +106,7 @@ fn edit_after_resolve_slow(manager: &mut PackageManager) -> crate::Result<()> { result } -/// `bun update -r` / `bun update --filter`: every targeted root/workspace package.json. +/// Bare `bun update -r` / `bun update --filter`: every targeted root/workspace package.json (named requests take the `pending` branch instead). fn edit_update_targets( manager: &mut PackageManager, targets: &[super::UpdateTargetWorkspace], @@ -211,14 +213,11 @@ fn edit_cwd( } else { let dependency_list: &'static [u8] = manager.options.update.prop; let mut slice: &mut [UpdateRequest] = &mut updates[..]; - PackageJSONEditor::edit(manager, &mut slice, &mut ast, dependency_list, options)?; + add_catalog::edit_target(manager, &mut slice, &mut ast, dependency_list, options)?; } let catalog_mode = manager.options.add_catalog.is_some(); - if catalog_mode { - add_catalog::rewrite_references(manager, updates); - if in_root { - add_catalog::edit_root_after_install(manager, &ast, updates)?; - } + if catalog_mode && in_root { + add_catalog::edit_root_after_install(manager, &ast)?; } store_entry(manager, &cwd_target, ast); if in_root { @@ -232,7 +231,7 @@ fn edit_cwd( changed |= PackageJSONEditor::edit_catalogs_after_update(manager, &root_ast)?; } if catalog_mode { - add_catalog::edit_root_after_install(manager, &root_ast, updates)?; + add_catalog::edit_root_after_install(manager, &root_ast)?; } if changed { store_entry(manager, &root, root_ast); @@ -327,29 +326,35 @@ fn sync_lockfile(manager: &mut PackageManager, edited: &[EditedPackageJson]) -> let changed: Vec<(usize, usize)> = { let lbuf = manager.lockfile.buffers.string_bytes.as_slice(); let row_deps = row.get(manager.lockfile.buffers.dependencies.as_slice()); - // Empty while every scratch dep so far matched the row at its own index. - let mut claimed: Vec = Vec::new(); + // Zero-length while every scratch dep so far matched the row at its own index. + let mut claimed = DynamicBitSet::default(); let mut changed = Vec::new(); for (si, s) in scratch_deps.iter().enumerate() { let same_index = si < row_deps.len() - && claimed.get(si).is_none_or(|&taken| !taken) + && !claimed.is_set_allow_out_of_bound(si, false) && same_row(s, &row_deps[si]); let ti = if same_index { - if let Some(taken) = claimed.get_mut(si) { - *taken = true; + if si < claimed.bit_length() { + claimed.set(si); } si } else { - if claimed.is_empty() { - claimed = vec![false; row_deps.len()]; - claimed[..si.min(row_deps.len())].fill(true); + if claimed.bit_length() == 0 && !row_deps.is_empty() { + claimed = DynamicBitSet::init_empty(row_deps.len())?; + claimed.set_range_value( + BitRange { + start: 0, + end: si.min(row_deps.len()), + }, + true, + ); } - let Some(ti) = - (0..row_deps.len()).find(|&ti| !claimed[ti] && same_row(s, &row_deps[ti])) + let Some(ti) = (0..row_deps.len()) + .find(|&ti| !claimed.is_set(ti) && same_row(s, &row_deps[ti])) else { continue; }; - claimed[ti] = true; + claimed.set(ti); ti }; if row_deps[ti].version.literal.slice(lbuf) != s.version.literal.slice(sbuf) { diff --git a/src/install/PackageManager/updatePackageJSONAndInstall.rs b/src/install/PackageManager/updatePackageJSONAndInstall.rs index f2b3b1d653d9..f216955afbf9 100644 --- a/src/install/PackageManager/updatePackageJSONAndInstall.rs +++ b/src/install/PackageManager/updatePackageJSONAndInstall.rs @@ -195,8 +195,11 @@ fn update_package_json_and_install_with_manager_with_updates( Global::crash(); } - if matches!(subcommand, Subcommand::Add | Subcommand::Remove) - && !manager.options.filter_patterns.is_empty() + if (matches!(subcommand, Subcommand::Add | Subcommand::Remove) + && !manager.options.filter_patterns.is_empty()) + || (subcommand == Subcommand::Update + && !updates.is_empty() + && (manager.options.do_.recursive() || !manager.options.filter_patterns.is_empty())) { return super::add_remove_with_filter::update_filtered_workspaces_and_install( manager, @@ -258,6 +261,8 @@ fn update_package_json_and_install_with_manager_with_updates( ); } + add_catalog::prepare(manager, &updates); + // reshaped for borrowck — `get_with_path` returns `&mut MapEntry` // borrowed from `manager.workspace_package_json_cache`, but we then need // `&mut *manager` for `PackageJSONEditor::edit` / `do_patch_commit` while still @@ -368,7 +373,7 @@ fn update_package_json_and_install_with_manager_with_updates( if !updates.is_empty() { let mut updates_slice: &mut [UpdateRequest] = &mut updates[..]; - PackageJSONEditor::edit( + add_catalog::edit_target( manager, &mut updates_slice, &mut current_package_json_root, @@ -379,18 +384,11 @@ fn update_package_json_and_install_with_manager_with_updates( ..Default::default() }, )?; - // `edit` may shrink the slice. + // `edit_target` may shrink the slice. let new_len = updates_slice.len(); updates.truncate(new_len); - if manager.options.add_catalog.is_some() { - add_catalog::rewrite_references(manager, &updates); - if manager.workspace_name_hash.is_none() { - add_catalog::edit_root_before_install( - manager, - ¤t_package_json_root, - &updates, - )?; - } + if manager.options.add_catalog.is_some() && manager.workspace_name_hash.is_none() { + add_catalog::edit_root_before_install(manager, ¤t_package_json_root)?; } } else if subcommand == Subcommand::Update && manager.update_target_workspaces.is_none() { @@ -769,6 +767,7 @@ pub fn update_package_json_and_install_and_cli( subcommand: Subcommand, cli: CommandLineArguments, ) -> Result<(), Error> { + let update_groups = cli.update_groups; let (manager_ptr, original_cwd) = 'brk: { match super::init(ctx, cli.clone(), subcommand) { Ok(v) => v, @@ -828,6 +827,10 @@ pub fn update_package_json_and_install_and_cli( } } + if subcommand == Subcommand::Update { + crate::update_scope::expand_positionals(manager, original_cwd, update_groups); + } + update_package_json_and_install_with_manager(manager, ctx, original_cwd)?; if matches!(manager.options.patch_features, PatchFeatures::Patch) { diff --git a/src/install/PackageManager/workspace_selection.rs b/src/install/PackageManager/workspace_selection.rs new file mode 100644 index 000000000000..5a8969b4bcca --- /dev/null +++ b/src/install/PackageManager/workspace_selection.rs @@ -0,0 +1,367 @@ +use std::collections::VecDeque; + +use bstr::BStr; +use bun_collections::HashMap; +use bun_core::{Global, Output, strings}; +use bun_paths::path_buffer_pool; +use bun_paths::resolve_path::{join_abs_string_buf, platform}; + +use crate::bun_fs::FileSystem; +use crate::lockfile::Lockfile; +use crate::lockfile::package::PackageColumns as _; +use crate::resolution::Tag as ResolutionTag; +use crate::{PackageID, PackageNameHash}; + +pub(crate) struct Candidate<'a> { + pub(crate) name: &'a [u8], + pub(crate) abs_posix_dir: &'a [u8], + pub(crate) is_root: bool, +} + +#[derive(Clone, Copy, PartialEq, Eq)] +pub(crate) enum RootSelection { + Implicit, + ExplicitOnly, +} + +pub(crate) struct Selection { + pub(crate) selected: Vec, + pub(crate) unmatched_patterns: Vec, +} + +pub(crate) struct WorkspaceGraph { + dependencies: Vec>, + dependents: Vec>, +} + +enum Base { + All, + Name(Box<[u8]>), + Path(Box<[u8]>), + Subtree(Box<[u8]>), +} + +struct Selector { + negated: bool, + dependencies: bool, + dependents: bool, + exclude_self: bool, + base: Base, +} + +fn strip_negations(raw: &[u8]) -> (&[u8], bool) { + let mut remain = raw; + let mut negated = false; + while let Some(rest) = remain.strip_prefix(b"!") { + negated = !negated; + remain = rest; + } + (remain, negated) +} + +/// Accepted shapes: `!`* then `...`/`...^` prefix and/or `...`/`^...` suffix around `*`, `{dir}`, `./path` or a name glob. +fn parse(raw: &[u8], original_cwd: &[u8], path_buf: &mut [u8]) -> Selector { + let (mut remain, negated) = strip_negations(raw); + let mut dependencies = false; + let mut dependents = false; + let mut exclude_self = false; + + if let Some(rest) = remain.strip_prefix(b"...") { + dependents = true; + remain = rest; + if let Some(rest) = remain.strip_prefix(b"^") { + exclude_self = true; + remain = rest; + } + } + if let Some(rest) = remain.strip_suffix(b"...") { + dependencies = true; + remain = rest; + if let Some(rest) = remain.strip_suffix(b"^") { + exclude_self = true; + remain = rest; + } + } + + if (dependencies || dependents) && remain.is_empty() { + Output::err_generic( + "--filter \"{}\" is missing a workspace name or path", + (BStr::new(raw),), + ); + Global::crash(); + } + + let resolve = |part: &[u8], path_buf: &mut [u8]| -> Box<[u8]> { + strings::without_trailing_slash(join_abs_string_buf::( + original_cwd, + path_buf, + &[part], + )) + .into() + }; + + let base = if remain == b"*" || remain == b"**" { + Base::All + } else if remain.len() >= 2 && remain[0] == b'{' && remain[remain.len() - 1] == b'}' { + Base::Subtree(resolve(&remain[1..remain.len() - 1], path_buf)) + } else if remain.first() == Some(&b'.') { + Base::Path(resolve(remain, path_buf)) + } else { + Base::Name(remain.into()) + }; + + Selector { + negated, + dependencies, + dependents, + exclude_self, + base, + } +} + +fn base_matches(base: &Base, c: &Candidate<'_>, explicit_root_only: bool) -> bool { + match base { + Base::All => !(c.is_root && explicit_root_only), + Base::Name(glob) => bun_glob::r#match(glob, c.name).matches(), + Base::Path(glob) => bun_glob::r#match(glob, c.abs_posix_dir).matches(), + Base::Subtree(glob) => { + let mut dir = c.abs_posix_dir; + loop { + if bun_glob::r#match(glob, dir).matches() { + return true; + } + match strings::last_index_of_char(dir, b'/') { + Some(i) if i > 0 => dir = &dir[..i], + _ => return false, + } + } + } + } +} + +fn walk(graph: &WorkspaceGraph, sel: &Selector, base: &[bool]) -> Vec { + let n = base.len(); + let mut reached = vec![false; n]; + let mut queue: VecDeque = VecDeque::new(); + + for adjacency in [ + sel.dependencies.then_some(&graph.dependencies), + sel.dependents.then_some(&graph.dependents), + ] + .into_iter() + .flatten() + { + let mut visited = vec![false; n]; + queue.clear(); + queue.extend((0..n).filter(|&i| base[i]).map(|i| i as u32)); + while let Some(u) = queue.pop_front() { + for &v in &adjacency[u as usize] { + let v_index = v as usize; + if !visited[v_index] { + visited[v_index] = true; + reached[v_index] = true; + queue.push_back(v); + } + } + } + } + + for (reached, &in_base) in reached.iter_mut().zip(base) { + if sel.exclude_self { + *reached &= !in_base; + } else { + *reached |= in_base; + } + } + reached +} + +pub(crate) fn first_relational<'a>(patterns: &[&'a [u8]]) -> Option<&'a [u8]> { + patterns.iter().copied().find(|raw| { + let (trimmed, _) = strip_negations(raw); + strings::has_prefix(trimmed, b"...") || trimmed.ends_with(b"...") + }) +} + +pub(crate) fn select( + patterns: &[&[u8]], + original_cwd: &[u8], + candidates: &[Candidate<'_>], + graph: Option<&WorkspaceGraph>, + root: RootSelection, +) -> Selection { + let n = candidates.len(); + let explicit_root_only = root == RootSelection::ExplicitOnly && n > 1; + let mut include = vec![false; n]; + let mut exclude = vec![false; n]; + let mut any_positive = false; + let mut unmatched_patterns: Vec = Vec::new(); + let mut path_buf = path_buffer_pool::get(); + + for (index, raw) in patterns.iter().enumerate() { + let sel = parse(raw, original_cwd, &mut path_buf.0); + let mut set: Vec = candidates + .iter() + .map(|c| base_matches(&sel.base, c, explicit_root_only)) + .collect(); + if sel.dependencies || sel.dependents { + debug_assert!(graph.is_some()); + if let Some(graph) = graph { + set = walk(graph, &sel, &set); + } + } + if sel.negated { + for (excluded, &hit) in exclude.iter_mut().zip(&set) { + *excluded |= hit; + } + } else { + any_positive = true; + if !set.iter().any(|&hit| hit) { + unmatched_patterns.push(index); + } + for (included, &hit) in include.iter_mut().zip(&set) { + *included |= hit; + } + } + } + + if !any_positive { + for (included, c) in include.iter_mut().zip(candidates) { + *included = !(c.is_root && explicit_root_only); + } + } + + let selected = include + .iter() + .zip(&exclude) + .map(|(&included, &excluded)| included && !excluded) + .collect(); + Selection { + selected, + unmatched_patterns, + } +} + +impl WorkspaceGraph { + pub(crate) fn from_lockfile( + lockfile: &Lockfile, + candidate_name_hashes: &[Option], + ) -> WorkspaceGraph { + let n = candidate_name_hashes.len(); + let root_candidate = candidate_name_hashes + .iter() + .position(Option::is_none) + .map(|i| i as u32); + let mut by_hash: HashMap = HashMap::with_capacity(n); + for (i, hash) in candidate_name_hashes.iter().enumerate() { + if let Some(hash) = hash { + by_hash.insert(*hash, i as u32); + } + } + + let pkg_resolutions = lockfile.packages.items_resolution(); + let name_hashes = lockfile.packages.items_name_hash(); + let res_lists = lockfile.packages.items_resolutions(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + + let candidate_of: Vec = pkg_resolutions + .iter() + .zip(name_hashes) + .map(|(res, name_hash)| match res.tag { + ResolutionTag::Root => root_candidate.unwrap_or(u32::MAX), + ResolutionTag::Workspace => by_hash.get(name_hash).copied().unwrap_or(u32::MAX), + _ => u32::MAX, + }) + .collect(); + + let mut graph = WorkspaceGraph { + dependencies: vec![Vec::new(); n], + dependents: vec![Vec::new(); n], + }; + for (pkg_id, &from) in candidate_of.iter().enumerate() { + if from == u32::MAX { + continue; + } + for &dep_pkg in res_lists[pkg_id].get(resolutions) { + let Some(&to) = candidate_of.get(dep_pkg as usize) else { + continue; + }; + if to == u32::MAX || to == from { + continue; + } + graph.dependencies[from as usize].push(to); + graph.dependents[to as usize].push(from); + } + } + graph + } +} + +pub(crate) fn select_lockfile_workspaces( + lockfile: &Lockfile, + patterns: &[&[u8]], + original_cwd: &[u8], + root: RootSelection, +) -> Vec { + let pkg_resolutions = lockfile.packages.items_resolution(); + + let ids: Vec = pkg_resolutions + .iter() + .enumerate() + .filter(|(_, res)| matches!(res.tag, ResolutionTag::Root | ResolutionTag::Workspace)) + .map(|(pkg_id, _)| pkg_id as PackageID) + .collect(); + + if patterns.is_empty() { + return ids; + } + + let pkg_names = lockfile.packages.items_name(); + let name_hashes = lockfile.packages.items_name_hash(); + let string_buf = lockfile.buffers.string_bytes.as_slice(); + let top_level_dir = FileSystem::instance().top_level_dir(); + + let mut path_buf = path_buffer_pool::get(); + let dirs: Vec> = ids + .iter() + .map(|&pkg_id| { + let res = &pkg_resolutions[pkg_id as usize]; + let rel: &[u8] = match res.tag { + ResolutionTag::Workspace => res.workspace().slice(string_buf), + _ => b".", + }; + strings::without_trailing_slash(join_abs_string_buf::( + top_level_dir, + &mut path_buf.0, + &[rel], + )) + .into() + }) + .collect(); + + let candidates: Vec> = ids + .iter() + .zip(&dirs) + .map(|(&pkg_id, dir)| Candidate { + name: pkg_names[pkg_id as usize].slice(string_buf), + abs_posix_dir: dir, + is_root: pkg_resolutions[pkg_id as usize].tag == ResolutionTag::Root, + }) + .collect(); + + let graph = first_relational(patterns).map(|_| { + let hashes: Vec> = candidates + .iter() + .zip(&ids) + .map(|(c, &pkg_id)| (!c.is_root).then(|| name_hashes[pkg_id as usize])) + .collect(); + WorkspaceGraph::from_lockfile(lockfile, &hashes) + }); + + let selection = select(patterns, original_cwd, &candidates, graph.as_ref(), root); + ids.into_iter() + .zip(selection.selected) + .filter(|(_, selected)| *selected) + .map(|(pkg_id, _)| pkg_id) + .collect() +} diff --git a/src/install/audit_fix.rs b/src/install/audit_fix.rs index 7084834782fe..b0f3d176b0d5 100644 --- a/src/install/audit_fix.rs +++ b/src/install/audit_fix.rs @@ -3,15 +3,16 @@ use core::mem::ManuallyDrop; use std::io::Write as _; use bstr::BStr; -use bun_collections::HashMap; -use bun_core::{Global, Output, prettyln, strings}; +use bun_collections::{DynamicBitSet, HashMap}; +use bun_core::{Global, Output, UnwrapOrOom as _, pretty, prettyln, strings}; use bun_semver::query::Group; use bun_semver::{self as Semver, SlicedString}; use crate::dependency::Behavior; +use crate::lockfile::Lockfile; use crate::lockfile::package::PackageColumns as _; use crate::npm::PackageManifest; -use crate::package_manager::Options::{Enable, LogLevel}; +use crate::package_manager::Options::{Do, Enable, LogLevel}; use crate::package_manager_real::enqueue_dependency_with_main; use crate::package_manager_real::populate_manifest_cache::{self, Packages}; use crate::{ @@ -19,6 +20,10 @@ use crate::{ PackageNameHash, ResolutionTag, dependency, invalid_package_id, }; +mod json; +mod package_json_edits; +pub use package_json_edits::PackageJsonEdit; + pub struct Advisory { pub package_name: Box<[u8]>, pub vulnerable_versions: Box<[u8]>, @@ -31,6 +36,10 @@ pub struct PlannedFix { pub from: Box<[u8]>, pub to: Box<[u8]>, pub to_version: Semver::Version, + pub dep_ids: Vec, + pub downgrade: bool, + pub too_recent: bool, + pub edits: Vec, } pub struct Blocker { @@ -43,19 +52,18 @@ pub struct BlockedFix { pub name: Box<[u8]>, pub from: Box<[u8]>, pub needs: Box<[u8]>, + pub needs_is_downgrade: bool, pub blockers: Vec, } -pub enum UnfixableReason { - NoSafeRelease, - TooRecent(Box<[u8]>), - ManifestUnavailable, +pub struct UnfixableFix { + pub name: Box<[u8]>, + pub from: Box<[u8]>, } -pub struct UnfixableFix { +pub struct ManifestUnavailable { pub name: Box<[u8]>, pub from: Box<[u8]>, - pub reason: UnfixableReason, } pub struct UnmatchedAdvisory { @@ -63,20 +71,52 @@ pub struct UnmatchedAdvisory { pub range: Box<[u8]>, } +pub struct UnauditedRegistry { + pub registry: Box<[u8]>, + pub packages: Vec>, +} + pub struct FixPlan { pub fixes: Vec, pub blocked: Vec, pub unfixable: Vec, + pub manifest_unavailable: Vec, pub unmatched: Vec, + pub unaudited: Vec, pub fixed_vulnerabilities: u32, pub remaining_vulnerabilities: u32, + pub(crate) advisories: AdvisoryIndex, + pub(crate) expected_gone: Vec<(PackageNameHash, Box<[u8]>)>, +} + +pub(crate) struct AdvisoryIndex { + pub(crate) range_buf: Vec, + pub(crate) groups: Vec>, + pub(crate) by_name: HashMap>, + pub(crate) matched_before_install: DynamicBitSet, +} + +impl AdvisoryIndex { + pub(crate) fn matches(&self, i: usize, version: Semver::Version, version_buf: &[u8]) -> bool { + self.groups[i].as_ref().is_some_and(|group| { + group.satisfies_including_prerelease(version, &self.range_buf, version_buf) + }) + } +} + +pub struct FixOutcome { + pub fixed_vulnerabilities: u32, + pub remaining_vulnerabilities: u32, + pub still_vulnerable: Vec<(Box<[u8]>, Box<[u8]>)>, } struct Edge { + dep_id: DependencyID, range: Option, literal: Box<[u8]>, dependent: Box<[u8]>, bundled: bool, + pin: Option, } struct Instance { @@ -89,6 +129,12 @@ struct Instance { edges: Vec, } +struct Candidate { + version: Semver::Version, + index: usize, + downgrade: bool, +} + fn fmt_version(version: Semver::Version, buf: &[u8]) -> Box<[u8]> { let mut out: Vec = Vec::new(); let _ = write!(out, "{}", version.fmt(buf)); @@ -138,6 +184,102 @@ pub fn exit_unless_lockfile_writable(manager: &PackageManager) { Global::exit(1); } +pub fn print_unaudited(groups: &[UnauditedRegistry]) { + if groups.is_empty() { + return; + } + for group in groups { + pretty!("Skipped "); + for (i, package) in group.packages.iter().enumerate() { + if i > 0 { + pretty!(", "); + } + pretty!("{}", BStr::new(package)); + } + prettyln!( + " because {} could not be audited", + BStr::new(&group.registry) + ); + } + prettyln!(""); + Output::flush(); +} + +fn pin_for( + lockfile: &Lockfile, + dep_id: usize, + dep: &Dependency, + parent: PackageID, +) -> Option { + if parent == invalid_package_id || dep.behavior.is_bundled() || dep.behavior.is_workspace() { + return None; + } + let buf = lockfile.buffers.string_bytes.as_slice(); + let res = lockfile.packages.items_resolution(); + let parent_res = &res[parent as usize]; + if !matches!( + parent_res.tag, + ResolutionTag::Root | ResolutionTag::Workspace + ) { + return None; + } + let is_alias = dep.version.tag == DependencyVersionTag::Npm && dep.version.npm().is_alias; + if !is_alias + && lockfile + .overrides + .get(lockfile, dep_id as DependencyID, dep.name_hash) + .is_some() + { + return None; + } + let key: Box<[u8]> = Box::from(dep.name.slice(buf)); + match dep.version.tag { + DependencyVersionTag::Catalog => { + let catalog_name = dep.version.catalog().slice(buf); + let entry = lockfile + .catalogs + .find(buf, catalog_name, dep.name.slice(buf))?; + if entry.version.tag != DependencyVersionTag::Npm + || entry.version.npm().version.get_exact_version().is_none() + { + return None; + } + Some(PackageJsonEdit { + owner: 0, + file: Box::from(&b"package.json"[..]), + catalog: Some(Box::from(catalog_name)), + key, + old_literal: Box::from(entry.version.literal.slice(buf)), + new_literal: Box::default(), + }) + } + DependencyVersionTag::Npm => { + dep.version.npm().version.get_exact_version()?; + let file: Box<[u8]> = if parent_res.tag == ResolutionTag::Root { + Box::from(&b"package.json"[..]) + } else { + let dir = strings::without_trailing_slash(parent_res.workspace().slice(buf)); + let mut file: Vec = Vec::with_capacity(dir.len() + b"/package.json".len()); + if !dir.is_empty() { + file.extend_from_slice(dir); + file.push(b'/'); + } + file.extend_from_slice(b"package.json"); + file.into_boxed_slice() + }; + Some(PackageJsonEdit { + owner: parent, + file, + catalog: None, + key, + old_literal: Box::from(dep.version.literal.slice(buf)), + new_literal: Box::default(), + }) + } + _ => None, + } +} + pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crate::Result { let mut range_buf: Vec = Vec::new(); let mut range_spans: Vec<(usize, usize)> = Vec::with_capacity(advisories.len()); @@ -150,7 +292,7 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat range_spans.push((range_buf.len(), range.len())); range_buf.extend_from_slice(range); } - let advisory_groups: Vec> = range_spans + let groups: Vec> = range_spans .iter() .map(|&(start, len)| { let input = &range_buf[start..start + len]; @@ -166,8 +308,13 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat .or_default() .push(i); } + let mut index = AdvisoryIndex { + range_buf, + groups, + by_name, + matched_before_install: DynamicBitSet::init_empty(advisories.len())?, + }; - let mut advisory_matched: Vec = vec![false; advisories.len()]; let mut instances: Vec = Vec::new(); { let lockfile = &*manager.lockfile; @@ -184,24 +331,20 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat if res[pkg_id].tag != ResolutionTag::Npm { continue; } - let Some(candidates) = by_name.get(&name_hashes[pkg_id]) else { + let Some(candidates) = index.by_name.get(&name_hashes[pkg_id]) else { continue; }; let current = res[pkg_id].npm().version; let matched: Vec = candidates .iter() .copied() - .filter(|&i| { - advisory_groups[i].as_ref().is_some_and(|group| { - group.satisfies_including_prerelease(current, &range_buf, buf) - }) - }) + .filter(|&i| index.matches(i, current, buf)) .collect(); if matched.is_empty() { continue; } for &i in &matched { - advisory_matched[i] = true; + index.matched_before_install.set(i); } instance_of[pkg_id] = instances.len() as u32; instances.push(Instance { @@ -254,6 +397,7 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat dependent.extend_from_slice(b"package.json"); } instances[instance as usize].edges.push(Edge { + dep_id: dep_id as DependencyID, range: crate::dedupe::effective_npm_range( lockfile, dep_id as DependencyID, @@ -262,6 +406,7 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat literal: Box::from(dep.version.literal.slice(buf)), dependent: dependent.into_boxed_slice(), bundled: dep.behavior.is_bundled(), + pin: pin_for(lockfile, dep_id, dep, parent), }); } } @@ -270,11 +415,11 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat let mut unmatched: Vec = advisories .iter() .zip(&range_spans) - .zip(&advisory_matched) - .filter(|&(_, &matched)| !matched) - .map(|((advisory, &(start, len)), _)| UnmatchedAdvisory { + .enumerate() + .filter(|&(i, _)| !index.matched_before_install.is_set(i)) + .map(|(_, (advisory, &(start, len)))| UnmatchedAdvisory { name: advisory.package_name.clone(), - range: Box::from(&range_buf[start..start + len]), + range: Box::from(&index.range_buf[start..start + len]), }) .collect(); unmatched.sort_by(|a, b| order_name_from(&a.name, &a.range, &b.name, &b.range)); @@ -285,9 +430,13 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat fixes: Vec::new(), blocked: Vec::new(), unfixable: Vec::new(), + manifest_unavailable: Vec::new(), unmatched, + unaudited: Vec::new(), fixed_vulnerabilities: 0, remaining_vulnerabilities: advisories.len() as u32, + advisories: index, + expected_gone: Vec::new(), }); } @@ -311,7 +460,10 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat let mut fixes: Vec = Vec::new(); let mut blocked: Vec = Vec::new(); let mut unfixable: Vec = Vec::new(); - let mut advisory_still_present: Vec = advisory_matched.iter().map(|&m| !m).collect(); + let mut manifest_unavailable: Vec = Vec::new(); + let mut expected_gone: Vec<(PackageNameHash, Box<[u8]>)> = Vec::new(); + let mut advisory_still_present = index.matched_before_install.clone()?; + advisory_still_present.toggle_all(); for inst in instances { let mut expired = false; @@ -325,12 +477,11 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat min_age.is_some(), ) else { for &a in &inst.advisories { - advisory_still_present[a] = true; + advisory_still_present.set(a); } - unfixable.push(UnfixableFix { + manifest_unavailable.push(ManifestUnavailable { name: inst.name, from: inst.from, - reason: UnfixableReason::ManifestUnavailable, }); continue; }; @@ -339,88 +490,154 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat let releases = manifest.pkg.releases.keys.get(&manifest.versions); let release_pkgs = manifest.pkg.releases.values.get(&manifest.package_versions); let age_limit = min_age.filter(|_| !manifest.should_exclude_from_age_filter(excludes)); - let name_advisories: &[usize] = match by_name.get(&inst.name_hash) { + let name_advisories: &[usize] = match index.by_name.get(&inst.name_hash) { Some(indices) => indices, None => &[], }; + let is_safe = |v: Semver::Version| { + !name_advisories + .iter() + .any(|&a| index.matches(a, v, manifest_buf)) + }; - let mut needs: Option = None; - let mut too_recent: Option = None; - let mut target: Option = None; + let mut candidates: Vec = Vec::new(); for (i, &v) in releases.iter().enumerate() { - if v.order(inst.current, manifest_buf, buf) != Ordering::Greater { - continue; + if v.order(inst.current, manifest_buf, buf) == Ordering::Greater && is_safe(v) { + candidates.push(Candidate { + version: v, + index: i, + downgrade: false, + }); } - let still_vulnerable = name_advisories.iter().any(|&a| { - advisory_groups[a].as_ref().is_some_and(|group| { - group.satisfies_including_prerelease(v, &range_buf, manifest_buf) - }) - }); - if still_vulnerable { - continue; + } + let upgrade_count = candidates.len(); + for (i, &v) in releases.iter().enumerate().rev() { + if v.order(inst.current, manifest_buf, buf) == Ordering::Less && is_safe(v) { + candidates.push(Candidate { + version: v, + index: i, + downgrade: true, + }); } - if let Some(limit) = age_limit - && PackageManifest::is_package_version_too_recent(&release_pkgs[i], limit) - { - too_recent.get_or_insert(v); - continue; + } + if candidates.is_empty() { + for &a in &inst.advisories { + advisory_still_present.set(a); } - if needs.is_none() { - needs = Some(v); + unfixable.push(UnfixableFix { + name: inst.name, + from: inst.from, + }); + continue; + } + + let mut caret_buf: Vec = Vec::new(); + let mut caret: Option = None; + if inst.edges.iter().any(|edge| edge.pin.is_some()) { + caret_buf.reserve_exact(inst.from.len() + 1); + caret_buf.push(b'^'); + caret_buf.extend_from_slice(&inst.from); + caret = Some(Semver::query::parse( + &caret_buf, + SlicedString::init(&caret_buf, &caret_buf), + )?); + } + let accepts = |edge: &Edge, v: Semver::Version| -> bool { + if edge.bundled { + return false; + } + if edge.pin.is_some() { + return caret + .as_ref() + .is_some_and(|caret| caret.satisfies(v, &caret_buf, manifest_buf)); } - let all_dependents_accept = inst.edges.iter().all(|edge| { - !edge.bundled - && edge - .range - .as_ref() - .is_some_and(|range| range.npm().version.satisfies(v, buf, manifest_buf)) + edge.range + .as_ref() + .is_some_and(|range| range.npm().version.satisfies(v, buf, manifest_buf)) + }; + + let mut target: Vec> = inst + .edges + .iter() + .map(|edge| candidates.iter().position(|c| accepts(edge, c.version))) + .collect(); + let has_upgrader = target.iter().any(|t| t.is_some_and(|c| c < upgrade_count)); + if has_upgrader { + let common = (0..upgrade_count).find(|&c| { + inst.edges + .iter() + .zip(&target) + .filter(|(_, t)| t.is_some_and(|b| b < upgrade_count)) + .all(|(edge, _)| accepts(edge, candidates[c].version)) }); - if all_dependents_accept { - target = Some(v); - break; + if let Some(common) = common { + for t in target.iter_mut().flatten() { + if *t < upgrade_count { + *t = common; + } + } } } - if let Some(v) = target { + let mut chosen: Vec = if inst.edges.is_empty() { + vec![0] + } else { + target.iter().flatten().copied().collect() + }; + chosen.sort_unstable(); + chosen.dedup(); + for &c in &chosen { + let candidate = &candidates[c]; + let to = fmt_version(candidate.version, manifest_buf); + let mut dep_ids: Vec = Vec::new(); + let mut edits: Vec = Vec::new(); + for (edge, _) in inst + .edges + .iter() + .zip(&target) + .filter(|(_, t)| **t == Some(c)) + { + match &edge.pin { + None => dep_ids.push(edge.dep_id), + Some(pin) => { + if !edits.iter().any(|edit| edit.same_site(pin)) { + let mut edit = pin.clone(); + edit.new_literal = + package_json_edits::new_literal_for(&pin.old_literal, &to); + edits.push(edit); + } + } + } + } fixes.push(PlannedFix { - name: inst.name, + name: inst.name.clone(), name_hash: inst.name_hash, - from: inst.from, - to: fmt_version(v, manifest_buf), + from: inst.from.clone(), + to, to_version: Semver::Version { - major: v.major, - minor: v.minor, - patch: v.patch, + major: candidate.version.major, + minor: candidate.version.minor, + patch: candidate.version.patch, ..Default::default() }, + dep_ids, + downgrade: candidate.downgrade, + too_recent: age_limit.is_some_and(|limit| { + PackageManifest::is_package_version_too_recent( + &release_pkgs[candidate.index], + limit, + ) + }), + edits, }); - continue; } - for &a in &inst.advisories { - advisory_still_present[a] = true; - } - let Some(needs) = needs else { - unfixable.push(UnfixableFix { - name: inst.name, - from: inst.from, - reason: match too_recent { - Some(v) => UnfixableReason::TooRecent(fmt_version(v, manifest_buf)), - None => UnfixableReason::NoSafeRelease, - }, - }); - continue; - }; let blockers: Vec = inst .edges .iter() - .filter(|edge| { - edge.bundled - || !edge.range.as_ref().is_some_and(|range| { - range.npm().version.satisfies(needs, buf, manifest_buf) - }) - }) - .map(|edge| Blocker { + .zip(&target) + .filter(|(_, t)| t.is_none()) + .map(|(edge, _)| Blocker { dependent: edge.dependent.clone(), range: if edge.bundled { inst.from.clone() @@ -433,29 +650,42 @@ pub fn plan_fixes(manager: &mut PackageManager, advisories: &[Advisory]) -> crat bundled: edge.bundled, }) .collect(); + if blockers.is_empty() { + expected_gone.push((inst.name_hash, inst.from)); + continue; + } + for &a in &inst.advisories { + advisory_still_present.set(a); + } blocked.push(BlockedFix { name: inst.name, from: inst.from, - needs: fmt_version(needs, manifest_buf), + needs: fmt_version(candidates[0].version, manifest_buf), + needs_is_downgrade: candidates[0].downgrade, blockers, }); } - fixes.sort_by(|a, b| order_name_from(&a.name, &a.from, &b.name, &b.from)); + fixes.sort_by(|a, b| { + order_name_from(&a.name, &a.from, &b.name, &b.from) + .then_with(|| a.to_version.order(b.to_version, b"", b"")) + }); blocked.sort_by(|a, b| order_name_from(&a.name, &a.from, &b.name, &b.from)); unfixable.sort_by(|a, b| order_name_from(&a.name, &a.from, &b.name, &b.from)); + manifest_unavailable.sort_by(|a, b| order_name_from(&a.name, &a.from, &b.name, &b.from)); - let remaining = advisory_still_present - .iter() - .filter(|&&present| present) - .count() as u32; + let remaining = advisory_still_present.count() as u32; Ok(FixPlan { fixes, blocked, unfixable, + manifest_unavailable, unmatched, + unaudited: Vec::new(), fixed_vulnerabilities: advisories.len() as u32 - remaining, remaining_vulnerabilities: remaining, + advisories: index, + expected_gone, }) } @@ -464,24 +694,53 @@ impl FixPlan { if !self.fixes.is_empty() { prettyln!("fixing:"); for fix in &self.fixes { - prettyln!( + pretty!( " {}@{} → {}", BStr::new(&fix.name), BStr::new(&fix.from), BStr::new(&fix.to) ); + if fix.downgrade { + pretty!(" (downgrade)"); + } + if fix.too_recent { + pretty!(" (newer than --minimum-release-age)"); + } + prettyln!(""); + for edit in &fix.edits { + let file = BStr::new(&edit.file); + let old = BStr::new(&edit.old_literal); + let new = BStr::new(&edit.new_literal); + match edit.catalog.as_deref() { + None => prettyln!(" {}: {} → {}", file, old, new), + Some(b"" | b"default") => { + prettyln!(" {} (catalog): {} → {}", file, old, new) + } + Some(catalog) => prettyln!( + " {} (catalog {}): {} → {}", + file, + BStr::new(catalog), + old, + new + ), + } + } } prettyln!(""); } if !self.blocked.is_empty() { - prettyln!("requires a semver-major update:"); + prettyln!("blocked by a dependent's range:"); for item in &self.blocked { - prettyln!( + pretty!( " {}@{} → {}", BStr::new(&item.name), BStr::new(&item.from), BStr::new(&item.needs) ); + if item.needs_is_downgrade { + pretty!(" (downgrade)"); + } + prettyln!(""); for blocker in &item.blockers { prettyln!( " {} {} {}@{}", @@ -501,20 +760,14 @@ impl FixPlan { if !self.unfixable.is_empty() { prettyln!("no fix available:"); for item in &self.unfixable { - let name = BStr::new(&item.name); - let from = BStr::new(&item.from); - match &item.reason { - UnfixableReason::NoSafeRelease => prettyln!(" {}@{}", name, from), - UnfixableReason::TooRecent(version) => prettyln!( - " {}@{} ({} is newer than --minimum-release-age)", - name, - from, - BStr::new(version) - ), - UnfixableReason::ManifestUnavailable => { - prettyln!(" {}@{} (failed to fetch the manifest)", name, from) - } - } + prettyln!(" {}@{}", BStr::new(&item.name), BStr::new(&item.from)); + } + prettyln!(""); + } + if !self.manifest_unavailable.is_empty() { + prettyln!("manifest could not be fetched:"); + for item in &self.manifest_unavailable { + prettyln!(" {}@{}", BStr::new(&item.name), BStr::new(&item.from)); } prettyln!(""); } @@ -528,7 +781,7 @@ impl FixPlan { Output::flush(); } - pub fn print_summary(&self, dry_run: bool) { + fn print_summary_lines(&self, verb: &str, fixed: u32, remaining: u32) { if self.fixes.is_empty() { prettyln!("No fixable vulnerabilities"); } else { @@ -541,29 +794,149 @@ impl FixPlan { + 1; prettyln!( "{} {} {} in {} {}", - if dry_run { "Would fix" } else { "Fixed" }, - self.fixed_vulnerabilities, - vuln_word(self.fixed_vulnerabilities), + verb, + fixed, + vuln_word(fixed), packages, pkg_word(packages) ); } - if self.remaining_vulnerabilities > 0 { - prettyln!( - "{} {} remaining", + if remaining > 0 { + prettyln!("{} {} remaining", remaining, vuln_word(remaining)); + } + } + + pub fn finish_planned(&self, json: bool, dry_run: bool) -> u32 { + if json { + json::write(self, None, dry_run); + } else { + self.print_summary_lines( + if dry_run { "Would fix" } else { "Fixed" }, + self.fixed_vulnerabilities, self.remaining_vulnerabilities, - vuln_word(self.remaining_vulnerabilities) ); } + Output::flush(); + u32::from(self.remaining_vulnerabilities > 0) } - pub fn exit_code(&self) -> u32 { - u32::from(self.remaining_vulnerabilities > 0) + pub fn finish_installed(&self, lockfile: &Lockfile, json: bool) -> u32 { + let outcome = self.outcome(lockfile); + if json { + json::write(self, Some(&outcome), false); + } else { + if !outcome.still_vulnerable.is_empty() { + prettyln!("vulnerable after install:"); + for (name, version) in &outcome.still_vulnerable { + prettyln!(" {}@{}", BStr::new(name), BStr::new(version)); + } + prettyln!(""); + } + self.print_summary_lines( + "Fixed", + outcome.fixed_vulnerabilities, + outcome.remaining_vulnerabilities, + ); + } + Output::flush(); + u32::from(outcome.remaining_vulnerabilities > 0) } - pub fn pins(&self) -> Box<[PlannedFix]> { - self.fixes.clone().into_boxed_slice() + pub fn outcome(&self, lockfile: &Lockfile) -> FixOutcome { + let buf = lockfile.buffers.string_bytes.as_slice(); + let names = lockfile.packages.items_name(); + let name_hashes = lockfile.packages.items_name_hash(); + let res = lockfile.packages.items_resolution(); + let index = &self.advisories; + + let mut remaining = index.matched_before_install.clone().unwrap_or_oom(); + remaining.toggle_all(); + let staying: Vec<(PackageNameHash, &[u8])> = self + .blocked + .iter() + .map(|b| (&*b.name, &*b.from)) + .chain(self.unfixable.iter().map(|u| (&*u.name, &*u.from))) + .chain( + self.manifest_unavailable + .iter() + .map(|m| (&*m.name, &*m.from)), + ) + .map(|(name, from)| (Semver::string::Builder::string_hash(name), from)) + .collect(); + + let mut still_vulnerable: Vec<(Box<[u8]>, Box<[u8]>)> = Vec::new(); + for pkg_id in 0..res.len() { + if res[pkg_id].tag != ResolutionTag::Npm { + continue; + } + let name_hash = name_hashes[pkg_id]; + let Some(list) = index.by_name.get(&name_hash) else { + continue; + }; + let version = res[pkg_id].npm().version; + let mut hit = false; + for &i in list { + if index.matches(i, version, buf) { + remaining.set(i); + hit = true; + } + } + if !hit { + continue; + } + let ver = fmt_version(version, buf); + let planned_gone = self + .expected_gone + .iter() + .any(|(hash, from)| *hash == name_hash && **from == *ver); + let reported_staying = staying + .iter() + .any(|&(hash, from)| hash == name_hash && *from == *ver); + if planned_gone || !reported_staying { + still_vulnerable.push((Box::from(names[pkg_id].slice(buf)), ver)); + } + } + still_vulnerable.sort_by(|a, b| order_name_from(&a.0, &a.1, &b.0, &b.1)); + still_vulnerable.dedup(); + + let remaining_vulnerabilities = remaining.count() as u32; + FixOutcome { + fixed_vulnerabilities: remaining.bit_length() as u32 - remaining_vulnerabilities, + remaining_vulnerabilities, + still_vulnerable, + } + } +} + +pub fn prepare_install(manager: &mut PackageManager, plan: &FixPlan) -> crate::Result<()> { + package_json_edits::apply(manager, plan)?; + + if plan.fixes.iter().any(|fix| fix.too_recent) { + let mut names: Vec<&'static [u8]> = manager + .options + .minimum_release_age_excludes + .map_or_else(Vec::new, <[_]>::to_vec); + names.extend( + plan.fixes + .iter() + .filter(|fix| fix.too_recent) + .map(|fix| &*bun_core::heap::release(fix.name.clone())), + ); + manager.options.minimum_release_age_excludes = + Some(&*bun_core::heap::release(names.into_boxed_slice())); + } + + manager.audit_fix_pins = plan + .fixes + .iter() + .filter(|fix| !fix.dep_ids.is_empty()) + .cloned() + .collect(); + + if manager.options.json_output { + manager.options.do_.set(Do::SUMMARY, false); } + Ok(()) } pub fn enqueue_planned_fixes(manager: &mut PackageManager) -> crate::Result<()> { @@ -574,50 +947,32 @@ pub fn enqueue_planned_fixes(manager: &mut PackageManager) -> crate::Result<()> .options .enable .set(Enable::FORCE_SAVE_LOCKFILE, true); - // Counted as updates so clean drops the versions the peer slots skipped below still point at. - manager.summary.update += pins.len() as u32; - let target_of: Vec = { - let lockfile = &*manager.lockfile; - let buf = lockfile.buffers.string_bytes.as_slice(); - let name_hashes = lockfile.packages.items_name_hash(); - let res = lockfile.packages.items_resolution(); - (0..res.len()) - .map(|pkg_id| { - if res[pkg_id].tag != ResolutionTag::Npm { - return u32::MAX; - } - let mut version: Option> = None; - for (i, pin) in pins.iter().enumerate() { - if pin.name_hash != name_hashes[pkg_id] { - continue; - } - let version = - version.get_or_insert_with(|| fmt_version(res[pkg_id].npm().version, buf)); - if version[..] == pin.from[..] { - return i as u32; - } + for pin in &pins { + for &dep_id in &pin.dep_ids { + let target = manager.lockfile.buffers.resolutions[dep_id as usize]; + if target == invalid_package_id { + continue; + } + let target = target as usize; + { + let lockfile = &*manager.lockfile; + let res = lockfile.packages.items_resolution(); + if target >= res.len() + || res[target].tag != ResolutionTag::Npm + || lockfile.packages.items_name_hash()[target] != pin.name_hash + || fmt_version( + res[target].npm().version, + lockfile.buffers.string_bytes.as_slice(), + )[..] + != pin.from[..] + { + continue; } - u32::MAX - }) - .collect() - }; - - let n = manager.lockfile.buffers.resolutions.len(); - for dep_id in 0..n { - let target = manager.lockfile.buffers.resolutions[dep_id]; - if target == invalid_package_id - || target as usize >= target_of.len() - || target_of[target as usize] == u32::MAX - { - continue; - } - let behavior = manager.lockfile.buffers.dependencies[dep_id].behavior; - if behavior.is_optional_peer() || behavior.is_bundled() { - continue; + } + enqueue_pinned(manager, dep_id, pin.to_version)?; + manager.summary.update += 1; } - let pin = &pins[target_of[target as usize] as usize]; - enqueue_pinned(manager, dep_id as DependencyID, pin.to_version)?; } Ok(()) } diff --git a/src/install/audit_fix/json.rs b/src/install/audit_fix/json.rs new file mode 100644 index 000000000000..393d1e8bf947 --- /dev/null +++ b/src/install/audit_fix/json.rs @@ -0,0 +1,149 @@ +use std::io::Write as _; + +use bun_core::Output; + +use super::{FixOutcome, FixPlan, PackageJsonEdit}; + +pub(super) fn write(plan: &FixPlan, outcome: Option<&FixOutcome>, dry_run: bool) { + let (fixed, remaining) = match outcome { + Some(outcome) => ( + outcome.fixed_vulnerabilities, + outcome.remaining_vulnerabilities, + ), + None => (plan.fixed_vulnerabilities, plan.remaining_vulnerabilities), + }; + + let mut out: Vec = Vec::new(); + let _ = write!( + out, + "{{\"dryRun\":{dry_run},\"fixed\":{fixed},\"remaining\":{remaining},\"fixes\":[" + ); + for (i, fix) in plan.fixes.iter().enumerate() { + comma(&mut out, i); + out.extend_from_slice(b"{\"name\":"); + s(&mut out, &fix.name); + out.extend_from_slice(b",\"from\":"); + s(&mut out, &fix.from); + out.extend_from_slice(b",\"to\":"); + s(&mut out, &fix.to); + let _ = write!( + out, + ",\"downgrade\":{},\"newerThanMinimumReleaseAge\":{},\"packageJson\":[", + fix.downgrade, fix.too_recent + ); + for (j, edit) in fix.edits.iter().enumerate() { + comma(&mut out, j); + write_edit(&mut out, edit); + } + out.extend_from_slice(b"]}"); + } + + out.extend_from_slice(b"],\"blocked\":["); + for (i, blocked) in plan.blocked.iter().enumerate() { + comma(&mut out, i); + out.extend_from_slice(b"{\"name\":"); + s(&mut out, &blocked.name); + out.extend_from_slice(b",\"from\":"); + s(&mut out, &blocked.from); + out.extend_from_slice(b",\"needs\":"); + s(&mut out, &blocked.needs); + let _ = write!( + out, + ",\"downgrade\":{},\"blockers\":[", + blocked.needs_is_downgrade + ); + for (j, blocker) in blocked.blockers.iter().enumerate() { + comma(&mut out, j); + out.extend_from_slice(b"{\"dependent\":"); + s(&mut out, &blocker.dependent); + out.extend_from_slice(b",\"range\":"); + s(&mut out, &blocker.range); + let _ = write!(out, ",\"bundled\":{}}}", blocker.bundled); + } + out.extend_from_slice(b"]}"); + } + + out.extend_from_slice(b"],\"unfixable\":["); + for (i, unfixable) in plan.unfixable.iter().enumerate() { + comma(&mut out, i); + name_pair(&mut out, b"from", &unfixable.name, &unfixable.from); + } + + out.extend_from_slice(b"],\"manifestUnavailable\":["); + for (i, unavailable) in plan.manifest_unavailable.iter().enumerate() { + comma(&mut out, i); + name_pair(&mut out, b"from", &unavailable.name, &unavailable.from); + } + + out.extend_from_slice(b"],\"unmatched\":["); + for (i, unmatched) in plan.unmatched.iter().enumerate() { + comma(&mut out, i); + name_pair(&mut out, b"range", &unmatched.name, &unmatched.range); + } + + out.extend_from_slice(b"],\"unaudited\":["); + for (i, group) in plan.unaudited.iter().enumerate() { + comma(&mut out, i); + out.extend_from_slice(b"{\"registry\":"); + s(&mut out, &group.registry); + out.extend_from_slice(b",\"packages\":["); + for (j, package) in group.packages.iter().enumerate() { + comma(&mut out, j); + s(&mut out, package); + } + out.extend_from_slice(b"]}"); + } + + out.extend_from_slice(b"],\"vulnerableAfterInstall\":["); + if let Some(outcome) = outcome { + for (i, (name, version)) in outcome.still_vulnerable.iter().enumerate() { + comma(&mut out, i); + name_pair(&mut out, b"version", name, version); + } + } + out.extend_from_slice(b"]}\n"); + + let _ = Output::writer().write_all(&out); + Output::flush(); +} + +fn write_edit(out: &mut Vec, edit: &PackageJsonEdit) { + out.extend_from_slice(b"{\"file\":"); + s(out, &edit.file); + if let Some(catalog) = &edit.catalog { + out.extend_from_slice(b",\"catalog\":"); + s(out, catalog); + } + out.extend_from_slice(b",\"key\":"); + s(out, &edit.key); + out.extend_from_slice(b",\"from\":"); + s(out, &edit.old_literal); + out.extend_from_slice(b",\"to\":"); + s(out, &edit.new_literal); + out.push(b'}'); +} + +fn name_pair(out: &mut Vec, second_key: &[u8], name: &[u8], second: &[u8]) { + out.extend_from_slice(b"{\"name\":"); + s(out, name); + out.extend_from_slice(b",\""); + out.extend_from_slice(second_key); + out.extend_from_slice(b"\":"); + s(out, second); + out.push(b'}'); +} + +#[inline] +fn comma(out: &mut Vec, index: usize) { + if index > 0 { + out.push(b','); + } +} + +fn s(out: &mut Vec, bytes: &[u8]) { + let _ = write!( + out, + "{}", + bun_core::fmt::format_json_string_utf8(bytes, Default::default()) + ); +} diff --git a/src/install/audit_fix/package_json_edits.rs b/src/install/audit_fix/package_json_edits.rs new file mode 100644 index 000000000000..a034a1675f51 --- /dev/null +++ b/src/install/audit_fix/package_json_edits.rs @@ -0,0 +1,181 @@ +use bun_ast::{E, Expr}; +use bun_collections::VecExt as _; +use bun_core::strings; +use bun_paths::path_buffer_pool; +use bun_paths::resolve_path::{join_abs_string_buf, platform}; + +use crate::bun_fs::FileSystem; +use crate::lockfile::CatalogMap; +use crate::lockfile::package::PackageColumns as _; +use crate::package_manager_real::add_remove_with_filter::{ + WorkspaceTarget, fetch_entry_root, root_package_json_path, store_entry, +}; +use crate::package_manager_real::package_json_editor::for_each_catalog_object; +use crate::package_manager_real::package_json_write_back; +use crate::{PackageID, PackageManager, ResolutionTag}; + +const DEPENDENCY_GROUPS: [&[u8]; 4] = [ + b"dependencies", + b"devDependencies", + b"optionalDependencies", + b"peerDependencies", +]; + +#[derive(Clone)] +pub struct PackageJsonEdit { + pub owner: PackageID, + pub file: Box<[u8]>, + pub catalog: Option>, + pub key: Box<[u8]>, + pub old_literal: Box<[u8]>, + pub new_literal: Box<[u8]>, +} + +impl PackageJsonEdit { + pub(crate) fn same_site(&self, other: &PackageJsonEdit) -> bool { + self.owner == other.owner + && self.catalog == other.catalog + && self.key == other.key + && self.old_literal == other.old_literal + } +} + +pub(super) fn new_literal_for(old_literal: &[u8], to: &[u8]) -> Box<[u8]> { + let old = strings::trim(old_literal, &strings::WHITESPACE_CHARS); + let mut out: Vec = Vec::with_capacity(old.len() + to.len() + 1); + match old.strip_prefix(b"npm:") { + Some(alias) => match strings::last_index_of_char(alias, b'@') { + Some(i) if i > 0 => out.extend_from_slice(&old[..b"npm:".len() + i + 1]), + _ => { + out.extend_from_slice(old); + out.push(b'@'); + } + }, + None => { + if old.starts_with(b"=") { + out.push(b'='); + } + } + } + out.extend_from_slice(to); + out.into_boxed_slice() +} + +pub(super) fn apply(manager: &mut PackageManager, plan: &super::FixPlan) -> crate::Result<()> { + let mut edits: Vec<&PackageJsonEdit> = + plan.fixes.iter().flat_map(|fix| fix.edits.iter()).collect(); + if edits.is_empty() { + return Ok(()); + } + edits.sort_by_key(|edit| edit.owner); + + let mut start = 0; + while start < edits.len() { + let owner = edits[start].owner; + let end = start + edits[start..].partition_point(|edit| edit.owner == owner); + let owned = &edits[start..end]; + start = end; + + let Some(target) = target_for(manager, owner) else { + continue; + }; + apply_to_target(manager, &target, owned)?; + package_json_write_back::record(manager, target, false); + } + Ok(()) +} + +fn target_for(manager: &PackageManager, owner: PackageID) -> Option { + if owner == 0 { + return Some(WorkspaceTarget { + name: Box::default(), + name_hash: None, + package_json_path: root_package_json_path(), + }); + } + let lockfile = &manager.lockfile; + let res = lockfile.packages.items_resolution()[owner as usize]; + match res.tag { + ResolutionTag::Root => Some(WorkspaceTarget { + name: Box::default(), + name_hash: None, + package_json_path: root_package_json_path(), + }), + ResolutionTag::Workspace => { + let buf = lockfile.buffers.string_bytes.as_slice(); + let top_level = strings::without_trailing_slash(FileSystem::instance().top_level_dir()); + let mut path_buf = path_buffer_pool::get(); + Some(WorkspaceTarget { + name: Box::from(lockfile.packages.items_name()[owner as usize].slice(buf)), + name_hash: Some(lockfile.packages.items_name_hash()[owner as usize]), + package_json_path: join_abs_string_buf::( + top_level, + &mut path_buf.0, + &[res.workspace().slice(buf), b"package.json"], + ) + .into(), + }) + } + _ => { + debug_assert!(false, "audit fix edit owned by a non-importer package"); + None + } + } +} + +fn apply_to_target( + manager: &mut PackageManager, + target: &WorkspaceTarget, + edits: &[&PackageJsonEdit], +) -> crate::Result<()> { + let root = fetch_entry_root(manager, target); + { + let _guard = bun_ast::expr::Disabler::scope(); + let arena = &manager.ast_arena; + for edit in edits { + match &edit.catalog { + None => { + for group in DEPENDENCY_GROUPS { + let Some(mut query) = root.as_property(group) else { + continue; + }; + rewrite_property(arena, &mut query.expr, edit); + } + } + Some(catalog) => for_each_catalog_object(&root, |catalog_name, mut object| { + if CatalogMap::same_name(catalog_name, catalog) { + rewrite_property(arena, &mut object, edit); + } + Ok(()) + })?, + } + } + } + store_entry(manager, target, root); + Ok(()) +} + +fn rewrite_property(arena: &bun_alloc::Arena, object: &mut Expr, edit: &PackageJsonEdit) { + let Some(object) = object.data.e_object_mut() else { + return; + }; + for prop in object.properties.slice_mut() { + let Some(key) = prop.key.as_ref().and_then(Expr::as_utf8_string_literal) else { + continue; + }; + if key != &*edit.key { + continue; + } + let Some(value) = prop.value.as_ref().and_then(Expr::as_utf8_string_literal) else { + continue; + }; + if strings::trim(value, &strings::WHITESPACE_CHARS) != &*edit.old_literal { + continue; + } + prop.value = Some(Expr::allocate( + arena, + E::EString::init(arena.alloc_slice_copy(&edit.new_literal)), + bun_ast::Loc::EMPTY, + )); + } +} diff --git a/src/install/dedupe.rs b/src/install/dedupe.rs index d4569ae249b6..6a84f907cf6f 100644 --- a/src/install/dedupe.rs +++ b/src/install/dedupe.rs @@ -1,8 +1,9 @@ -use core::cmp::Ordering; use std::io::Write as _; use bstr::BStr; -use bun_core::{Global, Output, strings}; +use bun_collections::DynamicBitSet; +use bun_collections::bit_set::Range; +use bun_core::{Global, Output, UnwrapOrOom as _, strings}; use crate::lockfile::package::PackageColumns as _; use crate::lockfile::{LoadResult, LoadStep, Lockfile, PackageIndexEntry}; @@ -16,23 +17,124 @@ struct Pass<'a> { lockfile: &'a Lockfile, groups: &'a [Vec], group_of: &'a [u32], - pinned: &'a [bool], - edges: Vec>, + pinned: &'a DynamicBitSet, + edges: Vec>, candidates: Vec, - sat: Vec, - movable: Vec, + edge_count: usize, + sat: DynamicBitSet, + movable: DynamicBitSet, + cur_c: Vec, + required: DynamicBitSet, + chosen: DynamicBitSet, + survivors: DynamicBitSet, + covered: DynamicBitSet, + voters: DynamicBitSet, + voted: DynamicBitSet, count: Vec, + protected: Vec, } -impl Pass<'_> { - // Re-points the group edges owned by `voters` onto the best `live` version; also returns who voted. - fn vote( - &mut self, - cur: &[PackageID], - live: &[bool], - voters: &[bool], - ) -> (Vec, Vec) { +impl<'a> Pass<'a> { + fn new( + lockfile: &'a Lockfile, + groups: &'a [Vec], + group_of: &'a [u32], + pinned: &'a DynamicBitSet, + max_candidates: usize, + max_edges: usize, + ) -> Pass<'a> { + let package_count = lockfile.packages.len(); + Pass { + lockfile, + groups, + group_of, + pinned, + edges: vec![Vec::new(); groups.len()], + candidates: Vec::with_capacity(max_candidates), + edge_count: 0, + sat: DynamicBitSet::init_empty(max_edges * max_candidates).unwrap_or_oom(), + movable: DynamicBitSet::init_empty(max_edges).unwrap_or_oom(), + cur_c: Vec::with_capacity(max_edges), + required: DynamicBitSet::init_empty(max_candidates).unwrap_or_oom(), + chosen: DynamicBitSet::init_empty(max_candidates).unwrap_or_oom(), + survivors: DynamicBitSet::init_empty(max_candidates).unwrap_or_oom(), + covered: DynamicBitSet::init_empty(max_edges).unwrap_or_oom(), + voters: DynamicBitSet::init_empty(package_count).unwrap_or_oom(), + voted: DynamicBitSet::init_empty(package_count).unwrap_or_oom(), + count: Vec::with_capacity(max_candidates), + protected: Vec::new(), + } + } + + // Greedy minimum cover of the group's edges seeded with `required`; candidates are ordered highest version first. + fn cover(&mut self) -> usize { + let n = self.candidates.len(); + let m = self.edge_count; + self.required.copy_into(&mut self.chosen); + self.covered.unmanaged.set_all(false); + let mut uncovered = m; + let mut size = 0; + for c in 0..n { + if self.chosen.is_set(c) { + size += 1; + uncovered = self.mark_covered(c, uncovered); + } + } + while uncovered > 0 { + self.count.clear(); + self.count.resize(n, 0); + for e in 0..m { + if self.covered.is_set(e) { + continue; + } + let row = e * n; + for c in 0..n { + self.count[c] += self.sat.is_set(row + c) as u32; + } + } + let mut best: Option = None; + for c in 0..n { + if self.chosen.is_set(c) || self.count[c] == 0 { + continue; + } + if best.is_none_or(|b| self.count[c] > self.count[b]) { + best = Some(c); + } + } + let Some(b) = best else { + debug_assert!(false, "every edge is satisfied by its own live target"); + break; + }; + self.chosen.set(b); + size += 1; + uncovered = self.mark_covered(b, uncovered); + } + size + } + + fn mark_covered(&mut self, c: usize, mut uncovered: usize) -> usize { + let n = self.candidates.len(); + for e in 0..self.edge_count { + if !self.covered.is_set(e) && self.sat.is_set(e * n + c) { + self.covered.set(e); + uncovered -= 1; + } + } + uncovered + } + + fn placement(&self, e: usize) -> Option { + let n = self.candidates.len(); + let row = e * n; + (0..n).find(|&c| self.survivors.is_set(c) && self.sat.is_set(row + c)) + } + + // Keeps the fewest versions that satisfy every group edge owned by `voters` and re-points only edges whose version is dropped; `voted` records who voted. + fn vote(&mut self, cur: &[PackageID], live: &DynamicBitSet) -> Vec { let lockfile = self.lockfile; + let groups = self.groups; + let group_of = self.group_of; + let pinned = self.pinned; let buf = lockfile.buffers.string_bytes.as_slice(); let pkg_res = lockfile.packages.items_resolution(); let dep_slices = lockfile.packages.items_dependencies(); @@ -41,137 +143,149 @@ impl Pass<'_> { for edges in &mut self.edges { edges.clear(); } - let mut voted = vec![false; dep_slices.len()]; + self.voted.unmanaged.set_all(false); for (pkg_id, slice) in dep_slices.iter().enumerate() { - if !voters[pkg_id] { + if !self.voters.is_set(pkg_id) { continue; } + let direct = matches!( + pkg_res[pkg_id].tag, + ResolutionTag::Root | ResolutionTag::Workspace + ); for dep_id in slice.begin() as usize..slice.end() as usize { let target = cur[dep_id]; if target == invalid_package_id { continue; } - let Some(&g) = self.group_of.get(target as usize) else { + let Some(&g) = group_of.get(target as usize) else { continue; }; if g == u32::MAX { continue; } - self.edges[g as usize].push(dep_id as DependencyID); - voted[pkg_id] = true; + self.edges[g as usize].push((dep_id as DependencyID, direct)); + self.voted.set(pkg_id); } } let mut next: Vec = cur.to_vec(); - for (g, group) in self.groups.iter().enumerate() { - let edges = &self.edges[g]; - if edges.is_empty() { + for (g, group) in groups.iter().enumerate() { + if self.edges[g].is_empty() { continue; } self.candidates.clear(); self.candidates - .extend(group.iter().copied().filter(|&id| live[id as usize])); - let candidates = self.candidates.as_slice(); - let n = candidates.len(); + .extend(group.iter().copied().filter(|&id| live.is_set(id as usize))); + let n = self.candidates.len(); if n < 2 { continue; } + let edges = core::mem::take(&mut self.edges[g]); let m = edges.len(); - self.sat.clear(); - self.sat.resize(m * n, false); - self.movable.clear(); - self.movable.resize(m, false); - self.count.clear(); - self.count.resize(n, 0); + self.edge_count = m; + self.sat.set_range_value( + Range { + start: 0, + end: m * n, + }, + false, + ); + self.movable.unmanaged.set_all(false); + self.cur_c.clear(); + self.cur_c.resize(m, 0); + self.required.unmanaged.set_all(false); - for (e, &dep_id) in edges.iter().enumerate() { + for (e, &(dep_id, _)) in edges.iter().enumerate() { let dep = &deps[dep_id as usize]; let target = cur[dep_id as usize]; - let cur_c = candidates + let cur_c = self + .candidates .iter() .position(|&c| c == target) .expect("edge target is a live candidate of its group"); - let row = &mut self.sat[e * n..(e + 1) * n]; + self.cur_c[e] = cur_c; + let row = e * n; - let range = if self.pinned[target as usize] || dep.behavior.is_bundled() { + let range = if pinned.is_set(target as usize) || dep.behavior.is_bundled() { None } else { effective_npm_range(lockfile, dep_id, dep) }; match range { - None => row[cur_c] = true, + None => { + self.sat.set(row + cur_c); + self.required.set(cur_c); + } Some(range) => { - self.movable[e] = true; + self.movable.set(e); let query = &range.npm().version; - for (c, &id) in candidates.iter().enumerate() { - row[c] = query.satisfies(pkg_res[id as usize].npm().version, buf, buf); + for c in 0..n { + let id = self.candidates[c]; + if query.satisfies(pkg_res[id as usize].npm().version, buf, buf) { + self.sat.set(row + c); + } } } } - for (c, &ok) in row.iter().enumerate() { - self.count[c] += ok as u32; - } } - for (e, &dep_id) in edges.iter().enumerate() { - if !self.movable[e] { + let base_size = self.cover(); + self.chosen.copy_into(&mut self.survivors); + + self.protected.clear(); + for (e, &(_, direct)) in edges.iter().enumerate() { + if !direct || !self.movable.is_set(e) || self.survivors.is_set(self.cur_c[e]) { continue; } - let row = &self.sat[e * n..(e + 1) * n]; - let mut best: Option = None; - for c in 0..n { - if !row[c] { + if self.placement(e).is_some_and(|p| p > self.cur_c[e]) { + self.protected.push(self.cur_c[e]); + } + } + if !self.protected.is_empty() { + self.protected.sort_unstable(); + self.protected.dedup(); + for i in 0..self.protected.len() { + let k = self.protected[i]; + if self.survivors.is_set(k) { continue; } - let Some(b) = best else { - best = Some(c); - continue; - }; - let better = match self.count[c].cmp(&self.count[b]) { - Ordering::Greater => true, - Ordering::Less => false, - Ordering::Equal => { - let vc = pkg_res[candidates[c] as usize].npm().version; - let vb = pkg_res[candidates[b] as usize].npm().version; - match vc.order(vb, buf, buf) { - Ordering::Greater => true, - Ordering::Less => false, - Ordering::Equal => candidates[c] < candidates[b], - } - } - }; - if better { - best = Some(c); + self.required.set(k); + if self.cover() == base_size { + self.chosen.copy_into(&mut self.survivors); + } else { + self.required.unset(k); } } - if let Some(b) = best { - next[dep_id as usize] = candidates[b]; + } + + for (e, &(dep_id, _)) in edges.iter().enumerate() { + if !self.movable.is_set(e) || self.survivors.is_set(self.cur_c[e]) { + continue; + } + if let Some(p) = self.placement(e) { + next[dep_id as usize] = self.candidates[p]; } } + self.edges[g] = edges; } - (next, voted) + next } // Packages the pass itself removes must not vote (pnpm/pnpm#9213): shrink voters until the outcome agrees. - fn settle(&mut self, cur: &[PackageID], live: &[bool]) -> Vec { - let mut voters = live.to_vec(); + fn settle(&mut self, cur: &[PackageID], live: &DynamicBitSet) -> Vec { + live.copy_into(&mut self.voters); let mut best: Option> = None; loop { - let (next, voted) = self.vote(cur, live, &voters); + let next = self.vote(cur, live); let after = reachable(self.lockfile, &next); - if (0..after.len()).any(|p| after[p] && !voters[p]) { + if !after.unmanaged.subset_of(&self.voters.unmanaged) { return best.unwrap_or(next); } - let mut died = false; - for p in 0..after.len() { - if voted[p] && !after[p] { - voters[p] = false; - died = true; - } - } - if !died { + self.voted.unmanaged.set_exclude(&after.unmanaged); + if self.voted.count() == 0 { return next; } + self.voters.unmanaged.set_exclude(&self.voted.unmanaged); best = Some(next); } } @@ -193,18 +307,20 @@ pub(crate) fn label(lockfile: &Lockfile, id: PackageID) -> Vec { } pub fn dedupe_lockfile(lockfile: &mut Lockfile) -> Vec> { + let buf = lockfile.buffers.string_bytes.as_slice(); let pkg_res = lockfile.packages.items_resolution(); let has_patches = lockfile.patched_dependencies.count() > 0; let mut groups: Vec> = Vec::new(); let mut group_of: Vec = vec![u32::MAX; pkg_res.len()]; - let mut pinned: Vec = vec![false; pkg_res.len()]; + let mut pinned = DynamicBitSet::init_empty(pkg_res.len()).unwrap_or_oom(); + let mut max_candidates = 0; for entry in lockfile.package_index.values() { let PackageIndexEntry::Ids(ids) = entry else { continue; }; - let candidates: Vec = ids + let mut candidates: Vec = ids .iter() .copied() .filter(|&id| pkg_res[id as usize].tag == ResolutionTag::Npm) @@ -212,13 +328,24 @@ pub fn dedupe_lockfile(lockfile: &mut Lockfile) -> Vec> { if candidates.len() < 2 { continue; } + candidates.sort_by(|&a, &b| { + pkg_res[b as usize] + .npm() + .version + .order(pkg_res[a as usize].npm().version, buf, buf) + .then(a.cmp(&b)) + }); for &id in &candidates { group_of[id as usize] = groups.len() as u32; - pinned[id as usize] = has_patches + if has_patches && lockfile.patched_dependencies.contains( &bun_semver::string::Builder::string_hash(&label(lockfile, id)), - ); + ) + { + pinned.set(id as usize); + } } + max_candidates = max_candidates.max(candidates.len()); groups.push(candidates); } @@ -226,25 +353,33 @@ pub fn dedupe_lockfile(lockfile: &mut Lockfile) -> Vec> { return Vec::new(); } + // Re-pointing keeps an edge inside its group, so the initial per-group edge counts bound every pass. + let mut edge_counts: Vec = vec![0; groups.len()]; + for &target in lockfile.buffers.resolutions.iter() { + if let Some(&g) = group_of.get(target as usize) + && g != u32::MAX + { + edge_counts[g as usize] += 1; + } + } + let max_edges = edge_counts.iter().copied().max().unwrap_or(0) as usize; + let initial = reachable(lockfile, &lockfile.buffers.resolutions); - let mut live = initial.clone(); + let mut live = initial.clone().unwrap_or_oom(); let mut cur: Vec = lockfile.buffers.resolutions.clone(); { - let mut pass = Pass { + let mut pass = Pass::new( lockfile, - groups: &groups, - group_of: &group_of, - pinned: &pinned, - edges: vec![Vec::new(); groups.len()], - candidates: Vec::new(), - sat: Vec::new(), - movable: Vec::new(), - count: Vec::new(), - }; + &groups, + &group_of, + &pinned, + max_candidates, + max_edges, + ); loop { cur = pass.settle(&cur, &live); let after = reachable(lockfile, &cur); - if after == live { + if after.unmanaged.eql(&live.unmanaged) { break; } live = after; @@ -255,13 +390,12 @@ pub fn dedupe_lockfile(lockfile: &mut Lockfile) -> Vec> { .iter() .flatten() .copied() - .filter(|&id| initial[id as usize] && !live[id as usize]) + .filter(|&id| initial.is_set(id as usize) && !live.is_set(id as usize)) .collect(); if removed.is_empty() { return Vec::new(); } - let buf = lockfile.buffers.string_bytes.as_slice(); let names = lockfile.packages.items_name(); removed.sort_by(|&a, &b| { let (a, b) = (a as usize, b as usize); @@ -306,7 +440,7 @@ pub(crate) fn effective_npm_range( } // Optional-peer edges are followed too: with an in-sync package.json `clean` runs with `keep_optional_peer_targets`. -fn reachable(lockfile: &Lockfile, resolutions: &[PackageID]) -> Vec { +fn reachable(lockfile: &Lockfile, resolutions: &[PackageID]) -> DynamicBitSet { crate::lockfile::reachable::packages( lockfile, resolutions, @@ -353,17 +487,45 @@ pub fn dedupe_before_install( LoadResult::Ok(_) => {} } - let removed = dedupe_lockfile(&mut manager.lockfile); + if manager + .lockfile + .root_package() + .is_none_or(|root| root.dependencies.len == 0) + { + report_already_deduplicated(manager); + } + Ok(()) +} - if removed.is_empty() { +fn report_already_deduplicated(manager: &PackageManager) { + if manager.options.log_level != LogLevel::Silent { + bun_core::prettyln!("Already deduplicated."); + Output::flush(); + } + if manager.options.dry_run { + Global::exit(0); + } +} + +pub fn dedupe_after_differ(manager: &mut PackageManager) { + let quiet = manager.options.log_level == LogLevel::Silent; + + if manager.summary.changes_dependencies() { if !quiet { - bun_core::prettyln!("Already deduplicated."); + Output::err_generic( + "the lockfile is out of date with package.json, nothing was deduplicated", + (), + ); + bun_core::note!("run 'bun install' first"); Output::flush(); } - if manager.options.dry_run { - Global::exit(0); - } - return Ok(()); + Global::exit(1); + } + + let removed = dedupe_lockfile(&mut manager.lockfile); + if removed.is_empty() { + report_already_deduplicated(manager); + return; } let n = removed.len(); @@ -423,5 +585,4 @@ pub fn dedupe_before_install( .options .enable .set(Enable::FORCE_SAVE_LOCKFILE, true); - Ok(()) } diff --git a/src/install/dependency.rs b/src/install/dependency.rs index 749904124596..b559c4ffeec8 100644 --- a/src/install/dependency.rs +++ b/src/install/dependency.rs @@ -440,7 +440,14 @@ fn is_github_tarball_path(dependency: &[u8]) -> bool { // before I add that. #[inline] fn is_tarball(dependency: &[u8]) -> bool { - dependency.ends_with(b".tgz") || dependency.ends_with(b".tar.gz") + has_suffix_ignore_ascii_case(dependency, b".tgz") + || has_suffix_ignore_ascii_case(dependency, b".tar.gz") + || has_suffix_ignore_ascii_case(dependency, b".tar") +} + +#[inline] +fn has_suffix_ignore_ascii_case(s: &[u8], suffix: &[u8]) -> bool { + s.len() >= suffix.len() && s[s.len() - suffix.len()..].eq_ignore_ascii_case(suffix) } /// the input is assumed to be either a remote or local tarball diff --git a/src/install/isolated_install.rs b/src/install/isolated_install.rs index f0b3ea2ea181..f74bcb0a9bda 100644 --- a/src/install/isolated_install.rs +++ b/src/install/isolated_install.rs @@ -2089,47 +2089,6 @@ pub(crate) fn install_isolated_packages( ); } - let force_install = installer.manager().options.enable.force_install(); - let store_hash_path = paths::path_literal!("node_modules/.bun/.store-hash"); - let store_graph_hex: [u8; 16] = { - let mut hasher = Wyhash::init(0); - for _entry_id in 0..store.entries.len() { - let entry_id = store::entry::Id::from(u32::try_from(_entry_id).expect("int cast")); - { - let mut hw = WyhashWriter { - hasher: &mut hasher, - }; - write!( - hw, - "{}\0", - store::entry::fmt_store_path(entry_id, &store, lockfile_ro) - ) - .expect("unreachable"); - } - for dep in entry_dependencies[_entry_id].slice() { - hasher.update( - lockfile_ro.buffers.dependencies[dep.dep_id as usize] - .name - .slice(string_buf), - ); - hasher.update(b"\0"); - hasher.update(&dep.entry_id.get().to_le_bytes()); - } - } - let mut hex = [0u8; 16]; - write!(&mut hex[..], "{:016x}", hasher.final_()).expect("unreachable"); - hex - }; - let relink_needed = !is_new_bun_modules && !force_install && { - let mut stamp = [0u8; 17]; - let read = sys::File::openat(Fd::cwd(), store_hash_path, sys::O::RDONLY, 0) - .and_then(|file| file.read_all(&mut stamp)); - !matches!(read, Ok(n) if n == store_graph_hex.len() && stamp[..n] == store_graph_hex) - }; - if relink_needed { - let _ = sys::unlinkat(Fd::cwd(), store_hash_path); - } - // add the pending task count upfront installer .manager_mut() @@ -2339,7 +2298,7 @@ pub(crate) fn install_isolated_packages( if entry_hoisted[entry_id.get() as usize] { installer.link_to_hidden_node_modules(entry_id); } - if relink_needed && !uses_global_store { + if !uses_global_store { installer.start_relink_task(entry_id); continue; } @@ -2682,11 +2641,6 @@ pub(crate) fn install_isolated_packages( debug_assert!(done); } - if installer.summary.fail == 0 && (relink_needed || is_new_bun_modules || force_install) { - let _ = sys::File::create(Fd::cwd(), store_hash_path, true) - .and_then(|file| file.write_all(&store_graph_hex)); - } - let mut summary = core::mem::take(&mut installer.summary); summary.successfully_installed = Some(core::mem::take(&mut installer.installed)); diff --git a/src/install/lib.rs b/src/install/lib.rs index 4f156ed065de..710be9bc025c 100644 --- a/src/install/lib.rs +++ b/src/install/lib.rs @@ -125,6 +125,7 @@ pub mod pnpm; pub mod prune; #[path = "repository.rs"] pub mod repository_real; +pub mod update_scope; pub mod update_transitive; pub mod yarn; diff --git a/src/install/lockfile/CatalogMap.rs b/src/install/lockfile/CatalogMap.rs index 3c5a6a5db952..7a75f37d61e9 100644 --- a/src/install/lockfile/CatalogMap.rs +++ b/src/install/lockfile/CatalogMap.rs @@ -5,7 +5,9 @@ use bun_alloc::AllocError; use bun_collections::ArrayHashMap; use bun_collections::array_hash_map::ArrayHashAdapter; use bun_install::dependency::DependencyExt as _; -use bun_install::dependency::{Tag as DependencyVersionTag, Version as DependencyVersion}; +use bun_install::dependency::{ + Tag as DependencyVersionTag, Value as DependencyVersionValue, Version as DependencyVersion, +}; use bun_install::lockfile::{Buffers, StringBuilder}; use bun_install::{Dependency, Lockfile, PackageManager}; // Layering: every install-side caller (Package.rs / pnpm.rs) parses JSON/YAML @@ -157,6 +159,19 @@ impl CatalogMap { } } + /// Only the root and its workspaces may reference catalogs; anywhere else a `catalog:` spec is left unresolvable. + pub(crate) fn strip_reference(version: DependencyVersion) -> DependencyVersion { + if version.tag != DependencyVersionTag::Catalog { + return version; + } + let literal = version.literal; + DependencyVersion { + tag: DependencyVersionTag::Uninitialized, + literal, + value: DependencyVersionValue::default(), + } + } + /// Takes `buf: &[u8]` (the lockfile's string buffer, used for the hash /// context) rather than the whole `Lockfile` so callers can hold /// `&mut lockfile.catalogs` while only borrowing `buffers.string_bytes` diff --git a/src/install/lockfile/Package.rs b/src/install/lockfile/Package.rs index a27695bbfb6e..99ae39de7499 100644 --- a/src/install/lockfile/Package.rs +++ b/src/install/lockfile/Package.rs @@ -812,15 +812,17 @@ impl Package { name: name.value, name_hash: name.hash, behavior, - version: Dependency::parse( - name.value, - Some(name.hash), - sliced.slice, - &sliced, - Some(&mut *log), - Some(&mut *pm), - ) - .unwrap_or_default(), + version: lockfile::CatalogMap::strip_reference( + Dependency::parse( + name.value, + Some(name.hash), + sliced.slice, + &sliced, + Some(&mut *log), + Some(&mut *pm), + ) + .unwrap_or_default(), + ), }; // If a dependency appears in both "dependencies" and "optionalDependencies", it is considered optional! @@ -911,6 +913,7 @@ pub struct DiffSummary { pub(crate) add: u32, pub(crate) remove: u32, pub(crate) update: u32, + pub(crate) script_only_updates: u32, pub(crate) overrides_changed: bool, pub(crate) catalogs_changed: bool, @@ -940,6 +943,15 @@ impl DiffSummary { || self.removed_trusted_dependencies.count() > 0 || self.patched_dependencies_changed } + + #[inline] + pub(crate) fn changes_dependencies(&self) -> bool { + self.add > 0 + || self.remove > 0 + || self.overrides_changed + || self.catalogs_changed + || self.update > self.script_only_updates + } } impl Diff { @@ -959,6 +971,16 @@ impl Diff { ) -> crate::Result { let mut summary = DiffSummary::default(); let is_root = id_mapping.is_some(); + let named_update_here = match update_requests { + Some(updates) if !updates.is_empty() => crate::update_scope::UpdateScope::of(&*pm) + .contains_workspace( + from.resolution.tag == ResolutionTag::Root, + from.name_hash, + from.name + .slice(from_lockfile.buffers.string_bytes.as_slice()), + ), + _ => true, + }; // `parseWithJSON` may grow `to_lockfile.buffers.dependencies` and // invalidate the old slice, so `to_deps` is re-derived after it. Held as raw fat // pointers so the `&mut to_lockfile`/`&mut from_lockfile` reborrows below @@ -1002,15 +1024,24 @@ impl Diff { } if is_root { + let tolerate_catalog_subset = pm.options.enable.frozen_lockfile(); 'catalogs: { // don't sort if lengths are different if from_lockfile.catalogs.default.count() != to_lockfile.catalogs.default.count() { - summary.catalogs_changed = true; + summary.catalogs_changed = !(tolerate_catalog_subset + && lockfile::pruned_workspaces::lockfile_catalogs_are_subset( + &*from_lockfile, + &*to_lockfile, + )); break 'catalogs; } if from_lockfile.catalogs.groups.count() != to_lockfile.catalogs.groups.count() { - summary.catalogs_changed = true; + summary.catalogs_changed = !(tolerate_catalog_subset + && lockfile::pruned_workspaces::lockfile_catalogs_are_subset( + &*from_lockfile, + &*to_lockfile, + )); break 'catalogs; } @@ -1069,7 +1100,11 @@ impl Diff { } if from_catalog_deps.count() != to_catalog_deps.count() { - summary.catalogs_changed = true; + summary.catalogs_changed = !(tolerate_catalog_subset + && lockfile::pruned_workspaces::lockfile_catalogs_are_subset( + &*from_lockfile, + &*to_lockfile, + )); break 'catalogs; } @@ -1347,13 +1382,14 @@ impl Diff { ) { if let Some(updates) = update_requests { if updates.is_empty() - || UpdateRequest::contains_name( - updates, - from_dep.name_hash, - from_dep - .name - .slice(from_lockfile.buffers.string_bytes.as_slice()), - ) + || (named_update_here + && UpdateRequest::contains_name( + updates, + from_dep.name_hash, + from_dep + .name + .slice(from_lockfile.buffers.string_bytes.as_slice()), + )) { // Listed as to be updated summary.update += 1; @@ -1362,6 +1398,7 @@ impl Diff { } if let Some(mapping) = id_mapping.as_deref_mut() { + let mut workspace_hooks_only = false; let update_mapping = 'update_mapping: { if !is_root || !from_dep.behavior.is_workspace() { break 'update_mapping true; @@ -1456,6 +1493,7 @@ impl Diff { ); } + workspace_hooks_only = !diff.changes_dependencies(); !diff.changes_resolutions() }; @@ -1463,35 +1501,23 @@ impl Diff { mapping[cur_to_i] = i as PackageID; continue; } + if workspace_hooks_only { + summary.script_only_updates += 1; + } } else { continue; } } - // We found a changed dependency! - // - // If only the *version literal* changed and the previously-resolved - // package still satisfies the new range, keep the existing - // resolution. Otherwise widening a range (e.g. `"4.0.0"` → `"*"`) - // re-resolves to latest on the next `bun add `, which - // surprises migrations from npm/pnpm lockfiles whose package.json - // range diverged from the locked version. This matches npm's - // sticky-lockfile behaviour and lets `Lockfile::get_package_id` - // apply its order-independence guard without overriding a locked - // pin. - // - // Skipped when the dependency is an explicit update target - // (`bun update ` or bare `bun update`): the user is asking - // for a fresh resolve and the old resolution must not be - // preserved. Same gate as the `Dependency::eql == true` branch - // above. + // Changed literal: keep the locked resolution while it still satisfies the new range (npm's sticky rule), unless this row is being updated. let is_explicit_update_target = matches!(update_requests, Some(updates) if updates.is_empty() - || UpdateRequest::contains_name( - updates, - from_dep.name_hash, - from_dep.name.slice(from_lockfile.buffers.string_bytes.as_slice()), - )); + || (named_update_here + && UpdateRequest::contains_name( + updates, + from_dep.name_hash, + from_dep.name.slice(from_lockfile.buffers.string_bytes.as_slice()), + ))); if !is_explicit_update_target { if let Some(mapping) = id_mapping.as_deref_mut() { let from_res_id = from_resolutions[i]; @@ -1527,6 +1553,14 @@ impl Diff { .saturating_sub(summary.remove as usize + summary.pruned_workspaces.len()), )) as u32; + if !summary.pruned_workspaces.is_empty() { + lockfile::pruned_workspaces::exit_if_survivor_depends_on_pruned( + &*from_lockfile, + &summary.pruned_workspaces, + pm.options.log_level.is_silent(), + ); + } + if !summary.pruned_workspaces.is_empty() && !pm.options.log_level.is_silent() { let count = summary.pruned_workspaces.len(); bun_core::note!( @@ -1546,6 +1580,7 @@ impl Diff { ) { // We found a changed life-cycle script summary.update += 1; + summary.script_only_updates += 1; } } } @@ -1679,6 +1714,9 @@ impl Package { Some(&mut *pm), ) .unwrap_or_default(); + if !(features.is_main || features.is_workspace) { + dependency_version = lockfile::CatalogMap::strip_reference(dependency_version); + } let mut workspace_range: Option = None; #[allow(non_snake_case)] let FEATURES = features; diff --git a/src/install/lockfile/Tree.rs b/src/install/lockfile/Tree.rs index 129fdb5403cf..dc63c57e252a 100644 --- a/src/install/lockfile/Tree.rs +++ b/src/install/lockfile/Tree.rs @@ -4,7 +4,7 @@ use bun_alloc::AllocError; use bun_collections::{ArrayHashMap, DynamicBitSet, MultiArrayList}; use bun_core::Output; use bun_core::ZStr; -use bun_paths::{self, MAX_PATH_BYTES, PathBuffer, SEP}; +use bun_paths::{MAX_PATH_BYTES, PathBuffer, SEP}; use crate::lockfile::package::PackageColumns as _; use crate::lockfile::{DepSorter, DependencyIDList, DependencyIDSlice, Lockfile}; @@ -572,7 +572,6 @@ pub(crate) fn is_filtered_dependency_or_workspace( let pkg_resolutions = pkgs.items_resolution(); let dep = &lockfile.buffers.dependencies.as_slice()[dep_id as usize]; - let res = &pkg_resolutions[pkg_id as usize]; let parent_res = &pkg_resolutions[parent_pkg_id as usize]; if pkg_metas[pkg_id as usize].is_disabled(manager.options.cpu, manager.options.os) { @@ -632,50 +631,7 @@ pub(crate) fn is_filtered_dependency_or_workspace( return true; } - let mut workspace_matched = workspace_filters.is_empty(); - - for filter in workspace_filters { - // Separator is a const generic on `bun_paths::AbsPath`. - let mut filter_path = bun_paths::AbsPath::< - u8, - { bun_paths::path_options::PathSeparators::POSIX }, - >::init_top_level_dir(); - // filter_path drops at end of iteration. - - let (pattern, name_or_path): (&[u8], &[u8]) = match filter { - WorkspaceFilter::All => { - workspace_matched = true; - continue; - } - WorkspaceFilter::Name(name_pattern) => ( - name_pattern, - pkg_names[pkg_id as usize].slice(lockfile.buffers.string_bytes.as_slice()), - ), - WorkspaceFilter::Path(path_pattern) => 'path_pattern: { - if res.tag != crate::resolution::Tag::Workspace { - return false; - } - - // path-buffer overflow unreachable for bounded inputs - let _ = filter_path.join(&[res - .workspace() - .slice(lockfile.buffers.string_bytes.as_slice())]); - - break 'path_pattern (path_pattern, filter_path.slice()); - } - }; - - let result = bun_glob::r#match(pattern, name_or_path); - if result.matches() { - workspace_matched = true; - } else if result.is_negated() { - // always skip if a pattern specifically says "!" - workspace_matched = false; - break; - } - } - - !workspace_matched + !WorkspaceFilter::is_selected(workspace_filters, pkg_id) } // ────────────────────────────────────────────────────────────────────────── diff --git a/src/install/lockfile/bun.lock.rs b/src/install/lockfile/bun.lock.rs index f01b8460724a..576f002e1c15 100644 --- a/src/install/lockfile/bun.lock.rs +++ b/src/install/lockfile/bun.lock.rs @@ -2374,6 +2374,7 @@ pub(crate) fn parse_into_binary_lockfile( None, None, Some(&workspaces_obj), + true, )?; let mut root_pkg = Package::default(); @@ -2442,6 +2443,7 @@ pub(crate) fn parse_into_binary_lockfile( None, None, None, + true, )?; pkg.dependencies = DependencySlice::new(off, len); @@ -2795,6 +2797,7 @@ pub(crate) fn parse_into_binary_lockfile( Some(pkg_path), Some(&bundled_pkgs), None, + res.tag == ResolutionTag::Workspace, )?; pkg.dependencies = DependencySlice::new(off, len); @@ -3334,7 +3337,7 @@ fn deferred_peer_range<'a>( /// so the isolated store's ancestor walk and the hoisted tree's dedupe /// both resolve the name through the root's workspace entry before the /// edge value is ever consulted. -fn resolve_peer_dep_version_based( +pub(crate) fn resolve_peer_dep_version_based( dep: &Dependency, catalogs: &CatalogMap, package_index: &PackageIndexMap, @@ -3503,6 +3506,7 @@ fn parse_append_dependencies pkg_path: Option<&[u8]>, bundled_pkgs: Option<&PkgPathSet>, workspaces_obj: Option<&Expr>, + catalogs_apply: bool, ) -> Result<(u32, u32), ParseError> { // Clearing on entry is equivalent to clearing on every exit path for all // callers (none read the buf between calls) and also covers early-error exits. @@ -3586,7 +3590,8 @@ fn parse_append_dependencies &mut *log, None, ) { - Some(v) => v, + Some(v) if catalogs_apply => v, + Some(v) => CatalogMap::strip_reference(v), None => { log.add_error( Some(source), diff --git a/src/install/lockfile/pruned_workspaces.rs b/src/install/lockfile/pruned_workspaces.rs index 399f58fd460f..9cd8074ddf40 100644 --- a/src/install/lockfile/pruned_workspaces.rs +++ b/src/install/lockfile/pruned_workspaces.rs @@ -1,7 +1,11 @@ +use bstr::BStr; use bun_paths::AutoAbsPath; +use bun_semver::string::Builder as StringBuilderNs; -use crate::PackageNameHash; -use crate::lockfile_real::Lockfile; +use crate::dependency::{Dependency, Tag as DependencyVersionTag, VersionExt as _}; +use crate::lockfile::package::PackageColumns as _; +use crate::lockfile_real::{CatalogMap, Lockfile}; +use crate::{PackageNameHash, ResolutionTag}; pub(crate) fn workspace_is_missing_on_disk( lockfile: &Lockfile, @@ -25,3 +29,163 @@ pub(crate) fn lockfile_lists_workspace_path(lockfile: &Lockfile, workspace_path: .iter() .any(|path| path.slice(string_bytes) == workspace_path) } + +pub(crate) fn exit_if_survivor_depends_on_pruned( + lockfile: &Lockfile, + pruned: &[PackageNameHash], + silent: bool, +) { + let pkgs = lockfile.packages.slice(); + let names = pkgs.items_name(); + let name_hashes = pkgs.items_name_hash(); + let pkg_res = pkgs.items_resolution(); + let dep_slices = pkgs.items_dependencies(); + let res_slices = pkgs.items_resolutions(); + let deps = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + let buf = lockfile.buffers.string_bytes.as_slice(); + + let is_pruned = |id: usize| { + pkg_res[id].tag == ResolutionTag::Workspace && pruned.contains(&name_hashes[id]) + }; + + let mut found = false; + for pkg_id in 0..pkgs.len() { + let tag = pkg_res[pkg_id].tag; + if (tag != ResolutionTag::Root && tag != ResolutionTag::Workspace) || is_pruned(pkg_id) { + continue; + } + for (dep, &target) in dep_slices[pkg_id] + .get(deps) + .iter() + .zip(res_slices[pkg_id].get(resolutions)) + { + if pkg_id == 0 && dep.behavior.is_workspace() { + continue; + } + let target = target as usize; + if target >= pkgs.len() || !is_pruned(target) { + continue; + } + found = true; + if silent { + continue; + } + let target_name = BStr::new(names[target].slice(buf)); + let target_path = BStr::new(pkg_res[target].workspace().slice(buf)); + if pkg_id == 0 { + bun_core::pretty_errorln!( + "error: the root package depends on workspace \"{}\" ({}), which is listed in bun.lock but not on disk", + target_name, + target_path, + ); + } else { + bun_core::pretty_errorln!( + "error: workspace \"{}\" depends on workspace \"{}\" ({}), which is listed in bun.lock but not on disk", + BStr::new(names[pkg_id].slice(buf)), + target_name, + target_path, + ); + } + } + } + + if found { + if !silent { + bun_core::note!( + "a pruned checkout must keep every workspace that its remaining workspaces depend on" + ); + } + bun_core::Global::crash(); + } +} + +pub(crate) fn lockfile_catalogs_are_subset(from: &Lockfile, to: &Lockfile) -> bool { + let from_buf = from.buffers.string_bytes.as_slice(); + let to_buf = to.buffers.string_bytes.as_slice(); + + let mut matched = 0usize; + let all_kept = for_each_entry(&from.catalogs, from_buf, &mut |group, entry| match to + .catalogs + .find(to_buf, group, entry.name.slice(from_buf)) + { + Some(to_entry) if to_entry.version.eql(&entry.version, to_buf, from_buf) => { + matched += 1; + true + } + _ => false, + }); + if !all_kept { + return false; + } + + let to_total = to.catalogs.default.count() + + to.catalogs + .groups + .values() + .iter() + .map(|group| group.count()) + .sum::(); + if matched == to_total { + return true; + } + + for_each_entry(&to.catalogs, to_buf, &mut |group, entry| { + let name = entry.name.slice(to_buf); + from.catalogs.find(from_buf, group, name).is_some() + || !catalog_entry_is_referenced(from, group, StringBuilderNs::string_hash(name)) + }) +} + +fn for_each_entry( + catalogs: &CatalogMap, + buf: &[u8], + f: &mut dyn FnMut(&[u8], &Dependency) -> bool, +) -> bool { + if !catalogs.default.values().iter().all(|entry| f(b"", entry)) { + return false; + } + catalogs + .groups + .keys() + .iter() + .zip(catalogs.groups.values()) + .all(|(group_key, group)| { + let group_name = group_key.slice(buf); + group.values().iter().all(|entry| f(group_name, entry)) + }) +} + +fn catalog_entry_is_referenced( + lockfile: &Lockfile, + group: &[u8], + name_hash: PackageNameHash, +) -> bool { + let buf = lockfile.buffers.string_bytes.as_slice(); + let hits = |dep: &Dependency| { + dep.name_hash == name_hash + && dep.version.tag == DependencyVersionTag::Catalog + && CatalogMap::same_name(dep.version.catalog().slice(buf), group) + }; + + if lockfile.overrides.map.values().iter().any(&hits) + || lockfile.overrides.scoped.iter().any(|rule| hits(&rule.dep)) + { + return true; + } + + let pkgs = lockfile.packages.slice(); + let name_hashes = pkgs.items_name_hash(); + let pkg_res = pkgs.items_resolution(); + let dep_slices = pkgs.items_dependencies(); + let deps = lockfile.buffers.dependencies.as_slice(); + + (0..pkgs.len()).any(|pkg_id| match pkg_res[pkg_id].tag { + ResolutionTag::Root => dep_slices[pkg_id].get(deps).iter().any(&hits), + ResolutionTag::Workspace => { + dep_slices[pkg_id].get(deps).iter().any(&hits) + && !workspace_is_missing_on_disk(lockfile, name_hashes[pkg_id]) + } + _ => false, + }) +} diff --git a/src/install/lockfile/reachable.rs b/src/install/lockfile/reachable.rs index 0fb72ee8b983..c4e86629b8e1 100644 --- a/src/install/lockfile/reachable.rs +++ b/src/install/lockfile/reachable.rs @@ -1,7 +1,11 @@ -use crate::lockfile::package::PackageColumns as _; +use crate::dependency::{Behavior, Dependency}; +use crate::lockfile::DependencySlice; +use crate::lockfile::package::{Meta, PackageColumns as _}; use crate::lockfile_real::Lockfile; use crate::npm::{Architecture, OperatingSystem}; use crate::{PackageID, PackageManager}; +use bun_collections::DynamicBitSet; +use bun_core::UnwrapOrOom; #[derive(Clone, Copy)] pub struct Options { @@ -43,57 +47,161 @@ impl Options { } // `resolutions` is passed separately so callers can walk a candidate resolution buffer (dedupe). -pub fn packages(lockfile: &Lockfile, resolutions: &[PackageID], options: Options) -> Vec { - let pkgs = lockfile.packages.slice(); - let dep_slices = pkgs.items_dependencies(); - let metas = pkgs.items_meta(); - let deps = lockfile.buffers.dependencies.as_slice(); +pub fn packages(lockfile: &Lockfile, resolutions: &[PackageID], options: Options) -> DynamicBitSet { + packages_from( + lockfile, + resolutions, + core::slice::from_ref(&options.root), + true, + options, + ) +} - let mut seen = vec![false; dep_slices.len()]; - if (options.root as usize) >= seen.len() { - return seen; +// `options.root` is ignored; `follow_workspace_edges: false` keeps a root's workspace edges out of the walk (`--filter`). +pub fn packages_from( + lockfile: &Lockfile, + resolutions: &[PackageID], + roots: &[PackageID], + follow_workspace_edges: bool, + options: Options, +) -> DynamicBitSet { + let walk = Walk::new(lockfile, resolutions, follow_workspace_edges, options); + let mut seen = walk.empty_seen(); + let mut worklist: Vec = Vec::new(); + for &root in roots { + if mark(&mut seen, root) { + worklist.push(root); + } } - seen[options.root as usize] = true; - let follow_all = - options.dev && options.optional && options.peer && options.optional_peer && options.bundled; - let mut worklist: Vec = vec![options.root]; - while let Some(parent) = worklist.pop() { - let slice = dep_slices[parent as usize]; + walk.drain(&mut seen, &mut worklist); + seen +} + +// Out-of-range ids (invalid_package_id) are treated as already seen. +fn mark(seen: &mut DynamicBitSet, id: PackageID) -> bool { + let index = id as usize; + if seen.is_set_allow_out_of_bound(index, true) { + return false; + } + seen.set(index); + true +} + +// What the devDependencies of `roots` (and of the workspaces they own) pull in — `bun pm licenses --dev`. +pub fn dev_packages_from( + lockfile: &Lockfile, + resolutions: &[PackageID], + roots: &[PackageID], + follow_workspace_edges: bool, + options: Options, +) -> DynamicBitSet { + let walk = Walk::new(lockfile, resolutions, follow_workspace_edges, options); + let mut seen = walk.empty_seen(); + let mut importers: Vec = Vec::new(); + for &root in roots { + if mark(&mut seen, root) { + importers.push(root); + } + } + let mut worklist: Vec = Vec::new(); + while let Some(importer) = importers.pop() { + let slice = walk.dep_slices[importer as usize]; for dep_id in slice.begin() as usize..slice.end() as usize { - let followed = follow_all || { - let behavior = deps[dep_id].behavior; - if behavior.is_bundled() && !options.bundled { - false - } else if behavior.is_optional_peer() { - options.optional_peer - } else if behavior.is_peer() { - options.peer - } else if behavior.is_optional() { - options.optional - } else if behavior.is_dev() { - options.dev - } else { - true + let behavior = walk.deps[dep_id].behavior; + let target = walk.resolutions[dep_id]; + if behavior.is_workspace() { + if follow_workspace_edges && mark(&mut seen, target) { + importers.push(target); } - }; - if !followed { - continue; - } - let target = resolutions[dep_id]; - let Some(slot) = seen.get_mut(target as usize) else { - continue; - }; - if *slot { continue; } - if let Some((cpu, os)) = options.platform - && metas[target as usize].is_disabled(cpu, os) - { - continue; + if behavior.is_dev() && walk.follows(behavior) { + walk.admit(target, &mut seen, &mut worklist); } - *slot = true; - worklist.push(target); } } + walk.drain(&mut seen, &mut worklist); seen } + +struct Walk<'a> { + dep_slices: &'a [DependencySlice], + metas: &'a [Meta], + deps: &'a [Dependency], + resolutions: &'a [PackageID], + follow_workspace_edges: bool, + follow_all: bool, + options: Options, +} + +impl<'a> Walk<'a> { + fn new( + lockfile: &'a Lockfile, + resolutions: &'a [PackageID], + follow_workspace_edges: bool, + options: Options, + ) -> Walk<'a> { + Walk { + dep_slices: lockfile.packages.items_dependencies(), + metas: lockfile.packages.items_meta(), + deps: lockfile.buffers.dependencies.as_slice(), + resolutions, + follow_workspace_edges, + follow_all: follow_workspace_edges + && options.dev + && options.optional + && options.peer + && options.optional_peer + && options.bundled, + options, + } + } + + fn empty_seen(&self) -> DynamicBitSet { + DynamicBitSet::init_empty(self.dep_slices.len()).unwrap_or_oom() + } + + fn follows(&self, behavior: Behavior) -> bool { + let options = &self.options; + if (!self.follow_workspace_edges && behavior.is_workspace()) + || (behavior.is_bundled() && !options.bundled) + { + false + } else if behavior.is_optional_peer() { + options.optional_peer + } else if behavior.is_peer() { + options.peer + } else if behavior.is_optional() { + options.optional + } else if behavior.is_dev() { + options.dev + } else { + true + } + } + + fn admit(&self, target: PackageID, seen: &mut DynamicBitSet, worklist: &mut Vec) { + if seen.is_set_allow_out_of_bound(target as usize, true) { + return; + } + if let Some((cpu, os)) = self.options.platform + && self.metas[target as usize].is_disabled(cpu, os) + { + return; + } + seen.set(target as usize); + worklist.push(target); + } + + fn drain(&self, seen: &mut DynamicBitSet, worklist: &mut Vec) { + while let Some(parent) = worklist.pop() { + let slice = self.dep_slices[parent as usize]; + for dep_id in slice.begin() as usize..slice.end() as usize { + if !(self.follow_all || self.follows(self.deps[dep_id].behavior)) { + continue; + } + self.admit(self.resolutions[dep_id], seen, worklist); + } + } + } +} diff --git a/src/install/pnpm.rs b/src/install/pnpm.rs index 6a8829341baa..1adf8dc0fa67 100644 --- a/src/install/pnpm.rs +++ b/src/install/pnpm.rs @@ -127,6 +127,21 @@ fn write_pnpm_dep_path( .map_err(|_| AllocError) } +/// Binds a peer edge the way bun.lock's reader does, so the migrated tree matches the reloaded one. +fn resolve_peer_like_bun_lock(lockfile: &Lockfile, dep: &Dependency) -> Option { + if !dep.behavior.is_peer() { + return None; + } + lockfile::bun_lock::resolve_peer_dep_version_based( + dep, + &lockfile.catalogs, + &lockfile.package_index, + &lockfile.overrides, + lockfile.packages.items_resolution(), + string_bytes!(lockfile), + ) +} + fn missing_package_entry( log: &mut bun_ast::Log, dep_path: &[u8], @@ -201,6 +216,48 @@ fn read_config_patch_paths( Ok(paths) } +/// pnpm 11's built-in named registries; `namedRegistries` in pnpm-workspace.yaml overrides them. +const BUILTIN_NAMED_REGISTRIES: [(&[u8], &[u8]); 2] = [ + (b"gh", b"https://npm.pkg.github.com/"), + (b"npmjs", b"https://registry.npmjs.org/"), +]; + +fn read_named_registries( + log: &mut bun_ast::Log, +) -> Result>, AllocError> { + let mut registries: StringArrayHashMap> = StringArrayHashMap::new(); + + if let Ok(contents) = sys::File::read_from(Fd::cwd(), b"pnpm-workspace.yaml") { + let contents: &'static [u8] = js_ast::data_store_dupe_str(&contents); + let source = bun_ast::Source::init_path_string(b"pnpm-workspace.yaml", contents); + let arena = bun_alloc::Arena::new(); + if let Ok(ws_root) = bun_parsers::yaml::YAML::parse( + &source, + log, + &arena, + bun_parsers::yaml::CyclicAliases::Reject, + ) { + if let Some(named) = ws_root.get_object(b"namedRegistries") { + for prop in e_object(&named).properties.slice() { + let key = prop.key.as_ref().expect("infallible: prop has key"); + let value = prop.value.as_ref().expect("infallible: prop has value"); + if let (Some(name_str), Some(url_str)) = (as_string(key), as_string(value)) { + registries.put(name_str, Box::from(url_str))?; + } + } + } + } + } + + for (name, url) in BUILTIN_NAMED_REGISTRIES { + if !registries.contains(name) { + registries.put(name, Box::from(url))?; + } + } + + Ok(registries) +} + /// `work:1.0.0` -> `1.0.0` for pnpm's registry-qualified dep paths (pnpm11/deps/path parseRegistryQualifiedVersion). fn split_registry_qualified_version(res_str: &[u8]) -> Option<(&[u8], &[u8])> { let colon = strings::index_of_char_usize(res_str, b':')?; @@ -526,8 +583,16 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( ); } + let exclude_links = root + .get(b"settings") + .and_then(|settings| settings.get(b"excludeLinksFromLockfile")) + .and_then(|e| e.as_bool()) + == Some(true); + let mut found_patches: StringArrayHashMap> = StringArrayHashMap::new(); let mut snapshot_dep_paths = SnapshotDepPaths::new(); + let mut named_registries: Option>> = None; + let mut warned_registries: StringArrayHashMap<()> = StringArrayHashMap::new(); let (pkg_map, importer_dep_res_versions, workspace_pkgs_off, workspace_pkgs_end) = 'build: { if let Some(mut catalogs_expr) = root.get_object(b"catalogs") { @@ -774,9 +839,11 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( Err(_) => return Err(invalid_pnpm_lockfile()), }; + let root_manifest: Expr = pkg_json.root; + let mut root_pkg = lockfile::Package::default(); - if let Some((name, _)) = get_string(&pkg_json.root, b"name") { + if let Some((name, _)) = get_string(&root_manifest, b"name") { let name_hash = semver::string::Builder::string_hash(name); root_pkg.name = sbuf!(lockfile).append_with_hash(name, name_hash)?; root_pkg.name_hash = name_hash; @@ -789,6 +856,9 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( lockfile, manager, &root_pkg_expr, + &root_manifest, + b".", + exclude_links, log, true, &importers_obj, @@ -865,6 +935,9 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( lockfile, manager, value, + &workspace_root, + path, + exclude_links, log, false, &importers_obj, @@ -940,6 +1013,9 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( let Some(version_str) = importer_versions.get(dep.name.slice(string_bytes!(lockfile))) else { + if dep.behavior.is_peer() { + continue; + } return Err(invalid_pnpm_lockfile()); }; let version_without_suffix = remove_suffix(version_str); @@ -1021,36 +1097,27 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( } } - struct SnapshotEntry { - obj: Expr, - patch_hash: Option<&'static [u8]>, - } - impl Default for SnapshotEntry { - fn default() -> Self { - Self { - obj: Expr::EMPTY, - patch_hash: None, - } - } + let packages_obj = root.get_object(b"packages"); + let snapshots_obj = root.get_object(b"snapshots"); + + if packages_obj.is_some() && snapshots_obj.is_none() { + log.add_error( + None, + bun_ast::Loc::EMPTY, + b"pnpm-lock.yaml has 'packages' but missing 'snapshots' field", + ); + return Err(MigratePnpmLockfileError::PnpmLockfileInvalidSnapshot); } - let mut snapshots: StringArrayHashMap = StringArrayHashMap::new(); - if let Some(packages_obj) = root.get_object(b"packages") { - let Some(snapshots_obj) = root.get_object(b"snapshots") else { - log.add_error( - None, - bun_ast::Loc::EMPTY, - b"pnpm-lock.yaml has 'packages' but missing 'snapshots' field", - ); - return Err(MigratePnpmLockfileError::PnpmLockfileInvalidSnapshot); - }; + let mut packages_by_key: StringArrayHashMap = StringArrayHashMap::new(); - for snapshot_prop in e_object(&snapshots_obj).properties.slice() { - let key = snapshot_prop + if let Some(packages_obj) = &packages_obj { + for packages_prop in e_object(packages_obj).properties.slice() { + let key = packages_prop .key .as_ref() .expect("infallible: prop has key"); - let value = snapshot_prop + let package_obj = packages_prop .value .as_ref() .expect("infallible: prop has value"); @@ -1059,47 +1126,26 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( return Err(invalid_pnpm_lockfile()); }; - if !value.is_object() { + if !package_obj.is_object() { return Err(invalid_pnpm_lockfile()); } - let (peer_hash_idx, patch_hash_idx) = index_of_dep_path_suffix(key_str); - - let key_str_without_suffix = if let Some(idx) = patch_hash_idx.or(peer_hash_idx) { - &key_str[0..idx] - } else { - key_str - }; - - let patch_hash = match patch_hash_idx { - Some(idx) => { - let patch_hash_str = &key_str[idx + b"(patch_hash=".len()..]; - let Some(end_idx) = strings::index_of_char_usize(patch_hash_str, b')') - else { - return Err(invalid_pnpm_lockfile()); - }; - Some(&patch_hash_str[..end_idx]) - } - None => None, - }; - - let entry = snapshots.get_or_put(key_str_without_suffix)?; + // Pruned lockfiles (`turbo prune`) can leave peer-suffixed keys in `packages:`. + let entry = packages_by_key.get_or_put(remove_suffix(key_str))?; if entry.found_existing { continue; } - - *entry.value_ptr = SnapshotEntry { - obj: *value, - patch_hash, - }; + *entry.value_ptr = *package_obj; } + } - for packages_prop in e_object(&packages_obj).properties.slice() { - let key = packages_prop + if let Some(snapshots_obj) = &snapshots_obj { + for snapshot_prop in e_object(snapshots_obj).properties.slice() { + let key = snapshot_prop .key .as_ref() .expect("infallible: prop has key"); - let package_obj = packages_prop + let snapshot_obj = snapshot_prop .value .as_ref() .expect("infallible: prop has value"); @@ -1108,23 +1154,33 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( return Err(invalid_pnpm_lockfile()); }; - if !package_obj.is_object() { + if !snapshot_obj.is_object() { return Err(invalid_pnpm_lockfile()); } - // Pruned lockfiles (`turbo prune`) can leave peer-suffixed keys in `packages:`. - let key_str = remove_suffix(key_str); + let (peer_hash_idx, patch_hash_idx) = index_of_dep_path_suffix(key_str); + + let patch_hash = match patch_hash_idx { + Some(idx) => { + let patch_hash_str = &key_str[idx + b"(patch_hash=".len()..]; + let Some(end_idx) = strings::index_of_char_usize(patch_hash_str, b')') + else { + return Err(invalid_pnpm_lockfile()); + }; + Some(&patch_hash_str[..end_idx]) + } + None => None, + }; + + let key_str = match patch_hash_idx.or(peer_hash_idx) { + Some(idx) => &key_str[0..idx], + None => key_str, + }; + if pkg_map.contains(key_str) { continue; } - // Like pnpm, a `packages:` entry without a snapshot is unreachable and ignored. - let Some(snapshot) = snapshots.get(key_str) else { - continue; - }; - let snapshot_obj = snapshot.obj; - let snapshot_patch_hash = snapshot.patch_hash; - let Ok((name_str, res_str)) = dependency::split_name_and_version(key_str) else { return Err(invalid_pnpm_lockfile()); }; @@ -1133,20 +1189,25 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( continue; } - let res_str = match split_registry_qualified_version(res_str) { - Some((registry, version)) => { - log.add_warning_fmt( - None, - bun_ast::Loc::EMPTY, - format_args!( - "pnpm-lock.yaml package '{}' is from pnpm registry '{}', resolving it from the configured registry instead", - bstr::BStr::new(key_str), - bstr::BStr::new(registry) - ), - ); - version + let package_obj: Expr = match packages_by_key.get(key_str) { + Some(obj) => *obj, + None => { + // Like pnpm, only a `file:` directory is rebuilt from a snapshot whose `packages:` entry was pruned. + let Some(dir) = + strings::without_prefix_if_possible_comptime(res_str, b"file:") + else { + continue; + }; + if Dependency::is_tarball(dir) { + continue; + } + Expr::EMPTY } - None => res_str, + }; + + let (res_str, registry_name) = match split_registry_qualified_version(res_str) { + Some((registry, version)) => (version, Some(registry)), + None => (res_str, None), }; let resolution_expr: Option = package_obj.get(b"resolution"); @@ -1196,11 +1257,7 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( .copied() .filter(|id| (*id as usize) < workspace_pkgs_end) { - let entry = pkg_map.get_or_put(key_str)?; - if entry.found_existing { - return Err(invalid_pnpm_lockfile()); - } - *entry.value_ptr = workspace_pkg_id; + pkg_map.put(key_str, workspace_pkg_id)?; continue; } } @@ -1208,7 +1265,7 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( let name_hash = semver::string::Builder::string_hash(name_str); let name = sbuf!(lockfile).append_with_hash(name_str, name_hash)?; - if let Some(patch_list) = snapshot_patch_hash.and_then(|hash| patches.get(hash)) { + if let Some(patch_list) = patch_hash.and_then(|hash| patches.get(hash)) { if let Some(patch) = patch_list.iter().find(|patch| { Dependency::split_name_and_maybe_version(&patch.key).0 == name_str }) { @@ -1237,7 +1294,59 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( } if res.tag == resolution::Tag::Npm { - let registry = manager.scope_for_package_name(name_str).url.href(); + let scope_registry: &[u8] = manager.scope_for_package_name(name_str).url.href(); + let registry: &[u8] = match registry_name { + None => scope_registry, + Some(registry_name) => { + if named_registries.is_none() { + named_registries = Some(read_named_registries(log)?); + } + match named_registries + .as_ref() + .expect("set above") + .get(registry_name) + { + Some(url) + if !(lockfile::bun_lock::url_is_under_registry( + url, + scope_registry, + ) && lockfile::bun_lock::url_is_under_registry( + scope_registry, + url, + )) => + { + if !warned_registries.get_or_put(registry_name)?.found_existing + { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml packages from pnpm registry '{}' will be fetched from {}; add that registry to bunfig.toml or .npmrc if it needs authentication", + bstr::BStr::new(registry_name), + bstr::BStr::new(url) + ), + ); + } + &**url + } + Some(_) => scope_registry, + None => { + if !warned_registries.get_or_put(registry_name)?.found_existing + { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml packages from pnpm registry '{}' are not in namedRegistries of pnpm-workspace.yaml, resolving them from the configured registry instead", + bstr::BStr::new(registry_name) + ), + ); + } + scope_registry + } + } + } + }; // Registries like GitHub Packages serve tarballs off the canonical `/-/` path (pnpm/pnpm#13534). let recorded = resolution_expr .as_ref() @@ -1286,8 +1395,8 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( let (off, len) = parse_append_package_dependencies( lockfile, - package_obj, - &snapshot_obj, + &package_obj, + snapshot_obj, log, &mut snapshot_dep_paths, )?; @@ -1298,12 +1407,7 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( let pkg_id = lockfile.append_package_dedupe(&mut pkg)?; - let entry = pkg_map.get_or_put(key_str)?; - if entry.found_existing { - return Err(invalid_pnpm_lockfile()); - } - - *entry.value_ptr = pkg_id; + pkg_map.put(key_str, pkg_id)?; } } @@ -1352,8 +1456,15 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( } let dep_name = dep.name.slice(string_buf); + if let Some(peer_pkg_id) = resolve_peer_like_bun_lock(lockfile, &dep) { + lockfile.buffers.resolutions[dep_id as usize] = peer_pkg_id; + continue; + } let Some(mut version_maybe_alias) = importer_versions.get(dep_name).map(|v| &**v) else { + if dep.behavior.is_peer() { + continue; + } log.add_error_fmt( None, bun_ast::Loc::EMPTY, @@ -1412,8 +1523,15 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( let dep = lockfile.buffers.dependencies[dep_id as usize].clone(); let string_buf = string_bytes!(lockfile); let dep_name = dep.name.slice(string_buf); + if let Some(peer_pkg_id) = resolve_peer_like_bun_lock(lockfile, &dep) { + lockfile.buffers.resolutions[dep_id as usize] = peer_pkg_id; + continue; + } let Some(mut version_maybe_alias) = importer_versions.get(dep_name).map(|v| &**v) else { + if dep.behavior.is_peer() { + continue; + } log.add_error_fmt( None, bun_ast::Loc::EMPTY, @@ -1465,15 +1583,20 @@ pub(crate) fn migrate_pnpm_lockfile<'a>( let dep = lockfile.buffers.dependencies[dep_id as usize].clone(); let string_buf = string_bytes!(lockfile); let dep_name = dep.name.slice(string_buf); + if let Some(peer_pkg_id) = resolve_peer_like_bun_lock(lockfile, &dep) { + lockfile.buffers.resolutions[dep_id as usize] = peer_pkg_id; + continue; + } let mut version_maybe_alias = dep.version.literal.slice(string_buf); if strings::has_prefix(version_maybe_alias, b"npm:") { version_maybe_alias = &version_maybe_alias[b"npm:".len()..]; } let reference = remove_suffix(version_maybe_alias); - if let Some(dep_path) = snapshot_dep_paths.get(&dep_id) { - res_buf.clear(); - res_buf.extend_from_slice(dep_path); + if let Some(snapshot_reference) = snapshot_dep_paths.get(&dep_id) { + write_pnpm_dep_path(&mut res_buf, dep_name, snapshot_reference)?; + } else if dep.behavior.is_peer() { + continue; } else { match dep.version.tag { dependency::VersionTag::Folder @@ -1564,20 +1687,20 @@ impl From for MigratePnpmLockfileError { } } -/// dep -> full pnpm dep-path for aliases whose version isn't a registry version (`cfg: hi2@file:x` has no `npm:` spelling) +/// dep -> pnpm reference whose dep-path cannot be rebuilt from the dep's literal (non-registry aliases and peer edges) type SnapshotDepPaths = bun_collections::HashMap>; fn append_snapshot_dependency_version( lockfile: &mut Lockfile, reference: &[u8], version_buf: &mut Vec, - aliased_dep_paths: &mut StringArrayHashMap>, + references_by_name: &mut StringArrayHashMap>, dep_name: &[u8], ) -> Result<(String, Option), AllocError> { if pnpm_reference_is_dep_path(reference) { if let Ok((alias_str, version_str)) = dependency::split_name_and_version(reference) { if !version_str.first().is_some_and(u8::is_ascii_digit) { - aliased_dep_paths.put(dep_name, Box::from(reference))?; + references_by_name.put(dep_name, Box::from(reference))?; return Ok((sbuf!(lockfile).append(version_str)?, None)); } let alias = sbuf!(lockfile).append_external(alias_str)?; @@ -1590,6 +1713,69 @@ fn append_snapshot_dependency_version( Ok((sbuf!(lockfile).append(reference)?, None)) } +struct PeerDecl { + range: &'static [u8], + optional: bool, + seen: bool, +} + +fn declared_package_peers( + package_obj: &Expr, +) -> Result, ParseAppendDependenciesError> { + let mut peers: StringArrayHashMap = StringArrayHashMap::new(); + + if let Some(declared) = package_obj.get(b"peerDependencies") { + if !declared.is_object() { + return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); + } + for prop in e_object(&declared).properties.slice() { + let key = prop.key.as_ref().expect("infallible: prop has key"); + let value = prop.value.as_ref().expect("infallible: prop has value"); + let (Some(name_str), Some(range)) = (as_string(key), as_string(value)) else { + return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); + }; + peers.put( + name_str, + PeerDecl { + range, + optional: false, + seen: false, + }, + )?; + } + } + + if let Some(meta) = package_obj.get(b"peerDependenciesMeta") { + if !meta.is_object() { + return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); + } + for prop in e_object(&meta).properties.slice() { + let key = prop.key.as_ref().expect("infallible: prop has key"); + let value = prop.value.as_ref().expect("infallible: prop has value"); + let Some(name_str) = as_string(key) else { + return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); + }; + if !value.is_object() || value.get(b"optional").and_then(|e| e.as_bool()) != Some(true) + { + continue; + } + match peers.get_mut(name_str) { + Some(decl) => decl.optional = true, + None => peers.put( + name_str, + PeerDecl { + range: b"*", + optional: true, + seen: false, + }, + )?, + } + } + } + + Ok(peers) +} + fn parse_append_package_dependencies( lockfile: &mut Lockfile, package_obj: &Expr, @@ -1598,78 +1784,27 @@ fn parse_append_package_dependencies( snapshot_dep_paths: &mut SnapshotDepPaths, ) -> Result<(u32, u32), ParseAppendDependenciesError> { let mut version_buf: Vec = Vec::new(); - let mut aliased_dep_paths: StringArrayHashMap> = StringArrayHashMap::new(); + let mut references_by_name: StringArrayHashMap> = StringArrayHashMap::new(); + let mut peers = declared_package_peers(package_obj)?; let off = lockfile.buffers.dependencies.len(); - const SNAPSHOT_DEPENDENCY_GROUPS: [(&[u8], dependency::Behavior); 2] = [ + // pnpm records resolved required peers under `dependencies` and resolved optional peers under `optionalDependencies`. + const SNAPSHOT_DEPENDENCY_GROUPS: [(&[u8], dependency::Behavior); 3] = [ + (b"dependencies", dependency::Behavior::PROD), (b"devDependencies", dependency::Behavior::DEV), (b"optionalDependencies", dependency::Behavior::OPTIONAL), ]; for (group_name, group_behavior) in SNAPSHOT_DEPENDENCY_GROUPS { - if let Some(deps) = snapshot_obj.get(group_name) { - if !deps.is_object() { - return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); - } - - for prop in e_object(&deps).properties.slice() { - let key = prop.key.as_ref().expect("infallible: prop has key"); - let value = prop.value.as_ref().expect("infallible: prop has value"); - - let Some(name_str) = as_string(key) else { - return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); - }; - - let name_hash = semver::string::Builder::string_hash(name_str); - let name = sbuf!(lockfile).append_external_with_hash(name_str, name_hash)?; - - let Some(version_str) = as_string(value) else { - return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); - }; - - let version_without_suffix = remove_suffix(version_str); - - let (version, alias) = append_snapshot_dependency_version( - lockfile, - version_without_suffix, - &mut version_buf, - &mut aliased_dep_paths, - name_str, - )?; - let version_sliced = version.sliced(string_bytes!(lockfile)); - - let behavior: dependency::Behavior = group_behavior; - - let dep = Dependency { - name: name.value, - name_hash, - behavior, - version: match Dependency::parse( - alias.map(|a| a.value).unwrap_or(name.value), - alias.map(|a| a.hash).unwrap_or(name.hash), - version_sliced.slice, - &version_sliced, - Some(&mut *log), - None, - ) { - Some(v) => v, - None => return Err(ParseAppendDependenciesError::InvalidPnpmLockfile), - }, - }; - - lockfile.buffers.dependencies.push(dep); - } - } - } - - if let Some(deps) = snapshot_obj.get(b"dependencies") { + let Some(deps) = snapshot_obj.get(group_name) else { + continue; + }; if !deps.is_object() { return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); } - // for each dependency first look it up in peerDependencies in package_obj - 'next_prod_dep: for prop in e_object(&deps).properties.slice() { + for prop in e_object(&deps).properties.slice() { let key = prop.key.as_ref().expect("infallible: prop has key"); let value = prop.value.as_ref().expect("infallible: prop has value"); @@ -1684,97 +1819,52 @@ fn parse_append_package_dependencies( return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); }; - let version_without_suffix = remove_suffix(version_str); + let reference = remove_suffix(version_str); + + let mut behavior: dependency::Behavior = group_behavior; + + if let Some(decl) = peers.get_mut(name_str) { + if !decl.seen { + decl.seen = true; + if !strings::has_prefix_comptime(reference, b"link:") { + behavior = dependency::Behavior::PEER; + behavior.set_optional(decl.optional); + let range = sbuf!(lockfile).append(decl.range)?; + let range_sliced = range.sliced(string_bytes!(lockfile)); + references_by_name.put(name_str, Box::from(reference))?; + if let Some(version) = Dependency::parse( + name.value, + name.hash, + range_sliced.slice, + &range_sliced, + None, + None, + ) { + lockfile.buffers.dependencies.push(Dependency { + name: name.value, + name_hash: name.hash, + behavior, + version, + }); + continue; + } + } + } + } let (version, alias) = append_snapshot_dependency_version( lockfile, - version_without_suffix, + reference, &mut version_buf, - &mut aliased_dep_paths, + &mut references_by_name, name_str, )?; let version_sliced = version.sliced(string_bytes!(lockfile)); - if let Some(peers) = package_obj.get(b"peerDependencies") { - if !peers.is_object() { - return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); - } - - for peer_prop in e_object(&peers).properties.slice() { - let Some(peer_name_str) = - as_string(peer_prop.key.as_ref().expect("infallible: prop has key")) - else { - return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); - }; - - let mut behavior = dependency::Behavior::PEER; - - if strings::eql_long(name_str, peer_name_str, true) { - if let Some(peers_meta) = package_obj.get(b"peerDependenciesMeta") { - if !peers_meta.is_object() { - return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); - } - - for peer_meta_prop in e_object(&peers_meta).properties.slice() { - let Some(peer_meta_name_str) = as_string( - peer_meta_prop - .key - .as_ref() - .expect("infallible: prop has key"), - ) else { - return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); - }; - - if strings::eql_long(name_str, peer_meta_name_str, true) { - let meta_obj = peer_meta_prop - .value - .as_ref() - .expect("infallible: prop has value"); - if !meta_obj.is_object() { - return Err( - ParseAppendDependenciesError::InvalidPnpmLockfile, - ); - } - - behavior.set_optional( - meta_obj - .get(b"optional") - .and_then(|e| e.as_bool()) - .unwrap_or(false), - ); - break; - } - } - } - let dep = Dependency { - name: name.value, - name_hash: name.hash, - behavior, - version: match Dependency::parse( - alias.map(|a| a.value).unwrap_or(name.value), - alias.map(|a| a.hash).unwrap_or(name.hash), - version_sliced.slice, - &version_sliced, - Some(&mut *log), - None, - ) { - Some(v) => v, - None => { - return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); - } - }, - }; - - lockfile.buffers.dependencies.push(dep); - continue 'next_prod_dep; - } - } - } - let dep = Dependency { name: name.value, name_hash: name.hash, - behavior: dependency::Behavior::PROD, + behavior, version: match Dependency::parse( alias.map(|a| a.value).unwrap_or(name.value), alias.map(|a| a.hash).unwrap_or(name.hash), @@ -1792,6 +1882,35 @@ fn parse_append_package_dependencies( } } + for (peer_name, decl) in peers.iter() { + if decl.seen { + continue; + } + let peer_name: &[u8] = peer_name; + let name_hash = semver::string::Builder::string_hash(peer_name); + let name = sbuf!(lockfile).append_external_with_hash(peer_name, name_hash)?; + let range = sbuf!(lockfile).append(decl.range)?; + let range_sliced = range.sliced(string_bytes!(lockfile)); + let Some(version) = Dependency::parse( + name.value, + name.hash, + range_sliced.slice, + &range_sliced, + None, + None, + ) else { + continue; + }; + let mut behavior = dependency::Behavior::PEER; + behavior.set_optional(decl.optional); + lockfile.buffers.dependencies.push(Dependency { + name: name.value, + name_hash: name.hash, + behavior, + version, + }); + } + let end = lockfile.buffers.dependencies.len(); { @@ -1799,12 +1918,12 @@ fn parse_append_package_dependencies( lockfile.buffers.dependencies[off..].sort_by(|a, b| Dependency::cmp(bytes, a, b)); } - if aliased_dep_paths.count() > 0 { + if references_by_name.count() > 0 { let bytes = lockfile.buffers.string_bytes.as_slice(); for (i, dep) in lockfile.buffers.dependencies[off..end].iter().enumerate() { - if let Some(dep_path) = aliased_dep_paths.get(dep.name.slice(bytes)) { + if let Some(reference) = references_by_name.get(dep.name.slice(bytes)) { let dep_id = u32::try_from(off + i).expect("int cast"); - snapshot_dep_paths.put(dep_id, dep_path.clone())?; + snapshot_dep_paths.put(dep_id, reference.clone())?; } } } @@ -1815,20 +1934,124 @@ fn parse_append_package_dependencies( )) } +fn append_importer_dependency( + lockfile: &mut Lockfile, + log: &mut bun_ast::Log, + name_str: &[u8], + specifier_str: &[u8], + behavior: dependency::Behavior, +) -> Result<(), ParseAppendDependenciesError> { + let name_hash = semver::string::Builder::string_hash(name_str); + let name = sbuf!(lockfile).append_external_with_hash(name_str, name_hash)?; + + if strings::has_prefix(specifier_str, b"catalog:") { + let mut catalog_group_name_str = specifier_str[b"catalog:".len()..].trim_ascii(); + if catalog_group_name_str == b"default" { + catalog_group_name_str = b""; + } + let catalog_group_name = sbuf!(lockfile).append(catalog_group_name_str)?; + // `CatalogMap::get` borrows `&self` and the whole lockfile, so move catalogs out for the call. + let catalogs = core::mem::take(&mut lockfile.catalogs); + let dep_result = catalogs.get(lockfile, catalog_group_name, name.value); + lockfile.catalogs = catalogs; + let Some(mut dep) = dep_result else { + // catalog is missing an entry in the "catalogs" object in the lockfile + log.add_error_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml catalog '{}' missing entry for dependency '{}'", + bstr::BStr::new(specifier_str[b"catalog:".len()..].trim_ascii()), + bstr::BStr::new(name_str) + ), + ); + return Err(ParseAppendDependenciesError::PnpmLockfileMissingCatalogEntry); + }; + + dep.behavior = behavior; + + lockfile.buffers.dependencies.push(dep); + return Ok(()); + } + + let specifier = sbuf!(lockfile).append(specifier_str)?; + let specifier_sliced = specifier.sliced(string_bytes!(lockfile)); + + let dep = Dependency { + name: name.value, + name_hash: name.hash, + behavior, + version: match Dependency::parse( + name.value, + name.hash, + specifier_sliced.slice, + &specifier_sliced, + Some(&mut *log), + None, + ) { + Some(v) => v, + None => return Err(ParseAppendDependenciesError::InvalidPnpmLockfile), + }, + }; + + lockfile.buffers.dependencies.push(dep); + Ok(()) +} + +const IMPORTER_DEPENDENCY_GROUPS: [(&[u8], dependency::Behavior); 3] = [ + (b"dependencies", dependency::Behavior::PROD), + (b"devDependencies", dependency::Behavior::DEV), + (b"optionalDependencies", dependency::Behavior::OPTIONAL), +]; + +fn collect_manifest_peers( + manifest: &Expr, +) -> Result, AllocError> { + let mut peers: StringArrayHashMap<(&'static [u8], bool)> = StringArrayHashMap::new(); + + if let Some(declared) = manifest.get_object(b"peerDependencies") { + for prop in e_object(&declared).properties.slice() { + let key = prop.key.as_ref().expect("infallible: prop has key"); + let value = prop.value.as_ref().expect("infallible: prop has value"); + if let (Some(name_str), Some(range)) = (as_string(key), as_string(value)) { + peers.put(name_str, (range, false))?; + } + } + } + + if let Some(meta) = manifest.get_object(b"peerDependenciesMeta") { + for prop in e_object(&meta).properties.slice() { + let key = prop.key.as_ref().expect("infallible: prop has key"); + let value = prop.value.as_ref().expect("infallible: prop has value"); + let Some(name_str) = as_string(key) else { + continue; + }; + if value.get(b"optional").and_then(|e| e.as_bool()) != Some(true) { + continue; + } + match peers.get_mut(name_str) { + Some(entry) => entry.1 = true, + None => peers.put(name_str, (b"*", true))?, + } + } + } + + Ok(peers) +} + fn parse_append_importer_dependencies( lockfile: &mut Lockfile, manager: &mut PackageManager, pkg_expr: &Expr, + manifest: &Expr, + importer_path: &[u8], + exclude_links_from_lockfile: bool, log: &mut bun_ast::Log, is_root: bool, importers_obj: &Expr, importer_versions: &mut StringArrayHashMap>, ) -> Result<(u32, u32), ParseAppendDependenciesError> { - const IMPORTER_DEPENDENCY_GROUPS: [(&[u8], dependency::Behavior); 3] = [ - (b"dependencies", dependency::Behavior::PROD), - (b"devDependencies", dependency::Behavior::DEV), - (b"optionalDependencies", dependency::Behavior::OPTIONAL), - ]; + let manifest_peers = collect_manifest_peers(manifest)?; let off = lockfile.buffers.dependencies.len(); @@ -1846,9 +2069,6 @@ fn parse_append_importer_dependencies( return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); }; - let name_hash = semver::string::Builder::string_hash(name_str); - let name = sbuf!(lockfile).append_external_with_hash(name_str, name_hash)?; - let Some(specifier_expr) = value.get(b"specifier") else { log.add_error_fmt( None, @@ -1896,67 +2116,68 @@ fn parse_append_importer_dependencies( } *entry.value_ptr = Box::from(remove_suffix(version_str)); + // pnpm records an importer's auto-installed peers under `dependencies`; the peer edge comes from package.json below. + if manifest_peers.count() > 0 + && manifest_peers.contains(name_str) + && manifest + .get(group_name) + .is_none_or(|group| group.get(name_str).is_none()) + { + continue; + } + let Some(specifier_str) = as_string(&specifier_expr) else { return Err(ParseAppendDependenciesError::InvalidPnpmLockfile); }; - if strings::has_prefix(specifier_str, b"catalog:") { - let mut catalog_group_name_str = - specifier_str[b"catalog:".len()..].trim_ascii(); - if catalog_group_name_str == b"default" { - catalog_group_name_str = b""; - } - let catalog_group_name = sbuf!(lockfile).append(catalog_group_name_str)?; - // `CatalogMap::get` needs both `&mut self.catalogs` and - // `&self`; temporarily move catalogs out so the disjoint - // fields can be borrowed. - let catalogs = core::mem::take(&mut lockfile.catalogs); - let dep_result = catalogs.get(lockfile, catalog_group_name, name.value); - lockfile.catalogs = catalogs; - let Some(mut dep) = dep_result else { - // catalog is missing an entry in the "catalogs" object in the lockfile - log.add_error_fmt( - None, - bun_ast::Loc::EMPTY, - format_args!( - "pnpm-lock.yaml catalog '{}' missing entry for dependency '{}'", - bstr::BStr::new(specifier_str[b"catalog:".len()..].trim_ascii()), - bstr::BStr::new(name_str) - ), - ); - return Err(ParseAppendDependenciesError::PnpmLockfileMissingCatalogEntry); - }; + append_importer_dependency(lockfile, log, name_str, specifier_str, group_behavior)?; + } + } + } - dep.behavior = group_behavior; + for (peer_name, (range, optional)) in manifest_peers.iter() { + let peer_name: &[u8] = peer_name; + if !*optional && !importer_versions.contains(peer_name) { + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml does not record peer dependency '{}' of importer '{}'; the next bun install will resolve it", + bstr::BStr::new(peer_name), + bstr::BStr::new(importer_path) + ), + ); + } + let mut behavior = dependency::Behavior::PEER; + behavior.set_optional(*optional); + append_importer_dependency(lockfile, log, peer_name, *range, behavior)?; + } - lockfile.buffers.dependencies.push(dep); + if exclude_links_from_lockfile { + for (group_name, _) in IMPORTER_DEPENDENCY_GROUPS { + let Some(group) = manifest.get_object(group_name) else { + continue; + }; + for prop in e_object(&group).properties.slice() { + let key = prop.key.as_ref().expect("infallible: prop has key"); + let value = prop.value.as_ref().expect("infallible: prop has value"); + let (Some(name_str), Some(spec)) = (as_string(key), as_string(value)) else { continue; - } - - let specifier = sbuf!(lockfile).append(specifier_str)?; - let specifier_sliced = specifier.sliced(string_bytes!(lockfile)); - - let behavior: dependency::Behavior = group_behavior; - - // TODO: find peerDependencies from package.json - let dep = Dependency { - name: name.value, - name_hash: name.hash, - behavior, - version: match Dependency::parse( - name.value, - name.hash, - specifier_sliced.slice, - &specifier_sliced, - Some(&mut *log), - None, - ) { - Some(v) => v, - None => return Err(ParseAppendDependenciesError::InvalidPnpmLockfile), - }, }; - - lockfile.buffers.dependencies.push(dep); + if !strings::has_prefix_comptime(spec, b"link:") + || importer_versions.contains(name_str) + { + continue; + } + log.add_warning_fmt( + None, + bun_ast::Loc::EMPTY, + format_args!( + "pnpm-lock.yaml omits linked dependency '{}' of importer '{}' (excludeLinksFromLockfile); it is not migrated", + bstr::BStr::new(name_str), + bstr::BStr::new(importer_path) + ), + ); } } } diff --git a/src/install/prune.rs b/src/install/prune.rs index 9ff7c4a610b0..1617eeca2eed 100644 --- a/src/install/prune.rs +++ b/src/install/prune.rs @@ -3,7 +3,8 @@ use core::ops::Range; use std::io::Write as _; use bstr::BStr; -use bun_core::{Global, Output, ZStr, strings}; +use bun_collections::DynamicBitSet; +use bun_core::{Global, Output, ZStr, handle_oom, strings}; use bun_install_types::NodeLinker::NodeLinker; use bun_paths::SEP; use bun_sys::{self as sys, Dir, E, EntryKind, O}; @@ -12,8 +13,10 @@ use crate::config_version::ConfigVersion; use crate::lockfile::package::PackageColumns as _; use crate::lockfile::tree::is_filtered_dependency_or_workspace; use crate::lockfile::{LoadResult, Lockfile, reachable, tree}; -use crate::package_manager::Options::LogLevel; -use crate::{PackageID, PackageManager, ResolutionTag, invalid_package_id}; +use crate::lockfile_real::package::{Diff, Package}; +use crate::package_manager::Options::{Enable, LogLevel}; +use crate::package_manager::{ROOT_PACKAGE_JSON_PATH, WorkspaceFilter}; +use crate::{Features, PackageID, PackageManager, ResolutionTag, invalid_package_id}; const STORE_DIR: &[u8] = b"node_modules/.bun"; @@ -123,7 +126,7 @@ fn plural(n: usize) -> &'static str { if n == 1 { "" } else { "s" } } -pub fn prune(manager: &mut PackageManager) -> crate::Result<()> { +pub fn prune(manager: &mut PackageManager, original_cwd: &[u8]) -> crate::Result<()> { let quiet = manager.options.log_level == LogLevel::Silent; let dry_run = manager.options.dry_run; @@ -147,16 +150,14 @@ pub fn prune(manager: &mut PackageManager) -> crate::Result<()> { Err(Some(name)) => { if !quiet { Output::err_generic("failed to load lockfile: {s}", (name,)); - if manager.log_mut().has_errors() { - let _ = manager - .log_mut() - .print(core::ptr::from_mut(Output::error_writer())); - } + print_log_errors(manager.log_mut()); } Global::exit(1); } }; + refuse_unless_lockfile_matches_package_json(manager)?; + let store_present = match Dir::open(b"node_modules") { Ok(node_modules) => lstat_kind(&node_modules, b".bun") == EntryKind::Directory, Err(err) if err.get_errno() == E::ENOENT => { @@ -188,11 +189,12 @@ pub fn prune(manager: &mut PackageManager) -> crate::Result<()> { }; let workspace_names = collect_workspace_names(&manager.lockfile); + let selection = select_importers(manager, original_cwd); let mut plan = Plan::default(); match layout { - Layout::Hoisted => plan_hoisted(manager, &workspace_names, &mut plan), - Layout::Isolated => plan_isolated(manager, &workspace_names, &mut plan), + Layout::Hoisted => plan_hoisted(manager, &workspace_names, selection.as_ref(), &mut plan), + Layout::Isolated => plan_isolated(manager, &workspace_names, selection.as_ref(), &mut plan), } if layout_mismatch(&plan, layout, store_present) { @@ -266,6 +268,166 @@ fn collect_workspace_names(lockfile: &Lockfile) -> Vec> { out } +fn refuse_unless_lockfile_matches_package_json(manager: &mut PackageManager) -> crate::Result<()> { + let Some(root) = manager.lockfile.root_package() else { + return Ok(()); + }; + let quiet = manager.options.log_level == LogLevel::Silent; + manager.options.enable.set(Enable::FROZEN_LOCKFILE, true); + + let log = manager.log_mut(); + // SAFETY: written once inside `PackageManager::init` on this thread; only read afterwards. + let path: &[u8] = unsafe { ROOT_PACKAGE_JSON_PATH.read() }.as_bytes(); + let (source, json) = match manager + .workspace_package_json_cache + .get_with_path(log, path, Default::default()) + .unwrap() + { + Ok(entry) => (entry.source.clone(), entry.root), + Err(err) => { + if !quiet { + print_log_errors(log); + Output::err(err, "failed to read {s}", (BStr::new(path),)); + } + Global::exit(1); + } + }; + + let mut to_lockfile = Lockfile::default(); + let mut to_root = Package::default(); + let mut resolver: () = (); + let pm: *mut PackageManager = manager; + // SAFETY: same split as `hoist_filtered`; neither call reaches `lockfile` through `pm`. + let summary = unsafe { + let parsed = to_root.parse_with_json::<()>( + &mut to_lockfile, + &mut *pm, + log, + &source, + json, + &mut resolver, + Features::main(), + ); + match parsed { + Ok(()) => { + let mut mapping = vec![invalid_package_id; to_root.dependencies.len as usize]; + let from_lockfile: *mut Lockfile = &raw mut *(*pm).lockfile; + Diff::generate( + &mut *pm, + log, + &mut *from_lockfile, + &mut to_lockfile, + &root, + &to_root, + None, + Some(&mut mapping[..]), + ) + } + Err(err) => Err(err), + } + }; + let summary = match summary { + Ok(summary) => summary, + Err(err) => { + if !quiet { + print_log_errors(log); + } + return Err(err); + } + }; + + if summary.changes_dependencies() { + if !quiet { + Output::err_generic("bun.lock does not match package.json", ()); + bun_core::note!("run 'bun install' first, then run 'bun prune' again"); + } + Global::exit(1); + } + Ok(()) +} + +fn print_log_errors(log: &bun_ast::Log) { + if log.has_errors() { + let _ = log.print(core::ptr::from_mut(Output::error_writer())); + } +} + +struct Selection { + selected: DynamicBitSet, + protected_packages: DynamicBitSet, + protected_aliases: Vec>, +} + +fn select_importers(manager: &PackageManager, original_cwd: &[u8]) -> Option { + if manager.options.filter_patterns.is_empty() { + return None; + } + let lockfile: &Lockfile = &manager.lockfile; + let ids = + WorkspaceFilter::select_workspaces(lockfile, manager.options.filter_patterns, original_cwd); + if ids.is_empty() { + if manager.options.log_level != LogLevel::Silent { + Output::err_generic("No packages matched the filter", ()); + } + Global::exit(1); + } + + let buf = lockfile.buffers.string_bytes.as_slice(); + let deps = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + let pkg_res = lockfile.packages.items_resolution(); + let dep_slices = lockfile.packages.items_dependencies(); + let is_importer = |pkg_id: usize| { + matches!( + pkg_res[pkg_id].tag, + ResolutionTag::Root | ResolutionTag::Workspace + ) + }; + + let mut selected = handle_oom(DynamicBitSet::init_empty(pkg_res.len())); + for id in ids { + if (id as usize) < pkg_res.len() { + selected.set(id as usize); + } + } + + let mut protected_packages = handle_oom(DynamicBitSet::init_empty(pkg_res.len())); + let mut protected_aliases: Vec> = Vec::new(); + let mut worklist: Vec = Vec::new(); + for importer in 0..pkg_res.len() { + if selected.is_set(importer) || !is_importer(importer) { + continue; + } + protected_packages.set(importer); + worklist.push(importer as PackageID); + while let Some(pkg_id) = worklist.pop() { + let slice = dep_slices[pkg_id as usize]; + for dep_id in slice.begin() as usize..slice.end() as usize { + protected_aliases.push(deps[dep_id].name.slice(buf).into()); + let target = resolutions[dep_id]; + if target == invalid_package_id || (target as usize) >= pkg_res.len() { + continue; + } + if protected_packages.is_set(target as usize) { + continue; + } + protected_packages.set(target as usize); + if !is_importer(target as usize) { + worklist.push(target); + } + } + } + } + protected_aliases.sort_unstable(); + protected_aliases.dedup(); + + Some(Selection { + selected, + protected_packages, + protected_aliases, + }) +} + fn hoist_filtered(manager: &mut PackageManager) { let pm: *mut PackageManager = manager; // SAFETY: same split as `PackageManager::load_lockfile_from_cwd` — `lockfile` is its own `Box` allocation and the Filter builder only reads `manager.options`/`subcommand`/`summary`. @@ -468,8 +630,19 @@ fn without_build(version: &[u8]) -> &[u8] { &version[..strings::last_index_of_char(version, b'+').unwrap_or(version.len())] } -fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], plan: &mut Plan) { - let keep_workspaces = |entry: &Entry| contains(workspace_names, entry.alias); +fn plan_hoisted( + manager: &mut PackageManager, + workspace_names: &[Box<[u8]>], + selection: Option<&Selection>, + plan: &mut Plan, +) { + let root_protected: &[Box<[u8]>] = match selection { + Some(sel) => &sel.protected_aliases, + None => &[], + }; + let keep_workspaces = |entry: &Entry| { + contains(workspace_names, entry.alias) || contains(root_protected, entry.alias) + }; if manager.lockfile.packages.len() == 0 { if let Ok(dir) = Dir::open(b"node_modules") { scan_folder(dir, b"node_modules", false, &keep_workspaces, plan); @@ -501,19 +674,48 @@ fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], pla .collect(); nested_trees.sort_unstable(); - let mut visited = vec![false; pkg_res.len()]; + let tree_owner = |tree_idx: usize| -> PackageID { + match trees[tree_idx].dependency_id { + tree::ROOT_DEP_ID => 0, + dep_id => resolutions[dep_id as usize], + } + }; + let mut tree_importer: Vec = Vec::new(); + if selection.is_some() { + tree_importer.reserve_exact(trees.len()); + for tree_idx in 0..trees.len() { + let owner = tree_owner(tree_idx); + let importer = if tree_idx == 0 { + 0 + } else if (owner as usize) < pkg_res.len() + && pkg_res[owner as usize].tag == ResolutionTag::Workspace + { + owner + } else { + tree_importer + .get(trees[tree_idx].parent as usize) + .copied() + .unwrap_or(0) + }; + tree_importer.push(importer); + } + } + + let mut visited = handle_oom(DynamicBitSet::init_empty(pkg_res.len())); for tree_idx in 0..hoisted.folders.len() { let folder_path = hoisted.path(tree_idx); if folder_path.is_empty() { continue; } + let importer = tree_importer.get(tree_idx).copied().unwrap_or(0); + if selection.is_some_and(|sel| importer != 0 && !sel.selected.is_set(importer as usize)) { + continue; + } + let protected: &[Box<[u8]>] = if importer == 0 { root_protected } else { &[] }; let tree_id = tree_idx as tree::Id; - let owner: PackageID = match trees[tree_idx].dependency_id { - tree::ROOT_DEP_ID => 0, - dep_id => resolutions[dep_id as usize], - }; + let owner = tree_owner(tree_idx); if owner != invalid_package_id && (owner as usize) < pkg_res.len() { - visited[owner as usize] = true; + visited.set(owner as usize); let tag = pkg_res[owner as usize].tag; if tag != ResolutionTag::Root && tag != ResolutionTag::Workspace @@ -558,6 +760,7 @@ fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], pla false, &|entry: &Entry| { contains(workspace_names, entry.alias) + || contains(protected, entry.alias) || !hoisted.removable(tree_id, entry.alias, &nested_path) }, plan, @@ -574,19 +777,23 @@ fn plan_hoisted(manager: &mut PackageManager, workspace_names: &[Box<[u8]>], pla .binary_search_by(|(name, _)| (*name).cmp(entry.alias)) .is_ok() || contains(workspace_names, entry.alias) + || contains(protected, entry.alias) || !hoisted.removable(parent, entry.alias, folder_path) }, plan, ); } - if !visited[0] { + if !visited.is_set(0) { if let Ok(dir) = Dir::open(b"node_modules") { scan_folder(dir, b"node_modules", false, &keep_workspaces, plan); } } for (pkg_id, res) in pkg_res.iter().enumerate() { - if visited[pkg_id] || res.tag != ResolutionTag::Workspace { + if visited.is_set(pkg_id) + || res.tag != ResolutionTag::Workspace + || selection.is_some_and(|sel| !sel.selected.is_set(pkg_id)) + { continue; } let path = strings::without_trailing_slash(res.workspace().slice(buf)); @@ -760,7 +967,7 @@ fn scan_folder( folder_idx } -fn wanted_packages(manager: &PackageManager) -> Vec { +fn wanted_packages(manager: &PackageManager) -> DynamicBitSet { let lockfile: &Lockfile = &manager.lockfile; reachable::packages( lockfile, @@ -769,14 +976,15 @@ fn wanted_packages(manager: &PackageManager) -> Vec { ) } -fn store_keys(lockfile: &Lockfile, wanted: &[bool]) -> Vec> { +fn store_keys(lockfile: &Lockfile, wanted: &DynamicBitSet) -> Vec> { let buf = lockfile.buffers.string_bytes.as_slice(); let names = lockfile.packages.items_name(); let pkg_res = lockfile.packages.items_resolution(); - let mut keys: Vec> = Vec::new(); + let mut keys: Vec> = Vec::with_capacity(wanted.count()); let mut key: Vec = Vec::new(); - for pkg_id in 0..wanted.len() { - if !wanted[pkg_id] || pkg_res[pkg_id].tag == ResolutionTag::Workspace { + let mut set_bits = wanted.iterator::(); + while let Some(pkg_id) = set_bits.next() { + if pkg_res[pkg_id].tag == ResolutionTag::Workspace { continue; } let name = names[pkg_id]; @@ -862,8 +1070,18 @@ fn public_hoist_matches(manager: &PackageManager, alias: &[u8]) -> bool { .is_some_and(|pattern| pattern.is_match(alias)) } -fn plan_isolated(manager: &PackageManager, workspace_names: &[Box<[u8]>], plan: &mut Plan) { - let wanted = wanted_packages(manager); +fn plan_isolated( + manager: &PackageManager, + workspace_names: &[Box<[u8]>], + selection: Option<&Selection>, + plan: &mut Plan, +) { + let mut wanted = wanted_packages(manager); + if let Some(sel) = selection { + wanted + .unmanaged + .set_union(&sel.protected_packages.unmanaged); + } let keys = store_keys(&manager.lockfile, &wanted); let keep_store_entry = |name: &[u8]| { contains(&keys, name) || strip_peer_hash(name).is_some_and(|base| contains(&keys, base)) @@ -900,6 +1118,9 @@ fn plan_isolated(manager: &PackageManager, workspace_names: &[Box<[u8]>], plan: } _ => continue, }; + if selection.is_some_and(|sel| !sel.selected.is_set(pkg_id)) { + continue; + } let Ok(dir) = Dir::open(&folder_path) else { continue; }; @@ -910,7 +1131,7 @@ fn plan_isolated(manager: &PackageManager, workspace_names: &[Box<[u8]>], plan: store_touched, &|entry| { contains(&direct, entry.alias) - || contains(workspace_names, entry.alias) + || (entry.kind != EntryKind::SymLink && contains(workspace_names, entry.alias)) || public_hoist_matches(manager, entry.alias) || (entry.kind == EntryKind::SymLink && store_link_target(entry.dir, entry.name) @@ -1103,7 +1324,6 @@ fn prune_bins(dir: &Dir) { } fn housekeeping(plan: &Plan, layout: Layout, manager: &PackageManager) { - let workspace_names = collect_workspace_names(&manager.lockfile); for (idx, folder) in plan.folders.iter().enumerate() { if !folder.touched { continue; @@ -1123,7 +1343,7 @@ fn housekeeping(plan: &Plan, layout: Layout, manager: &PackageManager) { if let (Layout::Isolated, Some(direct)) = (layout, &folder.direct) { if let Ok(dir) = Dir::open(&folder.path) { unlink_links(&dir, &|dir, alias, name| { - if contains(direct, alias) || contains(&workspace_names, alias) { + if contains(direct, alias) { return false; } if public_hoist_matches(manager, alias) { diff --git a/src/install/update_scope.rs b/src/install/update_scope.rs new file mode 100644 index 000000000000..edb71f16772d --- /dev/null +++ b/src/install/update_scope.rs @@ -0,0 +1,361 @@ +use std::sync::OnceLock; + +use bstr::BStr; +use bun_collections::HashMap; +use bun_core::{Global, Output, prettyln, strings}; +use bun_semver::string::Builder as StringBuilder; + +use crate::dependency::Behavior; +use crate::lockfile::Lockfile; +use crate::lockfile::package::PackageColumns as _; +use crate::package_manager::Options::LogLevel; +use crate::package_manager::{UpdateTargetWorkspace, WorkspaceFilter}; +use crate::package_manager_real::command_line_arguments::UpdateGroups; +use crate::resolution::Tag as ResolutionTag; +use crate::{DependencyID, PackageManager, PackageNameHash, invalid_package_id}; + +/// Which workspaces a named `bun update` may re-resolve and rewrite; rows owned by non-workspace packages are always in scope. +pub struct UpdateScope<'a> { + pub targets: Option<&'a [UpdateTargetWorkspace]>, + pub invoking: Option, +} + +impl<'a> UpdateScope<'a> { + pub fn of(manager: &'a PackageManager) -> UpdateScope<'a> { + UpdateScope { + targets: manager.update_target_workspaces.as_deref(), + invoking: manager.workspace_name_hash, + } + } +} + +impl UpdateScope<'_> { + pub fn contains_workspace( + &self, + is_root: bool, + name_hash: PackageNameHash, + name: &[u8], + ) -> bool { + match self.targets { + Some(targets) => targets.iter().any(|t| t.matches(is_root, name_hash, name)), + None => match self.invoking { + None => is_root, + Some(hash) => !is_root && hash == name_hash, + }, + } + } + + pub fn contains_dependency(&self, lockfile: &Lockfile, dep_id: DependencyID) -> bool { + let owner = lockfile.get_workspace_pkg_if_workspace_dep(dep_id); + if owner == invalid_package_id { + return true; + } + let owner = owner as usize; + let is_root = lockfile.packages.items_resolution()[owner].tag == ResolutionTag::Root; + let name = + lockfile.packages.items_name()[owner].slice(lockfile.buffers.string_bytes.as_slice()); + self.contains_workspace(is_root, lockfile.packages.items_name_hash()[owner], name) + } + + /// One flag per dependency row; rows covered by no package's slice (orphans left by the differ) stay false. + pub fn walkable_rows(&self, lockfile: &Lockfile) -> Vec { + let mut walk = vec![false; lockfile.buffers.dependencies.len()]; + let pkg_res = lockfile.packages.items_resolution(); + let name_hashes = lockfile.packages.items_name_hash(); + let names = lockfile.packages.items_name(); + let buf = lockfile.buffers.string_bytes.as_slice(); + for (id, slice) in lockfile.packages.items_dependencies().iter().enumerate() { + if slice.len == 0 { + continue; + } + let res = &pkg_res[id]; + let in_scope = match res.tag { + ResolutionTag::Root | ResolutionTag::Workspace => self.contains_workspace( + res.tag == ResolutionTag::Root, + name_hashes[id], + names[id].slice(buf), + ), + _ => true, + }; + walk[slice.begin() as usize..slice.end() as usize].fill(in_scope); + } + walk + } +} + +fn selects(groups: UpdateGroups, behavior: Behavior) -> bool { + if groups.no_optional && behavior.is_optional() { + return false; + } + if !groups.dev && !groups.prod { + return true; + } + (groups.dev && behavior.is_dev()) + || (groups.prod && (behavior.is_prod() || behavior.is_optional())) +} + +fn is_pattern(arg: &[u8]) -> bool { + arg.starts_with(b"!") || strings::contains_char(arg, b'*') +} + +struct Pattern { + raw: &'static [u8], + negated: bool, + glob: &'static [u8], + hit: bool, +} + +impl Pattern { + fn parse(raw: &'static [u8]) -> Pattern { + let mut negated = false; + let mut glob = raw; + while let Some(rest) = glob.strip_prefix(b"!") { + negated = !negated; + glob = rest; + } + Pattern { + raw, + negated, + glob, + hit: false, + } + } +} + +fn strip_negations(arg: &[u8]) -> &[u8] { + let mut rest = arg; + while let Some(r) = rest.strip_prefix(b"!") { + rest = r; + } + rest +} + +fn has_version_suffix(arg: &[u8]) -> bool { + let rest = strip_negations(arg); + rest.len() > 1 && strings::contains_char(&rest[1..], b'@') +} + +fn name_of_plain_arg(arg: &[u8]) -> &[u8] { + match strings::index_of_char_usize(&arg[1.min(arg.len())..], b'@') { + Some(i) => &arg[..i + 1], + None => arg, + } +} + +fn matches(glob: &[u8], name: &[u8]) -> bool { + if glob == b"*" { + true + } else if strings::contains_char(glob, b'*') { + bun_glob::r#match(glob, name).matches() + } else { + glob == name + } +} + +fn exit_on_lockfile_load_failure(manager: &mut PackageManager) { + fn missing(silent: bool) -> ! { + if !silent { + Output::err_generic("missing lockfile, nothing to update", ()); + } + Global::exit(1); + } + let silent = manager.options.log_level == LogLevel::Silent; + if !manager.options.do_.load_lockfile() { + missing(silent); + } + match manager.load_lockfile_from_cwd::() { + crate::lockfile::LoadResult::Ok(_) => {} + crate::lockfile::LoadResult::NotFound => missing(silent), + crate::lockfile::LoadResult::Err(cause) => { + if !silent { + let what: &str = match cause.step { + crate::lockfile::LoadStep::OpenFile => "open", + crate::lockfile::LoadStep::ReadFile => "read", + crate::lockfile::LoadStep::ParseFile => "parse", + crate::lockfile::LoadStep::Migrating => "migrate", + }; + Output::err_generic("failed to {s} lockfile: {s}", (what, cause.value.name())); + if manager.log_mut().has_errors() { + let _ = manager + .log_mut() + .print(std::ptr::from_mut(Output::error_writer())); + } + } + Global::exit(1); + } + } +} + +/// Turns `bun update` patterns and `--dev`/`--prod`/`--no-optional` into the concrete names the named path expects; a plain `bun update [name]` returns before doing anything. +pub fn expand_positionals(manager: &mut PackageManager, original_cwd: &[u8], groups: UpdateGroups) { + let positionals = manager.options.positionals; + let args = positionals.get(1..).unwrap_or(&[]); + let selecting = !groups.is_default(); + if !selecting && !args.iter().any(|a| is_pattern(a)) { + return; + } + + let mut patterns: Vec = Vec::new(); + let mut passthrough: Vec<&'static [u8]> = Vec::new(); + for &arg in args { + if selecting { + if has_version_suffix(arg) { + Output::err_generic( + "a version cannot be combined with --dev, --prod or --no-optional: {}", + (BStr::new(arg),), + ); + Global::exit(1); + } + patterns.push(Pattern::parse(arg)); + } else if is_pattern(arg) { + if has_version_suffix(arg) { + Output::err_generic( + "a version cannot be combined with a pattern: {}", + (BStr::new(arg),), + ); + Global::exit(1); + } + patterns.push(Pattern::parse(arg)); + } else { + passthrough.push(arg); + } + } + + exit_on_lockfile_load_failure(manager); + + let selection: Option> = (manager.options.do_.recursive() + || !manager.options.filter_patterns.is_empty()) + .then(|| { + let lockfile = &*manager.lockfile; + let name_hashes = lockfile.packages.items_name_hash(); + let names = lockfile.packages.items_name(); + let resolutions = lockfile.packages.items_resolution(); + let buf = lockfile.buffers.string_bytes.as_slice(); + WorkspaceFilter::select_workspaces(lockfile, manager.options.filter_patterns, original_cwd) + .into_iter() + .map(|id| UpdateTargetWorkspace { + is_root: resolutions[id as usize].tag == ResolutionTag::Root, + name_hash: name_hashes[id as usize], + name: Box::from(names[id as usize].slice(buf)), + }) + .collect() + }); + let scope = UpdateScope { + targets: selection.as_deref(), + invoking: manager.workspace_name_hash, + }; + + let mut names: Vec> = Vec::new(); + { + let lockfile = &*manager.lockfile; + let walk = scope.walkable_rows(lockfile); + let pkg_res = lockfile.packages.items_resolution(); + let pkg_names = lockfile.packages.items_name(); + let pkg_name_hashes = lockfile.packages.items_name_hash(); + let deps = lockfile.buffers.dependencies.as_slice(); + let resolutions = lockfile.buffers.resolutions.as_slice(); + let buf = lockfile.buffers.string_bytes.as_slice(); + let include_transitive = !selecting; + + let mut decided: HashMap = HashMap::new(); + for &arg in &passthrough { + decided.insert(StringBuilder::string_hash(name_of_plain_arg(arg)), ()); + } + + for (owner, slice) in lockfile.packages.items_dependencies().iter().enumerate() { + if slice.len == 0 { + continue; + } + let owner_is_ws = matches!( + pkg_res[owner].tag, + ResolutionTag::Root | ResolutionTag::Workspace + ); + if !owner_is_ws && !include_transitive { + continue; + } + for i in slice.begin() as usize..slice.end() as usize { + if !walk[i] { + continue; + } + let target = resolutions[i]; + if target == invalid_package_id + || matches!( + pkg_res[target as usize].tag, + ResolutionTag::Root | ResolutionTag::Workspace + ) + { + continue; + } + let dep = &deps[i]; + if owner_is_ws && !selects(groups, dep.behavior) { + continue; + } + let real_hash = pkg_name_hashes[target as usize]; + if decided.insert(real_hash, ()).is_some() { + continue; + } + let real = pkg_names[target as usize].slice(buf); + let alias = dep.name.slice(buf); + let mut positive_hit = patterns.iter().all(|p| p.negated); + let mut excluded = false; + for pattern in patterns.iter_mut() { + if !matches(pattern.glob, real) + && !(alias != real && matches(pattern.glob, alias)) + { + continue; + } + if pattern.negated { + excluded = true; + } else { + pattern.hit = true; + positive_hit = true; + } + } + if positive_hit && !excluded { + // The named path matches `npm:` aliases through the real name, so the real name reaches both spellings. + names.push(Box::from(real)); + } + } + } + } + + let mut failed = false; + for pattern in patterns.iter().filter(|p| !p.negated && !p.hit) { + failed = true; + if selecting { + Output::err_generic( + "no dependencies in the selected groups match \"{}\"", + (BStr::new(pattern.raw),), + ); + } else { + Output::err_generic( + "no packages in bun.lock match \"{}\"", + (BStr::new(pattern.raw),), + ); + } + } + if failed { + Global::exit(1); + } + if names.is_empty() && passthrough.is_empty() { + if manager.options.log_level != LogLevel::Silent { + prettyln!("No packages to update"); + Output::flush(); + } + Global::exit(0); + } + + names.sort_unstable(); + static EXPANDED_NAMES: OnceLock>> = OnceLock::new(); + static EXPANDED_POSITIONALS: OnceLock> = OnceLock::new(); + let expanded = EXPANDED_NAMES.get_or_init(|| names); + let expanded_positionals = EXPANDED_POSITIONALS.get_or_init(|| { + let mut out: Vec<&'static [u8]> = + Vec::with_capacity(1 + passthrough.len() + expanded.len()); + out.push(positionals[0]); + out.extend(passthrough.iter().copied()); + out.extend(expanded.iter().map(|name| &**name)); + out + }); + manager.options.positionals = expanded_positionals.as_slice(); +} diff --git a/src/install/update_transitive.rs b/src/install/update_transitive.rs index c07f9ee73822..6ba6e3301e9e 100644 --- a/src/install/update_transitive.rs +++ b/src/install/update_transitive.rs @@ -2,7 +2,9 @@ use core::cmp::Ordering; use std::io::Write as _; use bstr::BStr; -use bun_core::{Output, prettyln}; +use bun_collections::DynamicBitSet; +use bun_collections::bit_set::Range as BitRange; +use bun_core::{Output, UnwrapOrOom as _, prettyln}; use bun_semver as Semver; use crate::audit_fix; @@ -82,7 +84,7 @@ impl DirectDependencies { continue; } let rows = &self.rows[start as usize..(start + len) as usize]; - let mut claimed: Option> = None; + let mut claimed: Option = None; let current = dep_slices[owner] .get(deps) .iter() @@ -96,19 +98,25 @@ impl DirectDependencies { } else { // Every row before the first miss was a same-index hit. let taken = claimed.get_or_insert_with(|| { - let mut taken = vec![false; rows.len()]; - taken[..i.min(rows.len())].fill(true); + let mut taken = DynamicBitSet::init_empty(rows.len()).unwrap_or_oom(); + taken.set_range_value( + BitRange { + start: 0, + end: i.min(rows.len()), + }, + true, + ); taken }); - let hit = if !taken.get(i).copied().unwrap_or(true) && same(&rows[i]) { + let hit = if !taken.is_set_allow_out_of_bound(i, true) && same(&rows[i]) { Some(i) } else { - (0..rows.len()).find(|&k| !taken[k] && same(&rows[k])) + (0..rows.len()).find(|&k| !taken.is_set(k) && same(&rows[k])) }; let Some(k) = hit else { continue; }; - taken[k] = true; + taken.set(k); k }; let old = rows[index].2; @@ -276,18 +284,24 @@ struct Instance { wants: Vec, } -fn workspace_owned_dependencies(lockfile: &Lockfile) -> Vec { - let mut owned = vec![false; lockfile.buffers.dependencies.len()]; +fn workspace_owned_dependencies(lockfile: &Lockfile) -> crate::Result { + let mut owned = DynamicBitSet::init_empty(lockfile.buffers.dependencies.len())?; let pkg_res = lockfile.packages.items_resolution(); for (owner, slice) in lockfile.packages.items_dependencies().iter().enumerate() { if matches!( pkg_res[owner].tag, ResolutionTag::Root | ResolutionTag::Workspace ) { - owned[slice.begin() as usize..slice.end() as usize].fill(true); + owned.set_range_value( + BitRange { + start: slice.begin() as usize, + end: slice.end() as usize, + }, + true, + ); } } - owned + Ok(owned) } fn plan_edges(manager: &mut PackageManager) -> crate::Result<(Vec, Vec)> { @@ -323,7 +337,7 @@ fn plan_edges(manager: &mut PackageManager) -> crate::Result<(Vec, Vec return Ok((Vec::new(), Vec::new())); } - let workspace_owned = workspace_owned_dependencies(lockfile); + let workspace_owned = workspace_owned_dependencies(lockfile)?; let no_overrides = lockfile.overrides.is_empty(); let deps = lockfile.buffers.dependencies.as_slice(); for (dep_id, &target) in lockfile.buffers.resolutions.iter().enumerate() { @@ -332,7 +346,7 @@ fn plan_edges(manager: &mut PackageManager) -> crate::Result<(Vec, Vec }; let dep = &deps[dep_id]; if instance == u32::MAX - || workspace_owned[dep_id] + || workspace_owned.is_set(dep_id) || dep.behavior.is_peer() || dep.behavior.is_bundled() { diff --git a/src/runtime/cli/audit_command.rs b/src/runtime/cli/audit_command.rs index 190dca81d9fb..c0a2badf1d64 100644 --- a/src/runtime/cli/audit_command.rs +++ b/src/runtime/cli/audit_command.rs @@ -2,7 +2,7 @@ use bstr::BStr; use std::io::Write as _; use bun_ast::{ExprData, e as E}; -use bun_collections::{StringArrayHashMap, StringHashMap}; +use bun_collections::{DynamicBitSet, StringArrayHashMap, StringHashMap}; use bun_core::{Global, Output, pretty, prettyln}; use bun_core::{MutableString, strings}; use bun_http::{self as http, HeaderBuilder}; @@ -99,11 +99,14 @@ impl AuditCommand { let json_output = manager.options.json_output; if fix { - if json_output { - Output::err_generic("--json is not supported by bun audit fix", ()); - Global::exit(1); - } - return Self::audit_fix(ctx, manager, audit_level, audit_ignore_list, &original_cwd); + return Self::audit_fix( + ctx, + manager, + json_output, + audit_level, + audit_ignore_list, + &original_cwd, + ); } let code = Self::audit(ctx, manager, json_output, audit_level, audit_ignore_list)?; @@ -133,9 +136,9 @@ impl AuditCommand { let dependency_tree = build_dependency_tree(pm)?; - let packages_result = collect_packages_for_audit(pm, true)?; - - let response_text = send_audit_request(pm, &packages_result.audit_body)?; + let collected = collect_packages_for_audit(pm, true)?; + let responses = send_audit_requests(pm, &collected)?; + let response_text = responses.response_text; if json_output { let _ = Output::writer().write_all(&response_text); @@ -163,13 +166,13 @@ impl AuditCommand { ignore_list, )?; - print_skipped_packages(&packages_result.skipped_packages); + audit_fix::print_unaudited(&responses.unaudited); return Ok(exit_code); } else { prettyln!("No vulnerabilities found"); - print_skipped_packages(&packages_result.skipped_packages); + audit_fix::print_unaudited(&responses.unaudited); return Ok(0); } @@ -178,6 +181,7 @@ impl AuditCommand { fn audit_fix( ctx: Command::Context, pm: &mut PackageManager, + json_output: bool, audit_level: Option, ignore_list: &[&[u8]], original_cwd: &[u8], @@ -196,8 +200,9 @@ impl AuditCommand { audit_fix::exit_unless_lockfile_writable(pm); - let packages_result = collect_packages_for_audit(pm, false)?; - let response_text = send_audit_request(pm, &packages_result.audit_body)?; + let collected = collect_packages_for_audit(pm, false)?; + let responses = send_audit_requests(pm, &collected)?; + let response_text = responses.response_text; let vulnerabilities = if response_text.is_empty() { Vec::new() @@ -213,9 +218,11 @@ impl AuditCommand { } }; - print_skipped_packages(&packages_result.skipped_packages); + if !json_output { + audit_fix::print_unaudited(&responses.unaudited); + } - if vulnerabilities.is_empty() { + if vulnerabilities.is_empty() && !json_output { prettyln!("No vulnerabilities found"); Output::flush(); Global::exit(0); @@ -229,16 +236,17 @@ impl AuditCommand { }) .collect(); let dry_run = pm.options.dry_run; - let plan = audit_fix::plan_fixes(pm, &advisories)?; - plan.print_sections(); + let mut plan = audit_fix::plan_fixes(pm, &advisories)?; + plan.unaudited = responses.unaudited; + if !json_output { + plan.print_sections(); + } if dry_run || plan.fixes.is_empty() { - plan.print_summary(dry_run); - Output::flush(); - Global::exit(plan.exit_code()); + Global::exit(plan.finish_planned(json_output, dry_run)); } - pm.audit_fix_pins = plan.pins(); + audit_fix::prepare_install(pm, &plan)?; // SAFETY: `ROOT_PACKAGE_JSON_PATH` is written exactly once inside `PackageManager::init`; only read thereafter. let root_package_json_path = unsafe { ROOT_PACKAGE_JSON_PATH.read() }; @@ -251,29 +259,7 @@ impl AuditCommand { Global::exit(1); } - plan.print_summary(false); - Output::flush(); - Global::exit(plan.exit_code()); - } -} - -fn print_skipped_packages(skipped_packages: &[Box<[u8]>]) { - if !skipped_packages.is_empty() { - pretty!("Skipped "); - for (i, package_name) in skipped_packages.iter().enumerate() { - if i > 0 { - pretty!(", "); - } - pretty!("{}", BStr::new(package_name)); - } - - if skipped_packages.len() > 1 { - prettyln!(" because they do not come from the default registry"); - } else { - prettyln!(" because it does not come from the default registry"); - } - - prettyln!(""); + Global::exit(plan.finish_installed(&pm.lockfile, json_output)); } } @@ -317,9 +303,39 @@ fn build_dependency_tree( Ok(dependency_tree) } +struct AuditRegistry { + href: Box<[u8]>, + url_hash: u64, + token: Box<[u8]>, + auth: Box<[u8]>, + is_default: bool, +} + +impl AuditRegistry { + fn from_scope(scope: &bun_install::npm::registry::Scope, is_default: bool) -> AuditRegistry { + AuditRegistry { + href: Box::<[u8]>::from(strings::without_trailing_slash(scope.url.href())), + url_hash: scope.url_hash, + token: scope.token.clone(), + auth: scope.auth.clone(), + is_default, + } + } +} + +struct AuditRequest { + registry: AuditRegistry, + package_names: Vec>, + body: Box<[u8]>, +} + struct CollectPackagesResult { - audit_body: Box<[u8]>, - skipped_packages: Vec>, + requests: Vec, +} + +struct AuditResponses { + response_text: Box<[u8]>, + unaudited: Vec, } struct PackageVersions { @@ -333,15 +349,17 @@ fn collect_packages_for_audit( ) -> Result { let root_id = pm.root_package_id.get(&pm.lockfile, pm.workspace_name_hash); - let mut packages_list: Vec = Vec::new(); - let mut skipped_packages: Vec> = Vec::new(); + let mut groups: Vec<(AuditRegistry, Vec)> = vec![( + AuditRegistry::from_scope(&pm.options.scope, true), + Vec::new(), + )]; let features = pm.options.local_package_features; let omits_something = apply_omit && !(features.dev_dependencies && features.optional_dependencies && features.peer_dependencies); - let wanted_packages: Option> = omits_something.then(|| { + let wanted_packages: Option = omits_something.then(|| { reachable::packages( &pm.lockfile, pm.lockfile.buffers.resolutions.as_slice(), @@ -358,7 +376,6 @@ fn collect_packages_for_audit( }); let options = &pm.options; - let default_url_hash = options.scope.url_hash; let packages = pm.lockfile.packages.slice(); let pkg_names = packages.items_name(); let pkg_resolutions = packages.items_resolution(); @@ -372,17 +389,27 @@ fn collect_packages_for_audit( continue; } - if wanted_packages.as_ref().is_some_and(|wanted| !wanted[idx]) { + if wanted_packages + .as_ref() + .is_some_and(|wanted| !wanted.is_set(idx)) + { continue; } let name_slice = name.slice(buf); let package_scope = options.scope_for_package_name(name_slice); - if package_scope.url_hash != default_url_hash { - skipped_packages.push(Box::<[u8]>::from(name_slice)); - continue; - } + let group_idx = match groups + .iter() + .position(|(registry, _)| registry.url_hash == package_scope.url_hash) + { + Some(i) => i, + None => { + groups.push((AuditRegistry::from_scope(package_scope, false), Vec::new())); + groups.len() - 1 + } + }; + let packages_list = &mut groups[group_idx].1; let mut ver_str: Vec = Vec::new(); // `res.tag == ResolutionTag::Npm` checked above. @@ -415,6 +442,21 @@ fn collect_packages_for_audit( } } + let requests = groups + .into_iter() + .enumerate() + .filter(|(i, (_, list))| *i == 0 || !list.is_empty()) + .map(|(_, (registry, list))| AuditRequest { + registry, + package_names: list.iter().map(|package| package.name.clone()).collect(), + body: build_body(&list), + }) + .collect(); + + Ok(CollectPackagesResult { requests }) +} + +fn build_body(packages_list: &[PackageVersions]) -> Box<[u8]> { let mut body: Vec = Vec::with_capacity(1024); body.push(b'{'); @@ -445,16 +487,72 @@ fn collect_packages_for_audit( } body.push(b'}'); - Ok(CollectPackagesResult { - audit_body: body.into_boxed_slice(), - skipped_packages, + body.into_boxed_slice() +} + +fn send_audit_requests( + pm: &mut PackageManager, + collected: &CollectPackagesResult, +) -> Result { + let mut bodies: Vec> = Vec::with_capacity(collected.requests.len()); + let mut unaudited: Vec = Vec::new(); + + for request in &collected.requests { + match send_audit_request(pm, &request.registry, &request.body)? { + Some(body) => bodies.push(body), + None => unaudited.push(audit_fix::UnauditedRegistry { + registry: request.registry.href.clone(), + packages: request.package_names.clone(), + }), + } + } + + Ok(AuditResponses { + response_text: merge_bulk_bodies(&bodies), + unaudited, }) } +fn is_empty_bulk_body(body: &[u8]) -> bool { + let body = body.trim_ascii(); + body.is_empty() + || body + .strip_prefix(b"{") + .and_then(|rest| rest.strip_suffix(b"}")) + .is_some_and(|inner| inner.trim_ascii().is_empty()) +} + +fn merge_bulk_bodies(bodies: &[Box<[u8]>]) -> Box<[u8]> { + let mut non_empty = bodies.iter().filter(|body| !is_empty_bulk_body(body)); + let Some(first) = non_empty.next() else { + return Box::default(); + }; + let Some(second) = non_empty.next() else { + return first.clone(); + }; + + let mut merged: Vec = Vec::with_capacity(bodies.iter().map(|body| body.len()).sum()); + merged.push(b'{'); + for (i, body) in [first, second].into_iter().chain(non_empty).enumerate() { + let body = body.trim_ascii(); + let inner = body + .strip_prefix(b"{") + .and_then(|rest| rest.strip_suffix(b"}")) + .unwrap_or(body); + if i > 0 { + merged.push(b','); + } + merged.extend_from_slice(inner); + } + merged.push(b'}'); + merged.into_boxed_slice() +} + fn send_audit_request( pm: &mut PackageManager, + registry: &AuditRegistry, body: &[u8], -) -> Result, bun_alloc::AllocError> { +) -> Result>, bun_alloc::AllocError> { libdeflate::load(); let mut compressor = libdeflate::OwnedCompressor::new(6).ok_or(bun_alloc::AllocError)?; @@ -468,26 +566,26 @@ fn send_audit_request( headers.count(b"accept", b"application/json"); headers.count(b"content-type", b"application/json"); headers.count(b"content-encoding", b"gzip"); - if !pm.options.scope.token.is_empty() { + if !registry.token.is_empty() { headers.count(b"authorization", b""); - headers.content.cap += b"Bearer ".len() + pm.options.scope.token.len(); - } else if !pm.options.scope.auth.is_empty() { + headers.content.cap += b"Bearer ".len() + registry.token.len(); + } else if !registry.auth.is_empty() { headers.count(b"authorization", b""); - headers.content.cap += b"Basic ".len() + pm.options.scope.auth.len(); + headers.content.cap += b"Basic ".len() + registry.auth.len(); } headers.allocate()?; headers.append(b"accept", b"application/json"); headers.append(b"content-type", b"application/json"); headers.append(b"content-encoding", b"gzip"); - if !pm.options.scope.token.is_empty() { + if !registry.token.is_empty() { headers.append_fmt( b"authorization", - format_args!("Bearer {}", BStr::new(&pm.options.scope.token)), + format_args!("Bearer {}", BStr::new(®istry.token)), ); - } else if !pm.options.scope.auth.is_empty() { + } else if !registry.auth.is_empty() { headers.append_fmt( b"authorization", - format_args!("Basic {}", BStr::new(&pm.options.scope.auth)), + format_args!("Basic {}", BStr::new(®istry.auth)), ); } @@ -495,7 +593,7 @@ fn send_audit_request( write!( &mut url_str, "{}/-/npm/v1/security/advisories/bulk", - BStr::new(strings::without_trailing_slash(pm.options.scope.url.href())) + BStr::new(®istry.href) ) .expect("unreachable"); let url = URL::parse(&url_str); @@ -520,12 +618,18 @@ fn send_audit_request( let res = match req.send_sync(&mut response_buf) { Ok(r) => r, Err(err) => { + if !registry.is_default { + return Ok(None); + } Output::err(err, "audit request failed", ()); Global::crash(); } }; if res.status_code() >= 400 { + if !registry.is_default { + return Ok(None); + } bun_core::pretty_errorln!( "error: audit request failed (status {})", res.status_code() @@ -533,7 +637,15 @@ fn send_audit_request( Global::crash(); } - Ok(Box::<[u8]>::from(response_buf.list.as_slice())) + let response = response_buf.list.as_slice(); + if !registry.is_default { + let trimmed = response.trim_ascii(); + if !trimmed.is_empty() && trimmed[0] != b'{' { + return Ok(None); + } + } + + Ok(Some(Box::<[u8]>::from(response))) } fn parse_vulnerability( diff --git a/src/runtime/cli/mod.rs b/src/runtime/cli/mod.rs index a18d1134044d..f609dfdd4a0b 100644 --- a/src/runtime/cli/mod.rs +++ b/src/runtime/cli/mod.rs @@ -1015,7 +1015,7 @@ pub mod command { if x == RootCommandMatcher::case(b"add") || x == RootCommandMatcher::case(b"a") { return Tag::AddCommand; } - if x == RootCommandMatcher::case(b"update") { + if x == RootCommandMatcher::case(b"update") || x == RootCommandMatcher::case(b"up") { return Tag::UpdateCommand; } if x == RootCommandMatcher::case(b"patch") { diff --git a/src/runtime/cli/package_manager_command.rs b/src/runtime/cli/package_manager_command.rs index 1d267eecb449..63e6840a176a 100644 --- a/src/runtime/cli/package_manager_command.rs +++ b/src/runtime/cli/package_manager_command.rs @@ -1,6 +1,7 @@ use core::cmp::Ordering; use std::io::Write as _; +use bun_collections::DynamicBitSet; use bun_core::fmt::PathSep; use bun_core::strings; use bun_core::{Global, Output, env_var, fmt as bun_fmt}; @@ -17,7 +18,7 @@ use bun_resolver::fs as Fs; use bun_sys::{self, Dir, Fd, File}; use crate::cli::Command; -use crate::cli::pm_licenses_command::PmLicensesCommand; +use crate::cli::pm_licenses_command::{LicensesFlags, PmLicensesCommand}; use crate::cli::pm_pkg_command::PmPkgCommand; use crate::cli::pm_trusted_command::{DefaultTrustedCommand, TrustCommand, UntrustedCommand}; use crate::cli::pm_version_command::PmVersionCommand; @@ -162,7 +163,10 @@ impl PackageManagerCommand { bun pm why \\ show dependency tree explaining why a package is installed\n\ bun pm licenses list installed packages grouped by license\n\ ├ --json output as JSON\n\ - └ --prod omit devDependencies\n\ + ├ --prod omit devDependencies\n\ + ├ --dev list only what devDependencies pull in\n\ + ├ --long also print author, description and homepage\n\ + └ --filter \\ list only the matching workspaces' dependencies\n\ bun pm whoami print the current npm username\n\ bun pm view name[@version] view package metadata from the registry (use `bun info` instead)\n\ bun pm version [increment] bump the version in package.json and create a git tag\n\ @@ -206,7 +210,10 @@ Learn more about these at https://bun.com/docs/cli/pm.\n"; .is_some_and(|arg| strings::eql_comptime(arg.as_bytes(), b"whoami")); let cli = CommandLineArguments::parse(Subcommand::Pm)?; - let production = cli.production; + let licenses_flags = LicensesFlags { + dev_only: cli.dev_only, + long: cli.long, + }; let (pm, cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Pm) { Ok(v) => v, Err(err) => { @@ -250,6 +257,12 @@ Learn more about these at https://bun.com/docs/cli/pm.\n"; subcommand = b"ls"; } + if !pm.options.filter_patterns.is_empty() && !strings::eql_comptime(subcommand, b"licenses") + { + Output::err_generic("--filter is only supported by `bun pm licenses`", ()); + Global::exit(1); + } + if pm.options.global { setup_global_dir(pm, &&mut *ctx)?; } @@ -708,7 +721,7 @@ Learn more about these at https://bun.com/docs/cli/pm.\n"; Global::exit(0); } else if strings::eql_comptime(subcommand, b"licenses") { let positionals: &[&[u8]] = pm.options.positionals; - PmLicensesCommand::exec(pm, positionals, production)?; + PmLicensesCommand::exec(pm, positionals, &cwd, licenses_flags)?; Global::exit(0); } else if strings::eql_comptime(subcommand, b"pkg") { let positionals: &[&[u8]] = pm.options.positionals; @@ -936,7 +949,7 @@ fn print_trusted_dependencies_flat( let pkg_names = slice.items_name(); let pkg_count = lockfile.packages.len(); - let mut seen: Vec = vec![false; pkg_count]; + let mut seen = bun_core::handle_oom(DynamicBitSet::init_empty(pkg_count)); let mut trusted: Vec = Vec::new(); let mut visit = |dep_id: DependencyID| { @@ -944,13 +957,13 @@ fn print_trusted_dependencies_flat( if package_id as usize >= pkg_count { return; } - if seen[package_id as usize] { + if seen.is_set(package_id as usize) { return; } let alias = dependencies[dep_id as usize].name.slice(string_bytes); let pkg_name = pkg_names[package_id as usize].slice(string_bytes); if lockfile.has_trusted_dependency(alias, pkg_name, &resolutions[package_id as usize]) { - seen[package_id as usize] = true; + seen.set(package_id as usize); trusted.push(dep_id); } }; diff --git a/src/runtime/cli/pm_licenses_command.rs b/src/runtime/cli/pm_licenses_command.rs index 2a7ed4b65f79..38d1eb8f204a 100644 --- a/src/runtime/cli/pm_licenses_command.rs +++ b/src/runtime/cli/pm_licenses_command.rs @@ -1,13 +1,14 @@ +use std::borrow::Cow; use std::cmp::Ordering; use std::io::Write as _; use bstr::BStr; use bun_ast::{Expr, Log, Source}; -use bun_collections::StringHashMap; +use bun_collections::{DynamicBitSet, StringHashMap}; use bun_core::fmt::PathSep; use bun_core::{FileKind, Global, Output, strings}; use bun_install::lockfile::{Lockfile, package::PackageColumns as _, reachable, tree}; -use bun_install::{PackageManager, Resolution, ResolutionTag}; +use bun_install::{PackageID, PackageManager, Resolution, ResolutionTag, WorkspaceFilter}; use bun_parsers::json as JSON; use bun_paths::AutoAbsPath; use bun_sys::{self, Dir, Fd, File}; @@ -23,6 +24,7 @@ struct PackageInfo { license: Box<[u8]>, homepage: Option>, author: Option>, + description: Option>, } struct Entry { @@ -32,6 +34,8 @@ struct Entry { semver: Option, homepage: Option>, author: Option>, + description: Option>, + dev_only: bool, } /// Lazily-scanned, sorted entry names of `node_modules/.bun/`; `None` until first needed. @@ -43,13 +47,20 @@ struct DiskIndex { entries: Option>, } +#[derive(Clone, Copy)] +pub(crate) struct LicensesFlags { + pub(crate) dev_only: bool, + pub(crate) long: bool, +} + pub(crate) struct PmLicensesCommand; impl PmLicensesCommand { pub(crate) fn exec( pm: &mut PackageManager, positionals: &[&[u8]], - _production: bool, + original_cwd: &[u8], + flags: LicensesFlags, ) -> crate::Result<()> { if positionals.len() > 1 && !strings::eql_comptime(positionals[1], b"list") @@ -65,9 +76,13 @@ impl PmLicensesCommand { let json_output = pm.options.json_output; let features = pm.options.local_package_features; - let root_id = pm.root_package_id.get(&pm.lockfile, pm.workspace_name_hash); let lockfile: &Lockfile = &pm.lockfile; + if flags.dev_only && !features.dev_dependencies { + Output::err_generic("--dev cannot be combined with --prod or --omit=dev", ()); + Global::exit(1); + } + let mut path = AutoAbsPath::init_top_level_dir(); let top_len = path.len(); let _ = path.append(b"node_modules"); @@ -77,19 +92,60 @@ impl PmLicensesCommand { } path.set_length(top_len); - let wanted = reachable::packages( + let filter_patterns = pm.options.filter_patterns; + let (roots, follow_workspace_edges): (Vec, bool) = if filter_patterns.is_empty() + { + ( + vec![pm.root_package_id.get(lockfile, pm.workspace_name_hash)], + true, + ) + } else { + let roots = WorkspaceFilter::select_workspaces(lockfile, filter_patterns, original_cwd); + if roots.is_empty() { + Output::err_generic( + "No workspace packages matched the filter {}", + (BStr::new("ed_patterns(filter_patterns)),), + ); + Global::exit(1); + } + (roots, false) + }; + + let options = reachable::Options { + root: 0, + dev: features.dev_dependencies, + optional: features.optional_dependencies, + peer: features.peer_dependencies, + optional_peer: features.peer_dependencies, + bundled: true, + platform: Some((pm.options.cpu, pm.options.os)), + }; + let resolutions = lockfile.buffers.resolutions.as_slice(); + let walk = if flags.dev_only { + reachable::dev_packages_from + } else { + reachable::packages_from + }; + let wanted = walk( lockfile, - lockfile.buffers.resolutions.as_slice(), - reachable::Options { - root: root_id, - dev: features.dev_dependencies, - optional: features.optional_dependencies, - peer: features.peer_dependencies, - optional_peer: features.peer_dependencies, - bundled: true, - platform: Some((pm.options.cpu, pm.options.os)), - }, + resolutions, + &roots, + follow_workspace_edges, + options, ); + let production: Option = + (!json_output && features.dev_dependencies).then(|| { + reachable::packages_from( + lockfile, + resolutions, + &roots, + follow_workspace_edges, + reachable::Options { + dev: false, + ..options + }, + ) + }); let locations = tree_locations(lockfile); let packages = lockfile.packages.slice(); @@ -104,7 +160,7 @@ impl PmLicensesCommand { let mut missing: usize = 0; for pkg_id in 0..packages.len() { - if !wanted[pkg_id] { + if !wanted.is_set(pkg_id) { continue; } let resolution = &pkg_resolution[pkg_id]; @@ -173,6 +229,8 @@ impl PmLicensesCommand { semver: is_npm.then(|| resolution.npm().version), homepage: info.homepage, author: info.author, + description: info.description, + dev_only: production.as_ref().is_some_and(|prod| !prod.is_set(pkg_id)), }); } @@ -199,7 +257,7 @@ impl PmLicensesCommand { if json_output { print_json(&entries); } else { - print_text(&entries); + print_text(&entries, flags.long); } Output::flush(); @@ -215,6 +273,32 @@ fn sort_key(e: &Entry) -> (bool, &[u8], &[u8]) { ) } +fn quoted_patterns(patterns: &[&[u8]]) -> Vec { + let mut out: Vec = Vec::new(); + for (i, pattern) in patterns.iter().enumerate() { + if i > 0 { + out.extend_from_slice(b", "); + } + out.push(b'"'); + out.extend_from_slice(pattern); + out.push(b'"'); + } + out +} + +fn printable(s: &[u8]) -> Cow<'_, [u8]> { + if s.iter().any(u8::is_ascii_control) { + Cow::Owned( + s.iter() + .copied() + .filter(|b| !b.is_ascii_control()) + .collect(), + ) + } else { + Cow::Borrowed(s) + } +} + fn tree_locations(lockfile: &Lockfile) -> Vec>> { let len = lockfile.packages.len(); let mut out: Vec>> = vec![None; len]; @@ -269,6 +353,7 @@ fn read_package_info(path: &[u8], log: &mut Log) -> Option { license: license_of(&json), homepage: string_field(&json, b"homepage"), author: author_of(&json), + description: string_field(&json, b"description"), }, Err(_) => PackageInfo { name: None, @@ -276,6 +361,7 @@ fn read_package_info(path: &[u8], log: &mut Log) -> Option { license: UNKNOWN_LICENSE.into(), homepage: None, author: None, + description: None, }, }, ) @@ -521,7 +607,12 @@ impl DiskIndex { } } -fn print_text(entries: &[Entry]) { +fn print_text(entries: &[Entry], long: bool) { + if entries.is_empty() { + bun_core::prettyln!("No packages found"); + return; + } + let mut start = 0; while start < entries.len() { let license = &entries[start].license; @@ -533,20 +624,34 @@ fn print_text(entries: &[Entry]) { if start > 0 { Output::print(format_args!("\n")); } - bun_core::prettyln!("{} ({})", BStr::new(license), end - start); + bun_core::prettyln!( + "{} ({})", + BStr::new(&printable(license)), + end - start + ); for (i, entry) in entries[start..end].iter().enumerate() { - if start + i + 1 < end { - bun_core::prettyln!( - "├── {}@{}", - BStr::new(&entry.name), - BStr::new(&entry.version) - ); - } else { - bun_core::prettyln!( - "└── {}@{}", - BStr::new(&entry.name), - BStr::new(&entry.version) - ); + let last = start + i + 1 == end; + bun_core::pretty!( + "{} {}@{}", + if last { "└──" } else { "├──" }, + BStr::new(&entry.name), + BStr::new(&entry.version) + ); + if entry.dev_only { + bun_core::pretty!(" (dev)"); + } + Output::print(format_args!("\n")); + if long { + for field in [&entry.author, &entry.description, &entry.homepage] + .into_iter() + .flatten() + { + if last { + bun_core::prettyln!(" {}", BStr::new(&printable(field))); + } else { + bun_core::prettyln!("│ {}", BStr::new(&printable(field))); + } + } } } @@ -615,6 +720,8 @@ fn print_json(entries: &[Entry]) { previous = Some(&entry.version[..]); } out.push(b']'); + out.extend_from_slice(b",\n \"license\": "); + json_string(&mut out, license); let newest = &entries[group_end - 1]; if let Some(homepage) = &newest.homepage { out.extend_from_slice(b",\n \"homepage\": "); @@ -624,6 +731,10 @@ fn print_json(entries: &[Entry]) { out.extend_from_slice(b",\n \"author\": "); json_string(&mut out, author); } + if let Some(description) = &newest.description { + out.extend_from_slice(b",\n \"description\": "); + json_string(&mut out, description); + } out.extend_from_slice(b"\n }"); group_start = group_end; diff --git a/src/runtime/cli/prune_command.rs b/src/runtime/cli/prune_command.rs index 4849f4a7c08b..f575a2ca306e 100644 --- a/src/runtime/cli/prune_command.rs +++ b/src/runtime/cli/prune_command.rs @@ -19,7 +19,7 @@ impl PruneCommand { Global::exit(1); } - let (manager, _original_cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Prune) + let (manager, original_cwd) = match PackageManager::init(&mut *ctx, cli, Subcommand::Prune) { Ok(v) => v, Err(bun_install::Error::MissingPackageJSON) => { @@ -37,6 +37,6 @@ impl PruneCommand { Output::flush(); } - bun_install::prune::prune(manager).map_err(crate::Error::from) + bun_install::prune::prune(manager, &original_cwd).map_err(crate::Error::from) } } diff --git a/test/cli/install/__snapshots__/bun-audit.test.ts.snap b/test/cli/install/__snapshots__/bun-audit.test.ts.snap index ff068d15bc9d..f4c386a81c1a 100644 --- a/test/cli/install/__snapshots__/bun-audit.test.ts.snap +++ b/test/cli/install/__snapshots__/bun-audit.test.ts.snap @@ -69,6 +69,9 @@ serve-static <1.16.0 21 vulnerabilities (2 critical, 9 high, 4 moderate, 6 low) +To upgrade only the vulnerable packages, within their declared ranges: + bun audit fix + To update all dependencies to the latest compatible versions: bun update @@ -418,6 +421,9 @@ exports[`\`bun audit\` should exit 1 and behave exactly the same when there are 2 vulnerabilities (1 high, 1 moderate) +To upgrade only the vulnerable packages, within their declared ranges: + bun audit fix + To update all dependencies to the latest compatible versions: bun update @@ -435,6 +441,9 @@ exports[`\`bun audit\` when a project has some safe dependencies and some vulner 2 vulnerabilities (1 high, 1 moderate) +To upgrade only the vulnerable packages, within their declared ranges: + bun audit fix + To update all dependencies to the latest compatible versions: bun update diff --git a/test/cli/install/bun-add-catalog.test.ts b/test/cli/install/bun-add-catalog.test.ts index a325d6ac25c7..9e1cdbf0ea04 100644 --- a/test/cli/install/bun-add-catalog.test.ts +++ b/test/cli/install/bun-add-catalog.test.ts @@ -1,7 +1,7 @@ import { file, write } from "bun"; import { readTarball } from "bun:internal-for-testing"; import { afterAll, beforeAll, describe, expect, test } from "bun:test"; -import { VerdaccioRegistry, bunEnv, bunExe, runBunInstall } from "harness"; +import { VerdaccioRegistry, bunEnv, bunExe, readdirSorted, runBunInstall } from "harness"; import { join } from "path"; const registry = new VerdaccioRegistry(); @@ -15,6 +15,8 @@ afterAll(() => { }); const PKG1 = JSON.stringify({ name: "pkg1", version: "1.0.0" }); +// `bun add` re-prints the package.json it edits; fixtures asserted byte-for-byte afterwards are written in that shape. +const pretty = (json: Record) => JSON.stringify(json, null, 2); // CI's per-file BUN_INSTALL_CACHE_DIR overrides bunfig's cache; concurrent installs sharing it race on Windows. function envFor(packageDir: string) { @@ -38,9 +40,10 @@ async function createDir( root: Record | string, pkg1: Record | string = PKG1, extraFiles: Record = {}, + linker: "hoisted" | "isolated" = "hoisted", ) { const { packageDir } = await registry.createTestDir({ - bunfigOpts: { linker: "hoisted", saveTextLockfile: true }, + bunfigOpts: { linker, saveTextLockfile: true }, files: { "package.json": typeof root === "string" ? root : JSON.stringify(root), "packages/pkg1/package.json": typeof pkg1 === "string" ? pkg1 : JSON.stringify(pkg1), @@ -75,6 +78,14 @@ async function createDir( lock: async () => Bun.JSONC.parse(await file(join(packageDir, "bun.lock")).text()) as any, installed: (name: string) => file(join(packageDir, "node_modules", name, "package.json")).json(), installedExists: (name: string) => file(join(packageDir, "node_modules", name, "package.json")).exists(), + memberInstalled: (member: string, name: string) => + file(join(packageDir, "packages", member, "node_modules", name, "package.json")).json(), + store: () => readdirSorted(join(packageDir, "node_modules", ".bun")), + frozen: () => spawnBun(packageDir, ["install", "--frozen-lockfile"], env), + installSavesNothing: async () => { + const { err } = await runBunInstall(env, packageDir, { savesLockfile: false }); + expect(err).not.toContain("Saved lockfile"); + }, }; } @@ -210,26 +221,175 @@ describe.concurrent("bun add --catalog", () => { } }); - test("refreshes an existing catalog entry", async () => { - const dir = await createDir(workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), { - name: "pkg1", - version: "1.0.0", - dependencies: { "no-deps": "catalog:" }, - }); + // pnpm keeps an existing entry as written when a bare name is added to the catalog. + test("a bare name reuses an existing catalog entry", async () => { + const pkg1 = pretty({ name: "pkg1", version: "1.0.0", dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir(workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), pkg1); await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + const lockBefore = await dir.lockText(); expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); - expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); - expect((await dir.installed("no-deps")).version).toBe("2.0.0"); - expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect(await dir.pkg1Text()).toBe(pkg1); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + expect(await dir.lockText()).toBe(lockBefore); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "^1.0.0" }); + + await dir.installSavesNothing(); + }); + + test("an existing entry is reused by a new consumer without touching the other members", async () => { + const pkg1 = JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir(workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), pkg1, withPkg2()); + await dir.install(); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + expectOk(await dir.add(dir.pkg2Dir, "no-deps", "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect((await dir.pkg2()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect(await dir.pkg1Text()).toBe(pkg1); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + const lock = await dir.lock(); + expect(lock.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect(lock.workspaces["packages/pkg2"].dependencies).toEqual({ "no-deps": "catalog:" }); + expect(await dir.lockText()).not.toContain("no-deps@2.0.0"); + + await dir.installSavesNothing(); + const { stderr, exitCode } = await dir.frozen(); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + test("an existing entry wins over the range the target declares directly", async () => { + const pkg2 = JSON.stringify({ name: "pkg2", dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir( + workspacesObject({ catalog: { "no-deps": "1.0.0" } }), + { name: "pkg1", dependencies: { "no-deps": "^1.0.0" } }, + withPkg2(pkg2), + ); + await dir.install(); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "1.0.0" }); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect(await dir.pkg2Text()).toBe(pkg2); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); const lockText = await dir.lockText(); expect(lockText).not.toContain("no-deps@1.1.0"); - expect(lockText).toContain("no-deps@2.0.0"); - expect((await dir.lock()).catalog).toEqual({ "no-deps": "^2.0.0" }); + expect(lockText).not.toContain("no-deps@2.0.0"); + + await dir.installSavesNothing(); + }); + + test("an explicit version equal to the existing entry leaves the root as it was", async () => { + const root = pretty(workspacesObject({ catalog: { "a-dep": "1.0.1", "no-deps": "1.0.0" } })); + const dir = await createDir(root); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps@1.0.0", "--catalog")); + + expect(await dir.rootText()).toBe(root); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + expect((await dir.lock()).catalog).toEqual({ "a-dep": "1.0.1", "no-deps": "1.0.0" }); + }); + + describe("explicit version inside an existing range", () => { + for (const linker of ["hoisted", "isolated"] as const) { + test(`keeps the range and moves the installed version (${linker})`, async () => { + const member = (name: string) => pretty({ name, dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir( + workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), + member("pkg1"), + withPkg2(member("pkg2")), + linker, + ); + const installedVersions = async () => + linker === "isolated" + ? [ + (await dir.memberInstalled("pkg1", "no-deps")).version, + (await dir.memberInstalled("pkg2", "no-deps")).version, + ] + : [(await dir.installed("no-deps")).version]; + await dir.install(); + expect(await installedVersions()).toEqual(linker === "isolated" ? ["1.1.0", "1.1.0"] : ["1.1.0"]); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps@1.0.0", "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect(await dir.pkg1Text()).toBe(member("pkg1")); + expect(await dir.pkg2Text()).toBe(member("pkg2")); + expect(await installedVersions()).toEqual(linker === "isolated" ? ["1.0.0", "1.0.0"] : ["1.0.0"]); + if (linker === "isolated") { + expect(await dir.store()).toContain("no-deps@1.0.0"); + } + + const lock = await dir.lock(); + expect(lock.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect(lock.packages["no-deps"][0]).toBe("no-deps@1.0.0"); + expect(await dir.lockText()).not.toContain("no-deps@1.1.0"); + + await dir.installSavesNothing(); + expect(await installedVersions()).toEqual(linker === "isolated" ? ["1.0.0", "1.0.0"] : ["1.0.0"]); + const { stderr, exitCode } = await dir.frozen(); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + } + + test("a range that differs replaces the entry", async () => { + const member = (name: string) => JSON.stringify({ name, dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir( + workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), + member("pkg1"), + withPkg2(member("pkg2")), + ); + await dir.install(); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps@^2.0.0", "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect(await dir.pkg2Text()).toBe(member("pkg2")); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "^2.0.0" }); + }); + }); + + test("a dist-tag already in package.json is cataloged as that tag's range", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), { + name: "pkg1", + dependencies: { "dep-with-tags": "pre-1" }, + }); + + expectOk(await dir.add(dir.pkg1Dir, "dep-with-tags", "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "dep-with-tags": "^1.0.1" }); + expect((await dir.pkg1()).dependencies).toEqual({ "dep-with-tags": "catalog:" }); + expect((await dir.installed("dep-with-tags")).version).toBe("1.0.1"); + expect((await dir.lock()).catalog).toEqual({ "dep-with-tags": "^1.0.1" }); + + await dir.installSavesNothing(); + }); + + test("a direct exact pin in devDependencies is cataloged verbatim", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), { + name: "pkg1", + devDependencies: { "no-deps": "1.0.0" }, + }); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog", "--dev")); + + expect(await dir.pkg1()).toEqual({ name: "pkg1", devDependencies: { "no-deps": "catalog:" } }); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "1.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "1.0.0" }); + + await dir.installSavesNothing(); }); test("run from the workspace root", async () => { @@ -290,7 +450,7 @@ describe.concurrent("bun add --catalog", () => { expect(stderr).not.toContain("error:"); expect(await dir.rootText()).toBe(rootBefore); expect(await dir.pkg1Text()).toBe(pkg1Before); - expect(await file(join(dir.packageDir, "bun.lock")).exists()).toBeFalse(); + expect(await dir.lockExists()).toBeFalse(); expect(exitCode).toBe(0); }); } @@ -310,8 +470,11 @@ describe.concurrent("bun add --catalog", () => { devDependencies: { "no-deps": "catalog:" }, peerDependencies: { "no-deps": ">=1" }, }); - expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); - expect((await dir.lock()).workspaces["packages/pkg1"]).toEqual({ + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "1.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + const lock = await dir.lock(); + expect(lock.catalog).toEqual({ "no-deps": "1.0.0" }); + expect(lock.workspaces["packages/pkg1"]).toEqual({ name: "pkg1", devDependencies: { "no-deps": "catalog:" }, peerDependencies: { "no-deps": ">=1" }, @@ -349,31 +512,60 @@ describe.concurrent("bun add --catalog", () => { }); }); - test("target already on a named catalog reference is moved to the flag's catalog", async () => { - const dir = await createDir(workspacesObject({ catalogs: { testing: { "no-deps": "1.0.0" } } }), { - name: "pkg1", - dependencies: { "no-deps": "catalog:testing" }, - }); - await dir.install(); - expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + // pnpm: a dependency already on `catalog:` stays in that catalog; the flag only picks the catalog for new references. + describe("target already on a named catalog reference stays in that catalog", () => { + const pkg1 = pretty({ name: "pkg1", dependencies: { "no-deps": "catalog:testing" } }); + const testingRoot = workspacesObject({ catalogs: { testing: { "no-deps": "1.0.0" } } }); - expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); + for (const { args, entry, installed } of [ + { args: ["no-deps", "--catalog"], entry: "1.0.0", installed: "1.0.0" }, + { args: ["no-deps", "--catalog=other"], entry: "1.0.0", installed: "1.0.0" }, + { args: ["no-deps@1.1.0", "--catalog"], entry: "1.1.0", installed: "1.1.0" }, + ]) { + test(`bun add ${args.join(" ")}`, async () => { + const dir = await createDir(testingRoot, pkg1); + await dir.install(); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); - expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); - expect((await dir.root()).workspaces).toEqual({ - packages: ["packages/*"], - catalogs: { testing: { "no-deps": "1.0.0" } }, - catalog: { "no-deps": "^2.0.0" }, - }); - expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + expectOk(await dir.add(dir.pkg1Dir, ...args)); - const lock = await dir.lock(); - expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:" }); - expect(lock.catalog).toEqual({ "no-deps": "^2.0.0" }); - expect(lock.catalogs).toEqual({ testing: { "no-deps": "1.0.0" } }); + expect((await dir.root()).workspaces).toEqual({ + packages: ["packages/*"], + catalogs: { testing: { "no-deps": entry } }, + }); + expect(await dir.pkg1Text()).toBe(pkg1); + expect((await dir.installed("no-deps")).version).toBe(installed); - const { err } = await dir.install({ savesLockfile: false }); - expect(err).not.toContain("Saved lockfile"); + const lock = await dir.lock(); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:testing" }); + expect(lock.catalog).toBeUndefined(); + expect(lock.catalogs).toEqual({ testing: { "no-deps": entry } }); + + await dir.installSavesNothing(); + }); + } + + test("an explicit version inside the named entry's range moves the resolution within that catalog", async () => { + const dir = await createDir(workspacesObject({ catalogs: { testing: { "no-deps": "^1.0.0" } } }), pkg1); + await dir.install(); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps@1.0.1", "--catalog")); + + expect((await dir.root()).workspaces).toEqual({ + packages: ["packages/*"], + catalogs: { testing: { "no-deps": "^1.0.0" } }, + }); + expect(await dir.pkg1Text()).toBe(pkg1); + expect((await dir.installed("no-deps")).version).toBe("1.0.1"); + + const lock = await dir.lock(); + expect(lock.catalog).toBeUndefined(); + expect(lock.catalogs).toEqual({ testing: { "no-deps": "^1.0.0" } }); + expect(lock.packages["no-deps"][0]).toBe("no-deps@1.0.1"); + + await dir.installSavesNothing(); + }); }); describe("literals other than a bare name", () => { @@ -435,19 +627,104 @@ describe.concurrent("bun add --catalog", () => { }); describe("workspace sibling", () => { - test("bare name fails without writing anything", async () => { + const pkg2Message = 'error: --catalog cannot add a workspace package, but "pkg2" is the workspace at packages/pkg2'; + + for (const from of ["member", "root"] as const) { + test(`bare name is refused before anything is written (from ${from})`, async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const cwd = from === "member" ? dir.pkg1Dir : dir.packageDir; + const { stderr, exitCode } = await dir.add(cwd, "pkg2", "--catalog"); + + expect(stderr).toContain(pkg2Message); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + } + + for (const positional of ["pkg2", "pkg2@1.0.0"]) { + test(`${positional} with --filter is refused`, async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = await dir.add(dir.packageDir, positional, "--catalog", "--filter", "pkg1"); + + expect(stderr).toContain(pkg2Message); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + } + + test("name@version is refused", async () => { const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); - const { stderr, exitCode } = await dir.add(dir.pkg1Dir, "pkg2", "--catalog"); + const { stderr, exitCode } = await dir.add(dir.pkg1Dir, "pkg2@1.0.0", "--catalog"); - expect(stderr).toContain("error:"); + expect(stderr).toContain(pkg2Message); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + + test("the root package's name is refused", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = await dir.add(dir.pkg1Dir, "root", "--catalog"); + + expect(stderr).toContain('error: --catalog cannot add a workspace package, but "root" is the workspace root'); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + + test("a sibling whose name also exists on the registry is refused, not cataloged from the registry", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, { + "packages/no-deps/package.json": JSON.stringify({ name: "no-deps", version: "9.0.0" }), + }); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = await dir.add(dir.pkg1Dir, "no-deps", "--catalog"); + + expect(stderr).toContain( + 'error: --catalog cannot add a workspace package, but "no-deps" is the workspace at packages/no-deps', + ); expect(await dir.rootText()).toBe(rootBefore); expect(await dir.pkg1Text()).toBe(pkg1Before); expect(await dir.lockExists()).toBeFalse(); - expect(exitCode).not.toBe(0); + expect(await dir.installedExists("no-deps")).toBeFalse(); + expect(exitCode).toBe(1); }); + for (const from of ["member", "root --filter pkg1"] as const) { + test(`one refused name aborts the whole add before any network request (from ${from})`, async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = + from === "member" + ? await dir.add(dir.pkg1Dir, "a-dep", "pkg2", "--catalog") + : await dir.add(dir.packageDir, "a-dep", "pkg2", "--catalog", "--filter", "pkg1"); + + expect(stderr).toContain(pkg2Message); + expect(stderr).not.toContain("404"); + expect(stderr).not.toContain("failed to resolve"); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(await dir.installedExists("a-dep")).toBeFalse(); + expect(exitCode).toBe(1); + }); + } + for (const from of ["member", "root"] as const) { test(`name@workspace:* is rejected (from ${from})`, async () => { const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, withPkg2()); @@ -478,7 +755,159 @@ describe.concurrent("bun add --catalog", () => { }); }); + describe("local paths", () => { + for (const spec of ["foo@file:../vendor/foo", "foo@link:../vendor/foo", "foo@./vendor/foo"]) { + test(`${spec} is refused`, async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = await dir.add(dir.pkg1Dir, spec, "--catalog"); + + expect(stderr).toContain( + `error: --catalog cannot add "${spec}": a local path in the catalog would resolve from the workspace root, not from the package that added it`, + ); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + } + + test("a bare relative path is refused as a positional without a name", async () => { + const dir = await createDir(workspacesObject({ catalog: {} })); + const [rootBefore, pkg1Before] = await Promise.all([dir.rootText(), dir.pkg1Text()]); + + const { stderr, exitCode } = await dir.add(dir.pkg1Dir, "../../vendor/foo", "--catalog"); + + expect(stderr).toContain('error: --catalog can only add packages by name, but got "../../vendor/foo"'); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.pkg1Text()).toBe(pkg1Before); + expect(await dir.lockExists()).toBeFalse(); + expect(exitCode).toBe(1); + }); + + test("an absolute file: path is cataloged verbatim", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), PKG1, { + "vendor/foo/package.json": JSON.stringify({ name: "foo", version: "1.0.0" }), + }); + const literal = `file:${join(dir.packageDir, "vendor", "foo").replaceAll("\\", "/")}`; + + expectOk(await dir.add(dir.pkg1Dir, `foo@${literal}`, "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ foo: literal }); + expect((await dir.pkg1()).dependencies).toEqual({ foo: "catalog:" }); + expect((await dir.lock()).catalog).toEqual({ foo: literal }); + expect(await dir.installed("foo")).toMatchObject({ name: "foo", version: "1.0.0" }); + }); + }); + describe("--filter", () => { + test("'*' alone edits the members and the root catalog, not the root's dependencies", async () => { + const pkg1 = JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir(workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), pkg1, withPkg2()); + await dir.install(); + + expectOk(await dir.add(dir.packageDir, "a-dep", "--catalog", "--filter", "*")); + + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); + expect((await dir.pkg2()).dependencies).toEqual({ "a-dep": "catalog:" }); + const root = await dir.root(); + expect(root.dependencies).toBeUndefined(); + expect(root.workspaces.catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); + expect((await dir.lock()).catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); + + await dir.installSavesNothing(); + }); + + test("an existing entry is reused for every selected member", async () => { + const dir = await createDir(workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), PKG1, withPkg2()); + + expectOk(await dir.add(dir.packageDir, "no-deps", "--catalog", "--filter", "*")); + + const root = await dir.root(); + expect(root.workspaces.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect(root.dependencies).toBeUndefined(); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.pkg2()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "^1.0.0" }); + expect(await dir.lockText()).not.toContain("no-deps@2.0.0"); + + await dir.installSavesNothing(); + }); + + test("a range declared by any selected member is what gets cataloged", async () => { + const dir = await createDir( + workspacesObject({ catalog: {} }), + { name: "pkg1", dependencies: { "no-deps": "^1.0.0" } }, + withPkg2(), + ); + + expectOk(await dir.add(dir.packageDir, "no-deps", "--catalog", "--filter", "pkg1", "--filter", "pkg2")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.pkg2()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "^1.0.0" }); + + await dir.installSavesNothing(); + }); + + // Bun catalogs every selected member; the first selected member's range seeds the entry (pnpm leaves later members direct). + test("members declaring different ranges: the first selected member's range seeds the entry", async () => { + const dir = await createDir( + workspacesObject({ catalog: {} }), + { name: "pkg1", dependencies: { "no-deps": "1.0.0" } }, + withPkg2({ name: "pkg2", dependencies: { "no-deps": "1.0.1" } }), + ); + + expectOk(await dir.add(dir.packageDir, "no-deps", "--catalog", "--filter", "pkg*")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "1.0.0" }); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.pkg2()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + const lock = await dir.lock(); + expect(lock.catalog).toEqual({ "no-deps": "1.0.0" }); + expect(lock.packages["no-deps"][0]).toBe("no-deps@1.0.0"); + expect(await dir.lockText()).not.toContain("no-deps@1.0.1"); + + await dir.installSavesNothing(); + }); + + test("each member keeps its own catalog", async () => { + const pkg1 = pretty({ name: "pkg1", dependencies: { "no-deps": "catalog:legacy" } }); + const dir = await createDir( + workspacesObject({ catalogs: { legacy: { "no-deps": "1.0.0" } } }), + pkg1, + withPkg2({ name: "pkg2" }), + ); + + expectOk(await dir.add(dir.packageDir, "no-deps", "--catalog", "--filter", "pkg1", "--filter", "pkg2")); + + expect(await dir.pkg1Text()).toBe(pkg1); + expect((await dir.pkg2()).dependencies).toEqual({ "no-deps": "catalog:" }); + const root = await dir.root(); + expect(root.dependencies).toBeUndefined(); + expect(root.workspaces).toEqual({ + packages: ["packages/*"], + catalogs: { legacy: { "no-deps": "1.0.0" } }, + catalog: { "no-deps": "^2.0.0" }, + }); + + const lock = await dir.lock(); + expect(lock.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect(lock.catalogs).toEqual({ legacy: { "no-deps": "1.0.0" } }); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:legacy" }); + expect(lock.workspaces["packages/pkg2"].dependencies).toEqual({ "no-deps": "catalog:" }); + const lockText = await dir.lockText(); + expect(lockText).toContain("no-deps@1.0.0"); + expect(lockText).toContain("no-deps@2.0.0"); + + await dir.installSavesNothing(); + }); + test("edits only the filtered member and the root catalog", async () => { const pkg1 = JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:" } }); const dir = await createDir(workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), pkg1, withPkg2()); @@ -495,7 +924,7 @@ describe.concurrent("bun add --catalog", () => { expect((await dir.lock()).catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); expect((await dir.installed("a-dep")).version).toBe("1.0.10"); - expectOk(await dir.add(dir.packageDir, "a-dep", "--catalog", "--filter", "*")); + expectOk(await dir.add(dir.packageDir, "a-dep", "--catalog", "--filter", "*", "--filter", "root")); expect((await dir.root()).dependencies).toEqual({ "a-dep": "catalog:" }); expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); @@ -523,19 +952,10 @@ describe.concurrent("bun add --catalog", () => { const pkg1 = { name: "pkg1", dependencies: { "no-deps": "^1.0.0" } }; const dir = await createDir(workspacesObject({ catalog: {} }), pkg1); - const { stderr, exitCode } = await dir.add( - dir.packageDir, - "no-deps", - "--catalog", - "--only-missing", - "--filter", - "pkg1", - ); + expectOk(await dir.add(dir.packageDir, "no-deps", "--catalog", "--only-missing", "--filter", "pkg1")); - expect(stderr).not.toContain("panic:"); expect(await dir.pkg1()).toEqual(pkg1); expect(await dir.root()).toEqual(workspacesObject({ catalog: {} })); - expect(exitCode).toBe(0); }); }); @@ -576,6 +996,41 @@ describe.concurrent("bun add --catalog", () => { expect(err).not.toContain("Saved lockfile"); }); + // Bun deviates from pnpm here: pnpm keeps the entry and writes the version directly into the member. + describe("an explicit version the entry or the target's range does not cover replaces the entry", () => { + test("entry other members reference", async () => { + const member = (name: string) => JSON.stringify({ name, dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir( + workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), + member("pkg1"), + withPkg2(member("pkg2")), + ); + await dir.install(); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps@2.0.0", "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "2.0.0" }); + expect(await dir.pkg2Text()).toBe(member("pkg2")); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + expect(await dir.lockText()).not.toContain("no-deps@1.1.0"); + }); + + test("range declared directly by the target", async () => { + const dir = await createDir(workspacesObject({ catalog: {} }), { + name: "pkg1", + dependencies: { "no-deps": "^1.0.0" }, + }); + await dir.install(); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps@2.0.0", "--catalog")); + + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "2.0.0" }); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + }); + }); + test("other dependencies of the target are left alone", async () => { const dir = await createDir(workspacesObject({ catalog: {} }), { name: "pkg1", @@ -600,11 +1055,13 @@ describe.concurrent("bun add --catalog", () => { expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); - expect(await dir.pkg1Text()).toBe( - JSON.stringify({ name: "pkg1", dependencies: { "no-deps": "catalog:" } }, null, 2), - ); - expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); - expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + expect(await dir.pkg1Text()).toBe(pretty({ name: "pkg1", dependencies: { "no-deps": "catalog:" } })); + expect((await dir.root()).workspaces.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + expect((await dir.lock()).catalog).toEqual({ "no-deps": "^1.0.0" }); + expect(await dir.lockText()).not.toContain("no-deps@2.0.0"); + + await dir.installSavesNothing(); }); test("re-running the same add is idempotent", async () => { @@ -692,7 +1149,34 @@ describe.concurrent("bun add --catalog", () => { describe("default catalog alias", () => { const member = (name: string, ref = "catalog:") => JSON.stringify({ name, dependencies: { "no-deps": ref } }); - test("--catalog refreshes an existing catalogs.default instead of adding a second catalog", async () => { + const reuseRows: { flag: string; root: { catalog?: object; catalogs?: object }; reference: string }[] = [ + { flag: "--catalog", root: { catalogs: { default: { "no-deps": "1.0.0" } } }, reference: "catalog:" }, + { flag: "--catalog=default", root: { catalog: { "no-deps": "1.0.0" } }, reference: "catalog:default" }, + ]; + for (const { flag, root, reference } of reuseRows) { + test(`${flag} reuses the entry spelled ${Object.keys(root)[0]}`, async () => { + const otherReference = reference === "catalog:" ? "catalog:default" : "catalog:"; + const dir = await createDir(workspacesObject(root), PKG1, withPkg2(member("pkg2", otherReference))); + await dir.install(); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps", flag)); + + expect((await dir.root()).workspaces).toEqual({ packages: ["packages/*"], ...root }); + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": reference }); + expect(await dir.pkg2Text()).toBe(member("pkg2", otherReference)); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + + const lock = await dir.lock(); + expect(lock.catalog).toEqual(root.catalog); + expect(lock.catalogs).toEqual(root.catalogs); + expect(await dir.lockText()).not.toContain("no-deps@2.0.0"); + + await dir.installSavesNothing(); + }); + } + + test("--catalog with an explicit version replaces the catalogs.default entry instead of adding a second catalog", async () => { const dir = await createDir( workspacesObject({ catalogs: { default: { "no-deps": "1.0.0" } } }), member("pkg1"), @@ -701,11 +1185,11 @@ describe.concurrent("bun add --catalog", () => { await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.0.0"); - expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps@2.0.0", "--catalog")); expect((await dir.root()).workspaces).toEqual({ packages: ["packages/*"], - catalogs: { default: { "no-deps": "^2.0.0" } }, + catalogs: { default: { "no-deps": "2.0.0" } }, }); expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); expect(await dir.pkg2Text()).toBe(member("pkg2", "catalog:default")); @@ -713,15 +1197,15 @@ describe.concurrent("bun add --catalog", () => { const lock = await dir.lock(); expect(lock.catalog).toBeUndefined(); - expect(lock.catalogs).toEqual({ default: { "no-deps": "^2.0.0" } }); + expect(lock.catalogs).toEqual({ default: { "no-deps": "2.0.0" } }); expect(lock.packages["no-deps"][0]).toBe("no-deps@2.0.0"); - const { stderr, exitCode } = await dir.run(dir.packageDir, ["install", "--frozen-lockfile"]); + const { stderr, exitCode } = await dir.frozen(); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); }); - test("--catalog=default refreshes the singular catalog every catalog: reference resolves through", async () => { + test("--catalog=default with an explicit range replaces the singular catalog every catalog: reference resolves through", async () => { const dir = await createDir( workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), PKG1, @@ -730,7 +1214,7 @@ describe.concurrent("bun add --catalog", () => { await dir.install(); expect((await dir.installed("no-deps")).version).toBe("1.1.0"); - expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--catalog=default")); + expectOk(await dir.add(dir.pkg1Dir, "no-deps@^2.0.0", "--catalog=default")); expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:default" }); expect((await dir.root()).workspaces).toEqual({ @@ -745,8 +1229,7 @@ describe.concurrent("bun add --catalog", () => { expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:default" }); expect(await dir.lockText()).not.toContain("no-deps@1.1.0"); - const { err } = await dir.install({ savesLockfile: false }); - expect(err).not.toContain("Saved lockfile"); + await dir.installSavesNothing(); }); test("--catalog=default with no catalog defined creates the singular catalog", async () => { @@ -876,30 +1359,41 @@ describe.concurrent("bun add --catalog", () => { } // pnpm #12115 / #11591: update never replaces a `catalog:` reference, even with an override or an explicit spec. - test("catalog: references survive bun update with an override", async () => { + describe("catalog: references with an override", () => { const pkg1 = { name: "pkg1", dependencies: { "no-deps": "catalog:" } }; - const dir = await createDir( - { ...workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), overrides: { "no-deps": "1.0.0" } }, - pkg1, - ); - await dir.install(); - expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + const overriddenRoot = { + ...workspacesObject({ catalog: { "no-deps": "^1.0.0" } }), + overrides: { "no-deps": "1.0.0" }, + }; for (const args of [["update"], ["update", "--recursive"], ["update", "no-deps"], ["update", "no-deps@1.1.0"]]) { - expectOk(await dir.run(dir.pkg1Dir, args)); - expect(await dir.pkg1()).toEqual(pkg1); + test(`survive bun ${args.join(" ")}`, async () => { + const dir = await createDir(overriddenRoot, pkg1); + await dir.install(); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + + expectOk(await dir.run(dir.pkg1Dir, args)); + + expect(await dir.pkg1()).toEqual(pkg1); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + }); } - expectOk(await dir.add(dir.pkg1Dir, "a-dep", "--catalog")); - expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); - const lock = await dir.lock(); - expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); - expect(lock.catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); - expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + test("survive bun add --catalog of another package", async () => { + const dir = await createDir(overriddenRoot, pkg1); - const { stderr, exitCode } = await dir.run(dir.packageDir, ["install", "--frozen-lockfile"]); - expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); + expectOk(await dir.add(dir.pkg1Dir, "a-dep", "--catalog")); + + expect((await dir.pkg1()).dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); + const lock = await dir.lock(); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "a-dep": "catalog:", "no-deps": "catalog:" }); + expect(lock.catalog).toEqual({ "a-dep": "^1.0.10", "no-deps": "^1.0.0" }); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + + const { stderr, exitCode } = await dir.frozen(); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); }); // pnpm #9660: overrides win at resolution, so the catalog records the overridden version. @@ -917,6 +1411,178 @@ describe.concurrent("bun add --catalog", () => { expect(err).not.toContain("Saved lockfile"); }); + // pnpm: a bare `add ` in a workspace whose default catalog lists writes `catalog:`. + describe("plain bun add uses the default catalog", () => { + const defaultRoot = workspacesObject({ catalog: { "no-deps": "^1.0.0" } }); + + test("bare name from a member", async () => { + const dir = await createDir(defaultRoot); + const rootBefore = await dir.rootText(); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps")); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect(await dir.rootText()).toBe(rootBefore); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + const lock = await dir.lock(); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:" }); + expect(lock.catalog).toEqual({ "no-deps": "^1.0.0" }); + expect(await dir.lockText()).not.toContain("no-deps@2.0.0"); + + await dir.installSavesNothing(); + const { stderr, exitCode } = await dir.frozen(); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + }); + + test("bare name from the root", async () => { + const dir = await createDir(defaultRoot); + + expectOk(await dir.add(dir.packageDir, "no-deps")); + + const root = await dir.root(); + expect(root.dependencies).toEqual({ "no-deps": "catalog:" }); + expect(root.workspaces).toEqual(defaultRoot.workspaces); + expect(await dir.pkg1Text()).toBe(PKG1); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + const lock = await dir.lock(); + expect(lock.workspaces[""].dependencies).toEqual({ "no-deps": "catalog:" }); + expect(lock.catalog).toEqual({ "no-deps": "^1.0.0" }); + + await dir.installSavesNothing(); + }); + + test("--dev writes the reference into devDependencies", async () => { + const dir = await createDir(defaultRoot); + const rootBefore = await dir.rootText(); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps", "--dev")); + + const pkg1 = await dir.pkg1(); + expect(pkg1.devDependencies).toEqual({ "no-deps": "catalog:" }); + expect(pkg1.dependencies).toBeUndefined(); + expect(await dir.rootText()).toBe(rootBefore); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + expect((await dir.lock()).workspaces["packages/pkg1"].devDependencies).toEqual({ "no-deps": "catalog:" }); + }); + + for (const { title, args } of [ + { title: "bun install ", args: ["install", "no-deps"] }, + { title: "a spec equal to the entry text", args: ["add", "no-deps@^1.0.0"] }, + { title: "--exact", args: ["add", "no-deps", "--exact"] }, + { title: "-E", args: ["add", "no-deps", "-E"] }, + ]) { + test(`${title} writes the reference like a bare add`, async () => { + const dir = await createDir(defaultRoot); + const rootBefore = await dir.rootText(); + + expectOk(await dir.run(dir.pkg1Dir, args)); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect(await dir.rootText()).toBe(rootBefore); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + expect((await dir.lock()).workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:" }); + + await dir.installSavesNothing(); + }); + } + + test("--filter writes the reference into every selected member", async () => { + const dir = await createDir(defaultRoot, PKG1, withPkg2()); + const rootBefore = await dir.rootText(); + + expectOk(await dir.add(dir.packageDir, "no-deps", "--filter", "pkg1", "--filter", "pkg2")); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await dir.pkg2()).dependencies).toEqual({ "no-deps": "catalog:" }); + expect(await dir.rootText()).toBe(rootBefore); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + + const lock = await dir.lock(); + expect(lock.workspaces["packages/pkg1"].dependencies).toEqual({ "no-deps": "catalog:" }); + expect(lock.workspaces["packages/pkg2"].dependencies).toEqual({ "no-deps": "catalog:" }); + + await dir.installSavesNothing(); + }); + + test("re-adding a package already on catalog: keeps the reference", async () => { + const pkg1 = pretty({ name: "pkg1", dependencies: { "no-deps": "catalog:" } }); + const dir = await createDir(defaultRoot, pkg1); + await dir.install(); + const [rootBefore, lockBefore] = await Promise.all([dir.rootText(), dir.lockText()]); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps")); + + expect(await dir.pkg1Text()).toBe(pkg1); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.lockText()).toBe(lockBefore); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + }); + + test("an existing named reference is kept", async () => { + const pkg1 = pretty({ name: "pkg1", dependencies: { "no-deps": "catalog:libs" } }); + const dir = await createDir(workspacesObject({ catalogs: { libs: { "no-deps": "1.0.0" } } }), pkg1); + await dir.install(); + const [rootBefore, lockBefore] = await Promise.all([dir.rootText(), dir.lockText()]); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps")); + + expect(await dir.pkg1Text()).toBe(pkg1); + expect(await dir.rootText()).toBe(rootBefore); + expect(await dir.lockText()).toBe(lockBefore); + expect((await dir.installed("no-deps")).version).toBe("1.0.0"); + }); + + describe("explicit versions and unlisted names are added normally", () => { + for (const { spec, written, installed } of [ + { spec: "no-deps@1.0.0", written: { "no-deps": "1.0.0" }, installed: "1.0.0" }, + { spec: "no-deps@latest", written: { "no-deps": "^2.0.0" }, installed: "2.0.0" }, + { spec: "no-deps@^2.0.0", written: { "no-deps": "^2.0.0" }, installed: "2.0.0" }, + { spec: "a-dep", written: { "a-dep": "^1.0.10" }, installed: "1.0.10" }, + { spec: "@types/no-deps", written: { "@types/no-deps": "^2.0.0" }, installed: "2.0.0" }, + { spec: "foo@npm:no-deps", written: { foo: "npm:no-deps@^2.0.0" }, installed: "2.0.0" }, + ]) { + test(`bun add ${spec}`, async () => { + const dir = await createDir(defaultRoot); + const rootBefore = await dir.rootText(); + + expectOk(await dir.add(dir.pkg1Dir, spec)); + + expect((await dir.pkg1()).dependencies).toEqual(written); + expect(await dir.rootText()).toBe(rootBefore); + expect((await dir.installed(Object.keys(written)[0])).version).toBe(installed); + }); + } + + test("an explicit version replaces a named reference", async () => { + const dir = await createDir(workspacesObject({ catalogs: { libs: { "no-deps": "1.0.0" } } }), { + name: "pkg1", + dependencies: { "no-deps": "catalog:libs" }, + }); + const rootBefore = await dir.rootText(); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps@1.1.0")); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "1.1.0" }); + expect(await dir.rootText()).toBe(rootBefore); + expect((await dir.installed("no-deps")).version).toBe("1.1.0"); + }); + + test("named catalogs are not used without the flag", async () => { + const dir = await createDir(workspacesObject({ catalogs: { libs: { "no-deps": "1.0.0" } } })); + const rootBefore = await dir.rootText(); + + expectOk(await dir.add(dir.pkg1Dir, "no-deps")); + + expect((await dir.pkg1()).dependencies).toEqual({ "no-deps": "^2.0.0" }); + expect(await dir.rootText()).toBe(rootBefore); + expect((await dir.installed("no-deps")).version).toBe("2.0.0"); + }); + }); + }); + describe("errors", () => { test("root package.json without workspaces", async () => { const { packageDir } = await registry.createTestDir({ @@ -955,7 +1621,7 @@ describe.concurrent("bun add --catalog", () => { expect(stderr).toContain("error: --catalog requires at least one package to add"); expect(await dir.rootText()).toBe(rootBefore); expect(await dir.pkg1Text()).toBe(pkg1Before); - expect(await file(join(dir.packageDir, "bun.lock")).exists()).toBeFalse(); + expect(await dir.lockExists()).toBeFalse(); expect(exitCode).toBe(1); }); diff --git a/test/cli/install/bun-add-filter.test.ts b/test/cli/install/bun-add-filter.test.ts index e4cd03482fe5..a0bde27ee226 100644 --- a/test/cli/install/bun-add-filter.test.ts +++ b/test/cli/install/bun-add-filter.test.ts @@ -1,9 +1,8 @@ import { file, write } from "bun"; import { afterAll, beforeAll, expect, test } from "bun:test"; -import { existsSync } from "fs"; import { exists, mkdir } from "fs/promises"; import { VerdaccioRegistry, bunEnv, bunExe } from "harness"; -import { dirname, join } from "path"; +import { join } from "path"; const registry = new VerdaccioRegistry(); @@ -21,6 +20,12 @@ const WEB = { name: "web", dependencies: { "a-dep": "1.0.1" } }; const PKG_A = { name: "pkg-a" }; const PKG_B = { name: "pkg-b" }; +// Workspace edges web -> api -> pkg-a; pkg-b is isolated. +const GRAPH = { + api: { name: "api", dependencies: { "pkg-a": "workspace:*" } }, + web: { name: "web", dependencies: { api: "workspace:*" } }, +}; + type Workspace = "root" | "api" | "web" | "pkg-a" | "pkg-b"; type Linker = "hoisted" | "isolated"; @@ -42,25 +47,16 @@ async function makeMonorepo(extra: Partial> = } // CI exports BUN_INSTALL_CACHE_DIR (one per test file), which overrides the per-test-dir bunfig `cache`; concurrent cases racing on one cache fail on Windows. -function envFor(cwd: string) { - let root = cwd; - while (!existsSync(join(root, "bunfig.toml"))) { - const parent = dirname(root); - if (parent === root) { - root = cwd; - break; - } - root = parent; - } - return { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(root, ".bun-cache") }; +function envFor(dir: string) { + return { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }; } // `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. -async function run(args: string[], cwd: string, linker?: Linker) { +async function run(args: string[], dir: string, opts: { linker?: Linker; cwd?: string } = {}) { await using proc = Bun.spawn({ - cmd: [bunExe(), ...args, ...(linker ? ["--linker", linker] : [])], - cwd, - env: envFor(cwd), + cmd: [bunExe(), ...args, ...(opts.linker ? ["--linker", opts.linker] : [])], + cwd: opts.cwd ?? dir, + env: envFor(dir), stdout: "pipe", stderr: "pipe", }); @@ -90,18 +86,52 @@ function allPackageJsonTexts(dir: string) { return Promise.all(WORKSPACES.map(w => pkgText(dir, w))); } +function lockfileJson(dir: string) { + return file(join(dir, "bun.lock")) + .text() + .then(t => JSON.parse(t.replace(/,(\s*[}\]])/g, "$1"))); +} + async function installOk(dir: string, linker?: Linker) { - const { stderr, exitCode } = await run(["install"], dir, linker); + const { stderr, exitCode } = await run(["install"], dir, { linker }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); } +/** The workspaces (in WORKSPACES order) whose package.json declares `dep` in any dependency group. */ +async function declaring(dir: string, dep: string) { + const jsons = await allPackageJsons(dir); + return WORKSPACES.filter((_, i) => + ["dependencies", "devDependencies", "optionalDependencies", "peerDependencies"].some( + group => jsons[i][group]?.[dep] !== undefined, + ), + ); +} + +/** `edited` gained `dep: range` in dependencies; every other workspace's package.json is byte-identical to `before`. */ +async function expectAddedOnlyTo( + dir: string, + before: string[], + edited: Workspace[], + dep = "no-deps", + range = "^2.0.0", +) { + expect(await declaring(dir, dep)).toEqual(WORKSPACES.filter(w => edited.includes(w))); + for (const [i, workspace] of WORKSPACES.entries()) { + if (edited.includes(workspace)) { + expect((await pkg(dir, workspace)).dependencies[dep]).toBe(range); + } else { + expect(await pkgText(dir, workspace)).toBe(before[i]); + } + } +} + test.concurrent( 'add --filter targets one workspace by name (and does not install an npm package called "api")', async () => { const dir = await makeMonorepo(); - const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api"], dir); + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api"], dir, { linker: "hoisted" }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -182,20 +212,458 @@ test.concurrent("glob filter edits every match", async () => { ]); }); -test.concurrent("'*' edits every workspace including the root; '!' excludes", async () => { - const dir = await makeMonorepo(); +test.concurrent("'*' edits every workspace except the root; '!' excludes", async () => { + const dir = await makeMonorepo({ root: '{ "name": "root", "workspaces": ["packages/*"] }' }); + const rootBefore = await pkgText(dir, "root"); const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "*", "--filter", "!api"], dir); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); expect(await allPackageJsons(dir)).toEqual([ - { ...ROOT, dependencies: { "no-deps": "^2.0.0" } }, + ROOT, API, { name: "web", dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }, { name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }, { name: "pkg-b", dependencies: { "no-deps": "^2.0.0" } }, ]); + expect(await pkgText(dir, "root")).toBe(rootBefore); + expect((await lockfileJson(dir)).workspaces[""]).toEqual({ name: "root" }); +}); + +test.concurrent("a negation-only filter set skips the root", async () => { + const dir = await makeMonorepo({ root: '{ "name": "root", "workspaces": ["packages/*"] }' }); + const [rootBefore, apiBefore] = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "!api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkgText(dir, "root")).toBe(rootBefore); + expect(await pkgText(dir, "api")).toBe(apiBefore); + expect(await pkg(dir, "web")).toEqual({ name: "web", dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "pkg-a")).toEqual({ name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }); + expect(await pkg(dir, "pkg-b")).toEqual({ name: "pkg-b", dependencies: { "no-deps": "^2.0.0" } }); +}); + +test.concurrent("every member negated: zero targets is an error, the root is not silently edited", async () => { + const dir = await makeMonorepo({ root: { ...ROOT, dependencies: { "no-deps": "^2.0.0" } } }); + await installOk(dir, "hoisted"); + const before = await allPackageJsonTexts(dir); + const lockBefore = await file(join(dir, "bun.lock")).text(); + + { + const { stderr, exitCode } = await run( + ["add", "a-dep", "--filter", "!api", "--filter", "!web", "--filter", "!pkg-*"], + dir, + ); + expect(stderr).toContain("error: No workspace packages matched the filter"); + expect(exitCode).toBe(1); + } + + { + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "!./packages/*"], dir); + expect(stderr).toContain("error: No workspace packages matched the filter"); + expect(exitCode).toBe(1); + } + + expect(await allPackageJsonTexts(dir)).toEqual(before); + expect(await file(join(dir, "bun.lock")).text()).toBe(lockBefore); + expect(await exists(join(dir, "node_modules", "no-deps", "package.json"))).toBeTrue(); +}); + +test.concurrent("the root is included by naming it next to '*'", async () => { + const dir = await makeMonorepo({ root: '{ "name": "root", "workspaces": ["packages/*"] }' }); + const rootBefore = await pkgText(dir, "root"); + + { + const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "*"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkgText(dir, "root")).toBe(rootBefore); + expect(await declaring(dir, "a-dep")).toEqual(["api", "web", "pkg-a", "pkg-b"]); + } + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "*", "--filter", "root"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await declaring(dir, "no-deps")).toEqual(WORKSPACES); + expect(await pkg(dir, "root")).toEqual({ ...ROOT, dependencies: { "no-deps": "^2.0.0" } }); + } +}); + +test.concurrent("'{.}' from the root selects the root and every workspace", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "{.}"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await declaring(dir, "no-deps")).toEqual(WORKSPACES); + expect(await pkg(dir, "root")).toEqual({ ...ROOT, dependencies: { "no-deps": "^2.0.0" } }); +}); + +test.concurrent("'{dir}' selects every workspace under the directory", async () => { + const dir = await makeMonorepo({ root: '{ "name": "root", "workspaces": ["packages/*"] }' }); + const rootBefore = await pkgText(dir, "root"); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "{packages}"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await declaring(dir, "no-deps")).toEqual(["api", "web", "pkg-a", "pkg-b"]); + expect(await pkgText(dir, "root")).toBe(rootBefore); + } + + // The braces resolve against the invoking directory. + { + const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "{..}"], dir, { + cwd: join(dir, "packages", "web"), + }); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const [, api, web, pkgA, pkgB] = await allPackageJsons(dir); + for (const json of [api, web, pkgA, pkgB]) { + expect(json.dependencies["a-dep"]).toBe("^1.0.10"); + } + expect(await pkgText(dir, "root")).toBe(rootBefore); + } +}); + +test.concurrent("'{dir}' naming one workspace directory selects just it; '!{dir}' excludes the subtree", async () => { + const dir = await makeMonorepo(); + const before = await allPackageJsonTexts(dir); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "{./packages/pkg-a}"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + await expectAddedOnlyTo(dir, before, ["pkg-a"]); + } + + { + const after = await allPackageJsonTexts(dir); + const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "*", "--filter", "!{./packages}"], dir); + expect(stderr).toContain('error: No workspace packages matched the filter "*", "!{./packages}"'); + expect(exitCode).toBe(1); + expect(await allPackageJsonTexts(dir)).toEqual(after); + } +}); + +test.concurrent("a '{dir}' matching nothing is the usual no-match error", async () => { + const dir = await makeMonorepo(); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "{./tools}"], dir); + expect(stderr).toContain('error: No workspace packages matched the filter "{./tools}"'); + expect(exitCode).toBe(1); + + expect(await allPackageJsonTexts(dir)).toEqual(before); +}); + +test.concurrent.each<[string, Workspace[]]>([ + ["api...", ["api", "pkg-a"]], + ["api^...", ["pkg-a"]], + ["...api", ["api", "web"]], + ["...^api", ["web"]], + ["...api...", ["api", "web", "pkg-a"]], + ["...{./packages/pkg-a}", ["api", "web", "pkg-a"]], +])("relation selector '%s' edits %p", async (pattern, edited) => { + const dir = await makeMonorepo(GRAPH); + await installOk(dir); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", pattern], dir); + expect(stderr).not.toContain("error:"); + expect(stderr).not.toContain("warn:"); + expect(exitCode).toBe(0); + + await expectAddedOnlyTo(dir, before, edited); +}); + +test.concurrent("a negated relation subtracts the whole closure", async () => { + { + const dir = await makeMonorepo(GRAPH); + await installOk(dir); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "*", "--filter", "!...api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + await expectAddedOnlyTo(dir, before, ["pkg-a", "pkg-b"]); + } + + // Negation-only: everything except the closure, and (for add) except the root. + { + const dir = await makeMonorepo(GRAPH); + await installOk(dir); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "!...api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + await expectAddedOnlyTo(dir, before, ["pkg-a", "pkg-b"]); + } +}); + +test.concurrent("two relation selectors union", async () => { + const dir = await makeMonorepo(GRAPH); + await installOk(dir); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api...", "--filter", "...api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + await expectAddedOnlyTo(dir, before, ["api", "web", "pkg-a"]); +}); + +test.concurrent( + "dependents are found through devDependencies and plain-range edges; the root stays out of add", + async () => { + const dir = await makeMonorepo({ + root: { ...ROOT, dependencies: { api: "workspace:*" } }, + api: { name: "api", version: "1.0.0" }, + web: { name: "web", devDependencies: { api: "1.0.0" } }, + }); + await installOk(dir); + const before = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "...api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + await expectAddedOnlyTo(dir, before, ["api", "web"]); + expect((await lockfileJson(dir)).workspaces[""].dependencies).toEqual({ api: "workspace:*" }); + }, +); + +test.concurrent("relation selectors terminate on a workspace dependency cycle", async () => { + const files = { + "package.json": JSON.stringify(ROOT), + "packages/x/package.json": JSON.stringify({ name: "x", version: "1.0.0", dependencies: { y: "workspace:*" } }), + "packages/y/package.json": JSON.stringify({ name: "y", version: "1.0.0", dependencies: { x: "workspace:*" } }), + "packages/z/package.json": JSON.stringify({ name: "z", version: "1.0.0" }), + }; + const read = (dir: string, name: string) => + file(name === "root" ? join(dir, "package.json") : join(dir, "packages", name, "package.json")); + + { + const { packageDir: dir } = await registry.createTestDir({ files }); + await installOk(dir); + const [rootBefore, zBefore] = await Promise.all([read(dir, "root").text(), read(dir, "z").text()]); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "x..."], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect((await read(dir, "x").json()).dependencies).toEqual({ y: "workspace:*", "no-deps": "^2.0.0" }); + expect((await read(dir, "y").json()).dependencies).toEqual({ x: "workspace:*", "no-deps": "^2.0.0" }); + expect(await read(dir, "z").text()).toBe(zBefore); + expect(await read(dir, "root").text()).toBe(rootBefore); + } + + // x is reached again through y, but '^' still removes the selector's own base. + { + const { packageDir: dir } = await registry.createTestDir({ files }); + await installOk(dir); + const [rootBefore, xBefore, zBefore] = await Promise.all([ + read(dir, "root").text(), + read(dir, "x").text(), + read(dir, "z").text(), + ]); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "...^x"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect((await read(dir, "y").json()).dependencies).toEqual({ x: "workspace:*", "no-deps": "^2.0.0" }); + expect(await read(dir, "x").text()).toBe(xBefore); + expect(await read(dir, "z").text()).toBe(zBefore); + expect(await read(dir, "root").text()).toBe(rootBefore); + } +}); + +test.concurrent("remove with a relation", async () => { + const dir = await makeMonorepo({ + api: { name: "api", dependencies: { "pkg-a": "workspace:*", "no-deps": "^2.0.0" } }, + web: { name: "web", dependencies: { api: "workspace:*", "no-deps": "^2.0.0" } }, + "pkg-a": { name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }, + }); + await installOk(dir, "hoisted"); + const [rootBefore, , webBefore, , pkgBBefore] = await allPackageJsonTexts(dir); + + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api..."], dir, { linker: "hoisted" }); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "pkg-a": "workspace:*" } }); + expect(await pkg(dir, "pkg-a")).toEqual({ name: "pkg-a" }); + expect(await pkgText(dir, "web")).toBe(webBefore); + expect(await pkgText(dir, "root")).toBe(rootBefore); + expect(await pkgText(dir, "pkg-b")).toBe(pkgBBefore); + expect(await exists(join(dir, "node_modules", "no-deps", "package.json"))).toBeTrue(); +}); + +test.concurrent("a relation that reaches nothing warns like any other pattern", async () => { + const dir = await makeMonorepo(GRAPH); + await installOk(dir); + + { + const before = await allPackageJsonTexts(dir); + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api", "--filter", "...^web"], dir); + expect(stderr).toContain('warn: No workspace packages matched the filter "...^web"'); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + await expectAddedOnlyTo(dir, before, ["api"]); + } + + { + const before = await allPackageJsonTexts(dir); + const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "nope..."], dir); + expect(stderr).toContain('error: No workspace packages matched the filter "nope..."'); + expect(exitCode).toBe(1); + expect(await allPackageJsonTexts(dir)).toEqual(before); + } +}); + +test.concurrent("a relation needs a lockfile; a name selector does not", async () => { + const dir = await makeMonorepo(GRAPH); + const before = await allPackageJsonTexts(dir); + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api..."], dir); + expect(stderr).toContain( + 'error: --filter "api..." selects workspaces by their dependency graph, which needs a bun.lock; run bun install first', + ); + expect(exitCode).toBe(1); + + expect(await allPackageJsonTexts(dir)).toEqual(before); + expect(await exists(join(dir, "bun.lock"))).toBeFalse(); + expect(await exists(join(dir, "node_modules"))).toBeFalse(); + } + + { + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api"], dir); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + await expectAddedOnlyTo(dir, before, ["api"]); + } +}); + +test.concurrent("a bare '...' is rejected", async () => { + const dir = await makeMonorepo(); + await installOk(dir); + const before = await allPackageJsonTexts(dir); + const lockBefore = await file(join(dir, "bun.lock")).text(); + + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "..."], dir); + expect(stderr).toContain('error: --filter "..." is missing a workspace name or path'); + expect(exitCode).toBe(1); + + expect(await allPackageJsonTexts(dir)).toEqual(before); + expect(await file(join(dir, "bun.lock")).text()).toBe(lockBefore); + expect(await exists(join(dir, "node_modules", "no-deps"))).toBeFalse(); +}); + +test.concurrent( + "install --filter with a positive and a negated pattern installs only the positive matches", + async () => { + const dir = await makeMonorepo({ + root: { ...ROOT, dependencies: { "a-dep": "1.0.1" } }, + api: { name: "api", dependencies: { "no-deps": "2.0.0" } }, + "pkg-a": { name: "pkg-a", dependencies: { "is-number": "1.0.0" } }, + }); + + const { stderr, exitCode } = await run(["install", "--filter", "api", "--filter", "!web"], dir, { + linker: "hoisted", + }); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect( + await Promise.all([ + exists(join(dir, "node_modules", "no-deps", "package.json")), + exists(join(dir, "node_modules", "a-dep")), + exists(join(dir, "node_modules", "is-number")), + ]), + ).toEqual([true, false, false]); + }, +); + +test.concurrent("install --filter with relations installs the closure only", async () => { + const dir = await makeMonorepo({ + root: { ...ROOT, dependencies: { "basic-1": "1.0.0" } }, + api: { name: "api", dependencies: { "pkg-a": "workspace:*", "no-deps": "2.0.0" } }, + web: { name: "web", dependencies: { api: "workspace:*", "a-dep": "1.0.1" } }, + "pkg-a": { name: "pkg-a", dependencies: { "is-number": "1.0.0" } }, + "pkg-b": { name: "pkg-b", dependencies: { "left-pad": "1.0.0" } }, + }); + const installed = (name: string) => exists(join(dir, "node_modules", name)); + + { + const { stderr, exitCode } = await run(["install", "--filter", "api..."], dir, { linker: "hoisted" }); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect( + await Promise.all(["no-deps", "is-number", "api", "pkg-a", "a-dep", "left-pad", "basic-1"].map(installed)), + ).toEqual([true, true, true, true, false, false, false]); + } + + { + const { stderr, exitCode } = await run(["install", "--filter", "...^pkg-a"], dir, { linker: "hoisted" }); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await Promise.all(["a-dep", "left-pad", "basic-1"].map(installed))).toEqual([true, false, false]); + } +}); + +// add/remove error on zero matches; install keeps its documented silent no-op. +test.concurrent("install --filter with a pattern that matches nothing still exits 0", async () => { + const dir = await makeMonorepo(); + + const { stderr, exitCode } = await run(["install", "--filter", "nope"], dir, { linker: "hoisted" }); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await exists(join(dir, "node_modules", "a-dep"))).toBeFalse(); +}); + +test.concurrent("outdated --filter with relations", async () => { + const dir = await makeMonorepo({ + api: { name: "api", dependencies: { "pkg-a": "workspace:*", "no-deps": "1.0.0" } }, + web: { name: "web", dependencies: { api: "workspace:*", "is-number": "1.0.0" } }, + "pkg-a": { name: "pkg-a", dependencies: { "a-dep": "1.0.1" } }, + "pkg-b": { name: "pkg-b", dependencies: { "no-deps": "1.0.0" } }, + }); + await installOk(dir); + + { + const { stdout, stderr, exitCode } = await run(["outdated", "--filter", "api..."], dir); + expect(stderr).not.toContain("error:"); + expect(stdout).toContain("no-deps"); + expect(stdout).toContain("a-dep"); + expect(stdout).not.toContain("is-number"); + expect(exitCode).toBe(0); + } + + { + const { stdout, stderr, exitCode } = await run(["outdated", "--filter", "...^api"], dir); + expect(stderr).not.toContain("error:"); + expect(stdout).toContain("is-number"); + expect(stdout).not.toContain("no-deps"); + expect(stdout).not.toContain("a-dep"); + expect(exitCode).toBe(0); + } }); test.concurrent("path filter, run from inside another workspace", async () => { @@ -212,7 +680,9 @@ test.concurrent("path filter, run from inside another workspace", async () => { } { - const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "../api"], join(dir, "packages", "web")); + const { stderr, exitCode } = await run(["add", "a-dep", "--filter", "../api"], dir, { + cwd: join(dir, "packages", "web"), + }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -227,13 +697,13 @@ test.concurrent("path filter, run from inside another workspace", async () => { test.concurrent("no match is an error and nothing is written", async () => { const dir = await makeMonorepo(); - const before = await Promise.all(WORKSPACES.map(w => pkgText(dir, w))); + const before = await allPackageJsonTexts(dir); const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "does-not-exist"], dir); expect(stderr).toContain("error: No workspace packages matched the filter"); expect(exitCode).toBe(1); - expect(await Promise.all(WORKSPACES.map(w => pkgText(dir, w)))).toEqual(before); + expect(await allPackageJsonTexts(dir)).toEqual(before); expect(await exists(join(dir, "bun.lock"))).toBeFalse(); expect(await exists(join(dir, "node_modules"))).toBeFalse(); }); @@ -244,15 +714,11 @@ test.concurrent("remove --filter removes from the matched workspace only", async web: { name: "web", dependencies: { "no-deps": "^2.0.0" } }, }); - { - const { stderr, exitCode } = await run(["install"], dir); - expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); - expect(await exists(join(dir, "bun.lock"))).toBeTrue(); - } + await installOk(dir, "hoisted"); + expect(await exists(join(dir, "bun.lock"))).toBeTrue(); { - const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api"], dir); + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api"], dir, { linker: "hoisted" }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -262,7 +728,7 @@ test.concurrent("remove --filter removes from the matched workspace only", async } { - const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "web"], dir); + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "web"], dir, { linker: "hoisted" }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -271,37 +737,36 @@ test.concurrent("remove --filter removes from the matched workspace only", async } }); -test.concurrent("remove --filter '*' with multiple packages", async () => { +test.concurrent("remove --filter '*' leaves the root's entries alone", async () => { const deps = { "no-deps": "^2.0.0", "a-dep": "^1.0.10" }; const dir = await makeMonorepo({ - root: { ...ROOT, dependencies: deps }, + root: JSON.stringify({ ...ROOT, dependencies: deps }), api: { name: "api", dependencies: deps }, web: { name: "web", dependencies: deps }, }); + await installOk(dir, "hoisted"); + const rootBefore = await pkgText(dir, "root"); - { - const { stderr, exitCode } = await run(["install"], dir); - expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); - } - - const { stderr, exitCode } = await run(["remove", "no-deps", "a-dep", "--filter", "*"], dir); + const { stderr, exitCode } = await run(["remove", "no-deps", "a-dep", "--filter", "*"], dir, { linker: "hoisted" }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); - expect(await allPackageJsons(dir)).toEqual([ROOT, API, { name: "web" }, PKG_A, PKG_B]); + expect(await allPackageJsons(dir)).toEqual([{ ...ROOT, dependencies: deps }, API, { name: "web" }, PKG_A, PKG_B]); + expect(await pkgText(dir, "root")).toBe(rootBefore); + expect( + await Promise.all([ + exists(join(dir, "node_modules", "no-deps", "package.json")), + exists(join(dir, "node_modules", "a-dep", "package.json")), + ]), + ).toEqual([true, true]); + expect((await lockfileJson(dir)).workspaces[""].dependencies).toEqual(deps); }); test.concurrent("add --filter with an existing lockfile re-resolves only the added dep", async () => { const dir = await makeMonorepo(); + await installOk(dir, "hoisted"); - { - const { stderr, exitCode } = await run(["install"], dir); - expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); - } - - const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "web"], dir); + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "web"], dir, { linker: "hoisted" }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -331,8 +796,7 @@ test.concurrent("--dry-run writes nothing", async () => { expect(await pkg(dir, "root")).toEqual(ROOT); }); -// Ported from pnpm's filtered add/remove suites (installing/commands/test/miscRecursive.ts, addRecursive.ts, -// remove/workspace.ts, pnpm/test/recursive/filter.ts) and pacquet's install_filters.rs. +// Ported from pnpm's filtered add/remove suites and pacquet's install_filters.rs. test.concurrent( "add --only-missing --filter leaves an existing entry untouched in the target that has it", @@ -346,7 +810,7 @@ test.concurrent( expect(await pkgText(dir, "api")).toBe(apiBefore); expect(await allPackageJsons(dir)).toEqual([ - { ...ROOT, dependencies: { "no-deps": "^2.0.0" } }, + ROOT, { name: "api", dependencies: { "no-deps": "1.0.0" } }, { name: "web", dependencies: { "a-dep": "1.0.1", "no-deps": "^2.0.0" } }, { name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }, @@ -394,10 +858,12 @@ test.concurrent("remove --filter leaves targets that did not contain the depende web: '{"name":"web","dependencies":{"a-dep":"1.0.1"}}', "pkg-a": '{ "name": "pkg-a" }', }); - await installOk(dir); + await installOk(dir, "hoisted"); const [rootBefore, , webBefore, pkgABefore] = await allPackageJsonTexts(dir); - const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api", "--filter", "web"], dir); + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api", "--filter", "web"], dir, { + linker: "hoisted", + }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -425,7 +891,7 @@ test.concurrent("add/remove --filter with the isolated linker links only the tar const dir = await makeMonorepo({}, "isolated"); { - const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api"], dir, "isolated"); + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api"], dir, { linker: "isolated" }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -439,7 +905,7 @@ test.concurrent("add/remove --filter with the isolated linker links only the tar } { - const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api"], dir, "isolated"); + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api"], dir, { linker: "isolated" }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -470,11 +936,11 @@ test.concurrent("two name filters are unioned and unselected workspaces are byte test.concurrent("remove --filter with no match names the pattern and touches nothing", async () => { const dir = await makeMonorepo({ api: { name: "api", dependencies: { "no-deps": "^2.0.0" } } }); - await installOk(dir); + await installOk(dir, "hoisted"); const before = await allPackageJsonTexts(dir); const lockBefore = await file(join(dir, "bun.lock")).text(); - const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "nope"], dir); + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "nope"], dir, { linker: "hoisted" }); expect(stderr).toContain('error: No workspace packages matched the filter "nope"'); expect(exitCode).toBe(1); @@ -499,7 +965,10 @@ test.concurrent("negated filters that match nothing do not warn", async () => { const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "api", "--filter", "!nothing"], dir); expect(stderr).not.toContain("warn:"); + expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); + + expect(await pkg(dir, "api")).toEqual({ name: "api", dependencies: { "no-deps": "^2.0.0" } }); }); test.concurrent("an explicit version is written verbatim to every target", async () => { @@ -515,7 +984,7 @@ test.concurrent("an explicit version is written verbatim to every target", async } { - const { stderr, exitCode } = await run(["add", "no-deps@^1.0.0", "--filter", "web"], dir); + const { stderr, exitCode } = await run(["add", "no-deps@^1.0.0", "--filter", "web"], dir, { linker: "hoisted" }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -585,7 +1054,9 @@ test.concurrent("'.' selects the workspace of the invoking directory", async () { const rootBefore = await pkgText(dir, "root"); - const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "."], join(dir, "packages", "web")); + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "."], dir, { + cwd: join(dir, "packages", "web"), + }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -605,7 +1076,7 @@ test.concurrent("a directory excluded by a '!' entry in workspaces is never a ta expect(exitCode).toBe(0); expect(await allPackageJsons(dir)).toEqual([ - { name: "root", workspaces: ["packages/*", "!packages/web"], dependencies: { "no-deps": "^2.0.0" } }, + { name: "root", workspaces: ["packages/*", "!packages/web"] }, { name: "api", dependencies: { "no-deps": "^2.0.0" } }, WEB, { name: "pkg-a", dependencies: { "no-deps": "^2.0.0" } }, @@ -680,14 +1151,18 @@ test.concurrent("running from a non-package subdirectory selects by filter and s ]); { - const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "web"], join(dir, "packages", "api", "src")); + const { stderr, exitCode } = await run(["add", "no-deps", "--filter", "web"], dir, { + cwd: join(dir, "packages", "api", "src"), + }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); expect(await exists(join(dir, "packages", "api", "src", "package.json"))).toBeFalse(); } { - const { stderr, exitCode } = await run(["install", "a-dep", "--filter", "web"], join(dir, "tools", "scratch")); + const { stderr, exitCode } = await run(["install", "a-dep", "--filter", "web"], dir, { + cwd: join(dir, "tools", "scratch"), + }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); expect(await exists(join(dir, "tools", "scratch", "package.json"))).toBeFalse(); @@ -704,7 +1179,7 @@ test.concurrent.each(["add", "install"])( const { packageDir } = await registry.createTestDir(); const { stderr, exitCode } = await run([cmd, "no-deps", "--filter", "web"], packageDir); - expect(stderr).toContain("error:"); + expect(stderr).toContain("error: Bun could not find a package.json file to install from"); expect(exitCode).toBe(1); expect(await exists(join(packageDir, "package.json"))).toBeFalse(); @@ -716,17 +1191,13 @@ test.concurrent("filtered remove keeps the unselected workspace's lockfile entri api: { name: "api", dependencies: { "no-deps": "^2.0.0" } }, web: { name: "web", dependencies: { api: "workspace:*", "no-deps": "^2.0.0" } }, }); - await installOk(dir); + await installOk(dir, "hoisted"); - const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api"], dir); + const { stderr, exitCode } = await run(["remove", "no-deps", "--filter", "api"], dir, { linker: "hoisted" }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); - const lockfile = JSON.parse( - await file(join(dir, "bun.lock")) - .text() - .then(t => t.replace(/,(\s*[}\]])/g, "$1")), - ); + const lockfile = await lockfileJson(dir); expect(lockfile.workspaces["packages/api"]).toEqual({ name: "api" }); expect(lockfile.workspaces["packages/web"]).toEqual({ name: "web", @@ -747,6 +1218,7 @@ test.concurrent("bun install -F targets the workspace", async () => { expect(await pkg(dir, "root")).toEqual(ROOT); }); +// The root is only skipped by '*' when there are workspaces to select instead. test.concurrent("--filter outside a workspace: the lone package is the only candidate", async () => { const { packageDir } = await registry.createTestDir({ files: { "package.json": JSON.stringify({ name: "solo" }) } }); @@ -777,7 +1249,7 @@ test.concurrent("--filter with --global is rejected", async () => { cmd: [bunExe(), "add", "no-deps", "-g", "--filter", "api"], cwd: dir, env: { ...envFor(dir), BUN_INSTALL: globalDir, BUN_INSTALL_GLOBAL_DIR: join(globalDir, "install", "global") }, - stdout: "pipe", + stdout: "ignore", stderr: "pipe", }); const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); @@ -790,12 +1262,6 @@ test.concurrent("--filter with --global is rejected", async () => { // Round 3: bugs and non-bugs mined from pnpm's open issue tracker. -function lockfileJson(dir: string) { - return file(join(dir, "bun.lock")) - .text() - .then(t => JSON.parse(t.replace(/,(\s*[}\]])/g, "$1"))); -} - const VENDOR_FOO = { name: "foo", version: "1.0.0" }; async function addVendorFoo(dir: string) { @@ -821,7 +1287,7 @@ test.concurrent("a failed resolution leaves every target untouched", async () => const before = await allPackageJsonTexts(dir); const { stderr, exitCode } = await run(["add", "does-not-exist-anywhere", "--filter", "pkg-*"], dir); - expect(stderr).toContain("error:"); + expect(stderr).toContain(`error: GET ${registry.registryUrl()}does-not-exist-anywhere - 404`); expect(exitCode).toBe(1); expect(await allPackageJsonTexts(dir)).toEqual(before); @@ -864,7 +1330,9 @@ test.concurrent("a local path is relative to the cwd and re-spelled for each tar const dir = await makeMonorepo(); await addVendorFoo(dir); - const { stderr, exitCode } = await run(["add", "./vendor/foo", "--filter", "root", "--filter", "api"], dir); + const { stderr, exitCode } = await run(["add", "./vendor/foo", "--filter", "root", "--filter", "api"], dir, { + linker: "hoisted", + }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -876,7 +1344,7 @@ test.concurrent("a local path is relative to the cwd and re-spelled for each tar expect(workspaces["packages/api"].dependencies).toEqual({ foo: "../../vendor/foo" }); expect(await file(join(dir, "node_modules", "foo", "package.json")).json()).toEqual(VENDOR_FOO); - const frozen = await run(["install", "--frozen-lockfile"], dir); + const frozen = await run(["install", "--frozen-lockfile"], dir, { linker: "hoisted" }); expect(frozen.stderr).not.toContain("error:"); expect(frozen.exitCode).toBe(0); }); @@ -885,10 +1353,9 @@ test.concurrent("a file: path keeps its prefix and resolves from a nested cwd", const dir = await makeMonorepo(); await addVendorFoo(dir); - const { stderr, exitCode } = await run( - ["add", "file:../../vendor/foo", "--filter", "api"], - join(dir, "packages", "web"), - ); + const { stderr, exitCode } = await run(["add", "file:../../vendor/foo", "--filter", "api"], dir, { + cwd: join(dir, "packages", "web"), + }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -903,7 +1370,7 @@ test.concurrent("a local path that does not exist relative to the cwd fails and // Relative to packages/api this would exist; paths are relative to the cwd (the root) instead. const { stderr, exitCode } = await run(["add", "../../vendor/foo", "--filter", "api"], dir); - expect(stderr).toContain("error:"); + expect(stderr).toContain('error: Could not find package.json for "file:'); expect(exitCode).toBe(1); expect(await allPackageJsonTexts(dir)).toEqual(before); @@ -918,7 +1385,7 @@ test.concurrent("--filter writes the same entry as running bun add inside the wo expect(filtered.stderr).not.toContain("error:"); expect(filtered.exitCode).toBe(0); - const inside = await run(["add", dep, ...flags], join(dir, "packages", "api")); + const inside = await run(["add", dep, ...flags], dir, { cwd: join(dir, "packages", "api") }); expect(inside.stderr).not.toContain("error:"); expect(inside.exitCode).toBe(0); } @@ -978,6 +1445,7 @@ test.concurrent("a negated pattern wins regardless of flag order (pnpm#9354)", a expect(await allPackageJsonTexts(dir)).toEqual(before); }); +// Only the brace form `{./packages}` is a subtree selector; a bare path must name a workspace directory. test.concurrent("a path pattern naming the parent directory selects nothing (pnpm#5508)", async () => { const dir = await makeMonorepo(); const before = await allPackageJsonTexts(dir); @@ -1019,7 +1487,9 @@ test.concurrent("path filters resolve against --cwd (pnpm#5270)", async () => { const dir = await makeMonorepo(); const { packageDir: elsewhere } = await registry.createTestDir(); - const { stderr, exitCode } = await run(["add", "no-deps", "--cwd", dir, "--filter", "./packages/api"], elsewhere); + const { stderr, exitCode } = await run(["add", "no-deps", "--cwd", dir, "--filter", "./packages/api"], dir, { + cwd: elsewhere, + }); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); diff --git a/test/cli/install/bun-add.test.ts b/test/cli/install/bun-add.test.ts index d1c51013feee..8fc1d6efa8a4 100644 --- a/test/cli/install/bun-add.test.ts +++ b/test/cli/install/bun-add.test.ts @@ -2754,3 +2754,80 @@ it("should install tarball with tarball dependencies", async () => { await access(join(add_dir, "node_modules", "test-parent")); await access(join(add_dir, "node_modules", "test-child")); }); + +it("should add a local tarball with an uppercase .TGZ extension", async () => { + const urls: string[] = []; + setHandler(dummyRegistry(urls)); + await writeFile( + join(package_dir, "package.json"), + JSON.stringify({ + name: "foo", + version: "0.0.1", + }), + ); + await copyFile(join(__dirname, "baz-0.0.3.tgz"), join(package_dir, "BAZ-0.0.3.TGZ")); + const { stdout, stderr, exited } = spawn({ + cmd: [bunExe(), "add", "./BAZ-0.0.3.TGZ"], + cwd: package_dir, + stdout: "pipe", + stdin: "pipe", + stderr: "pipe", + env, + }); + const err = await stderr.text(); + expect(err).not.toContain("error:"); + expect(err).toContain("Saved lockfile"); + const out = await stdout.text(); + expect(out).toContain("installed baz@"); + expect(out).toContain("1 package installed"); + expect(await exited).toBe(0); + expect(urls).toBeEmpty(); + expect(requested).toBe(0); + const package_json = await file(join(package_dir, "node_modules", "baz", "package.json")).json(); + expect(package_json.name).toBe("baz"); + expect(package_json.version).toBe("0.0.3"); +}); + +it("should add an uncompressed .tar local tarball", async () => { + const urls: string[] = []; + setHandler(dummyRegistry(urls)); + await writeFile( + join(package_dir, "package.json"), + JSON.stringify({ + name: "foo", + version: "0.0.1", + }), + ); + await writeFile( + join(package_dir, "baz-0.0.3.tar"), + Bun.gunzipSync(await file(join(__dirname, "baz-0.0.3.tgz")).bytes()), + ); + const { stdout, stderr, exited } = spawn({ + cmd: [bunExe(), "add", "baz-0.0.3.tar"], + cwd: package_dir, + stdout: "pipe", + stdin: "pipe", + stderr: "pipe", + env, + }); + const err = await stderr.text(); + expect(err).not.toContain("error:"); + expect(err).toContain("Saved lockfile"); + const out = await stdout.text(); + expect(out.replace(/\s*\[[0-9\.]+m?s\]\s*$/, "").split(/\r?\n/)).toEqual([ + expect.stringContaining("bun add v1."), + "", + "installed baz@baz-0.0.3.tar with binaries:", + " - baz-run", + "", + "1 package installed", + ]); + expect(await exited).toBe(0); + expect(urls).toBeEmpty(); + expect(requested).toBe(0); + expect(await readdirSorted(join(package_dir, "node_modules", "baz"))).toEqual(["index.js", "package.json"]); + const package_json = await file(join(package_dir, "node_modules", "baz", "package.json")).json(); + expect(package_json.name).toBe("baz"); + expect(package_json.version).toBe("0.0.3"); + expect(await file(join(package_dir, "package.json")).text()).toInclude('"baz-0.0.3.tar"'); +}); diff --git a/test/cli/install/bun-audit.test.ts b/test/cli/install/bun-audit.test.ts index 5351b97d197f..108b4aaad9a2 100644 --- a/test/cli/install/bun-audit.test.ts +++ b/test/cli/install/bun-audit.test.ts @@ -7,7 +7,6 @@ import { bunEnv, bunExe, gunzipJsonRequest, - lazyPromiseLike, normalizeBunSnapshot, runBunInstall, tempDir, @@ -68,7 +67,7 @@ function doAuditTest( const url = server.url.toString().slice(0, -1); - const proc = spawn({ + await using proc = spawn({ cmd, stdout: "pipe", stderr: "pipe", @@ -79,25 +78,15 @@ function doAuditTest( }, }); - const stdout = lazyPromiseLike(() => proc.stdout.text()); - const stderr = lazyPromiseLike(() => proc.stderr.text()); - - const exitCode = await proc.exited; + const [out, err, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); try { + await options.fn({ stdout: Promise.resolve(out), stderr: Promise.resolve(err), dir }); expect(exitCode).toBe(options.exitCode); - await options.fn({ stdout, stderr, dir }); } catch (e) { - const err = await stderr; - const out = await stdout; - - // useful to see what went wrong otherwise - // we are just eating the rror silently - console.log("ERR:", err); console.log("OUT:", out); - - throw e; //but still rethrow so test fails + throw e; } }); } @@ -120,6 +109,10 @@ type RegistryOptions = { bulkStatus?: number; // Package names whose manifest requests answer 404; mutable so a test can break the registry after installing. denyManifests?: Set; + // Tarball file names (`a-dep-1.0.4.tgz`) whose downloads answer 404. + denyTarballs?: Set; + // Publish times overlaid on a package's manifest: { "a-dep": { "1.0.4": iso } }. + rewriteTime?: Record>; }; // Answers the bulk-advisory endpoint itself and proxies everything else to verdaccio. @@ -142,7 +135,11 @@ function startRegistry(advisories: Record, options: Registry return Response.json(out); } - if (options.denyManifests?.has(decodeURIComponent(url.pathname.slice(1)))) { + const packageName = decodeURIComponent(url.pathname.slice(1)); + if (options.denyManifests?.has(packageName)) { + return new Response("not found", { status: 404 }); + } + if (options.denyTarballs?.has(url.pathname.slice(url.pathname.lastIndexOf("/") + 1))) { return new Response("not found", { status: 404 }); } @@ -150,6 +147,12 @@ function startRegistry(advisories: Record, options: Registry method: req.method, headers: { accept: req.headers.get("accept") ?? "*/*" }, }); + const time = options.rewriteTime?.[packageName]; + if (time && up.ok) { + const manifest = await up.json(); + manifest.time = { ...manifest.time, ...time }; + return Response.json(manifest); + } return new Response(up.body, { status: up.status, headers: { "content-type": up.headers.get("content-type") ?? "application/octet-stream" }, @@ -160,11 +163,20 @@ function startRegistry(advisories: Record, options: Registry type Registry = ReturnType; -function writeBunfig(dir: string, server: Registry) { +function registryHref(server: Registry) { + return server.url.href.slice(0, -1); +} + +function writeBunfig(dir: string, server: Registry, scopes?: Record) { return write( join(dir, "bunfig.toml"), Bun.TOML.stringify({ - install: { cache: join(dir, ".bun-cache"), registry: server.url.href, saveTextLockfile: true }, + install: { + cache: join(dir, ".bun-cache"), + registry: server.url.href, + saveTextLockfile: true, + ...(scopes && { scopes }), + }, }), ); } @@ -174,13 +186,27 @@ function installEnv(dir: string) { return { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }; } -async function setup(server: Registry, pkgJson: object, extraFiles: Record = {}) { - const dir = tempDir("audit-fix-", { "package.json": JSON.stringify(pkgJson), ...extraFiles }); - await writeBunfig(dir, server); +async function setup( + server: Registry, + pkgJson: object | string, + extraFiles: Record = {}, + scopes?: Record, +) { + const text = typeof pkgJson === "string" ? pkgJson : JSON.stringify(pkgJson); + const dir = tempDir("audit-fix-", { "package.json": text, ...extraFiles }); + await writeBunfig(dir, server, scopes); await runBunInstall(installEnv(dir), dir); return dir; } +function pkgJson(dir: string, ...segments: string[]) { + return file(join(dir, ...segments, "package.json")).json(); +} + +function pkgJsonText(dir: string, ...segments: string[]) { + return file(join(dir, ...segments, "package.json")).text(); +} + async function reinstall(dir: string, pkgJson: object) { await write(join(dir, "package.json"), JSON.stringify(pkgJson)); await runBunInstall(installEnv(dir), dir); @@ -214,6 +240,20 @@ async function installedVersion(dir: string, ...segments: string[]) { return (await file(join(dir, "node_modules", ...segments, "package.json")).json()).version; } +// The version `dependent` resolves `name` to under the hoisted layout: its nested copy, else the root one. +async function resolvedVersion(dir: string, dependent: string, name: string) { + if (await exists(join(dir, "node_modules", dependent, "node_modules", name))) { + return installedVersion(dir, dependent, "node_modules", name); + } + return installedVersion(dir, name); +} + +async function expectInstall(dir: string, ...args: string[]) { + const { stderr, exitCode } = await run(dir, ["install", ...args]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); +} + // a-dep@1.0.2 stays installed after the range is widened because the still-satisfied edge is not re-resolved. async function setupVulnerableADep(server: Registry) { const dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2" } }); @@ -310,6 +350,35 @@ describe("`bun audit`", () => { }, }); + doAuditTest("--json exits 0 when --audit-level filters out every advisory, but still prints them all", { + exitCode: 0, + files: fixture("vuln-with-only-dev-dependencies"), + args: ["--json", "--audit-level", "critical"], + fn: async ({ stdout }) => { + const json = JSON.parse(await stdout); + expect(json.ms.map((a: { severity: string }) => a.severity).sort()).toEqual(["high", "moderate"]); + }, + }); + + doAuditTest("--json exits 0 when --ignore covers every advisory, but still prints them all", { + exitCode: 0, + files: fixture("vuln-with-only-dev-dependencies"), + args: ["--json", "--ignore", "GHSA-w9mr-4mfr-499f", "--ignore", "GHSA-3fx5-fwvr-xrjg"], + fn: async ({ stdout }) => { + const json = JSON.parse(await stdout); + expect(json.ms).toHaveLength(2); + }, + }); + + doAuditTest("--json still exits 1 when an advisory survives --audit-level and --ignore", { + exitCode: 1, + files: fixture("vuln-with-only-dev-dependencies"), + args: ["--json", "--audit-level", "high", "--ignore", "GHSA-w9mr-4mfr-499f"], + fn: async ({ stdout }) => { + expect(JSON.parse(await stdout).ms).toHaveLength(2); + }, + }); + doAuditTest( "should exit 1 and behave exactly the same when there are vulnerabilities when only devDependencies are specified", { @@ -327,8 +396,7 @@ describe("`bun audit`", () => { exitCode: 1, files: fixture("mix-of-safe-and-vulnerable-dependencies"), fn: async ({ stdout }) => { - // this fixture is using a safe version of is-number and an unsafe version of ms - // so we want to check that `is-number` is not included in the output and that `ms` is + // The fixture installs a safe is-number and a vulnerable ms. const out = await stdout; @@ -343,45 +411,64 @@ describe("`bun audit`", () => { const fakeIntegrity = // this is just random/fake data as the integrity check is not important for this test "sha512-V8E0l1jyyeSSS9R+J9oljx5eq2rqzClInuwaPcyuv0Mm3ViI/3/rcc4rCEO8i4eQ4I0O0FAGYDA2i5xWHHPhzg=="; - doAuditTest( - "packages that come from non-default registries should be ignored from the audit, however they should get surfaced at the bottom of the output that they got skipped", - { - exitCode: 0, - files: { - "package.json": JSON.stringify({ - name: "test", - version: "1.0.0", - dependencies: { - "@foo/bar": "1.0.0", - "@foo/baz": "1.0.0", - }, - }), - "bun.lock": JSON.stringify({ - "lockfileVersion": 1, - "workspaces": { - "": { - "name": "test", - }, - }, - "packages": { - "@foo/bar": ["@foo/bar@1.0.0", "", {}, fakeIntegrity], - "@foo/baz": ["@foo/baz@1.0.0", "", {}, fakeIntegrity], + function scopedRegistryProject(scoped: Registry) { + return { + "package.json": JSON.stringify({ + name: "test", + version: "1.0.0", + dependencies: { + "@foo/bar": "1.0.0", + "@foo/baz": "1.0.0", + }, + }), + "bun.lock": JSON.stringify({ + "lockfileVersion": 1, + "workspaces": { + "": { + "name": "test", }, - }), - //prettier-ignore - ".npmrc": [ - `registry=https://registry.npmjs.org`, - `@foo:registry=https://my-registry.example.com`, - ].join("\n"), - }, - fn: async ({ stdout }) => { - const out = await stdout; + }, + "packages": { + "@foo/bar": ["@foo/bar@1.0.0", "", {}, fakeIntegrity], + "@foo/baz": ["@foo/baz@1.0.0", "", {}, fakeIntegrity], + }, + }), + ".npmrc": `@foo:registry=${scoped.url.href}`, + }; + } - expect(out).toContain("Skipped @foo/bar, @foo/baz because they do not come from the default registry"); - expect(out).toContain("No vulnerabilities found"); - }, - }, - ); + async function auditWithDefaultRegistry(dir: string) { + await using proc = spawn({ + cmd: [bunExe(), "audit"], + stdout: "pipe", + stderr: "pipe", + cwd: dir, + env: { ...bunEnv, NPM_CONFIG_REGISTRY: registryHref(server) }, + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { stdout, stderr, exitCode }; + } + + test("packages served by a scoped registry are audited against that registry", async () => { + await using scoped = startRegistry({}, { bulkResponse: { "@foo/bar": [adv("<2.0.0")] } }); + using dir = tempDir("bun-test-audit-scoped-registry", scopedRegistryProject(scoped)); + + const { stdout, exitCode } = await auditWithDefaultRegistry(String(dir)); + expect(stdout).toContain("@foo/bar"); + expect(stdout).toContain("1 vulnerability (1 high)"); + expect(stdout).not.toContain("Skipped"); + expect(exitCode).toBe(1); + }); + + test("packages whose scoped registry does not answer the audit request are listed as skipped", async () => { + await using scoped = startRegistry({}, { bulkStatus: 404 }); + using dir = tempDir("bun-test-audit-scoped-registry-down", scopedRegistryProject(scoped)); + + const { stdout, exitCode } = await auditWithDefaultRegistry(String(dir)); + expect(stdout).toContain(`Skipped @foo/bar, @foo/baz because ${registryHref(scoped)} could not be audited`); + expect(stdout).toContain("No vulnerabilities found"); + expect(exitCode).toBe(0); + }); doAuditTest("workspaces print the path to the vulnerable package and include workspace:pkg in the name", { exitCode: 1, @@ -590,12 +677,41 @@ describe("`bun audit --prod`", () => { const prod = await audit(dir, "--prod"); expect(prod.stdout).toContain("no-deps"); - expect(prod.stdout).toContain("1 vulnerabilities (1 high)"); + expect(prod.stdout).toContain("1 vulnerability (1 high)"); expect(prod.exitCode).toBe(1); }, ); }); +describe("`bun audit --omit`", () => { + test.concurrent.each(["dev", "optional", "peer"] as const)( + "--omit=%s skips packages only reached that way", + async kind => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + const field = { dev: "devDependencies", optional: "optionalDependencies", peer: "peerDependencies" }[kind]; + using dir = await setup(server, { name: "foo", [field]: { "no-deps": "1.0.0" } }); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const all = await audit(dir); + expect(all.stdout).toContain("1 vulnerability (1 high)"); + expect(all.exitCode).toBe(1); + + const omitted = await audit(dir, `--omit=${kind}`); + expect(omitted.stdout).toBe("No vulnerabilities found\n"); + expect(omitted.exitCode).toBe(0); + }, + ); + + test.concurrent("--omit=optional keeps auditing dev dependencies", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", devDependencies: { "no-deps": "1.0.0" } }); + + const { stdout, exitCode } = await audit(dir, "--omit=optional"); + expect(stdout).toContain("1 vulnerability (1 high)"); + expect(exitCode).toBe(1); + }); +}); + describe("`bun audit fix`", () => { test.concurrent("fixes a direct dependency to the lowest safe version, not the newest", async () => { await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); @@ -651,10 +767,12 @@ describe("`bun audit fix`", () => { using dir = await setup(server, { name: "foo", dependencies: { "one-dep": "1.0.0" } }); const lockBefore = await lock(dir); expect(lockBefore).toContain('"no-deps@1.0.1"'); + const rootBefore = await pkgJsonText(dir); + const dependentBefore = await pkgJsonText(dir, "node_modules", "one-dep"); const { stdout, stderr, exitCode } = await auditFix(dir); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: + "blocked by a dependent's range: no-deps@1.0.1 → 1.1.0 one-dep@1.0.0 depends on no-deps@1.0.1 @@ -664,6 +782,28 @@ describe("`bun audit fix`", () => { expect(stderr).not.toContain("Saved lockfile"); expect(exitCode).toBe(1); expect(await lock(dir)).toBe(lockBefore); + expect(await pkgJsonText(dir)).toBe(rootBefore); + expect(await pkgJsonText(dir, "node_modules", "one-dep")).toBe(dependentBefore); + }); + + test.concurrent("a safe older release outside the dependent's range is not a downgrade candidate", async () => { + await using server = startRegistry({ "no-deps": [adv(">=1.0.1 <2.0.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-dep": "1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.1"'); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "blocked by a dependent's range: + no-deps@1.0.1 → 2.0.0 + one-dep@1.0.0 depends on no-deps@1.0.1 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); }); test.concurrent("--dry-run prints the plan and writes nothing", async () => { @@ -684,7 +824,7 @@ describe("`bun audit fix`", () => { expect(await installedVersion(dir, "a-dep")).toBe("1.0.2"); }); - test.concurrent("no fix available", async () => { + test.concurrent("a range that rejects every safe release is blocked on the highest safe downgrade", async () => { await using server = startRegistry({ "no-deps": [adv(">=2.0.0")] }); using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "^2.0.0" } }); const lockBefore = await lock(dir); @@ -692,8 +832,9 @@ describe("`bun audit fix`", () => { const { stdout, exitCode } = await auditFix(dir); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "no fix available: - no-deps@2.0.0 + "blocked by a dependent's range: + no-deps@2.0.0 → 1.1.0 (downgrade) + foo depends on no-deps@^2.0.0 No fixable vulnerabilities 1 vulnerability remaining" @@ -745,7 +886,7 @@ describe("`bun audit fix`", () => { const { stdout, exitCode } = await auditFix(dir); expect(stdout).toContain("fixing:"); expect(stdout).toContain("no-deps@1.0.0 → 1.1.0"); - expect(stdout).toContain("requires a semver-major update:"); + expect(stdout).toContain("blocked by a dependent's range:"); expect(stdout).toContain("no-deps@1.0.1 → 1.1.0"); expect(stdout).toContain("one-dep@1.0.0 depends on no-deps@1.0.1"); // pnpm#10646: the advisory still applies to no-deps@1.0.1, so it is remaining, not fixed, and `bun audit` agrees. @@ -760,7 +901,7 @@ describe("`bun audit fix`", () => { expect(await installedVersion(dir, "no-deps")).toBe("1.1.0"); const recheck = await audit(dir); - expect(recheck.stdout).toContain("1 vulnerabilities (1 high)"); + expect(recheck.stdout).toContain("1 vulnerability (1 high)"); expect(recheck.exitCode).toBe(1); await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); @@ -884,14 +1025,19 @@ describe("`bun audit fix`", () => { let lockfile = await lock(dir); expect(lockfile).toContain('"no-deps@1.0.0"'); expect(lockfile).not.toContain('"no-deps@1.0.1"'); + const rootBefore = await pkgJsonText(dir); + const memberBefore = await pkgJsonText(dir, "packages", "a"); const { stdout, exitCode } = await auditFix(dir); expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stdout).not.toContain("package.json"); expect(exitCode).toBe(0); lockfile = await lock(dir); expect(lockfile).toContain('"no-deps@1.0.1"'); expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(await pkgJsonText(dir)).toBe(rootBefore); + expect(await pkgJsonText(dir, "packages", "a")).toBe(memberBefore); }); test.concurrent("--ignore and --audit-level filter what gets fixed", async () => { @@ -915,15 +1061,11 @@ describe("`bun audit fix`", () => { expect(await lock(dir)).toContain('"a-dep@1.0.4"'); }); - test.concurrent("rejects --json and extra arguments", async () => { + test.concurrent("rejects extra arguments", async () => { await using server = startRegistry({}); using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); const lockBefore = await lock(dir); - const json = await auditFix(dir, "--json"); - expect(json.stderr).toContain("error: --json is not supported by bun audit fix"); - expect(json.exitCode).toBe(1); - const extra = await auditFix(dir, "extra"); expect(extra.stderr).toContain("error: bun audit fix does not take arguments"); expect(extra.exitCode).toBe(1); @@ -1007,23 +1149,23 @@ describe("`bun audit fix`", () => { }, ); - test.concurrent("an advisory for an installed prerelease is matched and reported", async () => { + test.concurrent("an advisory for an installed prerelease is matched and the pin is rewritten", async () => { await using server = startRegistry({}, { bulkResponse: { "no-deps-backward-tags": [adv("<1.1.0")] } }); using dir = await setup(server, { name: "foo", dependencies: { "no-deps-backward-tags": "1.0.0-rc.1" } }); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps-backward-tags@1.0.0-rc.1"'); + expect(await lock(dir)).toContain('"no-deps-backward-tags@1.0.0-rc.1"'); const { stdout, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: - no-deps-backward-tags@1.0.0-rc.1 → 1.1.0 - foo depends on no-deps-backward-tags@1.0.0-rc.1 + expect(stdout).toContain("no-deps-backward-tags@1.0.0-rc.1 → 1.1.0"); + expect(stdout).toContain("package.json: 1.0.0-rc.1 → 1.1.0"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); + expect((await pkgJson(dir)).dependencies["no-deps-backward-tags"]).toBe("1.1.0"); + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps-backward-tags@1.1.0"'); + expect(lockfile).not.toContain("1.0.0-rc.1"); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); }); test.concurrent("advisories that match no installed version are listed, not just counted", async () => { @@ -1068,7 +1210,7 @@ describe("`bun audit fix`", () => { const { stdout, stderr, exitCode } = await auditFix(dir); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: + "blocked by a dependent's range: no-deps@1.0.0 → 1.0.1 bundled-1@1.0.0 bundles no-deps@1.0.0 @@ -1114,8 +1256,7 @@ describe("`bun audit fix`", () => { expect(await lock(dir)).toContain('"no-deps@1.0.1"'); }); - // pnpm fixtures/update-single-depth-2/responses/unfixable-vulnerability.json: npm-style advisory objects carry - // fields (findings, patched_versions, ...) that must be ignored, and `>=0.0.0` is unfixable. + // npm-style advisory objects carry extra fields (findings, patched_versions, ...) that must be ignored. test.concurrent("ignores unknown advisory fields and treats >=0.0.0 as unfixable", async () => { await using server = startRegistry( {}, @@ -1149,10 +1290,9 @@ describe("`bun audit fix`", () => { expect(await lock(dir)).toBe(lockBefore); }); - // Ported verbatim from pnpm's test/audit/utils/responses/all-vulnerabilities-response.json (51 packages, 111 advisories). - test.concurrent("parses a real bulk response and only plans installed packages", async () => { + test.concurrent("parses a large bulk response and only plans installed packages", async () => { const bulkResponse = await file( - join(import.meta.dir, "registry/fixtures/audit/pnpm-all-vulnerabilities-response.json"), + join(import.meta.dirname, "registry", "fixtures", "audit", "pnpm-all-vulnerabilities-response.json"), ).json(); await using server = startRegistry({}, { bulkResponse }); using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); @@ -1231,7 +1371,7 @@ describe("`bun audit fix`", () => { await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); }); - test.concurrent("an overrides pin is reported as the blocker", async () => { + test.concurrent("a version held by an overrides entry is blocked and the override is not rewritten", async () => { await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); using dir = await setup(server, { name: "foo", @@ -1240,10 +1380,11 @@ describe("`bun audit fix`", () => { }); const lockBefore = await lock(dir); expect(lockBefore).toContain('"a-dep@1.0.2"'); + const pkgJsonBefore = await pkgJsonText(dir); const { stdout, exitCode } = await auditFix(dir); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: + "blocked by a dependent's range: a-dep@1.0.2 → 1.0.4 foo depends on a-dep@1.0.2 @@ -1252,52 +1393,84 @@ describe("`bun audit fix`", () => { `); expect(exitCode).toBe(1); expect(await lock(dir)).toBe(lockBefore); + expect(await pkgJsonText(dir)).toBe(pkgJsonBefore); }); - // pnpm fixtures/update-workspace-catalog-pinned: a pinned catalog entry blocks the fix. - test.concurrent("a pinned catalog entry is reported as the blocker", async () => { + test.concurrent("a pinned catalog entry is rewritten and the member keeps `catalog:`", async () => { await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + const member = JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }); using dir = await setup( server, { name: "root", workspaces: ["packages/*"], catalog: { "no-deps": "1.0.0" } }, - { "packages/a/package.json": JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }) }, + { "packages/a/package.json": member }, ); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps@1.0.0"'); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); const { stdout, exitCode } = await auditFix(dir); - expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: - no-deps@1.0.0 → 1.0.1 - a depends on no-deps@1.0.0 + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stdout).toContain("package.json (catalog): 1.0.0 → 1.0.1"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); + expect((await pkgJson(dir)).catalog).toEqual({ "no-deps": "1.0.1" }); + expect(await pkgJsonText(dir, "packages", "a")).toBe(member); + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps": "1.0.1"'); + expect(lockfile).toContain('"no-deps": "catalog:"'); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("rewrites a named catalog entry and leaves unused catalogs alone", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + const member = JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:build" } }); + using dir = await setup( + server, + { + name: "root", + workspaces: ["packages/*"], + catalogs: { build: { "no-deps": "1.0.0" }, other: { "no-deps": "1.0.0" } }, + }, + { "packages/a/package.json": member }, + ); + expect(await lock(dir)).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1"); + expect(stdout).toContain("package.json (catalog build): 1.0.0 → 1.0.1"); + expect(exitCode).toBe(0); + + expect((await pkgJson(dir)).catalogs).toEqual({ build: { "no-deps": "1.0.1" }, other: { "no-deps": "1.0.0" } }); + expect(await pkgJsonText(dir, "packages", "a")).toBe(member); + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); }); - // mismatched-peer-deps-lvl1 declares peer no-deps@<=1.0.1; its own dependency lvl2 declares peer no-deps@1.0.0, - // so the installs warn about an incorrect peer and cannot go through runBunInstall. - test.concurrent("a peer range is a blocker and is labelled with the peer dependent", async () => { + // mismatched-peer-deps-lvl1's own dependency declares a peer the install warns about, so runBunInstall cannot be used. + test.concurrent("a peer edge that rejects the fix is split off and labelled with the peer dependent", async () => { await using server = startRegistry({ "no-deps": [adv("<=1.0.1")] }); - const pkgJson = (noDeps: string) => + const rootPkgJson = (noDeps: string) => JSON.stringify({ name: "foo", dependencies: { "mismatched-peer-deps-lvl1": "1.0.0", "no-deps": noDeps } }); - using dir = tempDir("audit-fix-", { "package.json": pkgJson("1.0.1") }); + using dir = tempDir("audit-fix-", { "package.json": rootPkgJson("1.0.1") }); await writeBunfig(dir, server); - expect((await run(dir, ["install"])).exitCode).toBe(0); - await write(join(dir, "package.json"), pkgJson("^1.0.0")); - expect((await run(dir, ["install"])).exitCode).toBe(0); - const lockBefore = await lock(dir); - expect(lockBefore).toContain('"no-deps@1.0.1"'); + await expectInstall(dir); + await write(join(dir, "package.json"), rootPkgJson("^1.0.0")); + await expectInstall(dir); + expect(await lock(dir)).toContain('"no-deps@1.0.1"'); const { stdout, exitCode } = await auditFix(dir); - expect(stdout).toContain("no-deps@1.0.1 → 1.1.0"); + expect(stdout).toContain("fixing:\n no-deps@1.0.1 → 1.1.0"); + expect(stdout).toContain("blocked by a dependent's range:"); expect(stdout).toContain("mismatched-peer-deps-lvl1@1.0.0 depends on no-deps@<=1.0.1"); expect(stdout).not.toContain("foo depends on"); + expect(stdout).toContain("1 vulnerability remaining"); expect(exitCode).toBe(1); - expect(await lock(dir)).toBe(lockBefore); + expect(await lock(dir)).toContain('"no-deps@1.1.0"'); }); test.concurrent("a depth-3 blocker names the immediate dependent", async () => { @@ -1308,7 +1481,7 @@ describe("`bun audit fix`", () => { const { stdout, exitCode } = await auditFix(dir); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "requires a semver-major update: + "blocked by a dependent's range: no-deps@1.0.1 → 1.1.0 one-dep@1.0.0 depends on no-deps@1.0.1 @@ -1326,9 +1499,9 @@ describe("`bun audit fix`", () => { const rootPkgJson = JSON.stringify({ name: "root", workspaces: ["packages/*"] }); using dir = tempDir("audit-fix-", { "package.json": rootPkgJson, "packages/a/package.json": member("1.0.2") }); await writeBunfig(dir, server); - expect((await run(dir, ["install", "--linker", "hoisted"])).exitCode).toBe(0); + await expectInstall(dir, "--linker", "hoisted"); await write(join(dir, "packages", "a", "package.json"), member("^1.0.2")); - expect((await run(dir, ["install", "--linker", "hoisted"])).exitCode).toBe(0); + await expectInstall(dir, "--linker", "hoisted"); expect(await lock(dir)).toContain('"a-dep@1.0.2"'); const { stdout, exitCode } = await run(join(dir, "packages", "a"), ["audit", "fix", "--linker", "hoisted"], dir); @@ -1348,16 +1521,15 @@ describe("`bun audit fix`", () => { expect(frozen.exitCode).toBe(0); }); - // `--linker isolated` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. test.concurrent("isolated linker layout", async () => { await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); using dir = tempDir("audit-fix-", { "package.json": JSON.stringify({ name: "foo", dependencies: { "a-dep": "1.0.2" } }), }); await writeBunfig(dir, server); - expect((await run(dir, ["install", "--linker", "isolated"])).exitCode).toBe(0); + await expectInstall(dir, "--linker", "isolated"); await write(join(dir, "package.json"), JSON.stringify({ name: "foo", dependencies: { "a-dep": "^1.0.2" } })); - expect((await run(dir, ["install", "--linker", "isolated"])).exitCode).toBe(0); + await expectInstall(dir, "--linker", "isolated"); expect(await lock(dir)).toContain('"a-dep@1.0.2"'); expect(await readlink(join(dir, "node_modules", "a-dep"))).toContain("a-dep@1.0.2"); @@ -1375,28 +1547,47 @@ describe("`bun audit fix`", () => { }); // Every a-dep release was published in 2023, so a 100-year minimum age gates all of them. - test.concurrent("a fix gated by --minimum-release-age is reported distinctly", async () => { + test.concurrent("a fix newer than --minimum-release-age is installed anyway", async () => { await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); using dir = await setupVulnerableADep(server); const lockBefore = await lock(dir); - const gated = await auditFix(dir, "--minimum-release-age", "3153600000"); - expect(normalizeBunSnapshot(gated.stdout)).toMatchInlineSnapshot(` - "no fix available: - a-dep@1.0.2 (1.0.4 is newer than --minimum-release-age) - - No fixable vulnerabilities - 1 vulnerability remaining" - `); - expect(gated.exitCode).toBe(1); + const dryRun = await auditFix(dir, "--dry-run", "--minimum-release-age", "3153600000"); + expect(dryRun.stdout).toContain("a-dep@1.0.2 → 1.0.4 (newer than --minimum-release-age)"); + expect(dryRun.stdout).toContain("Would fix 1 vulnerability in 1 package"); + expect(dryRun.exitCode).toBe(0); expect(await lock(dir)).toBe(lockBefore); - const fixed = await auditFix(dir, "--minimum-release-age", "60"); - expect(fixed.stdout).toContain("a-dep@1.0.2 → 1.0.4"); - expect(fixed.exitCode).toBe(0); + const { stdout, stderr, exitCode } = await auditFix(dir, "--minimum-release-age", "3153600000"); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4 (newer than --minimum-release-age)"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); expect(await lock(dir)).toContain('"a-dep@1.0.4"'); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + + await expectInstall(dir, "--frozen-lockfile", "--minimum-release-age", "3153600000"); }); + test.concurrent( + "the lowest safe release is taken even when only it is newer than --minimum-release-age", + async () => { + await using server = startRegistry( + { "a-dep": [adv("<1.0.4")] }, + { rewriteTime: { "a-dep": { "1.0.4": new Date().toISOString() } } }, + ); + using dir = await setupVulnerableADep(server); + + const { stdout, exitCode } = await auditFix(dir, "--minimum-release-age", "86400"); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4 (newer than --minimum-release-age)"); + expect(stdout).not.toContain("1.0.5"); + expect(exitCode).toBe(0); + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).not.toContain('"a-dep@1.0.5"'); + }, + ); + test.concurrent("a failing bulk endpoint changes nothing", async () => { await using server = startRegistry({}, { bulkStatus: 500 }); using dir = await setupVulnerableADep(server); @@ -1418,8 +1609,8 @@ describe("`bun audit fix`", () => { const { stdout, stderr, exitCode } = await auditFix(dir); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` - "no fix available: - a-dep@1.0.2 (failed to fetch the manifest) + "manifest could not be fetched: + a-dep@1.0.2 No fixable vulnerabilities 1 vulnerability remaining" @@ -1430,6 +1621,42 @@ describe("`bun audit fix`", () => { expect(await lock(dir)).toBe(lockBefore); }); + test.concurrent("a manifest that fails to download does not stop the other fixes", async () => { + const denyManifests = new Set(); + await using server = startRegistry({ "a-dep": [adv("<1.0.4")], "no-deps": [adv("<1.0.1", 2)] }, { denyManifests }); + using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "a-dep": "^1.0.2", "no-deps": "^1.0.0" } }); + denyManifests.add("a-dep"); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).toContain("fixing:\n no-deps@1.0.0 → 1.0.1\n"); + expect(stdout).toContain("manifest could not be fetched:\n a-dep@1.0.2\n"); + expect(stdout).not.toContain("no fix available:"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(stdout).toContain("1 vulnerability remaining"); + expect(stderr).toContain("a-dep"); + expect(exitCode).toBe(1); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).toContain('"a-dep@1.0.2"'); + }); + + test.concurrent("a fix whose tarball fails to download is not reported as fixed", async () => { + const denyTarballs = new Set(); + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }, { denyTarballs }); + using dir = await setupVulnerableADep(server); + denyTarballs.add("a-dep-1.0.4.tgz"); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(stdout).not.toContain("Fixed 1 vulnerability"); + expect(stderr).toContain("a-dep"); + expect(stderr).toContain("1.0.4"); + expect(exitCode).toBe(1); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.2"); + }); + test.concurrent("refuses to run without a lockfile", async () => { await using server = startRegistry({}); using dir = tempDir("audit-fix-", { @@ -1455,4 +1682,551 @@ describe("`bun audit fix`", () => { expect(exitCode).toBe(1); expect(await lock(dir)).toBe(lockBefore); }); + + test.concurrent("rewrites an exact direct pin", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + const before = [ + "{", + ' "name": "foo",', + ' "scripts": {', + ' "check": "true"', + " },", + ' "dependencies": {', + ' "a-dep": "1.0.2"', + " }", + "}", + "", + ].join("\n"); + using dir = await setup(server, before); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toStartWith("fixing:\n a-dep@1.0.2 → 1.0.4\n package.json: 1.0.2 → 1.0.4"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + + expect(await pkgJsonText(dir)).toBe(before.replace('"a-dep": "1.0.2"', '"a-dep": "1.0.4"')); + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep": "1.0.4"'); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).not.toContain("a-dep@1.0.2"); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("--dry-run does not rewrite a pin", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2" } }); + const pkgJsonBefore = await pkgJsonText(dir); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await auditFix(dir, "--dry-run"); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(stdout).toContain("package.json: 1.0.2 → 1.0.4"); + expect(stdout).toContain("Would fix 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + expect(await pkgJsonText(dir)).toBe(pkgJsonBefore); + expect(await lock(dir)).toBe(lockBefore); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.2"); + }); + + test.concurrent("a pin is only widened within its major", async () => { + await using server = startRegistry({ "no-deps": [adv("<2.0.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const pkgJsonBefore = await pkgJsonText(dir); + const lockBefore = await lock(dir); + + const { stdout, stderr, exitCode } = await auditFix(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "blocked by a dependent's range: + no-deps@1.0.0 → 2.0.0 + foo depends on no-deps@1.0.0 + + No fixable vulnerabilities + 1 vulnerability remaining" + `); + expect(stderr).not.toContain("Saved lockfile"); + expect(exitCode).toBe(1); + expect(await pkgJsonText(dir)).toBe(pkgJsonBefore); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("rewrites an exact npm: alias pin", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setup(server, { name: "foo", dependencies: { nd: "npm:a-dep@1.0.2" } }); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(stdout).toContain("package.json: npm:a-dep@1.0.2 → npm:a-dep@1.0.4"); + expect(exitCode).toBe(0); + + expect((await pkgJson(dir)).dependencies).toEqual({ nd: "npm:a-dep@1.0.4" }); + expect(await installedVersion(dir, "nd")).toBe("1.0.4"); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("rewrites a pinned devDependency in its own group only", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")], "no-deps": [adv("<1.0.1", 2)] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "no-deps": "1.0.0" }, + devDependencies: { "a-dep": "1.0.2" }, + }); + await reinstall(dir, { name: "foo", dependencies: { "no-deps": "^1.0.0" }, devDependencies: { "a-dep": "1.0.2" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.2"'); + expect(lockfile).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4\n package.json: 1.0.2 → 1.0.4\n"); + expect(stdout).toContain("no-deps@1.0.0 → 1.0.1\n"); + expect(stdout).toContain("Fixed 2 vulnerabilities in 2 packages"); + expect(exitCode).toBe(0); + + const { dependencies, devDependencies } = await pkgJson(dir); + expect({ dependencies, devDependencies }).toEqual({ + dependencies: { "no-deps": "^1.0.0" }, + devDependencies: { "a-dep": "1.0.4" }, + }); + lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("rewrites a workspace member's pin and leaves the root alone", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + const rootPkgJson = JSON.stringify({ name: "root", workspaces: ["packages/*"] }); + using dir = await setup(server, rootPkgJson, { + "packages/a/package.json": JSON.stringify({ name: "a", dependencies: { "a-dep": "1.0.2" } }), + }); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4\n packages/a/package.json: 1.0.2 → 1.0.4\n"); + expect(exitCode).toBe(0); + + expect((await pkgJson(dir, "packages", "a")).dependencies).toEqual({ "a-dep": "1.0.4" }); + expect(await pkgJsonText(dir)).toBe(rootPkgJson); + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).toContain('"a-dep": "1.0.4"'); + expect(lockfile).not.toContain("a-dep@1.0.2"); + expect(await exists(join(dir, "packages", "a", "bun.lock"))).toBeFalse(); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("rewrites a workspace member's pin when run from the member directory", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + const rootPkgJson = JSON.stringify({ name: "root", workspaces: ["packages/*"] }); + using dir = tempDir("audit-fix-", { + "package.json": rootPkgJson, + "packages/a/package.json": JSON.stringify({ name: "a", dependencies: { "a-dep": "1.0.2" } }), + }); + await writeBunfig(dir, server); + await expectInstall(dir, "--linker", "hoisted"); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await run(join(dir, "packages", "a"), ["audit", "fix", "--linker", "hoisted"], dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4\n packages/a/package.json: 1.0.2 → 1.0.4\n"); + expect(exitCode).toBe(0); + + expect((await pkgJson(dir, "packages", "a")).dependencies).toEqual({ "a-dep": "1.0.4" }); + expect(await pkgJsonText(dir)).toBe(rootPkgJson); + expect(await exists(join(dir, "packages", "a", "bun.lock"))).toBeFalse(); + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + + await expectInstall(dir, "--frozen-lockfile", "--linker", "hoisted"); + }); + + test.concurrent("splits an instance when only some dependents accept the fix", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-fixed-dep": "1.0.0", "no-deps": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "one-fixed-dep": "1.0.0", "no-deps": "^1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("fixing:\n no-deps@1.0.0 → 1.0.1\n"); + expect(stdout).toContain("blocked by a dependent's range:"); + expect(stdout).toContain("one-fixed-dep@1.0.0 depends on no-deps@1.0.0"); + expect(stdout).toContain("Fixed 0 vulnerabilities in 1 package"); + expect(stdout).toContain("1 vulnerability remaining"); + expect(exitCode).toBe(1); + + lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); + expect(await installedVersion(dir, "one-fixed-dep", "node_modules", "no-deps")).toBe("1.0.0"); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("splits an instance shared by two transitive dependents", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + using dir = await setup(server, { + name: "foo", + dependencies: { "one-range-dep": "1.0.0", "one-fixed-dep": "1.0.0", "no-deps": "1.0.0" }, + }); + await reinstall(dir, { name: "foo", dependencies: { "one-range-dep": "1.0.0", "one-fixed-dep": "1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.0"'); + expect(lockBefore).not.toContain('"no-deps@1.0.1"'); + + const dryRun = await auditFix(dir, "--dry-run"); + expect(dryRun.stdout).toContain("fixing:\n no-deps@1.0.0 → 1.0.1\n"); + expect(dryRun.stdout).toContain("one-fixed-dep@1.0.0 depends on no-deps@1.0.0"); + expect(dryRun.stdout).not.toContain("one-range-dep@1.0.0 depends on"); + expect(dryRun.exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + + const { exitCode } = await auditFix(dir); + expect(exitCode).toBe(1); + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(await resolvedVersion(dir, "one-range-dep", "no-deps")).toBe("1.0.1"); + expect(await resolvedVersion(dir, "one-fixed-dep", "no-deps")).toBe("1.0.0"); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("a split instance under the isolated linker keeps both versions in the store", async () => { + await using server = startRegistry({ "no-deps": [adv("<1.0.1")] }); + const rootPkgJson = (deps: Record) => JSON.stringify({ name: "foo", dependencies: deps }); + using dir = tempDir("audit-fix-", { + "package.json": rootPkgJson({ "one-range-dep": "1.0.0", "one-fixed-dep": "1.0.0", "no-deps": "1.0.0" }), + }); + await writeBunfig(dir, server); + await expectInstall(dir, "--linker", "isolated"); + await write(join(dir, "package.json"), rootPkgJson({ "one-range-dep": "1.0.0", "one-fixed-dep": "1.0.0" })); + await expectInstall(dir, "--linker", "isolated"); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.0"'); + expect(lockBefore).not.toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir, "--linker", "isolated"); + expect(stdout).toContain("fixing:\n no-deps@1.0.0 → 1.0.1\n"); + expect(exitCode).toBe(1); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.1"'); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(await installedVersion(dir, ".bun", "one-range-dep@1.0.0", "node_modules", "no-deps")).toBe("1.0.1"); + expect(await installedVersion(dir, ".bun", "one-fixed-dep@1.0.0", "node_modules", "no-deps")).toBe("1.0.0"); + + await expectInstall(dir, "--frozen-lockfile", "--linker", "isolated"); + }); + + test.concurrent( + "a rewritten root pin moves even though a transitive dependent still pins the old version", + async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2", "uses-a-dep-2": "1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.2"'); + expect(lockfile).not.toContain('"a-dep@1.0.4"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("fixing:\n a-dep@1.0.2 → 1.0.4\n package.json: 1.0.2 → 1.0.4\n"); + expect(stdout).toContain("uses-a-dep-2@1.0.0 depends on a-dep@1.0.2"); + expect(stdout).toContain("1 vulnerability remaining"); + expect(exitCode).toBe(1); + + expect((await pkgJson(dir)).dependencies).toEqual({ "a-dep": "1.0.4", "uses-a-dep-2": "1.0.0" }); + lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).toContain('"a-dep@1.0.2"'); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + expect(await installedVersion(dir, "uses-a-dep-2", "node_modules", "a-dep")).toBe("1.0.2"); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }, + ); + + test.concurrent("downgrades when no newer release is safe", async () => { + await using server = startRegistry({ "a-dep": [adv(">=1.0.3")] }); + using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "^1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"a-dep@1.0.10"'); + const pkgJsonBefore = await pkgJsonText(dir); + + const dryRun = await auditFix(dir, "--dry-run"); + expect(normalizeBunSnapshot(dryRun.stdout)).toMatchInlineSnapshot(` + "fixing: + a-dep@1.0.10 → 1.0.2 (downgrade) + + Would fix 1 vulnerability in 1 package" + `); + expect(dryRun.exitCode).toBe(0); + expect(await lock(dir)).toBe(lockBefore); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.10 → 1.0.2 (downgrade)"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.2"'); + expect(lockfile).not.toContain('"a-dep@1.0.10"'); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.2"); + expect(await pkgJsonText(dir)).toBe(pkgJsonBefore); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + }); + + test.concurrent("downgrades a transitive dependency within its dependent's range", async () => { + await using server = startRegistry({ "no-deps": [adv(">=1.0.1 <2.0.0")] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-range-dep": "1.0.0" } }); + expect(await lock(dir)).toContain('"no-deps@1.1.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("no-deps@1.1.0 → 1.0.0 (downgrade)"); + expect(stdout).not.toContain("depends on"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.1.0"'); + expect(lockfile).not.toContain('"no-deps@2.0.0"'); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.0"); + + await runBunInstall(installEnv(dir), dir, { frozenLockfile: true }); + }); + + test.concurrent("prefers the lowest safe upgrade over any downgrade", async () => { + await using server = startRegistry({ "a-dep": [adv(">=1.0.2 <1.0.4")] }); + using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2" } }); + await reinstall(dir, { name: "foo", dependencies: { "a-dep": "^1.0.1" } }); + expect(await lock(dir)).toContain('"a-dep@1.0.2"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("a-dep@1.0.2 → 1.0.4"); + expect(stdout).not.toContain("downgrade"); + expect(exitCode).toBe(0); + + const lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(lockfile).not.toContain('"a-dep@1.0.1"'); + expect(lockfile).not.toContain('"a-dep@1.0.5"'); + }); + + test.concurrent("counts a vulnerability removed by another fix from the written lockfile", async () => { + await using server = startRegistry({ "one-fixed-dep": [adv("<2.0.0")], "no-deps": [adv("<1.0.1", 2)] }); + using dir = await setup(server, { name: "foo", dependencies: { "one-fixed-dep": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "one-fixed-dep": ">=1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"one-fixed-dep@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("one-fixed-dep@1.0.0 → 2.0.0"); + expect(stdout).toContain("one-fixed-dep@1.0.0 depends on no-deps@1.0.0"); + expect(stdout).toContain("Fixed 2 vulnerabilities in 1 package"); + expect(stdout).not.toContain("remaining"); + expect(exitCode).toBe(0); + + lockfile = await lock(dir); + expect(lockfile).toContain('"one-fixed-dep@2.0.0"'); + expect(lockfile).toContain('"no-deps@2.0.0"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + }); + + test.concurrent("reports vulnerable versions introduced by the fix from the written lockfile", async () => { + await using server = startRegistry( + {}, + { bulkResponse: { "one-fixed-dep": [adv("<2.0.0")], "no-deps": [adv(">=2.0.0", 2)] } }, + ); + using dir = await setup(server, { name: "foo", dependencies: { "one-fixed-dep": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "one-fixed-dep": "^1.0.0 || ^2.0.0" } }); + expect(await lock(dir)).toContain('"one-fixed-dep@1.0.0"'); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("one-fixed-dep@1.0.0 → 2.0.0"); + expect(stdout).toContain("vulnerable after install:\n no-deps@2.0.0\n"); + expect(stdout).toContain("Fixed 1 vulnerability in 1 package"); + expect(stdout).toContain("1 vulnerability remaining"); + expect(exitCode).toBe(1); + expect(await lock(dir)).toContain('"no-deps@2.0.0"'); + }); + + test.concurrent("--json prints a plan document with --dry-run", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2" } }); + const pkgJsonBefore = await pkgJsonText(dir); + const lockBefore = await lock(dir); + + const { stdout, exitCode } = await auditFix(dir, "--json", "--dry-run"); + expect(JSON.parse(stdout)).toEqual({ + dryRun: true, + fixed: 1, + remaining: 0, + fixes: [ + { + name: "a-dep", + from: "1.0.2", + to: "1.0.4", + downgrade: false, + newerThanMinimumReleaseAge: false, + packageJson: [{ file: "package.json", key: "a-dep", from: "1.0.2", to: "1.0.4" }], + }, + ], + blocked: [], + unfixable: [], + manifestUnavailable: [], + unmatched: [], + unaudited: [], + vulnerableAfterInstall: [], + }); + expect(exitCode).toBe(0); + expect(await pkgJsonText(dir)).toBe(pkgJsonBefore); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("--json prints the result after installing", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setupVulnerableADep(server); + + const { stdout, exitCode } = await auditFix(dir, "--json"); + expect(stdout.trim().split("\n")).toHaveLength(1); + const doc = JSON.parse(stdout); + expect(doc).toMatchObject({ dryRun: false, fixed: 1, remaining: 0 }); + expect(doc.fixes).toEqual([ + { + name: "a-dep", + from: "1.0.2", + to: "1.0.4", + downgrade: false, + newerThanMinimumReleaseAge: false, + packageJson: [], + }, + ]); + expect(exitCode).toBe(0); + expect(await lock(dir)).toContain('"a-dep@1.0.4"'); + + const clean = await auditFix(dir, "--json"); + expect(JSON.parse(clean.stdout)).toMatchObject({ dryRun: false, fixed: 0, remaining: 0, fixes: [] }); + expect(clean.exitCode).toBe(0); + }); + + test.concurrent("--json with a blocked and an unmatched advisory", async () => { + await using server = startRegistry({}, { bulkResponse: { "no-deps": [adv("<1.1.0"), adv(">=9.0.0", 2)] } }); + using dir = await setup(server, { name: "foo", dependencies: { "one-dep": "1.0.0" } }); + const lockBefore = await lock(dir); + expect(lockBefore).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir, "--json"); + const doc = JSON.parse(stdout); + expect(doc.blocked).toEqual([ + { + name: "no-deps", + from: "1.0.1", + needs: "1.1.0", + downgrade: false, + blockers: [{ dependent: "one-dep@1.0.0", range: "1.0.1", bundled: false }], + }, + ]); + expect(doc.unmatched).toEqual([{ name: "no-deps", range: ">=9.0.0" }]); + expect(doc).toMatchObject({ dryRun: false, fixed: 0, remaining: 2, fixes: [] }); + expect(exitCode).toBe(1); + expect(await lock(dir)).toBe(lockBefore); + }); + + test.concurrent("--json after installing carries the fixed and the blocked entries", async () => { + await using server = startRegistry({ "a-dep": [adv("<1.0.4")], "no-deps": [adv("<1.1.0", 2)] }); + using dir = await setup(server, { name: "foo", dependencies: { "a-dep": "1.0.2", "one-dep": "1.0.0" } }); + await reinstall(dir, { name: "foo", dependencies: { "a-dep": "^1.0.2", "one-dep": "1.0.0" } }); + let lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.2"'); + expect(lockfile).toContain('"no-deps@1.0.1"'); + + const { stdout, exitCode } = await auditFix(dir, "--json"); + expect(JSON.parse(stdout)).toMatchObject({ + dryRun: false, + fixed: 1, + remaining: 1, + fixes: [{ name: "a-dep", from: "1.0.2", to: "1.0.4" }], + blocked: [{ name: "no-deps", from: "1.0.1", needs: "1.1.0", blockers: [{ dependent: "one-dep@1.0.0" }] }], + vulnerableAfterInstall: [], + }); + expect(exitCode).toBe(1); + + lockfile = await lock(dir); + expect(lockfile).toContain('"a-dep@1.0.4"'); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.4"); + }); + + test.concurrent("audits and fixes a package served by a scoped registry", async () => { + await using scoped = startRegistry({ "@types/is-number": [adv("<2.0.0")] }); + await using server = startRegistry({}); + using dir = await setup( + server, + { name: "foo", dependencies: { "@types/is-number": "1.0.0" } }, + {}, + { + types: scoped.url.href, + }, + ); + await reinstall(dir, { name: "foo", dependencies: { "@types/is-number": ">=1.0.0" } }); + expect(await lock(dir)).toContain('"@types/is-number@1.0.0"'); + + const before = await audit(dir); + expect(before.stdout).toContain("@types/is-number"); + expect(before.stdout).not.toContain("Skipped"); + expect(before.exitCode).toBe(1); + + const { stdout, exitCode } = await auditFix(dir); + expect(stdout).toContain("@types/is-number@1.0.0 → 2.0.0"); + expect(stdout).not.toContain("Skipped"); + expect(exitCode).toBe(0); + expect(await installedVersion(dir, "@types", "is-number")).toBe("2.0.0"); + + const recheck = await audit(dir); + expect(recheck.stdout).toBe("No vulnerabilities found\n"); + expect(recheck.exitCode).toBe(0); + }); + + test.concurrent("a scoped registry that does not answer the audit request is reported", async () => { + await using scoped = startRegistry({}, { bulkStatus: 404 }); + await using server = startRegistry({ "a-dep": [adv("<1.0.4")] }); + using dir = await setup( + server, + { name: "foo", dependencies: { "@types/is-number": "1.0.0", "a-dep": "1.0.2" } }, + {}, + { + types: scoped.url.href, + }, + ); + const skipped = `Skipped @types/is-number because ${registryHref(scoped)} could not be audited`; + + const report = await audit(dir); + expect(report.stdout).toContain(skipped); + expect(report.stdout).toContain("a-dep"); + expect(report.stdout).toContain("1 vulnerability (1 high)"); + expect(report.exitCode).toBe(1); + + const dryRun = await auditFix(dir, "--dry-run"); + expect(dryRun.stdout).toContain(skipped); + expect(dryRun.stdout).toContain("Would fix 1 vulnerability in 1 package"); + expect(dryRun.exitCode).toBe(0); + }); }); diff --git a/test/cli/install/bun-dedupe.test.ts b/test/cli/install/bun-dedupe.test.ts index 99e928439664..e4d214cc226c 100644 --- a/test/cli/install/bun-dedupe.test.ts +++ b/test/cli/install/bun-dedupe.test.ts @@ -45,6 +45,54 @@ function nodeModulesVersion(packageDir: string, ...segments: string[]) { .then(pkg => pkg.version); } +async function expectAlreadyDeduplicated(dir: string) { + const check = await dedupe(dir, "--check"); + expect(check.stdout).toContain("Already deduplicated."); + expect(check.exitCode).toBe(0); +} + +async function expectRefused(dir: string, ...args: string[]) { + const { stdout, stderr, exitCode } = await dedupe(dir, ...args); + expect(normalizeBunSnapshot(stderr)).toContain( + "error: the lockfile is out of date with package.json, nothing was deduplicated", + ); + expect(normalizeBunSnapshot(stderr)).toContain("note: run 'bun install' first"); + expect(stdout).not.toContain("duplicate"); + expect(stdout).not.toContain("Removed"); + expect(exitCode).toBe(1); +} + +// bun.lock writes one package entry per line; edits the entry whose line starts with `entryPrefix`. +function editLockEntry(lockfile: string, entryPrefix: string, from: string, to: string) { + const lines = lockfile.split("\n"); + const i = lines.findIndex(line => line.trimStart().startsWith(entryPrefix)); + expect(i).not.toBe(-1); + expect(lines[i]).toContain(from); + lines[i] = lines[i].replace(from, to); + return lines.join("\n"); +} + +// Same trick as 'never upgrades past the locked version': widen the range in both files, keeping the locked resolution. +async function widen(packageDir: string, packageJsonPath: string, name: string, range: string) { + const [pkg, lockfile] = await Promise.all([file(packageJsonPath).json(), lock(packageDir)]); + const from = `"${name}": "${pkg.dependencies[name]}"`; + expect(lockfile.split(from)).toHaveLength(2); + pkg.dependencies[name] = range; + await Promise.all([ + write(packageJsonPath, JSON.stringify(pkg)), + write(join(packageDir, "bun.lock"), lockfile.replace(from, `"${name}": "${range}"`)), + ]); +} + +const noDepsPatch = `diff --git a/patched.txt b/patched.txt +new file mode 100644 +index 0000000000000000000000000000000000000000..3b18e512dba79e4c8300dd08aeb37f8e728b8dad +--- /dev/null ++++ b/patched.txt +@@ -0,0 +1 @@ ++hello world +`; + // A still-satisfied range edge is never re-resolved by a later install, so adding an exact pin afterwards leaves a duplicate. async function installTwice(packageDir: string, packageJson: string, first: object, second: object) { await write(packageJson, JSON.stringify(first)); @@ -254,7 +302,17 @@ test.concurrent("override range wins over the edge's own range", async () => { '\n "packages": {', '\n "overrides": {\n "no-deps": "1.1.0",\n },\n "packages": {', ); - await write(join(packageDir, "bun.lock"), withOverride); + await Promise.all([ + write(join(packageDir, "bun.lock"), withOverride), + write( + packageJson, + JSON.stringify({ + name: "foo", + dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" }, + overrides: { "no-deps": "1.1.0" }, + }), + ), + ]); const { stdout, stderr, exitCode } = await dedupe(packageDir, "--check"); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` @@ -304,7 +362,6 @@ test.concurrent("--help", async () => { test.concurrent("works with the isolated linker", async () => { const { packageDir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); - // `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. const install = async (...args: string[]) => { const result = await run(packageDir, "install", ...args, "--linker", "isolated"); expect(result.stderr).not.toContain("error:"); @@ -388,7 +445,7 @@ test.concurrent("duplicate held only by an optional peer edge is deduplicated", const after = await lock(packageDir); expect(after).toContain('"no-deps@1.0.0"'); expect(after).not.toContain('"no-deps@1.1.0"'); - expect((await dedupe(packageDir, "--check")).exitCode).toBe(0); + await expectAlreadyDeduplicated(packageDir); await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); @@ -551,7 +608,9 @@ test.concurrent("--check never creates node_modules", async () => { expect(await lock(packageDir)).toBe(lockBefore); await runBunInstall(installEnv(packageDir), packageDir, { savesLockfile: false }); - await dedupe(packageDir); + const apply = await dedupe(packageDir); + expect(apply.stdout).toContain("Removed 1 duplicate version: no-deps@1.1.0"); + expect(apply.exitCode).toBe(0); const lockDeduped = await lock(packageDir); await rm(join(packageDir, "node_modules"), { recursive: true }); @@ -589,9 +648,9 @@ test.concurrent("never upgrades past the locked version", async () => { }); test.concurrent.each([ - ["--ignore-scripts", false], - [undefined, true], -])("root lifecycle scripts with %s", async (flag, runs) => { + ["skipped with --ignore-scripts", ["--ignore-scripts"], false], + ["run by default", [], true], +])("root lifecycle scripts are %s", async (_, flags, runs) => { const { packageDir, packageJson } = await registry.createTestDir(); await setupRangeDuplicate(packageDir, packageJson); // Added after setup so only the dedupe run can create the marker. @@ -599,7 +658,7 @@ test.concurrent.each([ await write(packageJson, JSON.stringify({ ...pkg, scripts: { postinstall: "echo ran > postinstall.txt" } })); const marker = join(packageDir, "postinstall.txt"); - const { stdout, stderr, exitCode } = await dedupe(packageDir, ...(flag ? [flag] : [])); + const { stdout, stderr, exitCode } = await dedupe(packageDir, ...flags); expect(stdout).toContain("Removed 1 duplicate version: no-deps@1.1.0"); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); @@ -607,7 +666,7 @@ test.concurrent.each([ expect(await exists(marker)).toBe(runs); }); -// "Fewest versions" policy (pnpm/pnpm#4753, #6762): a direct dependency may be moved to an older version that still satisfies its range. +// A direct dependency only moves down when that is the only way to drop a version (pnpm would keep both, pnpm/pnpm#4753, #6762). test.concurrent("root range collapses onto a transitive exact pin", async () => { const { packageDir, packageJson } = await registry.createTestDir(); await write( @@ -739,25 +798,485 @@ test.concurrent("--silent keeps the exit codes", async () => { expect(await lock(packageDir)).not.toContain('"no-deps@1.1.0"'); }); -// dedupe runs against the ranges recorded in bun.lock; a stale package.json is applied by the install that follows. -test.concurrent("stale package.json is resolved after the dedupe", async () => { +// The workspace fixture: root pins no-deps@1.0.0, packages/a -> one-range-dep -> no-deps@1.1.0. +async function setupWorkspaceDuplicate(packageDir: string, packageJson: string, workspacePackageJson: object) { + const workspaceDir = join(packageDir, "packages", "a"); + await write(join(workspaceDir, "package.json"), JSON.stringify(workspacePackageJson)); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "root", workspaces: ["packages/*"] }, + { name: "root", workspaces: ["packages/*"], dependencies: { "no-deps": "1.0.0" } }, + ); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + return workspaceDir; +} + +type StaleFixture = { packageDir: string; cwds: string[]; afterRefusal?: () => Promise }; + +// Every edit is still satisfied by the locked versions, so the refusal is about the ranges, not about re-resolution. +test.concurrent.each<[string, () => Promise]>([ + [ + "a root dependency range", + async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + await write( + packageJson, + JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "^1.0.0" } }), + ); + const nested = () => nodeModulesVersion(packageDir, "one-range-dep", "node_modules", "no-deps"); + expect(await nested()).toBe("1.1.0"); + return { + packageDir, + cwds: [packageDir], + async afterRefusal() { + expect(await nested()).toBe("1.1.0"); + // The current range lets root move up onto 1.1.0; the recorded exact range would have removed it instead. + await runBunInstall(installEnv(packageDir), packageDir); + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 2)).toEqual([ + "bun dedupe ()", + "Removed 1 duplicate version: no-deps@1.0.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("1.1.0"); + }, + }; + }, + ], + [ + "an override", + async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + await write( + packageJson, + JSON.stringify({ + name: "foo", + dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" }, + overrides: { "no-deps": "1.1.0" }, + }), + ); + return { + packageDir, + cwds: [packageDir], + async afterRefusal() { + await runBunInstall(installEnv(packageDir), packageDir); + const lockfile = await lock(packageDir); + expect(lockfile).toContain('"overrides"'); + expect(lockfile).not.toContain('"no-deps@1.0.0"'); + await expectAlreadyDeduplicated(packageDir); + }, + }; + }, + ], + [ + "a catalog entry", + async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + // Same steps as 'honours catalog ranges', then the catalog entry is edited without installing. + const root = (catalogRange: string, dependencies?: Record) => + JSON.stringify({ + name: "root", + workspaces: ["packages/*"], + catalog: { "no-deps": catalogRange }, + dependencies, + }); + await Promise.all([ + write(packageJson, root("1.0.0")), + write( + join(packageDir, "packages", "a", "package.json"), + JSON.stringify({ name: "a", dependencies: { "no-deps": "catalog:" } }), + ), + ]); + await runBunInstall(installEnv(packageDir), packageDir); + await write(packageJson, root("^1.0.0")); + await runBunInstall(installEnv(packageDir), packageDir); + await write(packageJson, root("^1.0.0", { "no-deps": "1.1.0" })); + await runBunInstall(installEnv(packageDir), packageDir); + const lockfile = await lock(packageDir); + expect(lockfile).toContain('"no-deps@1.0.0"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + await write(packageJson, root(">=1.0.0", { "no-deps": "1.1.0" })); + return { packageDir, cwds: [packageDir] }; + }, + ], + [ + "a workspace package's dependencies", + async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const workspaceDir = await setupWorkspaceDuplicate(packageDir, packageJson, { + name: "a", + dependencies: { "one-range-dep": "1.0.0" }, + }); + await write( + join(workspaceDir, "package.json"), + JSON.stringify({ name: "a", dependencies: { "one-range-dep": "^1.0.0" } }), + ); + return { packageDir, cwds: [packageDir, workspaceDir] }; + }, + ], +])("refuses to dedupe when %s changed since the last install", async (_, setup) => { + const { packageDir, cwds, afterRefusal } = await setup(); + const lockBefore = await lock(packageDir); + + for (const cwd of cwds) { + await expectRefused(cwd, "--check"); + expect(await lock(packageDir)).toBe(lockBefore); + await expectRefused(cwd); + expect(await lock(packageDir)).toBe(lockBefore); + } + + await afterRefusal?.(); +}); + +// The gate is the dependency diff: diffs the differ reports on every install of an in-sync tree must not refuse. +test.concurrent.each([ + [ + "trustedDependencies", + async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + const pkg = await file(packageJson).json(); + await write(packageJson, JSON.stringify({ ...pkg, trustedDependencies: ["no-deps"] })); + return packageDir; + }, + ], + [ + "a workspace lifecycle script", + async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupWorkspaceDuplicate(packageDir, packageJson, { + name: "a", + dependencies: { "one-range-dep": "1.0.0" }, + scripts: { postinstall: "exit 0" }, + }); + return packageDir; + }, + ], + [ + "package.json formatting", + async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await setupRangeDuplicate(packageDir, packageJson); + await write( + packageJson, + JSON.stringify( + { description: "reformatted", dependencies: { "no-deps": "1.0.0", "one-range-dep": "1.0.0" }, name: "foo" }, + null, + 2, + ), + ); + return packageDir; + }, + ], +])("still dedupes when only %s changed", async (_, setup) => { + const packageDir = await setup(); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 2)).toEqual([ + "bun dedupe ()", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await lock(packageDir)).not.toContain('"no-deps@1.1.0"'); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); +}); + +test.concurrent("a direct dependency is not moved when its version survives anyway", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "foo", dependencies: { "no-deps": "1.0.1", "dwt": "npm:dep-with-tags@^1.0.0" } }, + { + name: "foo", + dependencies: { + "no-deps": "^1.0.0", + "one-dep": "1.0.0", + "one-fixed-dep": "1.0.0", + "has-bin-entries": "1.0.0", + "dwt": "npm:dep-with-tags@^1.0.0", + "dwt0": "npm:dep-with-tags@1.0.0", + }, + }, + ); + for (const label of ['"no-deps@1.0.0"', '"no-deps@1.0.1"', '"dep-with-tags@1.0.0"', '"dep-with-tags@1.0.1"']) { + expect(lockfile).toContain(label); + } + expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("1.0.1"); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 2)).toEqual([ + "bun dedupe ()", + "Removed 1 duplicate version: dep-with-tags@1.0.1", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).toContain('"no-deps@1.0.0"'); + expect(after).toContain('"no-deps@1.0.1"'); + expect(after).not.toContain('"dep-with-tags@1.0.1"'); + expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("1.0.1"); + expect(await nodeModulesVersion(packageDir, "dwt")).toBe("1.0.0"); + await expectAlreadyDeduplicated(packageDir); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); +}); + +// no-deps ends up as: direct ^1.0.0 @1.1.0, `<=1.0.1` @1.0.1, `>=1.0.1` @1.0.1, exact 1.0.0 — {1.0.0, 1.1.0} and {1.0.0, 1.0.1} are equally small covers. +test.concurrent.each(["root", "workspace"])( + "prefers dropping the version that keeps the direct dependency in place (%s)", + async variant => { + const { packageDir, packageJson } = await registry.createTestDir(); + const rootExtras = { "one-fixed-dep": "1.0.0", "one-dep": "1.0.0", "normal-dep-and-dev-dep": "1.0.0" }; + if (variant === "root") { + await write(packageJson, JSON.stringify({ name: "root", dependencies: { "no-deps": "^1.0.0" } })); + await runBunInstall(installEnv(packageDir), packageDir); + await write(packageJson, JSON.stringify({ name: "root", dependencies: { "no-deps": "^1.0.0", ...rootExtras } })); + } else { + await Promise.all([ + write(packageJson, JSON.stringify({ name: "root", workspaces: ["packages/*"] })), + write( + join(packageDir, "packages", "a", "package.json"), + JSON.stringify({ name: "a", dependencies: { "no-deps": "^1.0.0" } }), + ), + ]); + await runBunInstall(installEnv(packageDir), packageDir); + await write(packageJson, JSON.stringify({ name: "root", workspaces: ["packages/*"], dependencies: rootExtras })); + } + await runBunInstall(installEnv(packageDir), packageDir); + let lockfile = await lock(packageDir); + for (const label of ['"no-deps@1.0.0"', '"no-deps@1.0.1"', '"no-deps@1.1.0"']) { + expect(lockfile).toContain(label); + } + lockfile = editLockEntry(lockfile, '"one-dep": ["one-dep@1.0.0"', '"no-deps": "1.0.1"', '"no-deps": "<=1.0.1"'); + lockfile = editLockEntry( + lockfile, + '"normal-dep-and-dev-dep": ["normal-dep-and-dev-dep@1.0.0"', + '"no-deps": "1.0.1"', + '"no-deps": ">=1.0.1"', + ); + await write(join(packageDir, "bun.lock"), lockfile); + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + 1 duplicate version can be removed: no-deps@1.0.1" + `); + expect(check.exitCode).toBe(1); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 2)).toEqual([ + "bun dedupe ()", + "Removed 1 duplicate version: no-deps@1.0.1", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).toContain('"no-deps@1.1.0"'); + expect(after).toContain('"no-deps@1.0.0"'); + expect(after).not.toContain('"no-deps@1.0.1"'); + if (variant === "root") { + expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("1.1.0"); + } + await expectAlreadyDeduplicated(packageDir); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); + }, +); + +// pnpm/pnpm#4753 direct-dependency weighting: when the direct edge's own version is dropped, it moves to the highest survivor. +test.concurrent("a direct range whose version is dropped moves up, not down", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await Promise.all([ + write( + packageJson, + JSON.stringify({ + name: "root", + workspaces: ["packages/*"], + dependencies: { "a-dep": "1.0.5", "uses-a-dep-3": "1.0.0", "uses-a-dep-10": "1.0.0" }, + }), + ), + write( + join(packageDir, "packages", "b", "package.json"), + JSON.stringify({ name: "b", dependencies: { "a-dep": "1.0.3" } }), + ), + ]); + await runBunInstall(installEnv(packageDir), packageDir); + await widen(packageDir, packageJson, "a-dep", "^1.0.0"); + const lockfile = await lock(packageDir); + for (const label of ['"a-dep@1.0.3"', '"a-dep@1.0.5"', '"a-dep@1.0.10"']) { + expect(lockfile).toContain(label); + } + expect(await nodeModulesVersion(packageDir, "a-dep")).toBe("1.0.5"); + + const check = await dedupe(packageDir, "--check"); + expect(normalizeBunSnapshot(check.stdout)).toMatchInlineSnapshot(` + "bun dedupe () + 1 duplicate version can be removed: a-dep@1.0.5" + `); + expect(check.exitCode).toBe(1); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 2)).toEqual([ + "bun dedupe ()", + "Removed 1 duplicate version: a-dep@1.0.5", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).toContain('"a-dep@1.0.3"'); + expect(after).toContain('"a-dep@1.0.10"'); + expect(after).not.toContain('"a-dep@1.0.5"'); + expect(await nodeModulesVersion(packageDir, "a-dep")).toBe("1.0.10"); + await expectAlreadyDeduplicated(packageDir); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); +}); + +// a-dep@1.0.3 has more edges than root's 1.0.10, but both versions must stay, so removing no-deps@1.1.0 must not drag root down. +test.concurrent("removing one name does not downgrade an unrelated direct dependency", async () => { const { packageDir, packageJson } = await registry.createTestDir(); - await setupRangeDuplicate(packageDir, packageJson); await write( + join(packageDir, "packages", "b", "package.json"), + JSON.stringify({ name: "b", dependencies: { "a-dep": "1.0.3" } }), + ); + const rootDeps = { "a-dep": "^1.0.0", "uses-a-dep-3": "1.0.0", "uses-a-dep-10": "1.0.0", "one-range-dep": "1.0.0" }; + const lockfile = await installTwice( + packageDir, packageJson, - JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0", "no-deps": "^2.0.0" } }), + { name: "root", workspaces: ["packages/*"], dependencies: rootDeps }, + { name: "root", workspaces: ["packages/*"], dependencies: { ...rootDeps, "no-deps": "1.0.0" } }, ); + for (const label of ['"a-dep": ["a-dep@1.0.10"', '"a-dep@1.0.3"', '"no-deps@1.0.0"', '"no-deps@1.1.0"']) { + expect(lockfile).toContain(label); + } + expect(lockfile).not.toContain('"uses-a-dep-10/a-dep"'); + expect(await nodeModulesVersion(packageDir, "a-dep")).toBe("1.0.10"); const { stdout, stderr, exitCode } = await dedupe(packageDir); - expect(stdout).toContain("Removed 1 duplicate version: no-deps@1.1.0"); + expect(firstLines(stdout, 2)).toEqual([ + "bun dedupe ()", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); expect(stderr).not.toContain("error:"); expect(exitCode).toBe(0); const after = await lock(packageDir); - expect(after).toContain('"no-deps@2.0.0"'); + expect(after).toContain('"a-dep": ["a-dep@1.0.10"'); + expect(after).toContain('"a-dep@1.0.3"'); + expect(after).not.toContain('"uses-a-dep-10/a-dep"'); + expect(after).not.toContain('"no-deps@1.1.0"'); + expect(await nodeModulesVersion(packageDir, "a-dep")).toBe("1.0.10"); + await expectAlreadyDeduplicated(packageDir); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); +}); + +test.concurrent("a workspace member's own range moves up when run from the member directory", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const memberDir = join(packageDir, "packages", "b"); + const memberPackageJson = join(memberDir, "package.json"); + await Promise.all([ + write( + packageJson, + JSON.stringify({ + name: "root", + workspaces: ["packages/*"], + dependencies: { "a-dep": "1.0.3", "uses-a-dep-3": "1.0.0", "uses-a-dep-10": "1.0.0" }, + }), + ), + write(memberPackageJson, JSON.stringify({ name: "b", dependencies: { "a-dep": "1.0.5" } })), + ]); + await runBunInstall(installEnv(packageDir), packageDir); + await widen(packageDir, memberPackageJson, "a-dep", "^1.0.0"); + const lockfile = await lock(packageDir); + for (const label of ['"a-dep": ["a-dep@1.0.3"', '"b/a-dep": ["a-dep@1.0.5"', '"a-dep@1.0.10"']) { + expect(lockfile).toContain(label); + } + expect(await nodeModulesVersion(memberDir, "a-dep")).toBe("1.0.5"); + + const { stdout, stderr, exitCode } = await dedupe(memberDir); + expect(firstLines(stdout, 2)).toEqual([ + "bun dedupe ()", + "Removed 1 duplicate version: a-dep@1.0.5", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).toContain('"a-dep": ["a-dep@1.0.3"'); + expect(after).toContain('"b/a-dep": ["a-dep@1.0.10"'); + expect(after).not.toContain('"a-dep@1.0.5"'); + expect(await nodeModulesVersion(memberDir, "a-dep")).toBe("1.0.10"); + expect(await exists(join(memberDir, "bun.lock"))).toBeFalse(); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); +}); + +test.concurrent("ranges collapse onto the version a dist-tag resolved to", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + const lockfile = await installTwice( + packageDir, + packageJson, + { name: "foo", dependencies: { "dwt": "npm:dep-with-tags@>=1.0.0" } }, + { name: "foo", dependencies: { "dwt": "npm:dep-with-tags@>=1.0.0", "dep-with-tags": "latest" } }, + ); + expect(lockfile).toContain('"dep-with-tags@3.0.0"'); + expect(lockfile).toContain('"dep-with-tags@3.0.1"'); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 2)).toEqual([ + "bun dedupe ()", + "Removed 1 duplicate version: dep-with-tags@3.0.1", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + expect(await lock(packageDir)).not.toContain('"dep-with-tags@3.0.1"'); + expect(await nodeModulesVersion(packageDir, "dep-with-tags")).toBe("3.0.0"); + expect(await nodeModulesVersion(packageDir, "dwt")).toBe("3.0.0"); + await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); +}); + +// The patched version pins root's now-ranged direct edge, so the transitive edge collapses down onto it. +test.concurrent("a patched version wins over keeping the direct dependency's higher version", async () => { + const { packageDir, packageJson } = await registry.createTestDir(); + await Promise.all([ + write( + packageJson, + JSON.stringify({ + name: "foo", + dependencies: { "no-deps": "1.0.0", "one-range-dep": "1.0.0" }, + patchedDependencies: { "no-deps@1.0.0": "patches/no-deps@1.0.0.patch" }, + }), + ), + write(join(packageDir, "patches", "no-deps@1.0.0.patch"), noDepsPatch), + ]); + await runBunInstall(installEnv(packageDir), packageDir); + await widen(packageDir, packageJson, "no-deps", "^1.0.0"); + const lockfile = await lock(packageDir); + expect(lockfile).toContain('"no-deps@1.0.0": "patches/no-deps@1.0.0.patch"'); + expect(lockfile).toContain('"no-deps@1.1.0"'); + expect(await exists(join(packageDir, "node_modules", "no-deps", "patched.txt"))).toBeTrue(); + + const { stdout, stderr, exitCode } = await dedupe(packageDir); + expect(firstLines(stdout, 2)).toEqual([ + "bun dedupe ()", + "Removed 1 duplicate version: no-deps@1.1.0", + ]); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + + const after = await lock(packageDir); + expect(after).toContain('"no-deps@1.0.0": "patches/no-deps@1.0.0.patch"'); + expect(after).toContain('"no-deps": ["no-deps@1.0.0"'); expect(after).not.toContain('"no-deps@1.1.0"'); - expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("2.0.0"); - expect((await dedupe(packageDir, "--check")).exitCode).toBe(0); + expect(await nodeModulesVersion(packageDir, "no-deps")).toBe("1.0.0"); + expect(await exists(join(packageDir, "node_modules", "no-deps", "patched.txt"))).toBeTrue(); await runBunInstall(installEnv(packageDir), packageDir, { frozenLockfile: true }); }); @@ -865,19 +1384,24 @@ test.concurrent("bundled edges are never re-pointed", async () => { expect(after).not.toContain('"no-deps@1.1.0"'); }); -// pnpm/pnpm#11238, #10329, #8446: dedupe never re-resolves, so it works with the registry down and ignores minimumReleaseAge. +// pnpm/pnpm#11238, #10329, #8446: dedupe never re-resolves, so it sends no registry request and ignores minimumReleaseAge. test.concurrent("does not contact the registry", async () => { const { packageDir, packageJson } = await registry.createTestDir(); await setupRangeDuplicate(packageDir, packageJson); - const closed = Bun.listen({ hostname: "localhost", port: 0, socket: { data() {} } }); - const port = closed.port; - closed.stop(true); + const requests: string[] = []; + using server = Bun.serve({ + port: 0, + fetch(req) { + requests.push(new URL(req.url).pathname); + return new Response("registry must not be contacted", { status: 500 }); + }, + }); await write( join(packageDir, "bunfig.toml"), Bun.TOML.stringify({ install: { cache: join(packageDir, ".bun-cache"), - registry: `http://localhost:${port}/`, + registry: `http://localhost:${server.port}/`, minimumReleaseAge: 60 * 60 * 24 * 365 * 100, }, }), @@ -905,4 +1429,5 @@ test.concurrent("does not contact the registry", async () => { expect(recheck.stdout).toContain("Already deduplicated."); expect(recheck.stderr).not.toContain("error:"); expect(recheck.exitCode).toBe(0); + expect(requests).toEqual([]); }); diff --git a/test/cli/install/bun-install-registry.test.ts b/test/cli/install/bun-install-registry.test.ts index bf2495932341..83ef039204c4 100644 --- a/test/cli/install/bun-install-registry.test.ts +++ b/test/cli/install/bun-install-registry.test.ts @@ -5167,14 +5167,14 @@ describe("update", () => { }, }); - // Update with `a-dep` and `--latest`, `latest` should be replaced with the installed version + // a dist-tag literal is kept in every mode; only bun.lock follows the tag await runBunUpdate(env, packageDir, ["a-dep"]); assertManifestsPopulated(join(packageDir, ".bun-cache"), registryUrl()); expect(await file(packageJson).json()).toEqual({ name: "foo", dependencies: { - "a-dep": "^1.0.10", + "a-dep": "latest", }, }); await runBunUpdate(env, packageDir, ["--latest"]); @@ -5183,9 +5183,10 @@ describe("update", () => { expect(await file(packageJson).json()).toEqual({ name: "foo", dependencies: { - "a-dep": "^1.0.10", + "a-dep": "latest", }, }); + expect((await file(join(packageDir, "node_modules", "a-dep", "package.json")).json()).version).toBe("1.0.10"); }); test("exact versions stay exact", async () => { const runs = [ @@ -5949,11 +5950,23 @@ describe("update", () => { version: "1.0.0", }); - // updating a package the workspace does not declare re-resolves the root's pin in place and never adds it to the workspace's package.json - ({ out } = await runBunUpdate(env, join(packageDir, "packages", "pkg1"), ["no-deps"])); - assertManifestsPopulated(join(packageDir, ".bun-cache"), registryUrl()); - - expect(out[0]).toContain("bun update v1."); + // a name only the root declares is out of scope inside pkg1: error, nothing written + { + await using proc = spawn({ + cmd: [bunExe(), "update", "no-deps"], + cwd: join(packageDir, "packages", "pkg1"), + stdout: "pipe", + stderr: "pipe", + env, + }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + expect(stderr).toContain( + 'error: "no-deps" is only a dependency of other workspaces, so there is nothing to update here', + ); + expect(stderr).toContain("bun update -r no-deps"); + expect(stdout).not.toContain("installed no-deps"); + expect(exitCode).toBe(1); + } expect(await file(join(packageDir, "node_modules", "no-deps", "package.json")).json()).toMatchObject({ version: "1.0.0", }); @@ -8592,6 +8605,62 @@ describe("outdated", () => { expect(out).toContain("pkg1"); }); + test("--filter with relation selectors lists only the selected workspaces' dependencies", async () => { + await Promise.all([ + write( + packageJson, + JSON.stringify({ + name: "root", + workspaces: ["packages/*"], + dependencies: { app: "workspace:*", "left-pad": "1.0.0" }, + }), + ), + write( + join(packageDir, "packages", "app", "package.json"), + JSON.stringify({ name: "app", version: "1.0.0", dependencies: { lib: "workspace:*", "is-number": "1.0.0" } }), + ), + write( + join(packageDir, "packages", "lib", "package.json"), + JSON.stringify({ name: "lib", version: "1.0.0", dependencies: { util: "workspace:*", "no-deps": "1.0.0" } }), + ), + write( + join(packageDir, "packages", "util", "package.json"), + JSON.stringify({ name: "util", version: "1.0.0", dependencies: { "a-dep": "1.0.1" } }), + ), + write( + join(packageDir, "packages", "tool", "package.json"), + JSON.stringify({ + name: "tool", + version: "1.0.0", + devDependencies: { lib: "1.0.0" }, + dependencies: { "no-deps-bins": "1.0.0" }, + }), + ), + write( + join(packageDir, "packages", "lone", "package.json"), + JSON.stringify({ name: "lone", version: "1.0.0", dependencies: { "peer-no-deps": "1.0.0" } }), + ), + ]); + await runBunInstall(env, packageDir); + + // app... = app + lib + util + let out = await runBunOutdated(env, packageDir, "--filter", "app..."); + expect(out).toContain("Workspace"); + expect(out).toContain("is-number"); + expect(out).toMatch(/\bno-deps\s/); + expect(out).toContain("a-dep"); + expect(out).not.toContain("no-deps-bins"); + expect(out).not.toContain("peer-no-deps"); + + // ...^util = root + lib + app + tool, minus app + out = await runBunOutdated(env, packageDir, "--filter", "...^util", "--filter", "!app"); + expect(out).toContain("no-deps-bins"); + expect(out).toMatch(/\bno-deps\s/); + expect(out).not.toContain("is-number"); + expect(out).not.toContain("a-dep"); + expect(out).not.toContain("peer-no-deps"); + }); + test("dependency pattern args", async () => { await setupWorkspace(); await runBunInstall(env, packageDir); diff --git a/test/cli/install/bun-pm-licenses.test.ts b/test/cli/install/bun-pm-licenses.test.ts index 3fbcb7097fb0..112dddd023f1 100644 --- a/test/cli/install/bun-pm-licenses.test.ts +++ b/test/cli/install/bun-pm-licenses.test.ts @@ -7,6 +7,14 @@ import { pathToFileURL } from "url"; type Linker = "hoisted" | "isolated"; type Files = Record; +type LicenseEntry = { + name: string; + versions: string[]; + license: string; + homepage?: string; + author?: string; + description?: string; +}; const registry = new VerdaccioRegistry(); @@ -30,6 +38,8 @@ const gitEnv = { GIT_COMMITTER_EMAIL: "test@example.com", }; +const EMPTY_TEXT = "No packages found\n"; + const fixturePackageJson = JSON.stringify({ name: "licenses-fixture", version: "1.0.0", @@ -63,7 +73,6 @@ const monorepoFiles: Files = { }), }; -// `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. async function install(dir: string, linker: Linker, ...args: string[]) { await using proc = spawn({ cmd: [bunExe(), "install", "--linker", linker, ...args], @@ -103,12 +112,30 @@ async function licenses(dir: string, ...args: string[]): Promise<[string, string return await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); } +async function licensesText(dir: string, ...args: string[]) { + const [stdout, stderr, exitCode] = await licenses(dir, ...args); + expect(stderr).toBe(""); + expect(exitCode).toBe(0); + return stdout; +} + async function licensesJson(dir: string, ...args: string[]) { const [stdout, stderr, exitCode] = await licenses(dir, ...args, "--json"); expect(stderr).toBe(""); const parsed = JSON.parse(stdout); expect(exitCode).toBe(0); - return parsed as Record; + return parsed as Record; +} + +async function pm(dir: string, ...args: string[]): Promise<[string, string, number]> { + await using proc = spawn({ + cmd: [bunExe(), "pm", ...args], + env: bunEnv, + cwd: dir, + stdout: "pipe", + stderr: "pipe", + }); + return await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); } function names(parsed: Record) { @@ -118,6 +145,8 @@ function names(parsed: Record) { .sort(); } +const u = (name: string, ...versions: string[]): LicenseEntry => ({ name, versions, license: "Unknown" }); + // On Linux/Windows installed files are hardlinks into the cache; unlink first so the cache copy is left alone. function overwriteInstalledManifest(dir: string, pkg: string, contents: string) { const path = join(dir, "node_modules", pkg, "package.json"); @@ -132,43 +161,49 @@ function patchInstalledManifest(dir: string, pkg: string, fields: Record", + description: pathParseDescription, }, { name: "resolve", versions: ["1.9.0"], + license: "MIT", author: "James Halliday (http://substack.net)", + description: resolveDescription, }, ], - Unknown: [ - { name: "a-dep", versions: ["1.0.1"] }, - { name: "no-deps", versions: ["1.0.0", "1.0.1"] }, - { name: "one-dep", versions: ["1.0.0"] }, - ], + Unknown: [u("a-dep", "1.0.1"), u("no-deps", "1.0.0", "1.0.1"), u("one-dep", "1.0.0")], }; const prodJson = { MIT: fullJson.MIT, - Unknown: [ - { name: "no-deps", versions: ["1.0.0", "1.0.1"] }, - { name: "one-dep", versions: ["1.0.0"] }, - ], + Unknown: [u("no-deps", "1.0.0", "1.0.1"), u("one-dep", "1.0.0")], }; +const monoJson = { MIT: fullJson.MIT, Unknown: [u("a-dep", "1.0.1"), u("no-deps", "1.0.0")] }; +const fooJson = { Unknown: [u("a-dep", "1.0.1"), u("no-deps", "1.0.0")] }; +const barJson = { MIT: fullJson.MIT }; +const monoNames = ["a-dep", "no-deps", "path-parse", "resolve"]; + describe("bun pm licenses", () => { let hoistedDir: string; + let monoDir: string; beforeAll(async () => { - hoistedDir = await setup(); + [hoistedDir, monoDir] = await Promise.all([setup(), setup("hoisted", monorepoFiles)]); }); - test.concurrent("text output groups packages by license, Unknown last", async () => { + test.concurrent("text output groups packages by license, Unknown last, dev-only packages marked", async () => { const [stdout, stderr, exitCode] = await licenses(hoistedDir); expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` "MIT (2) @@ -176,22 +211,30 @@ describe("bun pm licenses", () => { └── resolve@1.9.0 Unknown (4) - ├── a-dep@1.0.1 + ├── a-dep@1.0.1 (dev) ├── no-deps@1.0.0 ├── no-deps@1.0.1 └── one-dep@1.0.0" `); - expect(stderr).not.toContain("error:"); + expect(stdout.split("\n").filter(line => line.endsWith(" (dev)"))).toEqual(["├── a-dep@1.0.1 (dev)"]); + expect(stderr).toBe(""); expect(exitCode).toBe(0); }); test.concurrent("--json shape", async () => { const [stdout, stderr, exitCode] = await licenses(hoistedDir, "--json"); + expect(stderr).toBe(""); const parsed = JSON.parse(stdout); expect(parsed).toEqual(fullJson); expect(Object.keys(parsed)).toEqual(["MIT", "Unknown"]); + expect(parsed.MIT[0].description).toBe(pathParseDescription); expect(parsed.MIT[1]).not.toHaveProperty("homepage"); - expect(stderr).not.toContain("error:"); + expect(parsed.Unknown[0]).not.toHaveProperty("description"); + for (const [key, entries] of Object.entries(parsed as Record)) { + expect(entries.map(entry => entry.license)).toEqual(entries.map(() => key)); + } + expect(stdout).not.toContain("(dev)"); + expect(stdout).not.toContain('"dev"'); expect(exitCode).toBe(0); }); @@ -203,10 +246,12 @@ describe("bun pm licenses", () => { const parsed = await licensesJson(dir); expect(Object.keys(parsed)).toEqual(["(MIT OR Apache-2.0)", "BSD-3-Clause", "ISC", "MIT", "Unknown"]); - expect(parsed["ISC"]).toEqual([{ name: "no-deps", versions: ["1.0.0"] }]); - expect(parsed["Unknown"]).toEqual([{ name: "no-deps", versions: ["1.0.1"] }]); - expect(parsed["(MIT OR Apache-2.0)"]).toEqual([{ name: "one-dep", versions: ["1.0.0"] }]); - expect(parsed["BSD-3-Clause"]).toEqual([{ name: "a-dep", versions: ["1.0.1"] }]); + expect(parsed["ISC"]).toEqual([{ name: "no-deps", versions: ["1.0.0"], license: "ISC" }]); + expect(parsed["Unknown"]).toEqual([u("no-deps", "1.0.1")]); + expect(parsed["(MIT OR Apache-2.0)"]).toEqual([ + { name: "one-dep", versions: ["1.0.0"], license: "(MIT OR Apache-2.0)" }, + ]); + expect(parsed["BSD-3-Clause"]).toEqual([{ name: "a-dep", versions: ["1.0.1"], license: "BSD-3-Clause" }]); expect(parsed["MIT"].map(entry => entry.name)).toEqual(["path-parse", "resolve"]); }); @@ -219,10 +264,12 @@ describe("bun pm licenses", () => { patchInstalledManifest(dir, "one-dep/node_modules/no-deps", { license: { name: "0BSD" } }); const parsed = await licensesJson(dir); - expect(parsed["(MIT OR Apache-2.0)"]).toEqual([{ name: "a-dep", versions: ["1.0.1"] }]); - expect(parsed["ISC"]).toEqual([{ name: "one-dep", versions: ["1.0.0"] }]); - expect(parsed["BSD-2-Clause"]).toEqual([{ name: "no-deps", versions: ["1.0.0"] }]); - expect(parsed["0BSD"]).toEqual([{ name: "no-deps", versions: ["1.0.1"] }]); + expect(parsed["(MIT OR Apache-2.0)"]).toEqual([ + { name: "a-dep", versions: ["1.0.1"], license: "(MIT OR Apache-2.0)" }, + ]); + expect(parsed["ISC"]).toEqual([{ name: "one-dep", versions: ["1.0.0"], license: "ISC" }]); + expect(parsed["BSD-2-Clause"]).toEqual([{ name: "no-deps", versions: ["1.0.0"], license: "BSD-2-Clause" }]); + expect(parsed["0BSD"]).toEqual([{ name: "no-deps", versions: ["1.0.1"], license: "0BSD" }]); expect(parsed).not.toHaveProperty("Unknown"); }); @@ -233,7 +280,7 @@ describe("bun pm licenses", () => { const parsed = await licensesJson(dir); expect(parsed["MIT"].map(entry => entry.name)).toEqual(["a-dep", "path-parse", "resolve"]); - expect(parsed["Apache-2.0"]).toEqual([{ name: "one-dep", versions: ["1.0.0"] }]); + expect(parsed["Apache-2.0"]).toEqual([{ name: "one-dep", versions: ["1.0.0"], license: "Apache-2.0" }]); }); test.concurrent("non-string license shapes are Unknown; entries with non-string type are skipped", async () => { @@ -245,7 +292,16 @@ describe("bun pm licenses", () => { const parsed = await licensesJson(dir); expect(parsed).toEqual({ - MIT: [fullJson.MIT[0], { name: "resolve", versions: ["1.9.0"], author: fullJson.MIT[1].author }], + MIT: [ + fullJson.MIT[0], + { + name: "resolve", + versions: ["1.9.0"], + license: "MIT", + author: fullJson.MIT[1].author, + description: resolveDescription, + }, + ], Unknown: fullJson.Unknown, }); }); @@ -255,7 +311,9 @@ describe("bun pm licenses", () => { patchInstalledManifest(dir, "one-dep", { licenses: [{ type: "MIT" }, { type: "MIT" }, { type: "Apache-2.0" }] }); const parsed = await licensesJson(dir); - expect(parsed["(MIT OR MIT OR Apache-2.0)"]).toEqual([{ name: "one-dep", versions: ["1.0.0"] }]); + expect(parsed["(MIT OR MIT OR Apache-2.0)"]).toEqual([ + { name: "one-dep", versions: ["1.0.0"], license: "(MIT OR MIT OR Apache-2.0)" }, + ]); }); test.concurrent("one package with two versions: per-license grouping, metadata from the newest version", async () => { @@ -268,13 +326,25 @@ describe("bun pm licenses", () => { const parsed = await licensesJson(dir); expect(parsed["MIT"]).toEqual([ - { name: "no-deps", versions: ["1.0.0", "1.0.1"], homepage: "https://example.com/new" }, + { name: "no-deps", versions: ["1.0.0", "1.0.1"], license: "MIT", homepage: "https://example.com/new" }, ...fullJson.MIT, ]); - expect(parsed["Unknown"]).toEqual([ - { name: "a-dep", versions: ["1.0.1"] }, - { name: "one-dep", versions: ["1.0.0"] }, - ]); + expect(parsed["Unknown"]).toEqual([u("a-dep", "1.0.1"), u("one-dep", "1.0.0")]); + }); + + test.concurrent("--json `license` follows each version's group; an empty description is omitted", async () => { + const dir = await setup(); + patchInstalledManifest(dir, "no-deps", { license: "ISC", description: "" }); + patchInstalledManifest(dir, "one-dep/node_modules/no-deps", { license: "0BSD", description: "newer" }); + patchInstalledManifest(dir, "a-dep", { licenses: [{ type: "MIT" }, { type: "Apache-2.0" }] }); + + expect(await licensesJson(dir)).toEqual({ + "(MIT OR Apache-2.0)": [{ name: "a-dep", versions: ["1.0.1"], license: "(MIT OR Apache-2.0)" }], + "0BSD": [{ name: "no-deps", versions: ["1.0.1"], license: "0BSD", description: "newer" }], + "ISC": [{ name: "no-deps", versions: ["1.0.0"], license: "ISC" }], + "MIT": fullJson.MIT, + "Unknown": [u("one-dep", "1.0.0")], + }); }); test.concurrent("versions are ordered by semver, names bytewise", async () => { @@ -294,13 +364,116 @@ describe("bun pm licenses", () => { expect(exitCode).toBe(0); const parsed = await licensesJson(dir); - expect(parsed["Unknown"][0]).toEqual({ name: "a-dep", versions: ["1.0.9", "1.0.10"] }); + expect(parsed["Unknown"][0]).toEqual(u("a-dep", "1.0.9", "1.0.10")); + }); + + test.concurrent("(dev) marks packages only reachable through devDependencies", async () => { + const dir = await setup("hoisted", { + "package.json": pkg({ dependencies: { "a-dep": "1.0.9" }, devDependencies: { "uses-a-dep-9": "1.0.0" } }), + }); + + const [stdout, stderr, exitCode] = await licenses(dir); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "Unknown (2) + ├── a-dep@1.0.9 + └── uses-a-dep-9@1.0.0 (dev)" + `); + expect(stderr).toBe(""); + expect(exitCode).toBe(0); + + expect(await licensesText(dir, "--prod")).not.toContain("(dev)"); + + const [json, jsonStderr, jsonExit] = await licenses(dir, "--json"); + expect(jsonStderr).toBe(""); + expect(json).not.toContain("(dev)"); + expect(json).not.toContain('"dev"'); + expect(JSON.parse(json)).toEqual({ Unknown: [u("a-dep", "1.0.9"), u("uses-a-dep-9", "1.0.0")] }); + expect(jsonExit).toBe(0); + + expect(await licensesJson(dir, "--dev")).toEqual({ Unknown: [u("a-dep", "1.0.9"), u("uses-a-dep-9", "1.0.0")] }); + expect(normalizeBunSnapshot(await licensesText(dir, "--dev"))).toMatchInlineSnapshot(` + "Unknown (2) + ├── a-dep@1.0.9 + └── uses-a-dep-9@1.0.0 (dev)" + `); + }); + + // pnpm detect-dep-types: the marker is per name@version and reaches the transitive dependencies of a devDependency. + test.concurrent("(dev) propagates through a devDependency's subtree; --dev lists that subtree", async () => { + const dir = await setup("hoisted", { + "package.json": pkg({ + dependencies: { "no-deps": "1.0.1", "uses-a-dep-10": "1.0.0" }, + devDependencies: { "one-dep": "1.0.0", "uses-a-dep-9": "1.0.0" }, + }), + }); + + expect(normalizeBunSnapshot(await licensesText(dir))).toMatchInlineSnapshot(` + "Unknown (6) + ├── a-dep@1.0.9 (dev) + ├── a-dep@1.0.10 + ├── no-deps@1.0.1 + ├── one-dep@1.0.0 (dev) + ├── uses-a-dep-10@1.0.0 + └── uses-a-dep-9@1.0.0 (dev)" + `); + + expect(normalizeBunSnapshot(await licensesText(dir, "--dev"))).toMatchInlineSnapshot(` + "Unknown (4) + ├── a-dep@1.0.9 (dev) + ├── no-deps@1.0.1 + ├── one-dep@1.0.0 (dev) + └── uses-a-dep-9@1.0.0 (dev)" + `); + expect(await licensesJson(dir, "--dev")).toEqual({ + Unknown: [u("a-dep", "1.0.9"), u("no-deps", "1.0.1"), u("one-dep", "1.0.0"), u("uses-a-dep-9", "1.0.0")], + }); + }); + + test.concurrent("--dev lists the devDependencies closure", async () => { + const dir = await setup("hoisted", { + "package.json": pkg({ dependencies: { "no-deps": "1.0.0" }, devDependencies: { "one-dep": "1.0.0" } }), + }); + + const expected = { Unknown: [u("no-deps", "1.0.1"), u("one-dep", "1.0.0")] }; + expect(await licensesJson(dir, "--dev")).toEqual(expected); + expect(await licensesJson(dir, "-D")).toEqual(expected); + expect(normalizeBunSnapshot(await licensesText(dir, "--dev"))).toMatchInlineSnapshot(` + "Unknown (2) + ├── no-deps@1.0.1 (dev) + └── one-dep@1.0.0 (dev)" + `); + + expect(normalizeBunSnapshot(await licensesText(hoistedDir, "--dev"))).toMatchInlineSnapshot(` + "Unknown (1) + └── a-dep@1.0.1 (dev)" + `); + expect(await licensesJson(hoistedDir, "--dev")).toEqual({ Unknown: [u("a-dep", "1.0.1")] }); + }); + + test.concurrent("--dev in a workspace", async () => { + expect(await licensesJson(monoDir, "--dev")).toEqual({ Unknown: [u("a-dep", "1.0.1")] }); + expect(await licensesJson(join(monoDir, "packages", "foo"), "--dev")).toEqual({ Unknown: [u("a-dep", "1.0.1")] }); + + const bar = join(monoDir, "packages", "bar"); + expect(await licensesText(bar, "--dev")).toBe(EMPTY_TEXT); + expect(await licensesJson(bar, "--dev")).toEqual({}); + }); + + test.concurrent("--dev cannot be combined with --prod", async () => { + for (const args of [ + ["--dev", "--prod"], + ["--dev", "--omit=dev"], + ["--prod", "--dev", "--json"], + ]) { + const [stdout, stderr, exitCode] = await licenses(hoistedDir, ...args); + expect(stdout).toBe(""); + expect(stderr).toContain("error: --dev cannot be combined with --prod or --omit=dev"); + expect(exitCode).toBe(1); + } }); test.concurrent.each(["--prod", "--production", "-p", "-P"])("%s omits devDependencies (--json)", async flag => { - const parsed = await licensesJson(hoistedDir, flag); - expect(JSON.stringify(parsed)).not.toContain("a-dep"); - expect(parsed).toEqual(prodJson); + expect(await licensesJson(hoistedDir, flag)).toEqual(prodJson); }); test.concurrent("--prod omits devDependencies (text)", async () => { @@ -315,7 +488,7 @@ describe("bun pm licenses", () => { ├── no-deps@1.0.1 └── one-dep@1.0.0" `); - expect(stderr).not.toContain("error:"); + expect(stderr).toBe(""); expect(exitCode).toBe(0); }); @@ -329,10 +502,7 @@ describe("bun pm licenses", () => { }); expect(await licensesJson(dir, "--prod")).toEqual({ - Unknown: [ - { name: "no-deps", versions: ["1.0.0", "1.0.1"] }, - { name: "one-dep", versions: ["1.0.0"] }, - ], + Unknown: [u("no-deps", "1.0.0", "1.0.1"), u("one-dep", "1.0.0")], }); expect(names(await licensesJson(dir))).toEqual(["a-dep", "no-deps", "one-dep"]); }); @@ -347,19 +517,139 @@ describe("bun pm licenses", () => { "native-libc-musl", ] .filter(name => existsSync(join(dir, "node_modules", name, "package.json"))) - .map(name => ({ name, versions: ["1.0.0"] })); + .map(name => u(name, "1.0.0")); expect(installedNatives.map(entry => entry.name)).not.toContain("native-foo-x64"); - const [stdout, stderr, exitCode] = await licenses(dir, "--json"); - expect(JSON.parse(stdout)).toEqual({ - Unknown: [...installedNatives, { name: "optional-native", versions: ["1.0.0"] }], + expect(await licensesJson(dir)).toEqual({ + Unknown: [...installedNatives, u("optional-native", "1.0.0")], }); + }); + + test.concurrent("--long prints author, description and homepage under each entry", async () => { + const [stdout, stderr, exitCode] = await licenses(hoistedDir, "--long"); + expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(` + "MIT (2) + ├── path-parse@1.0.6 + │ Javier Blanco + │ Node.js path.parse() ponyfill + │ https://github.com/jbgutierrez/path-parse#readme + └── resolve@1.9.0 + James Halliday (http://substack.net) + resolve like require.resolve() on behalf of files asynchronously and synchronously + + Unknown (4) + ├── a-dep@1.0.1 (dev) + ├── no-deps@1.0.0 + ├── no-deps@1.0.1 + └── one-dep@1.0.0" + `); expect(stderr).toBe(""); expect(exitCode).toBe(0); + + expect(await licensesText(hoistedDir, "ls", "--long")).toBe(stdout); + + const [plainJson, longJson] = await Promise.all([ + licensesText(hoistedDir, "--json"), + licensesText(hoistedDir, "--long", "--json"), + ]); + expect(longJson).toBe(plainJson); + expect(JSON.parse(longJson)).toEqual(fullJson); + }); + + test.concurrent("--long details are per version in text; --json takes them from the newest version", async () => { + const dir = await setup(); + patchInstalledManifest(dir, "no-deps", { description: "only a description" }); + patchInstalledManifest(dir, "a-dep", { author: { name: "Ann", email: "ann@example.com" } }); + + const first = await licensesText(dir, "--long"); + expect(first).toContain( + "├── a-dep@1.0.1 (dev)\n│ Ann \n├── no-deps@1.0.0\n│ only a description\n├── no-deps@1.0.1\n└── one-dep@1.0.0\n", + ); + expect(await licensesJson(dir)).toEqual({ + MIT: fullJson.MIT, + Unknown: [ + { ...u("a-dep", "1.0.1"), author: "Ann " }, + u("no-deps", "1.0.0", "1.0.1"), + u("one-dep", "1.0.0"), + ], + }); + + patchInstalledManifest(dir, "one-dep/node_modules/no-deps", { + description: "newest wins\nline two", + homepage: "https://example.com/new", + }); + + const second = await licensesText(dir, "--long"); + expect(second).toContain( + "├── no-deps@1.0.0\n│ only a description\n├── no-deps@1.0.1\n│ newest winsline two\n│ https://example.com/new\n└── one-dep@1.0.0\n", + ); + expect(second.split("\n").some(line => line.startsWith("line two"))).toBeFalse(); + const parsed = await licensesJson(dir); + expect(parsed.Unknown[1]).toEqual({ + ...u("no-deps", "1.0.0", "1.0.1"), + homepage: "https://example.com/new", + description: "newest wins\nline two", + }); + expect(JSON.stringify(parsed)).not.toContain("only a description"); }); - test.concurrent("isolated linker", async () => { - const dir = await setup("isolated"); + test.concurrent( + "control characters from package.json are stripped in text output but preserved in --json", + async () => { + const dir = await setup(); + const evilLicense = "MIT\u001b[31m\nEVIL"; + patchInstalledManifest(dir, "a-dep", { license: evilLicense, description: "tab\there\r\n" }); + patchInstalledManifest(dir, "one-dep", { license: "ISC\nGPL-3.0" }); + patchInstalledManifest(dir, "no-deps", { license: "BSD\t2" }); + + const stdout = await licensesText(dir, "--long"); + expect(stdout).not.toContain("\u001b"); + expect(stdout).not.toContain("\r"); + expect(stdout).not.toContain("\t"); + expect(stdout).toContain("MIT[31mEVIL (1)\n└── a-dep@1.0.1 (dev)\n tabhere\n"); + expect(stdout).toContain("ISCGPL-3.0 (1)\n└── one-dep@1.0.0\n"); + expect(stdout).toContain("BSD2 (1)\n└── no-deps@1.0.0\n"); + expect(stdout.split("\n").filter(line => / \(\d+\)$/.test(line))).toEqual([ + "BSD2 (1)", + "ISCGPL-3.0 (1)", + "MIT (2)", + "MIT[31mEVIL (1)", + "Unknown (1)", + ]); + expect(stdout.split("\n").some(line => line.startsWith("GPL-3.0") || line.startsWith("EVIL"))).toBeFalse(); + + const parsed = await licensesJson(dir); + expect(Object.keys(parsed)).toEqual(["BSD\t2", "ISC\nGPL-3.0", "MIT", evilLicense, "Unknown"]); + expect(parsed[evilLicense]).toEqual([ + { name: "a-dep", versions: ["1.0.1"], license: evilLicense, description: "tab\there\r\n" }, + ]); + expect(parsed["ISC\nGPL-3.0"]).toEqual([{ name: "one-dep", versions: ["1.0.0"], license: "ISC\nGPL-3.0" }]); + expect(parsed["BSD\t2"]).toEqual([{ name: "no-deps", versions: ["1.0.0"], license: "BSD\t2" }]); + expect(parsed.Unknown).toEqual([u("no-deps", "1.0.1")]); + }, + ); + + test.concurrent("--long strips control characters from author, description and homepage", async () => { + const dir = await setup(); + const author = "Eve\u001b]8;;https://evil.example\u0007click\u001b]8;;\u0007"; + const description = "first\r\nsecond"; + const homepage = "https://example.com/\u001b[2Jx"; + patchInstalledManifest(dir, "a-dep", { author, description, homepage }); + + const stdout = await licensesText(dir, "--long"); + expect(stdout).not.toContain("\u001b"); + expect(stdout).not.toContain("\u0007"); + expect(stdout).not.toContain("\r"); + expect(stdout).toContain( + "├── a-dep@1.0.1 (dev)\n│ Eve]8;;https://evil.exampleclick]8;;\n│ firstsecond\n│ https://example.com/[2Jx\n├── no-deps@1.0.0\n", + ); + expect(stdout.split("\n").some(line => line.startsWith("second"))).toBeFalse(); + + expect((await licensesJson(dir)).Unknown[0]).toEqual({ ...u("a-dep", "1.0.1"), author, description, homepage }); + }); + + test.concurrent("isolated linker matches hoisted: marker, --dev, --long and --filter", async () => { + const [dir, isoMono] = await Promise.all([setup("isolated"), setup("isolated", monorepoFiles)]); const [expected] = await licenses(hoistedDir); const [stdout, stderr, exitCode] = await licenses(dir); expect(stdout).toBe(expected); @@ -369,13 +659,22 @@ describe("bun pm licenses", () => { └── resolve@1.9.0 Unknown (4) - ├── a-dep@1.0.1 + ├── a-dep@1.0.1 (dev) ├── no-deps@1.0.0 ├── no-deps@1.0.1 └── one-dep@1.0.0" `); - expect(stderr).not.toContain("error:"); + expect(stderr).toBe(""); expect(exitCode).toBe(0); + + for (const args of [["--long"], ["--dev"], ["--dev", "--json"], ["--long", "--json"]]) { + expect(await licensesText(dir, ...args)).toBe(await licensesText(hoistedDir, ...args)); + } + expect(await licensesText(dir, "--long")).toContain("│ Javier Blanco \n"); + expect(await licensesText(isoMono, "--filter", "foo", "--json")).toBe( + await licensesText(monoDir, "--filter", "foo", "--json"), + ); + expect(await licensesText(isoMono, "--filter", "foo")).toContain("├── a-dep@1.0.1 (dev)\n└── no-deps@1.0.0\n"); }); test.concurrent("isolated linker: scoped transitive dependency is found through the store", async () => { @@ -384,11 +683,7 @@ describe("bun pm licenses", () => { expect(existsSync(join(dir, "node_modules", ".bun", "@types+is-number@2.0.0"))).toBeTrue(); expect(await licensesJson(dir)).toEqual({ - Unknown: [ - { name: "@types/is-number", versions: ["2.0.0"] }, - { name: "no-deps", versions: ["1.1.0"] }, - { name: "two-range-deps", versions: ["1.0.0"] }, - ], + Unknown: [u("@types/is-number", "2.0.0"), u("no-deps", "1.1.0"), u("two-range-deps", "1.0.0")], }); }); @@ -401,10 +696,10 @@ describe("bun pm licenses", () => { expect(await licensesJson(dir)).toEqual({ Unknown: [ - { name: "no-deps", versions: ["1.0.0"] }, - { name: "peer-deps-lvl0", versions: ["1.0.0"] }, - { name: "peer-deps-lvl1", versions: ["1.0.0"] }, - { name: "peer-deps-lvl2", versions: ["1.0.0"] }, + u("no-deps", "1.0.0"), + u("peer-deps-lvl0", "1.0.0"), + u("peer-deps-lvl1", "1.0.0"), + u("peer-deps-lvl2", "1.0.0"), ], }); }); @@ -417,10 +712,7 @@ describe("bun pm licenses", () => { }); expect(await licensesJson(dir)).toEqual({ - Unknown: [ - { name: "no-deps", versions: ["1.0.0"] }, - { name: "sub-dep", versions: ["sub-dep"] }, - ], + Unknown: [u("no-deps", "1.0.0"), u("sub-dep", "sub-dep")], }); }); @@ -465,8 +757,9 @@ describe("bun pm licenses", () => { stdout: "pipe", stderr: "pipe", }); - const [stdout, exitCode] = await Promise.all([proc.stdout.text(), proc.exited]); - expect(JSON.parse(stdout)).toEqual({ Unknown: [{ name: "no-deps", versions: ["1.0.0"] }] }); + const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + expect(stderr).toBe(""); + expect(JSON.parse(stdout)).toEqual({ Unknown: [u("no-deps", "1.0.0")] }); expect(exitCode).toBe(0); }); @@ -478,43 +771,139 @@ describe("bun pm licenses", () => { "package.json": pkg({ dependencies: { "no-deps": "1.0.0", "nd2": "npm:no-deps@1.0.1" } }), }); - expect(await licensesJson(dir)).toEqual({ Unknown: [{ name: "no-deps", versions: ["1.0.0", "1.0.1"] }] }); + expect(await licensesJson(dir)).toEqual({ Unknown: [u("no-deps", "1.0.0", "1.0.1")] }); }, ); // pnpm 'path should be correct for workspaces' / 'filter outputs'; pnpm#5689 (same output from every directory of a monorepo). test.concurrent("workspace root lists every member's dependencies; a member lists only its own closure", async () => { - const dir = await setup("hoisted", monorepoFiles); + expect(await licensesJson(monoDir)).toEqual(monoJson); + expect(await licensesJson(join(monoDir, "packages", "bar"))).toEqual(barJson); + expect(await licensesJson(join(monoDir, "packages", "foo"))).toEqual(fooJson); + expect(await licensesText(monoDir)).toContain("├── a-dep@1.0.1 (dev)\n└── no-deps@1.0.0\n"); + }); - const fromRoot = await licensesJson(dir); - expect(names(fromRoot)).toEqual(["a-dep", "no-deps", "path-parse", "resolve"]); - expect(await licensesJson(join(dir, "packages", "bar"))).toEqual({ MIT: fullJson.MIT }); - expect(await licensesJson(join(dir, "packages", "foo"))).toEqual({ - Unknown: [ - { name: "a-dep", versions: ["1.0.1"] }, - { name: "no-deps", versions: ["1.0.0"] }, - ], + test.concurrent("(dev) from the root is unmarked when another member needs the package in production", async () => { + const dir = await setup("hoisted", { + ...monorepoFiles, + "packages/foo/package.json": JSON.stringify({ + name: "foo", + version: "1.0.0", + dependencies: { "no-deps": "1.0.0" }, + devDependencies: { "a-dep": "1.0.1", "resolve": "1.9.0" }, + }), }); + + expect(normalizeBunSnapshot(await licensesText(dir))).toMatchInlineSnapshot(` + "MIT (2) + ├── path-parse@1.0.6 + └── resolve@1.9.0 + + Unknown (2) + ├── a-dep@1.0.1 (dev) + └── no-deps@1.0.0" + `); + expect(await licensesText(join(dir, "packages", "foo"))).toContain("├── a-dep@1.0.1 (dev)\n└── no-deps@1.0.0\n"); }); test.concurrent("--prod inside a workspace drops members' devDependencies", async () => { - const dir = await setup("hoisted", monorepoFiles); + expect(await licensesJson(monoDir, "--prod")).toEqual({ MIT: fullJson.MIT, Unknown: [u("no-deps", "1.0.0")] }); + expect(await licensesJson(join(monoDir, "packages", "foo"), "--prod")).toEqual({ + Unknown: [u("no-deps", "1.0.0")], + }); + }); - expect(names(await licensesJson(dir, "--prod"))).toEqual(["no-deps", "path-parse", "resolve"]); - expect(await licensesJson(join(dir, "packages", "foo"), "--prod")).toEqual({ - Unknown: [{ name: "no-deps", versions: ["1.0.0"] }], + test.concurrent("--filter selects workspaces from any directory", async () => { + const bar = join(monoDir, "packages", "bar"); + const [union, star, all, notFoo, starNotFoo, glob, parentGlob, packagesGlob, rootOnly, fooText] = await Promise.all( + [ + licensesJson(monoDir, "-F", "foo", "-F", "bar"), + licensesJson(monoDir, "--filter", "*"), + licensesJson(monoDir), + licensesJson(monoDir, "--filter", "!foo"), + licensesJson(monoDir, "--filter", "*", "--filter", "!foo"), + licensesJson(monoDir, "--filter", "b*"), + licensesJson(bar, "--filter", "../*"), + licensesJson(monoDir, "--filter", "./packages/*"), + licensesText(monoDir, "--filter", "mono"), + licensesText(monoDir, "--filter", "foo"), + ], + ); + expect(await licensesJson(monoDir, "--filter", "foo")).toEqual(fooJson); + expect(await licensesJson(monoDir, "--filter", "bar")).toEqual(barJson); + expect(await licensesJson(monoDir, "--filter", "./packages/bar")).toEqual(barJson); + expect(await licensesJson(bar, "--filter", "foo")).toEqual(fooJson); + expect(await licensesJson(bar, "--filter", "./")).toEqual(barJson); + expect(union).toEqual(monoJson); + expect(names(union)).toEqual(monoNames); + expect(star).toEqual(all); + expect(all).toEqual(monoJson); + expect(notFoo).toEqual(barJson); + expect(starNotFoo).toEqual(barJson); + expect(glob).toEqual(barJson); + expect(parentGlob).toEqual(monoJson); + expect(packagesGlob).toEqual(monoJson); + expect(rootOnly).toBe(EMPTY_TEXT); + expect(normalizeBunSnapshot(fooText)).toMatchInlineSnapshot(` + "Unknown (2) + ├── a-dep@1.0.1 (dev) + └── no-deps@1.0.0" + `); + expect(await licensesJson(monoDir, "--filter", "foo", "--prod")).toEqual({ Unknown: [u("no-deps", "1.0.0")] }); + expect(await licensesJson(monoDir, "--filter", "foo", "--dev")).toEqual({ Unknown: [u("a-dep", "1.0.1")] }); + expect(await licensesJson(monoDir, "--filter", "bar", "--dev")).toEqual({}); + }); + + test.concurrent("--filter selecting the root lists only the root's own dependencies", async () => { + const dir = await setup("hoisted", { + ...monorepoFiles, + "package.json": JSON.stringify({ + name: "mono", + private: true, + workspaces: ["packages/*"], + dependencies: { "one-dep": "1.0.0" }, + }), }); + + expect(await licensesJson(dir, "--filter", "mono")).toEqual({ + Unknown: [u("no-deps", "1.0.1"), u("one-dep", "1.0.0")], + }); + expect(await licensesJson(dir)).toEqual({ + MIT: fullJson.MIT, + Unknown: [u("a-dep", "1.0.1"), u("no-deps", "1.0.0", "1.0.1"), u("one-dep", "1.0.0")], + }); + }); + + test.concurrent("--filter matching nothing is an error", async () => { + const [stdout, stderr, exitCode] = await licenses(monoDir, "--filter", "nope", "--json"); + expect(stdout).toBe(""); + expect(normalizeBunSnapshot(stderr)).toMatchInlineSnapshot( + `"error: No workspace packages matched the filter "nope""`, + ); + expect(exitCode).toBe(1); }); - test.concurrent("nothing to list prints nothing / {}", async () => { + test.concurrent("--filter is rejected by other pm subcommands and works outside a monorepo", async () => { + const [stdout, stderr, exitCode] = await pm(hoistedDir, "ls", "--filter", "foo"); + expect(stdout).toBe(""); + expect(stderr).toContain("--filter is only supported by `bun pm licenses`"); + expect(exitCode).toBe(1); + + expect(await licensesJson(hoistedDir, "--filter", "licenses-fixture")).toEqual(fullJson); + }); + + test.concurrent('nothing to list prints "No packages found" / {}', async () => { const dir = await setup("hoisted", { "package.json": pkg({ devDependencies: { "no-deps": "1.0.0" } }) }); const [stdout, stderr, exitCode] = await licenses(dir, "--prod"); - expect(stdout).toBe(""); + expect(stdout).toBe(EMPTY_TEXT); expect(stderr).toBe(""); expect(exitCode).toBe(0); - expect(await licensesJson(dir, "--prod")).toEqual({}); - expect(await licensesJson(dir)).toEqual({ Unknown: [{ name: "no-deps", versions: ["1.0.0"] }] }); + const [json, jsonStderr, jsonExit] = await licenses(dir, "--prod", "--json"); + expect(json).toBe("{}\n"); + expect(jsonStderr).toBe(""); + expect(jsonExit).toBe(0); + expect(await licensesJson(dir)).toEqual({ Unknown: [u("no-deps", "1.0.0")] }); }); test.concurrent("licenses list / ls aliases", async () => { @@ -544,14 +933,7 @@ describe("bun pm licenses", () => { test.concurrent("missing node_modules", async () => { const { packageDir } = await registry.createTestDir({ files: { "package.json": fixturePackageJson } }); - await using install = spawn({ - cmd: [bunExe(), "install", "--lockfile-only"], - env: installEnv(packageDir), - cwd: packageDir, - stdout: "pipe", - stderr: "pipe", - }); - expect(await install.exited).toBe(0); + await install(packageDir, "hoisted", "--lockfile-only"); const [stdout, stderr, exitCode] = await licenses(packageDir); expect(stdout).toBe(""); @@ -568,17 +950,13 @@ describe("bun pm licenses", () => { rmSync(join(dir, "node_modules", "one-dep"), { recursive: true, force: true }); const [stdout, stderr, exitCode] = await licenses(dir, "--json"); - expect(JSON.parse(stdout)).toEqual({ - MIT: [fullJson.MIT[0]], - Unknown: [ - { name: "a-dep", versions: ["1.0.1"] }, - { name: "no-deps", versions: ["1.0.0"] }, - { name: "resolve", versions: ["1.9.0"] }, - ], - }); expect(normalizeBunSnapshot(stderr)).toMatchInlineSnapshot( `"warn: omitted 2 packages from the lockfile not found in node_modules"`, ); + expect(JSON.parse(stdout)).toEqual({ + MIT: [fullJson.MIT[0]], + Unknown: [u("a-dep", "1.0.1"), u("no-deps", "1.0.0"), u("resolve", "1.9.0")], + }); expect(exitCode).toBe(0); }, ); @@ -596,15 +974,9 @@ describe("bun pm licenses", () => { expect(JSON.parse(readFileSync(join(dir, "node_modules", "a-dep", "package.json"), "utf8")).version).toBe("1.0.10"); expect(existsSync(join(dir, "node_modules", "uses-a-dep-10", "node_modules"))).toBeFalse(); - const [stdout, stderr, exitCode] = await licenses(dir, "--prod", "--json"); - expect(JSON.parse(stdout)).toEqual({ - Unknown: [ - { name: "a-dep", versions: ["1.0.10"] }, - { name: "uses-a-dep-10", versions: ["1.0.0"] }, - ], + expect(await licensesJson(dir, "--prod")).toEqual({ + Unknown: [u("a-dep", "1.0.10"), u("uses-a-dep-10", "1.0.0")], }); - expect(stderr).toBe(""); - expect(exitCode).toBe(0); }); test.concurrent("pnpm#8589: a different version at the tree path is not misattributed", async () => { @@ -612,13 +984,13 @@ describe("bun pm licenses", () => { patchInstalledManifest(dir, "a-dep", { license: "MIT" }); const [stdout, stderr, exitCode] = await licenses(dir, "--json"); - expect(JSON.parse(stdout)).toEqual({ - MIT: [{ name: "a-dep", versions: ["1.0.10"] }], - Unknown: [{ name: "uses-a-dep-10", versions: ["1.0.0"] }], - }); expect(normalizeBunSnapshot(stderr)).toMatchInlineSnapshot( `"warn: omitted 2 packages from the lockfile not found in node_modules"`, ); + expect(JSON.parse(stdout)).toEqual({ + MIT: [{ name: "a-dep", versions: ["1.0.10"], license: "MIT" }], + Unknown: [u("uses-a-dep-10", "1.0.0")], + }); expect(exitCode).toBe(0); }); @@ -629,15 +1001,9 @@ describe("bun pm licenses", () => { const dir = await setup(linker, { "package.json": pkg({ dependencies: { "bundled-1": "1.0.0" } }) }); expect(existsSync(join(dir, "node_modules", "bundled-1", "node_modules", "no-deps", "package.json"))).toBeTrue(); - const [stdout, stderr, exitCode] = await licenses(dir, "--json"); - expect(JSON.parse(stdout)).toEqual({ - Unknown: [ - { name: "bundled-1", versions: ["1.0.0"] }, - { name: "no-deps", versions: ["1.0.0"] }, - ], + expect(await licensesJson(dir)).toEqual({ + Unknown: [u("bundled-1", "1.0.0"), u("no-deps", "1.0.0")], }); - expect(stderr).toBe(""); - expect(exitCode).toBe(0); }, ); @@ -662,25 +1028,25 @@ describe("bun pm licenses", () => { "package.json": pkg({ dependencies: { "no-deps": "1.0.0", "git-pkg": `git+${pathToFileURL(repo)}` } }), }); - const [stdout, stderr, exitCode] = await licenses(dir, "--json"); - const parsed = JSON.parse(stdout); + const parsed = await licensesJson(dir); expect(Object.keys(parsed)).toEqual(["ISC", "Unknown"]); - expect(parsed.ISC).toEqual([{ name: "git-pkg", versions: [expect.stringContaining("git+file://")] }]); - expect(parsed.Unknown).toEqual([{ name: "no-deps", versions: ["1.0.0"] }]); - expect(stderr).toBe(""); - expect(exitCode).toBe(0); + expect(parsed.ISC).toEqual([ + { name: "git-pkg", versions: [expect.stringContaining("git+file://")], license: "ISC" }, + ]); + expect(parsed.Unknown).toEqual([u("no-deps", "1.0.0")]); }); - test.concurrent("bun pm help lists licenses", async () => { - await using proc = spawn({ - cmd: [bunExe(), "pm"], - env: bunEnv, - cwd: hoistedDir, - stdout: "pipe", - stderr: "pipe", - }); - const [stdout, exitCode] = await Promise.all([proc.stdout.text(), proc.exited]); - expect(stdout).toContain("bun pm licenses"); - expect(exitCode).toBe(0); + test.concurrent("bun pm help lists licenses and its flags", async () => { + for (const args of [[], ["--help"]]) { + const [stdout, stderr, exitCode] = await pm(hoistedDir, ...args); + const start = stdout.indexOf("bun pm licenses"); + const end = stdout.indexOf("bun pm whoami"); + expect(start).toBeGreaterThan(-1); + expect(end).toBeGreaterThan(start); + const block = stdout.slice(start, end); + for (const flag of ["--json", "--prod", "--dev", "--long", "--filter"]) expect(block).toContain(flag); + expect(stderr).toBe(""); + expect(exitCode).toBe(0); + } }); }); diff --git a/test/cli/install/bun-prune.test.ts b/test/cli/install/bun-prune.test.ts index d0d95ed63c19..6af32442d580 100644 --- a/test/cli/install/bun-prune.test.ts +++ b/test/cli/install/bun-prune.test.ts @@ -31,12 +31,16 @@ const WARN = (expected: string, kept: string) => `warn: ${expected} is not the version bun.lock installs there; keeping ${kept}`; const NOTE = "note: run 'bun install' with the same flags to install the versions bun.lock expects, then run 'bun prune' again"; +const OUT_OF_SYNC = "bun.lock does not match package.json"; +const OUT_OF_SYNC_NOTE = "note: run 'bun install' first, then run 'bun prune' again"; +const linkers: Linker[] = ["hoisted", "isolated"]; -async function prune(dir: string, ...args: string[]) { +async function prune(where: string | { dir: string; cwd: string }, ...args: string[]) { + const { dir, cwd } = typeof where === "string" ? { dir: where, cwd: where } : where; await using proc = Bun.spawn({ cmd: [bunExe(), "prune", ...args], env: installEnv(dir), - cwd: dir, + cwd, stdout: "pipe", stderr: "pipe", }); @@ -106,7 +110,6 @@ async function install(dir: string, ...args: string[]) { type Linker = "hoisted" | "isolated"; -// `--linker` is passed on the command line as well: an `install-strategy` in ~/.npmrc overrides the bunfig linker. async function setupWithLinker(linker: Linker, pkgJson: Record, bunfigOpts?: BunfigOpts) { const { packageDir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker, ...bunfigOpts } }); await write(packageJson, JSON.stringify(pkgJson)); @@ -114,6 +117,38 @@ async function setupWithLinker(linker: Linker, pkgJson: Record, return packageDir; } +type Workspaces = { root?: Record; packages: Record> }; + +function writeWorkspaces(dir: string, packageJson: string, { root, packages }: Workspaces) { + return Promise.all([ + write(packageJson, JSON.stringify({ name: "root", workspaces: ["packages/*"], ...root })), + ...Object.entries(packages).map(([folder, pkg]) => + write(join(dir, "packages", folder, "package.json"), JSON.stringify({ name: folder, version: "1.0.0", ...pkg })), + ), + ]); +} + +async function setupWorkspaces(linker: Linker, workspaces: Workspaces) { + const { packageDir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker } }); + await writeWorkspaces(packageDir, packageJson, workspaces); + await install(packageDir, "--linker", linker); + return packageDir; +} + +function expectRefused({ stdout, stderr, exitCode }: Awaited>) { + expect(stderr).toContain(OUT_OF_SYNC); + expect(stderr).toContain(OUT_OF_SYNC_NOTE); + expect(out(stdout)).not.toMatch(/^- /m); + expect(stdout).not.toContain("Removed"); + expect(stdout).not.toContain("Would remove"); + expect(exitCode).toBe(1); +} + +async function expectProductionInstallIsNoop(dir: string) { + const { out: installOut } = await runBunInstall(installEnv(dir), dir, { production: true }); + expect(installOut).toContain("no changes"); +} + function linkOutside(dir: string, rel: string, contents: Record = {}) { const outside = join(dir, "outside", basename(rel)); mkdirSync(outside, { recursive: true }); @@ -218,8 +253,7 @@ test.concurrent.each([["--production"], ["--prod"], ["--omit=dev"]])( expectBinRemoved(nm, "what-bin"); expectBinInstalled(nm, "has-bin-entry"); - const { out: installOut } = await runBunInstall(installEnv(dir), dir, { production: true }); - expect(installOut).toContain("no changes"); + await expectProductionInstallIsNoop(dir); expect(await lock(dir)).toBe(lockBefore); }, ); @@ -344,11 +378,12 @@ test.concurrent("refuses to run without a lockfile", async () => { expect(help.stdout).toContain("bun prune"); expect(help.stdout).toContain("--production"); expect(help.stdout).toContain("--linker"); + expect(help.stdout).toContain("--filter"); expect(help.exitCode).toBe(0); expect(existsSync(junk)).toBeTrue(); }); -// pnpm#9796: --production never removes workspace links. +// pnpm#9796: hoisted keeps the root's workspace links under --production because they are root->workspace prod edges; the isolated per-importer dev link case is below. test.concurrent("workspaces: prunes workspace folders, keeps workspace links, runs from the root", async () => { const { packageDir: dir, packageJson } = await registry.createTestDir(); await Promise.all([ @@ -421,7 +456,8 @@ test.concurrent("isolated linker: removes unused store entries and their links", const junkReal = plant(dir, "node_modules/junk-real"); const hiddenHoist = join(store, "node_modules"); mkdirSync(hiddenHoist, { recursive: true }); - symlinkSync("../zzz@1.0.0/node_modules/zzz", join(hiddenHoist, "zzz")); + const zzz = plant(dir, "node_modules/.bun/zzz@1.0.0/node_modules/zzz"); + symlinkSync(zzz, join(hiddenHoist, "zzz"), "junction"); expect(isSymlink(join(hiddenHoist, "zzz"))).toBeTrue(); const lockBefore = await lock(dir); @@ -431,12 +467,14 @@ test.concurrent("isolated linker: removes unused store entries and their links", - node_modules/.bun/junk@1.0.0 - node_modules/.bun/no-deps@1.0.1 - node_modules/.bun/one-dep@1.0.0 + - node_modules/.bun/zzz@1.0.0 - node_modules/junk-real - Removed 4 packages" + Removed 5 packages" `); expect(exitCode).toBe(0); expect(existsSync(join(store, "junk@1.0.0"))).toBeFalse(); + expect(existsSync(join(store, "zzz@1.0.0"))).toBeFalse(); expect(existsSync(join(store, "no-deps@1.0.1"))).toBeFalse(); expect(existsSync(join(store, "one-dep@1.0.0"))).toBeFalse(); expect(existsSync(junkReal)).toBeFalse(); @@ -448,7 +486,7 @@ test.concurrent("isolated linker: removes unused store entries and their links", expect(() => lstatSync(join(hiddenHoist, "zzz"))).toThrow(); expect(await lock(dir)).toBe(lockBefore); - await runBunInstall(installEnv(dir), dir, { production: true }); + await expectProductionInstallIsNoop(dir); }); test.concurrent("isolated linker + global store: unlinks the store link, never deletes the shared entry", async () => { @@ -728,50 +766,602 @@ test.concurrent("hoisted: --production empties a workspace folder that only held expect(existsSync(nested)).toBeFalse(); expect(isSymlink(join(nm, "a"))).toBeTrue(); expect(await file(join(nm, "no-deps", "package.json")).json()).toMatchObject({ version: "2.0.0" }); - await runBunInstall(installEnv(dir), dir, { production: true }); + await expectProductionInstallIsNoop(dir); }); +const appLinksTool = { + packages: { + app: { + dependencies: { lib: "workspace:*", "no-deps": "1.0.0" }, + devDependencies: { tool: "workspace:*", "a-dep": "1.0.1" }, + }, + lib: {}, + tool: {}, + }, +}; + test.concurrent( - "isolated + workspaces: --production prunes a workspace's registry devDependency, keeps workspace links", + "isolated + workspaces: --production removes a workspace's registry devDependency and its dev-only workspace link, keeps prod links", async () => { - const { packageDir: dir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); - await Promise.all([ - write(packageJson, JSON.stringify({ name: "root", workspaces: ["packages/*"] })), - write( - join(dir, "packages", "app", "package.json"), - JSON.stringify({ - name: "app", - version: "1.0.0", - dependencies: { lib: "workspace:*", "no-deps": "1.0.0" }, - devDependencies: { tool: "workspace:*", "a-dep": "1.0.1" }, - }), - ), - write(join(dir, "packages", "lib", "package.json"), JSON.stringify({ name: "lib", version: "1.0.0" })), - write(join(dir, "packages", "tool", "package.json"), JSON.stringify({ name: "tool", version: "1.0.0" })), - ]); - await install(dir, "--linker", "isolated"); - const appNm = join(dir, "packages", "app", "node_modules"); + const dir = await setupWorkspaces("isolated", appLinksTool); + const app = join(dir, "packages", "app"); + const appNm = join(app, "node_modules"); expect(existsSync(join(dir, "node_modules", ".bun"))).toBeTrue(); expect(isSymlink(join(appNm, "a-dep"))).toBeTrue(); expect(isSymlink(join(appNm, "tool"))).toBeTrue(); - const { stdout, exitCode } = await prune(join(dir, "packages", "app"), "--production", "--linker", "isolated"); + const plain = await prune(dir, "--linker", "isolated"); + expect(out(plain.stdout)).toEndWith("Nothing to prune."); + expect(plain.exitCode).toBe(0); + + const dryRun = await prune(dir, "--production", "--dry-run", "--linker", "isolated"); + expect(out(dryRun.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/a-dep@1.0.1 + - packages/app/node_modules/tool + Would remove 2 packages" + `); + expect(dryRun.exitCode).toBe(0); + expect(isSymlink(join(appNm, "tool"))).toBeTrue(); + + const { stdout, exitCode } = await prune({ dir, cwd: app }, "--production", "--linker", "isolated"); expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/a-dep@1.0.1 + - packages/app/node_modules/tool + Removed 2 packages" + `); + expect(exitCode).toBe(0); + expect(() => lstatSync(join(appNm, "a-dep"))).toThrow(); + expect(() => lstatSync(join(appNm, "tool"))).toThrow(); + expect(existsSync(join(appNm, "no-deps", "package.json"))).toBeTrue(); + expect(existsSync(join(appNm, "lib", "package.json"))).toBeTrue(); + expect(existsSync(join(dir, "packages", "tool", "package.json"))).toBeTrue(); + + const again = await prune(dir, "--production", "--linker", "isolated"); + expect(out(again.stdout)).toEndWith("Nothing to prune."); + expect(again.exitCode).toBe(0); + await expectProductionInstallIsNoop(dir); + expect(() => lstatSync(join(appNm, "tool"))).toThrow(); + }, +); + +test.concurrent("isolated: a real directory named like a workspace is never deleted", async () => { + const dir = await setupWorkspaces("isolated", appLinksTool); + const appNm = join(dir, "packages", "app", "node_modules"); + rmSync(join(appNm, "tool")); + const planted = plant(dir, "packages/app/node_modules/tool"); + + const { stdout, exitCode } = await prune(dir, "--production", "--linker", "isolated"); + expect(out(stdout)).toMatchInlineSnapshot(` "bun prune () - node_modules/.bun/a-dep@1.0.1 Removed 1 package" `); + expect(exitCode).toBe(0); + expect(existsSync(join(planted, "package.json"))).toBeTrue(); + expect(existsSync(join(appNm, "lib", "package.json"))).toBeTrue(); +}); + +test.concurrent( + "isolated: a scoped dev-only workspace link goes away with its emptied scope dir, the prod scoped link stays", + async () => { + const scoped = (app: Record) => ({ + packages: { + app: { dependencies: { "no-deps": "1.0.0" }, ...app }, + lib: { name: "@scope/lib" }, + tool: { name: "@scope/tool" }, + }, + }); + const [mixedDir, devOnlyDir] = await Promise.all([ + setupWorkspaces( + "isolated", + scoped({ + dependencies: { "@scope/lib": "workspace:*", "no-deps": "1.0.0" }, + devDependencies: { "@scope/tool": "workspace:*" }, + }), + ), + setupWorkspaces("isolated", scoped({ devDependencies: { "@scope/tool": "workspace:*" } })), + ]); + const mixedScope = join(mixedDir, "packages", "app", "node_modules", "@scope"); + const devOnlyScope = join(devOnlyDir, "packages", "app", "node_modules", "@scope"); + expect(isSymlink(join(mixedScope, "tool"))).toBeTrue(); + expect(isSymlink(join(devOnlyScope, "tool"))).toBeTrue(); + + const mixed = await prune(mixedDir, "--production", "--linker", "isolated"); + expect(out(mixed.stdout)).toMatchInlineSnapshot(` + "bun prune () + - packages/app/node_modules/@scope/tool + Removed 1 package" + `); + expect(mixed.exitCode).toBe(0); + expect(() => lstatSync(join(mixedScope, "tool"))).toThrow(); + expect(existsSync(join(mixedScope, "lib", "package.json"))).toBeTrue(); + expect(existsSync(join(mixedDir, "packages", "tool", "package.json"))).toBeTrue(); + await expectProductionInstallIsNoop(mixedDir); + + const devOnly = await prune(devOnlyDir, "--production", "--linker", "isolated"); + expect(out(devOnly.stdout)).toMatchInlineSnapshot(` + "bun prune () + - packages/app/node_modules/@scope/tool + Removed 1 package" + `); + expect(devOnly.exitCode).toBe(0); + expect(existsSync(devOnlyScope)).toBeFalse(); + expect(existsSync(join(devOnlyDir, "packages", "app", "node_modules", "no-deps", "package.json"))).toBeTrue(); + expect(existsSync(join(devOnlyDir, "packages", "tool", "package.json"))).toBeTrue(); + await expectProductionInstallIsNoop(devOnlyDir); + }, +); + +test.concurrent( + "isolated: a workspace that is dev-only for one workspace and a prod dependency of another loses only the dev link", + async () => { + const dir = await setupWorkspaces("isolated", { + packages: { + app: { devDependencies: { tool: "workspace:*" } }, + b: { dependencies: { tool: "workspace:*", "no-deps": "1.0.0" } }, + tool: {}, + }, + }); + const appTool = join(dir, "packages", "app", "node_modules", "tool"); + const bTool = join(dir, "packages", "b", "node_modules", "tool"); + expect(isSymlink(appTool)).toBeTrue(); + expect(isSymlink(bTool)).toBeTrue(); + + const { stdout, exitCode } = await prune(dir, "--production", "--linker", "isolated"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - packages/app/node_modules/tool + Removed 1 package" + `); expect(exitCode).toBe(0); - expect(() => lstatSync(join(appNm, "a-dep"))).toThrow(); - expect(existsSync(join(appNm, "no-deps", "package.json"))).toBeTrue(); - expect(existsSync(join(appNm, "lib", "package.json"))).toBeTrue(); - // Diverges from pnpm on purpose: an alias that names a workspace is never removed. - expect(existsSync(join(appNm, "tool", "package.json"))).toBeTrue(); + expect(() => lstatSync(appTool)).toThrow(); + expect(await file(join(bTool, "package.json")).json()).toMatchObject({ name: "tool" }); + await expectProductionInstallIsNoop(dir); + expect(() => lstatSync(appTool)).toThrow(); + expect(isSymlink(bTool)).toBeTrue(); + }, +); + +test.concurrent.each(linkers)( + "%s: a workspace's own dependencies survive --production even when the root lists the workspace under devDependencies", + async linker => { + const dir = await setupWorkspaces(linker, { + root: { devDependencies: { tool: "workspace:*" } }, + packages: { + app: { devDependencies: { tool: "workspace:*" } }, + tool: { dependencies: { "no-deps": "1.0.0" } }, + }, + }); + const appTool = join(dir, "packages", "app", "node_modules", "tool"); + const rootTool = join(dir, "node_modules", "tool"); + expect(isSymlink(rootTool)).toBeTrue(); + const toolNoDeps = + linker === "hoisted" + ? join(dir, "node_modules", "no-deps", "package.json") + : join(dir, "packages", "tool", "node_modules", "no-deps", "package.json"); + expect(existsSync(toolNoDeps)).toBeTrue(); + + const first = await prune(dir, "--production", "--linker", linker); + if (linker === "hoisted") { + expect(out(first.stdout)).toEndWith("Nothing to prune."); + expect(isSymlink(rootTool)).toBeTrue(); + } else { + expect(out(first.stdout)).toContain("- packages/app/node_modules/tool"); + expect(() => lstatSync(appTool)).toThrow(); + } + expect(first.exitCode).toBe(0); + expect(existsSync(toolNoDeps)).toBeTrue(); expect(existsSync(join(dir, "packages", "tool", "package.json"))).toBeTrue(); - await runBunInstall(installEnv(dir), dir, { production: true }); + + const second = await prune(dir, "--production", "--linker", linker); + expect(out(second.stdout)).toEndWith("Nothing to prune."); + expect(second.exitCode).toBe(0); + await expectProductionInstallIsNoop(dir); + expect(existsSync(toolNoDeps)).toBeTrue(); + }, +); + +test.concurrent.each(linkers)("%s: refuses when package.json changed since bun.lock was written", async linker => { + const dir = await setupWithLinker(linker, { name: "foo", dependencies: { "no-deps": "1.0.0", "a-dep": "1.0.1" } }); + const junk = plant(dir, "node_modules/junk"); + const aDep = join(dir, "node_modules", "a-dep"); + await write(join(dir, "package.json"), JSON.stringify({ name: "foo", dependencies: { "no-deps": "1.0.0" } })); + const lockBefore = await lock(dir); + + expectRefused(await prune(dir, "--linker", linker)); + expectRefused(await prune(dir, "--dry-run", "--linker", linker)); + expect(existsSync(junk)).toBeTrue(); + expect(existsSync(join(aDep, "package.json"))).toBeTrue(); + + const silent = await prune(dir, "--silent", "--linker", linker); + expect(silent.stdout).toBe(""); + expect(silent.stderr).toBe(""); + expect(silent.exitCode).toBe(1); + expect(existsSync(junk)).toBeTrue(); + expect(await lock(dir)).toBe(lockBefore); + + await install(dir, "--lockfile-only", "--linker", linker); + const { stdout, stderr, exitCode } = await prune(dir, "--linker", linker); + expect(out(stdout)).toBe( + `bun prune ()\n- ${linker === "hoisted" ? "node_modules/a-dep" : "node_modules/.bun/a-dep@1.0.1"}\n- node_modules/junk\nRemoved 2 packages`, + ); + expect(stderr).not.toContain(OUT_OF_SYNC); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); + expect(() => lstatSync(aDep)).toThrow(); +}); + +test.concurrent.each([ + ["a dependency is added", { dependencies: { "no-deps": "1.0.0" }, devDependencies: { "a-dep": "1.0.1" } }], + ["a range changes but still matches the installed version", { dependencies: { "no-deps": "^1.0.0" } }], + ["an override is added", { dependencies: { "no-deps": "1.0.0" }, overrides: { "no-deps": "1.0.0" } }], +] as const)("refuses when %s", async (_, edited) => { + const dir = await setup({ name: "foo", dependencies: { "no-deps": "1.0.0" } }); + const junk = plant(dir, "node_modules/junk"); + const lockBefore = await lock(dir); + await write(join(dir, "package.json"), JSON.stringify({ name: "foo", ...edited })); + + expectRefused(await prune(dir)); + expect(existsSync(junk)).toBeTrue(); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("refuses when a catalog entry changed", async () => { + const catalogRoot = (version: string) => ({ + root: { workspaces: { packages: ["packages/*"], catalog: { "no-deps": version } } }, + packages: { a: { dependencies: { "no-deps": "catalog:" } } }, + }); + const dir = await setupWorkspaces("hoisted", catalogRoot("1.0.0")); + const junk = plant(dir, "node_modules/junk"); + const lockBefore = await lock(dir); + await writeWorkspaces(dir, join(dir, "package.json"), catalogRoot("1.0.1")); + + expectRefused(await prune(dir, "--linker", "hoisted")); + expect(existsSync(junk)).toBeTrue(); + expect(await lock(dir)).toBe(lockBefore); +}); + +test.concurrent("refuses when a workspace's package.json changed, from any cwd and under --filter", async () => { + const dir = await setupWorkspaces("hoisted", { + root: { dependencies: { "no-deps": "2.0.0" } }, + packages: { + a: { dependencies: { "no-deps": "1.0.0" } }, + b: { dependencies: { "a-dep": "1.0.1" } }, + }, + }); + const junk = plant(dir, "packages/a/node_modules/junk"); + const aDep = join(dir, "node_modules", "a-dep"); + expect(existsSync(aDep)).toBeTrue(); + await write(join(dir, "packages", "b", "package.json"), JSON.stringify({ name: "b", version: "1.0.0" })); + + expectRefused(await prune(dir, "--linker", "hoisted")); + expectRefused(await prune({ dir, cwd: join(dir, "packages", "a") }, "--linker", "hoisted")); + expectRefused(await prune(dir, "--filter", "a", "--linker", "hoisted")); + expect(existsSync(junk)).toBeTrue(); + expect(existsSync(aDep)).toBeTrue(); +}); + +test.concurrent.each(linkers)("%s: a workspace lifecycle script is not out of sync", async linker => { + const dir = await setupWorkspaces(linker, { + packages: { a: { dependencies: { "no-deps": "1.0.0" }, scripts: { postinstall: "echo ok" } } }, + }); + const junk = plant(dir, "node_modules/junk"); + + const { stdout, stderr, exitCode } = await prune(dir, "--linker", linker); + expect(stderr).not.toContain(OUT_OF_SYNC); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); +}); + +test.concurrent("trustedDependencies stripped from bun.lock is not out of sync", async () => { + const dir = await setup({ name: "foo", dependencies: { "no-deps": "1.0.0" }, trustedDependencies: ["no-deps"] }); + const before = await lock(dir); + expect(before).toContain('"trustedDependencies"'); + const stripped = before.replace(/\n "trustedDependencies": \[\n(?: [^\n]*\n)* \],/, ""); + expect(stripped).not.toContain('"trustedDependencies"'); + await write(join(dir, "bun.lock"), stripped); + const junk = plant(dir, "node_modules/junk"); + + const { stdout, stderr, exitCode } = await prune(dir); + expect(stderr).not.toContain(OUT_OF_SYNC); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/junk + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); +}); + +const prunedCheckout = (app: Record) => ({ + packages: { + app: { dependencies: { "no-deps": "1.0.0", ...app } }, + other: { dependencies: { "left-pad": "1.0.0" } }, + }, +}); + +test.concurrent("prunes a checkout whose bun.lock lists a workspace that is no longer on disk", async () => { + const dir = await setupWorkspaces("hoisted", prunedCheckout({})); + rmSync(join(dir, "packages", "other"), { recursive: true }); + const junk = plant(dir, "node_modules/junk"); + + const { stdout, stderr, exitCode } = await prune(dir, "--linker", "hoisted"); + expect(stderr).toContain("note: skipped 1 workspace listed in bun.lock but not on disk"); + expect(stderr).not.toContain(OUT_OF_SYNC); + expect(out(stdout)).toContain("- node_modules/junk"); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); +}); + +test.concurrent("a survivor depending on a missing workspace makes prune fail like install does", async () => { + const dir = await setupWorkspaces("hoisted", prunedCheckout({ other: "workspace:*" })); + rmSync(join(dir, "packages", "other"), { recursive: true }); + const junk = plant(dir, "node_modules/junk"); + + const { stdout, stderr, exitCode } = await prune(dir, "--linker", "hoisted"); + expect(stderr).toContain( + 'workspace "app" depends on workspace "other" (packages/other), which is listed in bun.lock but not on disk', + ); + expect(out(stdout)).not.toMatch(/^- /m); + expect(exitCode).toBe(1); + expect(existsSync(junk)).toBeTrue(); +}); + +test.concurrent( + "hoisted: --filter prunes only the selected workspaces, the root folder keeps what other workspaces use", + async () => { + const dir = await setupWorkspaces("hoisted", { + root: { dependencies: { "no-deps": "2.0.0" }, devDependencies: { "left-pad": "1.0.0" } }, + packages: { + a: { dependencies: { "no-deps": "1.0.0" }, devDependencies: { "a-dep": "1.0.1" } }, + b: { dependencies: { "no-deps": "1.0.0" }, devDependencies: { "one-fixed-dep": "1.0.0" } }, + }, + }); + const nm = join(dir, "node_modules"); + const aJunk = plant(dir, "packages/a/node_modules/junk"); + const bJunk = plant(dir, "packages/b/node_modules/junk"); + const stillInstalled = () => { + expect(isSymlink(join(nm, "a"))).toBeTrue(); + expect(isSymlink(join(nm, "b"))).toBeTrue(); + expect(existsSync(join(dir, "packages", "a", "node_modules", "no-deps", "package.json"))).toBeTrue(); + expect(existsSync(join(dir, "packages", "b", "node_modules", "no-deps", "package.json"))).toBeTrue(); + }; + + const onlyA = await prune(dir, "--production", "--filter", "a", "--linker", "hoisted"); + expect(out(onlyA.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/a-dep + - node_modules/a/node_modules/junk + Removed 2 packages" + `); + expect(onlyA.exitCode).toBe(0); + expect(existsSync(aJunk)).toBeFalse(); + expect(existsSync(bJunk)).toBeTrue(); + expect(existsSync(join(nm, "left-pad"))).toBeTrue(); + expect(existsSync(join(nm, "one-fixed-dep"))).toBeTrue(); + expect(await file(join(nm, "no-deps", "package.json")).json()).toMatchObject({ version: "2.0.0" }); + stillInstalled(); + + const onlyRoot = await prune(dir, "--production", "--filter", "root", "--linker", "hoisted"); + expect(out(onlyRoot.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/left-pad + Removed 1 package" + `); + expect(onlyRoot.exitCode).toBe(0); + expect(existsSync(bJunk)).toBeTrue(); + expect(existsSync(join(nm, "one-fixed-dep"))).toBeTrue(); + + const everything = await prune(dir, "--production", "--linker", "hoisted"); + expect(out(everything.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/b/node_modules/junk + - node_modules/one-fixed-dep + Removed 2 packages" + `); + expect(everything.exitCode).toBe(0); + expect(existsSync(bJunk)).toBeFalse(); + stillInstalled(); + await expectProductionInstallIsNoop(dir); + }, +); + +test.concurrent( + "isolated: --filter keeps store entries other workspaces link to and drops the selected workspace's exclusive ones", + async () => { + const dir = await setupWorkspaces("isolated", { + packages: { + a: { dependencies: { "no-deps": "1.0.0" }, devDependencies: { "a-dep": "1.0.1", "one-fixed-dep": "1.0.0" } }, + b: { devDependencies: { "a-dep": "1.0.1", "left-pad": "1.0.0" } }, + }, + }); + const store = join(dir, "node_modules", ".bun"); + const aNm = join(dir, "packages", "a", "node_modules"); + const bNm = join(dir, "packages", "b", "node_modules"); + + const onlyA = await prune(dir, "--production", "--filter", "a", "--linker", "isolated"); + expect(out(onlyA.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/one-fixed-dep@1.0.0 + - packages/a/node_modules/a-dep + Removed 2 packages" + `); + expect(onlyA.exitCode).toBe(0); + expect(existsSync(join(store, "a-dep@1.0.1"))).toBeTrue(); + expect(existsSync(join(store, "left-pad@1.0.0"))).toBeTrue(); + expect(existsSync(join(store, "no-deps@1.0.0"))).toBeTrue(); + expect(existsSync(join(bNm, "a-dep", "package.json"))).toBeTrue(); + expect(existsSync(join(bNm, "left-pad", "package.json"))).toBeTrue(); + expect(() => lstatSync(join(aNm, "a-dep"))).toThrow(); + expect(() => lstatSync(join(aNm, "one-fixed-dep"))).toThrow(); + expect(existsSync(join(aNm, "no-deps", "package.json"))).toBeTrue(); + + const everything = await prune(dir, "--production", "--linker", "isolated"); + expect(out(everything.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/a-dep@1.0.1 + - node_modules/.bun/left-pad@1.0.0 + Removed 2 packages" + `); + expect(everything.exitCode).toBe(0); + expect(() => lstatSync(join(bNm, "a-dep"))).toThrow(); + expect(() => lstatSync(join(bNm, "left-pad"))).toThrow(); + expect(existsSync(join(aNm, "no-deps", "package.json"))).toBeTrue(); + await expectProductionInstallIsNoop(dir); + }, +); + +test.concurrent( + "isolated: --production --filter unlinks one workspace's dev deps at a time, the store entry waits for an unfiltered run", + async () => { + const pkg = { dependencies: { "no-deps": "1.0.0" }, devDependencies: { "a-dep": "1.0.1" } }; + const dir = await setupWorkspaces("isolated", { packages: { selected: pkg, unselected: pkg } }); + const storeEntry = join(dir, "node_modules", ".bun", "a-dep@1.0.1"); + const selectedADep = join(dir, "packages", "selected", "node_modules", "a-dep"); + const unselectedADep = join(dir, "packages", "unselected", "node_modules", "a-dep"); + + const first = await prune(dir, "--production", "--filter", "selected", "--linker", "isolated"); + expect(out(first.stdout)).toMatchInlineSnapshot(` + "bun prune () + - packages/selected/node_modules/a-dep + Removed 1 package" + `); + expect(first.exitCode).toBe(0); + expect(() => lstatSync(selectedADep)).toThrow(); + expect(existsSync(join(unselectedADep, "package.json"))).toBeTrue(); + expect(existsSync(storeEntry)).toBeTrue(); + + const second = await prune(dir, "--production", "--filter", "unselected", "--linker", "isolated"); + expect(out(second.stdout)).toMatchInlineSnapshot(` + "bun prune () + - packages/unselected/node_modules/a-dep + Removed 1 package" + `); + expect(second.exitCode).toBe(0); + expect(() => lstatSync(unselectedADep)).toThrow(); + expect(existsSync(storeEntry)).toBeTrue(); + + const everything = await prune(dir, "--production", "--linker", "isolated"); + expect(out(everything.stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/.bun/a-dep@1.0.1 + Removed 1 package" + `); + expect(everything.exitCode).toBe(0); + expect(existsSync(storeEntry)).toBeFalse(); + for (const ws of ["selected", "unselected"]) { + expect(existsSync(join(dir, "packages", ws, "node_modules", "no-deps", "package.json"))).toBeTrue(); + } + await expectProductionInstallIsNoop(dir); + }, +); + +test.concurrent.each(linkers)( + "%s: --filter limits extraneous entries to the selected workspaces' own folders; name, path and glob selectors", + async linker => { + const dir = await setupWorkspaces(linker, { + root: { dependencies: { "no-deps": "2.0.0" } }, + packages: { + app: { dependencies: { "no-deps": "1.0.0" } }, + lib: { dependencies: { "no-deps": "1.0.0" } }, + }, + }); + const rootJunk = plant(dir, "node_modules/root-junk"); + const appJunk = plant(dir, "packages/app/node_modules/app-junk"); + const libJunk = plant(dir, "packages/lib/node_modules/lib-junk"); + const storeJunk = linker === "isolated" ? plant(dir, "node_modules/.bun/junk@1.0.0/node_modules/junk") : null; + const shown = (ws: string, name: string) => + linker === "hoisted" ? `- node_modules/${ws}/node_modules/${name}` : `- packages/${ws}/node_modules/${name}`; + const listing = (removed: string[], verb: string) => + [ + "bun prune ()", + ...removed, + `${verb} ${removed.length} package${removed.length === 1 ? "" : "s"}`, + ].join("\n"); + // The shared area is swept whole-repo under --filter: the isolated store, and under hoisted the root folder itself. + const sharedJunk = linker === "hoisted" ? ["- node_modules/root-junk"] : ["- node_modules/.bun/junk@1.0.0"]; + + const dryRun = await prune(dir, "--filter", "app", "--dry-run", "--linker", linker); + expect(out(dryRun.stdout)).toBe(listing([...sharedJunk, shown("app", "app-junk")], "Would remove")); + expect(dryRun.exitCode).toBe(0); + expect(existsSync(appJunk)).toBeTrue(); + expect(existsSync(rootJunk)).toBeTrue(); + + const byPath = await prune(dir, "--filter", "./packages/app", "--linker", linker); + expect(out(byPath.stdout)).toBe(listing([...sharedJunk, shown("app", "app-junk")], "Removed")); + expect(byPath.exitCode).toBe(0); + expect(existsSync(appJunk)).toBeFalse(); + expect(existsSync(libJunk)).toBeTrue(); + expect(existsSync(rootJunk)).toBe(linker === "isolated"); + if (storeJunk) { + expect(existsSync(storeJunk)).toBeFalse(); + } + + const byGlob = await prune({ dir, cwd: join(dir, "packages", "app") }, "--filter", "li*", "--linker", linker); + expect(out(byGlob.stdout)).toBe(listing([shown("lib", "lib-junk")], "Removed")); + expect(byGlob.exitCode).toBe(0); + expect(existsSync(libJunk)).toBeFalse(); + + if (linker === "isolated") { + const root = await prune(dir, "--filter", "./", "--linker", linker); + expect(out(root.stdout)).toBe(listing(["- node_modules/root-junk"], "Removed")); + expect(root.exitCode).toBe(0); + } + expect(existsSync(rootJunk)).toBeFalse(); + for (const ws of ["app", "lib"]) { + expect(existsSync(join(dir, "packages", ws, "node_modules", "no-deps", "package.json"))).toBeTrue(); + } }, ); +test.concurrent("hoisted: --filter with no match is an error; path filters resolve against the cwd", async () => { + const dir = await setupWorkspaces("hoisted", { + root: { dependencies: { "no-deps": "2.0.0" } }, + packages: { a: { dependencies: { "no-deps": "1.0.0" } } }, + }); + const junk = plant(dir, "packages/a/node_modules/junk"); + const listing = `bun prune ()\n- node_modules/a/node_modules/junk\nWould remove 1 package`; + + const noMatch = await prune(dir, "--filter", "nope", "--linker", "hoisted"); + expect(noMatch.stderr).toContain("No packages matched the filter"); + expect(out(noMatch.stdout)).not.toMatch(/^- /m); + expect(noMatch.exitCode).toBe(1); + expect(existsSync(junk)).toBeTrue(); + + const fromRoot = await prune(dir, "--filter", "./packages/a", "--dry-run", "--linker", "hoisted"); + expect(out(fromRoot.stdout)).toBe(listing); + expect(fromRoot.exitCode).toBe(0); + expect(existsSync(junk)).toBeTrue(); + + const fromInside = await prune( + { dir, cwd: join(dir, "packages", "a") }, + "--filter", + ".", + "--dry-run", + "--linker", + "hoisted", + ); + expect(out(fromInside.stdout)).toBe(listing); + expect(fromInside.exitCode).toBe(0); + expect(existsSync(junk)).toBeTrue(); + + const { stdout, exitCode } = await prune(dir, "--filter", "a", "--linker", "hoisted"); + expect(out(stdout)).toMatchInlineSnapshot(` + "bun prune () + - node_modules/a/node_modules/junk + Removed 1 package" + `); + expect(exitCode).toBe(0); + expect(existsSync(junk)).toBeFalse(); +}); + test.concurrent.each(["hoisted", "isolated"] as Linker[])( "%s: optionalDependencies survive --production and go away with --omit=optional", async linker => { @@ -1216,7 +1806,7 @@ test.concurrent( const again = await prune(dir, "--production", "--linker", "isolated"); expect(out(again.stdout)).toEndWith("Nothing to prune."); expect(again.exitCode).toBe(0); - await runBunInstall(installEnv(dir), dir, { production: true }); + await expectProductionInstallIsNoop(dir); }, ); @@ -1320,7 +1910,7 @@ test.concurrent.each(["hoisted", "isolated"] as Linker[])( if (linker === "isolated") { expect(existsSync(join(nm, ".bun", "no-deps@1.0.0"))).toBeTrue(); } - await runBunInstall(installEnv(dir), dir, { production: true }); + await expectProductionInstallIsNoop(dir); }, ); diff --git a/test/cli/install/bun-update-lockfile-sync.test.ts b/test/cli/install/bun-update-lockfile-sync.test.ts index fa4cd07dc358..8ffdf0bfa3e7 100644 --- a/test/cli/install/bun-update-lockfile-sync.test.ts +++ b/test/cli/install/bun-update-lockfile-sync.test.ts @@ -1,10 +1,10 @@ import { Archive, file, write } from "bun"; import { afterAll, beforeAll, describe, expect, test } from "bun:test"; -import { exists } from "fs/promises"; -import { VerdaccioRegistry, bunEnv, bunExe, runBunInstall, runBunUpdate, tempDir } from "harness"; +import { appendFile, exists } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe, runBunInstall } from "harness"; import { join } from "path"; -// Registry: no-deps 1.0.0/1.0.1/1.1.0/2.0.0, @types/no-deps 1.0.0/2.0.0, a-dep 1.0.1..1.0.10, one-range-dep@1.0.0 -> no-deps ^1.0.0. +// Registry: no-deps 1.0.0/1.0.1/1.1.0/2.0.0, @types/no-deps 1.0.0/2.0.0, a-dep 1.0.1..1.0.10, one-range-dep@1.0.0 -> no-deps ^1.0.0, dep-with-tags 1.0.0..3.0.1 (latest=3.0.0, pre-2=2.0.1). const verdaccio = new VerdaccioRegistry(); @@ -19,6 +19,9 @@ afterAll(() => { type Json = Record; const GROUPS = ["dependencies", "devDependencies", "optionalDependencies", "peerDependencies"] as const; +// CI exports one BUN_INSTALL_CACHE_DIR per file, which overrides bunfig's cache; concurrent cases sharing a cache race on Windows. +const envFor = (dir: string) => ({ ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }); + function json(contents: Json | string) { return typeof contents === "string" ? contents : JSON.stringify(contents, null, 2) + "\n"; } @@ -27,50 +30,55 @@ async function setup( files: Record, opts: { exact?: boolean; text?: boolean; install?: boolean; allowWarnings?: boolean } = {}, ): Promise { - const dir = String( - tempDir( - "lockfile-sync-", - Object.fromEntries(Object.entries(files).map(([path, contents]) => [path, json(contents)])), - ), - ); - await write( - join(dir, "bunfig.toml"), - Bun.TOML.stringify({ - install: { - cache: join(dir, ".bun-cache"), - registry: verdaccio.registryUrl(), - saveTextLockfile: opts.text ?? true, - linker: "hoisted", - exact: opts.exact, - }, - }), - ); - if (opts.install !== false) await runBunInstall(bunEnv, dir, { allowWarnings: opts.allowWarnings }); + const { packageDir: dir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted", saveTextLockfile: opts.text ?? true }, + files: Object.fromEntries(Object.entries(files).map(([path, contents]) => [path, json(contents)])), + }); + if (opts.exact) await appendFile(join(dir, "bunfig.toml"), "exact = true\n"); + if (opts.install !== false) await runBunInstall(envFor(dir), dir, { allowWarnings: opts.allowWarnings }); return dir; } -async function run(cwd: string, ...args: string[]) { +async function tryRun(dir: string, rel: string, ...args: string[]) { await using proc = Bun.spawn({ cmd: [bunExe(), ...args], - cwd, - env: bunEnv, + cwd: join(dir, rel), + env: envFor(dir), stdout: "pipe", stderr: "pipe", stdin: "ignore", }); const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); - expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); return { stdout, stderr, exitCode }; } +async function runIn(dir: string, rel: string, ...args: string[]) { + const result = await tryRun(dir, rel, ...args); + expect(result.stderr).not.toContain("error:"); + expect(result.exitCode).toBe(0); + return result; +} + +const run = (dir: string, ...args: string[]) => runIn(dir, "", ...args); + const pkg = (dir: string, rel = ""): Promise => file(join(dir, rel, "package.json")).json(); +const pkgText = (dir: string, rel = "") => file(join(dir, rel, "package.json")).text(); const writePkg = (dir: string, contents: Json, rel = "") => write(join(dir, rel, "package.json"), json(contents)); const installed = (dir: string, name: string): Promise => file(join(dir, "node_modules", name, "package.json")).json(); const lockText = (dir: string) => file(join(dir, "bun.lock")).text(); const lock = async (dir: string): Promise => Bun.JSONC.parse(await lockText(dir)) as Json; +async function resolutions(dir: string, name: string): Promise { + const entries = Object.values((await lock(dir)).packages) as [string, ...unknown[]][]; + return [...new Set(entries.map(entry => entry[0]).filter(res => res.startsWith(`${name}@`)))].sort(); +} + +async function reinstall(dir: string, contents: Json, rel = "") { + await writePkg(dir, contents, rel); + await runBunInstall(envFor(dir), dir); +} + function declaredLiteral(manifest: Json, name: string): string | undefined { for (const group of GROUPS) { const literal = manifest[group]?.[name]; @@ -78,7 +86,11 @@ function declaredLiteral(manifest: Json, name: string): string | undefined { } } -async function expectInSync(dir: string, workspaces: string[] = [""], allowWarnings = false) { +async function expectInSync( + dir: string, + workspaces: string[] = [""], + opts: { allowWarnings?: boolean; reinstall?: boolean } = {}, +) { const lockfile = await lock(dir); for (const key of workspaces) { const manifest = await pkg(dir, key); @@ -104,20 +116,28 @@ async function expectInSync(dir: string, workspaces: string[] = [""], allowWarni const catalogs = manifest.workspaces?.catalogs ?? manifest.catalogs; if (catalogs !== undefined) expect(lockfile.catalogs).toEqual(catalogs); } - await runBunInstall(bunEnv, dir, { frozenLockfile: true, allowWarnings }); + const env = envFor(dir); + await runBunInstall(env, dir, { frozenLockfile: true, allowWarnings: opts.allowWarnings }); + if (!opts.reinstall) return; const before = await lockText(dir); - const { err } = await runBunInstall(bunEnv, dir, { savesLockfile: false, allowWarnings }); + const { err } = await runBunInstall(env, dir, { savesLockfile: false, allowWarnings: opts.allowWarnings }); expect(err).not.toContain("Saved lockfile"); expect(await lockText(dir)).toBe(before); } const root = (fields: Json): Json => ({ name: "foo", ...fields }); - -const MONOREPO = (pkg1: Json = {}, rootFields: Json = {}) => ({ - "package.json": { name: "root", workspaces: ["packages/*"], ...rootFields }, - "packages/pkg1/package.json": { name: "pkg1", version: "1.0.0", ...pkg1 }, +const wsRoot = (fields: Json = {}): Json => ({ name: "root", workspaces: ["packages/*"], ...fields }); +const member = (name: string, fields: Json = {}): Json => ({ name, version: "1.0.0", ...fields }); + +const WORKSPACES = (rootFields: Json, members: Record) => ({ + "package.json": wsRoot(rootFields), + ...Object.fromEntries( + Object.entries(members).map(([name, fields]) => [`packages/${name}/package.json`, member(name, fields)]), + ), }); +const MONOREPO = (pkg1: Json = {}, rootFields: Json = {}) => WORKSPACES(rootFields, { pkg1 }); const PKG1 = "packages/pkg1"; +const PKG2 = "packages/pkg2"; describe.concurrent("bun update rewrites bun.lock together with package.json", () => { test("bun update", async () => { @@ -128,7 +148,7 @@ describe.concurrent("bun update rewrites bun.lock together with package.json", ( const expected = { "no-deps": "^1.1.0", aliased: "npm:no-deps@~1.0.1" }; expect((await pkg(dir)).dependencies).toEqual(expected); expect((await lock(dir)).workspaces[""].dependencies).toEqual(expected); - await expectInSync(dir); + await expectInSync(dir, [""], { reinstall: true }); }); test("bun update --latest", async () => { @@ -144,6 +164,43 @@ describe.concurrent("bun update rewrites bun.lock together with package.json", ( await expectInSync(dir); }); + test.each([ + ["*", "2.0.0"], + ["1", "1.1.0"], + ["1.x", "1.1.0"], + [">=1.0.0", "2.0.0"], + ["1.0.0 - 1.0.1", "1.0.1"], + ])("bun update leaves a %s range as written and moves bun.lock to %s", async (literal, resolved) => { + const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "1.0.0" } }) }); + await reinstall(dir, root({ dependencies: { "no-deps": literal } })); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.0"]); + const pkgBefore = await pkgText(dir); + const { stdout } = await run(dir, "update"); + expect(stdout).toMatch(new RegExp(`no-deps 1\\.0\\.0 (→|->) ${resolved.replaceAll(".", "\\.")}`)); + expect(await pkgText(dir)).toBe(pkgBefore); + expect((await lock(dir)).workspaces[""].dependencies).toEqual({ "no-deps": literal }); + expect(await resolutions(dir, "no-deps")).toEqual([`no-deps@${resolved}`]); + expect(await installed(dir, "no-deps")).toMatchObject({ version: resolved }); + await expectInSync(dir); + }); + + test("bun update on a * range that already resolves the newest version writes nothing", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "*" } }) }); + const [pkgBefore, lockBefore] = await Promise.all([pkgText(dir), lockText(dir)]); + const { stderr } = await run(dir, "update"); + expect(stderr).not.toContain("Saved lockfile"); + expect(await pkgText(dir)).toBe(pkgBefore); + expect(await lockText(dir)).toBe(lockBefore); + }); + + test("bun update --latest rewrites a * range", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "*" } }) }); + await run(dir, "update", "--latest"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "^2.0.0" }); + expect((await lock(dir)).workspaces[""].dependencies).toEqual({ "no-deps": "^2.0.0" }); + await expectInSync(dir); + }); + test("bun update keeps the pin style", async () => { const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "~1.0.0" } }) }); await run(dir, "update", "no-deps"); @@ -160,6 +217,18 @@ describe.concurrent("bun update rewrites bun.lock together with package.json", ( await expectInSync(dir); }); + test("bun update keeps a * literal and leaves the unnamed sibling alone", async () => { + const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "1.0.0", "a-dep": "1.0.1" } }) }); + await reinstall(dir, root({ dependencies: { "no-deps": "*", "a-dep": "^1.0.1" } })); + await run(dir, "update", "no-deps"); + const expected = { "no-deps": "*", "a-dep": "^1.0.1" }; + expect((await pkg(dir)).dependencies).toEqual(expected); + expect((await lock(dir)).workspaces[""].dependencies).toEqual(expected); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@2.0.0"]); + expect(await resolutions(dir, "a-dep")).toEqual(["a-dep@1.0.1"]); + await expectInSync(dir); + }); + test("bun update keeps the alias", async () => { const dir = await setup({ "package.json": root({ dependencies: { aliased: "npm:no-deps@~1.0.0" } }) }); await run(dir, "update", "aliased"); @@ -186,20 +255,20 @@ describe.concurrent("bun update rewrites bun.lock together with package.json", ( const manifest = await pkg(dir); expect(manifest.dependencies).toEqual({ "no-deps": "~1.0.1" }); expect(manifest.devDependencies).toEqual({ "no-deps": "~1.0.0" }); - await expectInSync(dir, [""], true); + await expectInSync(dir, [""], { allowWarnings: true }); }); test("bun update with the name in dependencies and devDependencies moves one group", async () => { const dir = await setup({ "package.json": inBothGroups("1.0.0") }, { allowWarnings: true }); await writePkg(dir, inBothGroups("~1.0.0")); - const { out } = await runBunUpdate(bunEnv, dir); - expect(out.join("\n")).toMatch(/no-deps 1\.0\.0 (→|->) 1\.0\.1/); + const { stdout } = await run(dir, "update"); + expect(stdout).toMatch(/no-deps 1\.0\.0 (→|->) 1\.0\.1/); const manifest = await pkg(dir); expect([manifest.dependencies["no-deps"], manifest.devDependencies["no-deps"]].sort()).toEqual([ "~1.0.0", "~1.0.1", ]); - await expectInSync(dir, [""], true); + await expectInSync(dir, [""], { allowWarnings: true }); }); test("install.exact", async () => { @@ -229,7 +298,7 @@ describe.concurrent("bun update rewrites bun.lock together with package.json", ( { "package/package.json": JSON.stringify({ name: "tgz-dep", version: "1.0.0" }) }, { compress: "gzip" }, ); - await runBunInstall(bunEnv, dir); + await runBunInstall(envFor(dir), dir); await run(dir, "update", ...args); const expected = { ...dependencies, "no-deps": args.length ? "^2.0.0" : "^1.1.0" }; expect((await pkg(dir)).dependencies).toEqual(expected); @@ -245,31 +314,195 @@ describe.concurrent("bun update rewrites bun.lock together with package.json", ( await expectInSync(dir, ["", PKG1]); }); + test("bun update -r keeps a member's 1.x literal", async () => { + const dir = await setup(MONOREPO({ dependencies: { "no-deps": "1.0.0" } })); + await reinstall(dir, member("pkg1", { dependencies: { "no-deps": "1.x" } }), PKG1); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.0"]); + await run(dir, "update", "-r"); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "1.x" }); + expect((await lock(dir)).workspaces[PKG1].dependencies).toEqual({ "no-deps": "1.x" }); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.1.0"]); + await expectInSync(dir, ["", PKG1]); + }); + + test("bun update -r --latest keeps a member's dist-tag literal", async () => { + const dir = await setup(MONOREPO({ dependencies: { "dep-with-tags": "pre-2", "no-deps": "~1.0.0" } })); + await run(dir, "update", "-r", "--latest"); + const expected = { "dep-with-tags": "pre-2", "no-deps": "~2.0.0" }; + expect((await pkg(dir, PKG1)).dependencies).toEqual(expected); + expect((await lock(dir)).workspaces[PKG1].dependencies).toEqual(expected); + expect(await resolutions(dir, "dep-with-tags")).toEqual(["dep-with-tags@2.0.1"]); + await expectInSync(dir, ["", PKG1]); + }); + + test("bun update --filter rewrites only the selected member", async () => { + const dir = await setup( + MONOREPO({ dependencies: { "no-deps": "~1.0.0", "a-dep": "^1.0.1" } }, { dependencies: { "no-deps": "~1.0.0" } }), + ); + await run(dir, "update", "no-deps", "--filter", "pkg1"); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "~1.0.1", "a-dep": "^1.0.1" }); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "~1.0.0" }); + await expectInSync(dir, ["", PKG1], { reinstall: true }); + }); + + test("bun update -r --latest rewrites only the workspaces that declare the name", async () => { + const dir = await setup( + WORKSPACES( + { dependencies: { "no-deps": "~1.0.0" } }, + { pkg1: { dependencies: { "a-dep": "1.0.1" } }, pkg2: { dependencies: { "no-deps": "~1.0.0" } } }, + ), + ); + await run(dir, "update", "no-deps", "-r", "--latest"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "~2.0.0" }); + expect((await pkg(dir, PKG2)).dependencies).toEqual({ "no-deps": "~2.0.0" }); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "a-dep": "1.0.1" }); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@2.0.0"]); + await expectInSync(dir, ["", PKG1, PKG2]); + }); + + test("bun update @ -r keeps each workspace's operator", async () => { + const dir = await setup( + WORKSPACES({ dependencies: { "no-deps": "1.0.0" } }, { pkg1: { dependencies: { "no-deps": "1.0.0" } } }), + ); + await writePkg(dir, wsRoot({ dependencies: { "no-deps": "^1.0.0" } })); + await reinstall(dir, member("pkg1", { dependencies: { "no-deps": "~1.0.0" } }), PKG1); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.0"]); + await run(dir, "update", "no-deps@1.0.1", "-r"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "^1.0.1" }); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "~1.0.1" }); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.1"]); + await expectInSync(dir, ["", PKG1]); + }); + + test("bun update -r --dry-run writes nothing", async () => { + const dir = await setup(MONOREPO({ dependencies: { "no-deps": "1.0.0" } })); + await reinstall(dir, member("pkg1", { dependencies: { "no-deps": "~1.0.0" } }), PKG1); + const [pkgBefore, lockBefore] = await Promise.all([pkgText(dir, PKG1), lockText(dir)]); + const { stderr } = await run(dir, "update", "no-deps", "-r", "--dry-run"); + expect(stderr).not.toContain("Saved lockfile"); + expect(await pkgText(dir, PKG1)).toBe(pkgBefore); + expect(await lockText(dir)).toBe(lockBefore); + }); + test("bun update from a workspace member", async () => { const dir = await setup(MONOREPO({ dependencies: { "no-deps": "~1.0.0" } })); - await run(join(dir, PKG1), "update"); + await runIn(dir, PKG1, "update"); expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "~1.0.1" }); expect((await lock(dir)).workspaces[PKG1].dependencies).toEqual({ "no-deps": "~1.0.1" }); await expectInSync(dir, ["", PKG1]); }); }); +describe.concurrent("named update is scoped to the invoking workspace", () => { + test("from the root, another workspace's row stays put; -r moves it too", async () => { + const dir = await setup( + WORKSPACES({ dependencies: { "no-deps": "1.0.0" } }, { pkg1: { dependencies: { "no-deps": "1.0.0" } } }), + ); + await writePkg(dir, wsRoot({ dependencies: { "no-deps": "^1.0.0" } })); + await reinstall(dir, member("pkg1", { dependencies: { "no-deps": "~1.0.0" } }), PKG1); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.0"]); + + await run(dir, "update", "no-deps"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "^1.1.0" }); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "~1.0.0" }); + expect((await lock(dir)).workspaces[PKG1].dependencies).toEqual({ "no-deps": "~1.0.0" }); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.0", "no-deps@1.1.0"]); + await expectInSync(dir, ["", PKG1]); + + await run(dir, "update", "no-deps", "-r"); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "^1.1.0" }); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "~1.0.1" }); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.1", "no-deps@1.1.0"]); + await expectInSync(dir, ["", PKG1]); + }); + + test("inside a member, a sibling's row is not re-resolved", async () => { + const dir = await setup( + WORKSPACES( + {}, + { pkg1: { dependencies: { "no-deps": "1.0.0" } }, pkg2: { dependencies: { "no-deps": "1.0.0" } } }, + ), + ); + await writePkg(dir, member("pkg1", { dependencies: { "no-deps": "^1.0.0" } }), PKG1); + await reinstall(dir, member("pkg2", { dependencies: { "no-deps": "~1.0.0" } }), PKG2); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.0"]); + + await runIn(dir, PKG1, "update", "no-deps"); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "^1.1.0" }); + expect((await pkg(dir, PKG2)).dependencies).toEqual({ "no-deps": "~1.0.0" }); + expect((await lock(dir)).workspaces[PKG2].dependencies).toEqual({ "no-deps": "~1.0.0" }); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.0", "no-deps@1.1.0"]); + await expectInSync(dir, ["", PKG1, PKG2]); + }); + + test("bun update --filter from the root leaves the root's own row out of scope", async () => { + const dir = await setup( + WORKSPACES( + { dependencies: { "no-deps": "1.0.0" } }, + { pkg1: { dependencies: { "no-deps": "1.0.0" } }, pkg2: { dependencies: { "no-deps": "1.0.0" } } }, + ), + ); + await writePkg(dir, wsRoot({ dependencies: { "no-deps": "^1.0.0" } })); + await writePkg(dir, member("pkg1", { dependencies: { "no-deps": "~1.0.0" } }), PKG1); + await reinstall(dir, member("pkg2", { dependencies: { "no-deps": "~1.0.0" } }), PKG2); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.0"]); + + await run(dir, "update", "no-deps", "--filter", "pkg1"); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "~1.0.1" }); + expect((await pkg(dir)).dependencies).toEqual({ "no-deps": "^1.0.0" }); + expect((await pkg(dir, PKG2)).dependencies).toEqual({ "no-deps": "~1.0.0" }); + const { workspaces } = await lock(dir); + expect(workspaces[""].dependencies).toEqual({ "no-deps": "^1.0.0" }); + expect(workspaces[PKG2].dependencies).toEqual({ "no-deps": "~1.0.0" }); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.1"]); + await expectInSync(dir, ["", PKG1, PKG2]); + }); + + test("a name declared only by another workspace is an error that points at -r", async () => { + const dir = await setup(WORKSPACES({}, { pkg1: { dependencies: { "no-deps": "1.0.0" } } })); + await reinstall(dir, member("pkg1", { dependencies: { "no-deps": "~1.0.0" } }), PKG1); + const [pkgBefore, lockBefore] = await Promise.all([pkgText(dir, PKG1), lockText(dir)]); + const { stderr, exitCode } = await tryRun(dir, "", "update", "no-deps"); + expect(stderr).toContain( + 'error: "no-deps" is only a dependency of other workspaces, so there is nothing to update here', + ); + expect(stderr).toContain("bun update -r no-deps"); + expect(stderr).not.toContain("is not in the lockfile"); + expect(await pkgText(dir, PKG1)).toBe(pkgBefore); + expect(await lockText(dir)).toBe(lockBefore); + expect(exitCode).toBe(1); + }); +}); + describe.concurrent("npm: aliases", () => { - test.each([ + const ROWS: { pinned?: string; before: string; args: string[]; after: string; installs: [string, string] }[] = [ { before: "npm:no-deps@~1.0.0", args: ["aliased", "--latest"], after: "npm:no-deps@~2.0.0", installs: ["no-deps", "2.0.0"], }, - { before: "npm:no-deps", args: [], after: "npm:no-deps@^2.0.0", installs: ["no-deps", "2.0.0"] }, - { before: "npm:no-deps", args: ["aliased"], after: "npm:no-deps@^2.0.0", installs: ["no-deps", "2.0.0"] }, + { before: "npm:no-deps", args: [], after: "npm:no-deps", installs: ["no-deps", "2.0.0"] }, + { before: "npm:no-deps", args: ["aliased"], after: "npm:no-deps", installs: ["no-deps", "2.0.0"] }, + { + pinned: "npm:no-deps@1.0.0", + before: "npm:no-deps@*", + args: [], + after: "npm:no-deps@*", + installs: ["no-deps", "2.0.0"], + }, { before: "npm:@types/no-deps", args: ["--latest"], after: "npm:@types/no-deps@^2.0.0", installs: ["@types/no-deps", "2.0.0"], }, + { + before: "npm:dep-with-tags@pre-2", + args: ["--latest"], + after: "npm:dep-with-tags@pre-2", + installs: ["dep-with-tags", "2.0.1"], + }, { before: "npm:no-deps@~1.0.0", args: ["aliased@2.0.0"], @@ -277,14 +510,23 @@ describe.concurrent("npm: aliases", () => { installs: ["no-deps", "2.0.0"], }, { before: "npm:no-deps@^1.0.0", args: ["aliased"], after: "npm:no-deps@^1.1.0", installs: ["no-deps", "1.1.0"] }, - ])('"aliased": "$before" + bun update $args -> "$after"', async ({ before, args, after, installs }) => { - const dir = await setup({ "package.json": root({ dependencies: { aliased: before } }) }); - await run(dir, "update", ...args); - expect((await pkg(dir)).dependencies.aliased).toBe(after); - const [name, version] = installs; - expect(await installed(dir, "aliased")).toMatchObject({ name, version }); - await expectInSync(dir); - }); + ]; + + test.each(ROWS)( + '"aliased": "$before" + bun update $args -> "$after"', + async ({ pinned, before, args, after, installs }) => { + const dir = await setup({ "package.json": root({ dependencies: { aliased: pinned ?? before } }) }); + if (pinned !== undefined) { + await reinstall(dir, root({ dependencies: { aliased: before } })); + expect(await installed(dir, "aliased")).toMatchObject({ version: "1.0.0" }); + } + await run(dir, "update", ...args); + expect((await pkg(dir)).dependencies.aliased).toBe(after); + const [name, version] = installs; + expect(await installed(dir, "aliased")).toMatchObject({ name, version }); + await expectInSync(dir); + }, + ); test("bun update @npm: retargets the alias", async () => { const dir = await setup({ "package.json": root({ dependencies: { aliased: "npm:no-deps@~1.0.0" } }) }); @@ -296,19 +538,11 @@ describe.concurrent("npm: aliases", () => { test("bun update @npm:@ refuses to add; bun add keeps the target", async () => { const dir = await setup({ "package.json": root({ dependencies: { "a-dep": "1.0.1" } }) }); - const [pkgBefore, lockBefore] = await Promise.all([file(join(dir, "package.json")).text(), lockText(dir)]); - await using proc = Bun.spawn({ - cmd: [bunExe(), "update", "new-alias@npm:@types/no-deps@^1.0.0"], - cwd: dir, - env: bunEnv, - stdout: "pipe", - stderr: "pipe", - stdin: "ignore", - }); - const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); + const [pkgBefore, lockBefore] = await Promise.all([pkgText(dir), lockText(dir)]); + const { stderr, exitCode } = await tryRun(dir, "", "update", "new-alias@npm:@types/no-deps@^1.0.0"); expect(stderr).toContain('error: "new-alias" is not in the lockfile, so there is nothing to update'); expect(exitCode).toBe(1); - expect(await file(join(dir, "package.json")).text()).toBe(pkgBefore); + expect(await pkgText(dir)).toBe(pkgBefore); expect(await lockText(dir)).toBe(lockBefore); await run(dir, "add", "new-alias@npm:@types/no-deps@^1.0.0"); @@ -336,7 +570,8 @@ describe.concurrent("bun add", () => { { args: ["no-deps@latest"], expected: { "no-deps": "^2.0.0" } }, { args: ["x@npm:no-deps@~1.0.0"], expected: { x: "npm:no-deps@~1.0.0" } }, { args: ["x@npm:no-deps@latest"], expected: { x: "npm:no-deps@^2.0.0" } }, - { args: ["x@npm:no-deps"], expected: { x: "npm:no-deps" } }, + { args: ["x@npm:no-deps"], expected: { x: "npm:no-deps@^2.0.0" } }, + { args: ["x@npm:no-deps", "--exact"], expected: { x: "npm:no-deps@2.0.0" } }, ])("bun add $args", async ({ args, expected }) => { const dir = await setup({ "package.json": root({}) }, { install: false }); await run(dir, "add", ...args); @@ -391,50 +626,61 @@ describe.concurrent("bun add", () => { expect(await exists(join(dir, "node_modules", "uses-what-bin", "what-bin.txt"))).toBe(true); await expectInSync(dir); }); - - test("bun add --dry-run writes neither file", async () => { - const dir = await setup({ "package.json": root({ dependencies: { "a-dep": "1.0.1" } }) }); - const [pkgBefore, lockBefore] = await Promise.all([file(join(dir, "package.json")).text(), lockText(dir)]); - await run(dir, "add", "no-deps", "--dry-run"); - expect(await file(join(dir, "package.json")).text()).toBe(pkgBefore); - expect(await lockText(dir)).toBe(lockBefore); - }); }); describe.concurrent("catalogs", () => { const CATALOG_REPO = (catalog: Json = { "no-deps": "^1.0.0", aliased: "npm:no-deps" }) => MONOREPO( - { dependencies: { "no-deps": "catalog:", aliased: "catalog:" } }, + { dependencies: Object.fromEntries(Object.keys(catalog).map(name => [name, "catalog:"])) }, { workspaces: { packages: ["packages/*"], catalog } }, ); + async function expectCatalog(dir: string, expected: Json) { + expect((await pkg(dir)).workspaces.catalog).toEqual(expected); + expect((await lock(dir)).catalog).toEqual(expected); + } + test("bun update", async () => { const dir = await setup(CATALOG_REPO()); await run(dir, "update"); - const expected = { "no-deps": "^1.1.0", aliased: "npm:no-deps@^2.0.0" }; - expect((await pkg(dir)).workspaces.catalog).toEqual(expected); - expect((await lock(dir)).catalog).toEqual(expected); + await expectCatalog(dir, { "no-deps": "^1.1.0", aliased: "npm:no-deps" }); expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "catalog:", aliased: "catalog:" }); await expectInSync(dir, ["", PKG1]); }); - test.each([[""], [PKG1]])("bun update --latest from '%s' installs offline afterwards", async cwd => { + test.each([[""], [PKG1]])("bun update --latest from '%s'", async cwd => { const dir = await setup(CATALOG_REPO()); - await run(join(dir, cwd), "update", "--latest"); - const expected = { "no-deps": "^2.0.0", aliased: "npm:no-deps@^2.0.0" }; - expect((await pkg(dir)).workspaces.catalog).toEqual(expected); - const lockfile = await lock(dir); - expect(lockfile.catalog).toEqual(expected); - expect(JSON.stringify(lockfile)).not.toContain('"latest"'); + await runIn(dir, cwd, "update", "--latest"); + await expectCatalog(dir, { "no-deps": "^2.0.0", aliased: "npm:no-deps@^2.0.0" }); + expect(await lockText(dir)).not.toContain('"latest"'); await expectInSync(dir, ["", PKG1]); + }); - await write( - join(dir, "bunfig.toml"), - Bun.TOML.stringify({ - install: { cache: join(dir, ".bun-cache"), registry: "http://127.0.0.1:1/", saveTextLockfile: true }, - }), - ); - await run(join(dir, cwd), "install", "--frozen-lockfile"); + test("bun update --latest keeps a dist-tag catalog entry", async () => { + const dir = await setup(CATALOG_REPO({ "dep-with-tags": "pre-2" })); + await run(dir, "update", "--latest"); + await expectCatalog(dir, { "dep-with-tags": "pre-2" }); + expect(await installed(dir, "dep-with-tags")).toMatchObject({ version: "2.0.1" }); + await expectInSync(dir, ["", PKG1]); + }); + + test("bun update keeps a 1.x catalog entry", async () => { + const dir = await setup(CATALOG_REPO({ "no-deps": "1.x" })); + await run(dir, "update"); + await expectCatalog(dir, { "no-deps": "1.x" }); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "catalog:" }); + await expectInSync(dir, ["", PKG1]); + }); + + test("bun update keeps a * catalog entry and moves only bun.lock", async () => { + const dir = await setup(CATALOG_REPO({ "no-deps": "1.0.0" })); + await reinstall(dir, wsRoot({ workspaces: { packages: ["packages/*"], catalog: { "no-deps": "*" } } })); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.0"]); + await run(dir, "update"); + await expectCatalog(dir, { "no-deps": "*" }); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@2.0.0"]); + expect((await pkg(dir, PKG1)).dependencies).toEqual({ "no-deps": "catalog:" }); + await expectInSync(dir, ["", PKG1]); }); test("bun add --catalog --filter", async () => { @@ -448,8 +694,8 @@ describe.concurrent("catalogs", () => { }); describe.concurrent("$ref overrides", () => { - test("$name follows the rewritten dependency", async () => { - const overrides = { "no-deps": "$no-deps" }; + test("$name follows the rewritten dependency; a literal override stays as written", async () => { + const overrides = { "no-deps": "$no-deps", "one-range-dep": "1.0.0" }; const dir = await setup({ "package.json": root({ dependencies: { "no-deps": "1.0.0", "one-range-dep": "1.0.0" }, overrides }), }); @@ -458,19 +704,7 @@ describe.concurrent("$ref overrides", () => { const manifest = await pkg(dir); expect(manifest.dependencies).toEqual({ "no-deps": "^1.1.0", "one-range-dep": "1.0.0" }); expect(manifest.overrides).toEqual(overrides); - expect((await lock(dir)).overrides).toEqual({ "no-deps": "^1.1.0" }); - await expectInSync(dir); - }); - - test.each([["^1.0.1"], ["^1.0.0"]])("literal override %s stays as written", async override => { - const dir = await setup({ - "package.json": root({ dependencies: { "no-deps": "^1.0.0" }, overrides: { "no-deps": override } }), - }); - await run(dir, "update"); - const manifest = await pkg(dir); - expect(manifest.dependencies).toEqual({ "no-deps": "^1.1.0" }); - expect(manifest.overrides).toEqual({ "no-deps": override }); - expect((await lock(dir)).overrides).toEqual({ "no-deps": override }); + expect((await lock(dir)).overrides).toEqual({ "no-deps": "^1.1.0", "one-range-dep": "1.0.0" }); await expectInSync(dir); }); @@ -487,7 +721,8 @@ describe.concurrent("$ref overrides", () => { describe.concurrent("bumping a direct dependency re-points its dependents", () => { const nested = (dir: string) => exists(join(dir, "node_modules", "one-range-dep", "node_modules")); - const deps = (noDeps: string) => root({ dependencies: { "one-range-dep": "1.0.0", "no-deps": noDeps } }); + const deps = (noDeps?: string) => + root({ dependencies: { "one-range-dep": "1.0.0", ...(noDeps === undefined ? {} : { "no-deps": noDeps }) } }); test.each([ ["text", true], @@ -498,7 +733,7 @@ describe.concurrent("bumping a direct dependency re-points its dependents", () = expect(await nested(dir)).toBe(false); await writePkg(dir, deps("1.0.1")); - await runBunInstall(bunEnv, dir); + await runBunInstall(envFor(dir), dir); expect(await installed(dir, "no-deps")).toMatchObject({ version: "1.0.1" }); expect(await nested(dir)).toBe(false); if (text) { @@ -508,26 +743,29 @@ describe.concurrent("bumping a direct dependency re-points its dependents", () = } await writePkg(dir, deps("2.0.0")); - await runBunInstall(bunEnv, dir); + await runBunInstall(envFor(dir), dir); expect(await installed(dir, "no-deps")).toMatchObject({ version: "2.0.0" }); expect(await installed(dir, "one-range-dep/node_modules/no-deps")).toMatchObject({ version: "1.0.1" }); - if (text) { - const { packages } = await lock(dir); - expect(packages["no-deps"][0]).toBe("no-deps@2.0.0"); - expect(packages["one-range-dep/no-deps"][0]).toBe("no-deps@1.0.1"); - } + if (!text) return; + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.1", "no-deps@2.0.0"]); + + await writePkg(dir, deps()); + await runBunInstall(envFor(dir), dir); + expect(await resolutions(dir, "no-deps")).toEqual(["no-deps@1.0.1"]); + + await writePkg(dir, deps("1.0.0")); + await runBunInstall(envFor(dir), dir); + const { packages } = await lock(dir); + expect(packages["no-deps"][0]).toBe("no-deps@1.0.0"); + expect(packages["one-range-dep/no-deps"][0]).toBe("no-deps@1.0.1"); + await expectInSync(dir); }); test("declared by a workspace member", async () => { const dir = await setup(MONOREPO({ dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.0" } })); expect((await lock(dir)).packages["no-deps"][0]).toBe("no-deps@1.0.0"); - await writePkg( - dir, - { name: "pkg1", version: "1.0.0", dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.1" } }, - PKG1, - ); - await runBunInstall(bunEnv, dir); + await reinstall(dir, member("pkg1", { dependencies: { "one-range-dep": "1.0.0", "no-deps": "1.0.1" } }), PKG1); const { packages } = await lock(dir); expect(packages["no-deps"][0]).toBe("no-deps@1.0.1"); expect(packages["one-range-dep/no-deps"]).toBeUndefined(); @@ -544,16 +782,4 @@ describe.concurrent("bumping a direct dependency re-points its dependents", () = expect(await nested(dir)).toBe(false); await expectInSync(dir); }); - - test("a dependency added later never drags dependents along", async () => { - const dir = await setup({ "package.json": root({ dependencies: { "one-range-dep": "1.0.0" } }) }); - expect((await lock(dir)).packages["no-deps"][0]).toBe("no-deps@1.1.0"); - - await writePkg(dir, deps("1.0.0")); - await runBunInstall(bunEnv, dir); - const { packages } = await lock(dir); - expect(packages["no-deps"][0]).toBe("no-deps@1.0.0"); - expect(packages["one-range-dep/no-deps"][0]).toBe("no-deps@1.1.0"); - await expectInSync(dir); - }); }); diff --git a/test/cli/install/bun-update-transitive.test.ts b/test/cli/install/bun-update-transitive.test.ts index 50dba8f44495..b2c6296f8ab2 100644 --- a/test/cli/install/bun-update-transitive.test.ts +++ b/test/cli/install/bun-update-transitive.test.ts @@ -3,9 +3,7 @@ import { afterAll, beforeAll, expect, test } from "bun:test"; import { VerdaccioRegistry, bunEnv, bunExe, normalizeBunSnapshot, tempDir } from "harness"; import { join } from "path"; -// Registry: no-deps 1.0.0 / 1.0.1 / 1.1.0 / 2.0.0; one-range-dep@1.0.0 depends on `no-deps: ^1.0.0`; -// one-fixed-dep@1.0.0 depends on `no-deps: 1.0.0`; dep-with-tags has 3.0.1 published above its `latest` (3.0.0); -// prereleases-1 has 1.0.0-future.7 published above its `latest` (1.0.0-future.4). +// Registry: no-deps 1.0.0/1.0.1/1.1.0/2.0.0, a-dep 1.0.1..1.0.10, @types/no-deps 1.0.0/2.0.0, one-range-dep@1.0.0 -> no-deps ^1.0.0, one-fixed-dep@1.0.0 -> no-deps 1.0.0. const registry = new VerdaccioRegistry(); @@ -26,11 +24,11 @@ const stringify = (json: Json) => JSON.stringify(json, null, 2) + "\n"; const linkerArgs = (layout: Layout) => ["--linker", layout.linker ?? "hoisted"]; -async function run(dir: string, ...args: string[]) { +async function runIn(dir: string, rel: string, ...args: string[]) { await using proc = Bun.spawn({ cmd: [bunExe(), ...args], - cwd: dir, - env: bunEnv, + cwd: join(dir, rel), + env: { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }, stdout: "pipe", stderr: "pipe", stdin: "ignore", @@ -39,6 +37,8 @@ async function run(dir: string, ...args: string[]) { return { stdout, stderr, exitCode }; } +const run = (dir: string, ...args: string[]) => runIn(dir, "", ...args); + async function install(dir: string, ...args: string[]) { const { stderr, exitCode } = await run(dir, "install", ...args); expect(stderr).not.toContain("error:"); @@ -46,6 +46,8 @@ async function install(dir: string, ...args: string[]) { return stderr; } +const frozen = (dir: string, layout: Layout = {}) => install(dir, "--frozen-lockfile", ...linkerArgs(layout)); + async function setup(files: Record, layout: Layout = {}) { const { packageDir } = await registry.createTestDir({ bunfigOpts: { saveTextLockfile: layout.text ?? true, linker: layout.linker ?? "hoisted" }, @@ -81,8 +83,7 @@ async function installedVersion(dir: string, ...segments: string[]) { const noDepsPath = (linker: Linker = "hoisted") => linker === "isolated" ? [".bun", "one-range-dep@1.0.0", "node_modules", "no-deps"] : ["no-deps"]; -// no-deps@1.0.0 survives being dropped from package.json because one-range-dep's `^1.0.0` edge is still satisfied, -// leaving a transitive dependency that a plain `bun install` never moves. +// Dropping the root's exact no-deps@1.0.0 leaves one-range-dep's `^1.0.0` edge on 1.0.0, which `bun install` never moves. async function stale(layout: Layout = {}) { const dir = await setup({ "package.json": pkgJson({ "one-range-dep": "1.0.0", "no-deps": "1.0.0" }) }, layout); const packageJson = pkgJson({ "one-range-dep": "1.0.0" }); @@ -109,11 +110,62 @@ async function expectTransitiveBump( expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); expect(await lockedVersions(dir, "one-range-dep")).toStrictEqual(["1.0.0"]); } - await install(dir, "--frozen-lockfile", ...linkerArgs(layout)); + await frozen(dir, layout); expect(exitCode).toBe(0); return stdout; } +type Groups = Record; + +function grouped(versions: Json, groups: Groups) { + const json: Json = { name: "foo" }; + for (const [name, version] of Object.entries(versions)) (json[groups[name] ?? "dependencies"] ??= {})[name] = version; + return json; +} + +// Exact pins widened to ranges after the install: both entries stay locked below the newest version their range allows. +async function staleSiblings(groups: Groups = {}) { + const dir = await setup({ "package.json": grouped({ "no-deps": "1.0.0", "a-dep": "1.0.1" }, groups) }); + const packageJson = grouped({ "no-deps": "^1.0.0", "a-dep": "^1.0.1" }, groups); + await reinstall(dir, packageJson); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + expect(await lockedVersions(dir, "a-dep")).toStrictEqual(["1.0.1"]); + return { dir, packageJson, groups }; +} + +async function expectOnlyADepMoved({ dir, groups }: Awaited>, ...args: string[]) { + const { stderr, exitCode } = await run(dir, "update", ...args); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(grouped({ "no-deps": "^1.0.0", "a-dep": "^1.0.10" }, groups)); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + expect(await lockedVersions(dir, "a-dep")).toStrictEqual(["1.0.10"]); + await frozen(dir); + expect(exitCode).toBe(0); +} + +async function expectRejected(dir: string, message: string, ...args: string[]) { + const packageJsonBefore = await packageJsonText(dir); + const lockBefore = await lockText(dir); + const { stdout, stderr, exitCode } = await run(dir, "update", ...args); + expect(stderr).toContain(message); + expect(await packageJsonText(dir)).toBe(packageJsonBefore); + expect(await lockText(dir)).toBe(lockBefore); + expect(exitCode).toBe(1); + return { stdout, stderr }; +} + +async function expectNothingToUpdate(dir: string, ...args: string[]) { + const before = await lockText(dir); + const { stdout, stderr, exitCode } = await run(dir, "update", ...args); + expect(stdout).toContain("No packages to update"); + expect(stderr).not.toContain("error:"); + expect(await lockText(dir)).toBe(before); + expect(exitCode).toBe(0); +} + +const ROOT = { name: "root", workspaces: ["packages/*"] }; +const member = (name: string, dependencies: Json = {}) => ({ name, version: "1.0.0", dependencies }); + test.concurrent.each<[string, Layout]>([ ["text lockfile", { text: true }], ["binary lockfile", { text: false }], @@ -140,6 +192,15 @@ test.concurrent("`bun update ` reaches a package that is only a transitive expect(stdout).not.toContain("updating:"); }); +test.concurrent.each([ + ["a pattern", ["no-*"]], + ["a pattern alongside a direct name", ["no-d*", "one-range-dep"]], +])("`bun update` with %s reaches a package that is only a transitive dependency", async (_, args) => { + const fixture = await stale(); + const stdout = await expectTransitiveBump(fixture, {}, ...args); + expect(stdout).not.toContain("updating:"); +}); + test.concurrent("`bun update ` naming a package with nothing newer changes nothing", async () => { const { dir, packageJson } = await stale(); const before = await lock(dir); @@ -176,10 +237,13 @@ test.concurrent("`bun update --dry-run` prints the transitive plan and writes no }); test.concurrent("a direct dependency's declared range is left alone when only its dependency moves", async () => { - const { dir } = await stale(); - await run(dir, "update"); - expect(await packageJsonOf(dir)).toStrictEqual(pkgJson({ "one-range-dep": "1.0.0" })); - expect((await lock(dir)).workspaces[""].dependencies).toStrictEqual({ "one-range-dep": "1.0.0" }); + const { dir, packageJson } = await stale(); + const { stderr, exitCode } = await run(dir, "update"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect((await lock(dir)).workspaces[""].dependencies).toStrictEqual(packageJson.dependencies); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + expect(exitCode).toBe(0); }); test.concurrent("a transitive dependency pinned exactly by its dependent stays put", async () => { @@ -193,8 +257,7 @@ test.concurrent("a transitive dependency pinned exactly by its dependent stays p expect(exitCode).toBe(0); }); -// A root `no-deps@1.0.0` dedupes both dependents' edges onto 1.0.0 before it is dropped (a fresh install would already -// fork); `bun update` then moves only the `^1.0.0` edge, forking away from the sibling's exact pin. +// The root's no-deps@1.0.0 dedupes both dependents onto 1.0.0 before it is dropped; the update forks only the `^1.0.0` edge. test.concurrent("dependents with different ranges are resolved independently", async () => { const dependents = { "one-fixed-dep": "1.0.0", "one-range-dep": "1.0.0" }; const dir = await setup({ "package.json": pkgJson({ "no-deps": "1.0.0", ...dependents }) }); @@ -211,7 +274,7 @@ test.concurrent("dependents with different ranges are resolved independently", a "no-deps@1.0.0", "no-deps@1.1.0", ]); - await install(dir, "--frozen-lockfile"); + await frozen(dir); expect(exitCode).toBe(0); }); @@ -252,28 +315,50 @@ test.concurrent("without a lockfile `bun update` resolves everything fresh", asy expect(exitCode).toBe(0); }); -test.concurrent("in a workspace, `bun update` from the root moves a member's transitive dependency", async () => { - const root = { name: "root", workspaces: ["packages/*"] }; - const member = (dependencies: Json) => ({ name: "pkg1", version: "1.0.0", dependencies }); +// The `stale()` recipe applied inside pkg1; pkg2, when present, pins no-deps@1.0.0 too and is widened to `pkg2Range`. +async function staleMemberTransitive(pkg2Range?: string) { const dir = await setup({ - "package.json": root, - "packages/pkg1/package.json": member({ "one-range-dep": "1.0.0", "no-deps": "1.0.0" }), + "package.json": ROOT, + "packages/pkg1/package.json": member("pkg1", { "one-range-dep": "1.0.0", "no-deps": "1.0.0" }), + ...(pkg2Range ? { "packages/pkg2/package.json": member("pkg2", { "no-deps": "1.0.0" }) } : {}), }); - const pkg1 = member({ "one-range-dep": "1.0.0" }); + if (pkg2Range) { + await write(join(dir, "packages/pkg2/package.json"), stringify(member("pkg2", { "no-deps": pkg2Range }))); + } + const pkg1 = member("pkg1", { "one-range-dep": "1.0.0" }); await reinstall(dir, pkg1, {}, "packages/pkg1"); expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + return { dir, pkg1 }; +} +test.concurrent("in a workspace, `bun update` from the root moves a member's transitive dependency", async () => { + const { dir, pkg1 } = await staleMemberTransitive(); const { stdout, stderr, exitCode } = await run(dir, "update"); expect(stdout).toContain(" no-deps@1.0.0 → 1.1.0\n"); expect(stderr).not.toContain("error:"); - expect(await packageJsonOf(dir)).toStrictEqual(root); + expect(await packageJsonOf(dir)).toStrictEqual(ROOT); expect(await packageJsonOf(dir, "packages/pkg1")).toStrictEqual(pkg1); expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); expect(await installedVersion(dir, "no-deps")).toBe("1.1.0"); - await install(dir, "--frozen-lockfile"); + await frozen(dir); + expect(exitCode).toBe(0); +}); + +test.concurrent.each([ + ["the root", ""], + ["the member", "packages/pkg1"], +])("`bun update ` run from %s still moves a member's transitive dependency", async (_, cwd) => { + const { dir, pkg1 } = await staleMemberTransitive(); + const { stderr, exitCode } = await runIn(dir, cwd, "update", "no-deps"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(ROOT); + expect(await packageJsonOf(dir, "packages/pkg1")).toStrictEqual(pkg1); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + await frozen(dir); expect(exitCode).toBe(0); }); +// dep-with-tags has 3.0.1 published above its `latest` (3.0.0); prereleases-1 has 1.0.0-future.7 above 1.0.0-future.4. test.concurrent.each([ ["dep-with-tags", "3.0.1"], ["prereleases-1", "1.0.0-future.7"], @@ -288,17 +373,41 @@ test.concurrent.each([ expect(await lockedVersions(dir, name)).toStrictEqual([version]); expect(await lockText(dir)).not.toContain('"latest"'); expect(await installedVersion(dir, name)).toBe(version); - await install(dir, "--frozen-lockfile"); + await frozen(dir); expect(exitCode).toBe(0); }); -test.concurrent("`bun update --help` no longer offers a transitive flag", async () => { +test.concurrent("`bun up --help` prints the update help", async () => { const { packageDir } = await registry.createTestDir(); - const { stdout, exitCode } = await run(packageDir, "update", "--help"); + const { stdout, exitCode } = await run(packageDir, "up", "--help"); + expect(stdout).toContain("bun update"); + expect(stdout).toContain("Alias: bun up"); + expect(stdout).toContain("-L, --latest"); + expect(stdout).toContain("--no-optional"); expect(stdout).not.toContain("--transitive"); expect(exitCode).toBe(0); }); +test.concurrent("`bun up` is `bun update`", async () => { + const { dir, packageJson } = await stale(); + const { stdout, stderr, exitCode } = await run(dir, "up"); + expect(stdout).toContain(" no-deps@1.0.0 → 1.1.0\n"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(packageJson); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + expect(exitCode).toBe(0); +}); + +test.concurrent("`-L` is `--latest`", async () => { + const dir = await setup({ "package.json": pkgJson({ "no-deps": "~1.0.0" }) }); + const { stderr, exitCode } = await run(dir, "update", "-L"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(pkgJson({ "no-deps": "~2.0.0" })); + expect(await installedVersion(dir, "no-deps")).toBe("2.0.0"); + await frozen(dir); + expect(exitCode).toBe(0); +}); + test.concurrent("`bun update --silent` prints no plan but still moves the transitive dependency", async () => { const { dir, packageJson, noDeps } = await stale(); const { stdout, stderr, exitCode } = await run(dir, "update", "--silent"); @@ -307,15 +416,18 @@ test.concurrent("`bun update --silent` prints no plan but still moves the transi expect(await packageJsonOf(dir)).toStrictEqual(packageJson); expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); expect(await installedVersion(dir, ...noDeps)).toBe("1.1.0"); - await install(dir, "--frozen-lockfile"); + await frozen(dir); expect(exitCode).toBe(0); }); -test.concurrent("`bun update --no-save` moves the transitive dependency in node_modules only", async () => { - const { dir, noDeps } = await stale(); +test.concurrent.each<[string, Layout]>([ + ["hoisted", {}], + ["isolated", { linker: "isolated" }], +])("`bun update --no-save` moves the transitive dependency in node_modules only (%s)", async (_, layout) => { + const { dir, noDeps } = await stale(layout); const packageJsonBefore = await packageJsonText(dir); const lockBefore = await lockText(dir); - const { stdout, stderr, exitCode } = await run(dir, "update", "--no-save"); + const { stdout, stderr, exitCode } = await run(dir, "update", "--no-save", ...linkerArgs(layout)); expect(stdout).toContain(" no-deps@1.0.0 → 1.1.0\n"); expect(stderr).not.toContain("error:"); expect(stderr).not.toContain("Saved lockfile"); @@ -333,7 +445,7 @@ async function expectNoop(dir: string, ...args: string[]) { expect(stderr).not.toContain("error:"); expect(await packageJsonOf(dir)).toStrictEqual(packageJson); expect(await lockText(dir)).toBe(before); - await install(dir, "--frozen-lockfile"); + await frozen(dir); expect(exitCode).toBe(0); } @@ -359,7 +471,7 @@ test.concurrent.each([ expect(await packageJsonOf(dir)).toStrictEqual(packageJson); expect((await lock(dir)).workspaces[""].dependencies).toStrictEqual(HOIST_DEPENDENTS); expect(await lockedVersions(dir, "hoist-lockfile-shared")).toStrictEqual(["1.0.2", "2.0.2"]); - await install(dir, "--frozen-lockfile"); + await frozen(dir); expect(exitCode).toBe(0); }); @@ -432,30 +544,32 @@ test.concurrent.each([ expect(exitCode).toBe(0); }); -// pkg1 and pkg2 declare the same range; only the workspace `bun update` runs in gets its package.json rewritten. -test.concurrent("in a workspace, `bun update` from one member also re-points a sibling's identical range", async () => { - const root = { name: "root", workspaces: ["packages/*"] }; - const member = (name: string, range: string) => ({ name, version: "1.0.0", dependencies: { "no-deps": range } }); +// pkg1 and pkg2 both install no-deps@1.0.0 exactly, then widen to the given ranges, which keep 1.0.0 locked for both. +async function staleMembers(pkg1Range: string, pkg2Range: string) { const dir = await setup({ - "package.json": root, - "packages/pkg1/package.json": member("pkg1", "1.0.0"), - "packages/pkg2/package.json": member("pkg2", "1.0.0"), + "package.json": ROOT, + "packages/pkg1/package.json": member("pkg1", { "no-deps": "1.0.0" }), + "packages/pkg2/package.json": member("pkg2", { "no-deps": "1.0.0" }), }); - await write(join(dir, "packages/pkg2/package.json"), stringify(member("pkg2", "~1.0.0"))); - await reinstall(dir, member("pkg1", "~1.0.0"), {}, "packages/pkg1"); + await write(join(dir, "packages/pkg2/package.json"), stringify(member("pkg2", { "no-deps": pkg2Range }))); + await reinstall(dir, member("pkg1", { "no-deps": pkg1Range }), {}, "packages/pkg1"); expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + return { dir, pkg2Text: await packageJsonText(dir, "packages/pkg2") }; +} - const { stderr, exitCode } = await run(join(dir, "packages/pkg1"), "update"); +test.concurrent("in a workspace, `bun update` from one member also re-points a sibling's identical range", async () => { + const { dir } = await staleMembers("~1.0.0", "~1.0.0"); + const { stderr, exitCode } = await runIn(dir, "packages/pkg1", "update"); expect(stderr).not.toContain("error:"); - expect(await packageJsonOf(dir)).toStrictEqual(root); - expect(await packageJsonOf(dir, "packages/pkg1")).toStrictEqual(member("pkg1", "~1.0.1")); - expect(await packageJsonOf(dir, "packages/pkg2")).toStrictEqual(member("pkg2", "~1.0.0")); + expect(await packageJsonOf(dir)).toStrictEqual(ROOT); + expect(await packageJsonOf(dir, "packages/pkg1")).toStrictEqual(member("pkg1", { "no-deps": "~1.0.1" })); + expect(await packageJsonOf(dir, "packages/pkg2")).toStrictEqual(member("pkg2", { "no-deps": "~1.0.0" })); const { workspaces } = await lock(dir); expect(workspaces["packages/pkg1"].dependencies).toStrictEqual({ "no-deps": "~1.0.1" }); expect(workspaces["packages/pkg2"].dependencies).toStrictEqual({ "no-deps": "~1.0.0" }); expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.1"]); expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); - await install(dir, "--frozen-lockfile"); + await frozen(dir); expect(exitCode).toBe(0); }); @@ -519,7 +633,7 @@ test.concurrent("`bun update ` leaves the named package's own dependencies expect(await lockedVersions(dir, "leaf")).toStrictEqual(["1.0.0"]); expect(await installedVersion(dir, "parent")).toBe("1.1.0"); expect(await installedVersion(dir, "leaf")).toBe("1.0.0"); - await install(dir, "--frozen-lockfile"); + await frozen(dir); expect(named.exitCode).toBe(0); const bare = await run(dir, "update"); @@ -528,6 +642,212 @@ test.concurrent("`bun update ` leaves the named package's own dependencies expect(await packageJsonOf(dir)).toStrictEqual(pkgJson({ parent: "^1.1.0" })); expect(await lockedVersions(dir, "leaf")).toStrictEqual(["1.1.0"]); expect(await installedVersion(dir, "leaf")).toBe("1.1.0"); - await install(dir, "--frozen-lockfile"); + await frozen(dir); expect(bare.exitCode).toBe(0); }); + +test.concurrent("`bun update ` from a member leaves a sibling's own entry alone but lets it follow", async () => { + const { dir, pkg2Text } = await staleMembers("~1.0.0", "^1.0.0"); + const { stderr, exitCode } = await runIn(dir, "packages/pkg1", "update", "no-deps"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir, "packages/pkg1")).toStrictEqual(member("pkg1", { "no-deps": "~1.0.1" })); + expect(await packageJsonText(dir, "packages/pkg2")).toBe(pkg2Text); + expect((await lock(dir)).workspaces["packages/pkg2"].dependencies).toStrictEqual({ "no-deps": "^1.0.0" }); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.1"]); + await frozen(dir); + expect(exitCode).toBe(0); +}); + +test.concurrent( + "`bun update ` from a member: a sibling whose range rejects the picked version stays put", + async () => { + const { dir, pkg1 } = await staleMemberTransitive("~1.0.0"); + const pkg2Text = await packageJsonText(dir, "packages/pkg2"); + const { stderr, exitCode } = await runIn(dir, "packages/pkg1", "update", "no-deps"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir, "packages/pkg1")).toStrictEqual(pkg1); + expect(await packageJsonText(dir, "packages/pkg2")).toBe(pkg2Text); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0", "1.1.0"]); + await frozen(dir); + expect(exitCode).toBe(0); + }, +); + +test.concurrent("`bun update ` from the root does not re-resolve a member's own entry", async () => { + const root = { ...ROOT, dependencies: { "no-deps": "^2.0.0" } }; + const dir = await setup({ + "package.json": root, + "packages/pkg1/package.json": member("pkg1", { "no-deps": "1.0.0" }), + }); + await reinstall(dir, member("pkg1", { "no-deps": "^1.0.0" }), {}, "packages/pkg1"); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0", "2.0.0"]); + const rootText = await packageJsonText(dir); + const pkg1Text = await packageJsonText(dir, "packages/pkg1"); + + const fromRoot = await run(dir, "update", "no-deps"); + expect(fromRoot.stderr).not.toContain("error:"); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0", "2.0.0"]); + expect(await packageJsonText(dir)).toBe(rootText); + expect(await packageJsonText(dir, "packages/pkg1")).toBe(pkg1Text); + expect(fromRoot.exitCode).toBe(0); + + const fromMember = await runIn(dir, "packages/pkg1", "update", "no-deps"); + expect(fromMember.stderr).not.toContain("error:"); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0", "2.0.0"]); + expect(await packageJsonText(dir)).toBe(rootText); + expect(await packageJsonOf(dir, "packages/pkg1")).toStrictEqual(member("pkg1", { "no-deps": "^1.1.0" })); + await frozen(dir); + expect(fromMember.exitCode).toBe(0); +}); + +test.concurrent("`bun update ` for a name only other workspaces depend on is an error", async () => { + const dir = await setup({ + "package.json": { ...ROOT, dependencies: { "no-deps": "1.0.0" } }, + "packages/pkg1/package.json": member("pkg1"), + }); + await reinstall(dir, { ...ROOT, dependencies: { "no-deps": "^1.0.0" } }); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + const rootText = await packageJsonText(dir); + const lockBefore = await lockText(dir); + const { stderr, exitCode } = await runIn(dir, "packages/pkg1", "update", "no-deps"); + expect(stderr).toContain( + 'error: "no-deps" is only a dependency of other workspaces, so there is nothing to update here', + ); + expect(stderr).toContain("bun update -r no-deps"); + expect(await lockText(dir)).toBe(lockBefore); + expect(await packageJsonText(dir)).toBe(rootText); + expect(exitCode).toBe(1); +}); + +async function staleScoped() { + const dir = await setup({ "package.json": pkgJson({ "no-deps": "1.0.0", "@types/no-deps": "^1.0.0" }) }); + await reinstall(dir, pkgJson({ "no-deps": "^1.0.0", "@types/no-deps": "^1.0.0" })); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + expect(await lockedVersions(dir, "@types/no-deps")).toStrictEqual(["1.0.0"]); + return dir; +} + +test.concurrent("a scoped glob selects only the matching names", async () => { + const dir = await staleScoped(); + const { stderr, exitCode } = await run(dir, "update", "--latest", "@types/*"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(pkgJson({ "no-deps": "^1.0.0", "@types/no-deps": "^2.0.0" })); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.0.0"]); + expect(await lockedVersions(dir, "@types/no-deps")).toStrictEqual(["2.0.0"]); + await frozen(dir); + expect(exitCode).toBe(0); +}); + +test.concurrent("a bare `*` names everything, scoped names included", async () => { + const dir = await staleScoped(); + const { stderr, exitCode } = await run(dir, "update", "--latest", "*"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(pkgJson({ "no-deps": "^2.0.0", "@types/no-deps": "^2.0.0" })); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["2.0.0"]); + await frozen(dir); + expect(exitCode).toBe(0); +}); + +test.concurrent.each([ + ["a negated pattern updates everything else", "!no-deps"], + ["an unscoped glob", "a-*"], +])("%s", async (_, pattern) => { + await expectOnlyADepMoved(await staleSiblings(), pattern); +}); + +test.concurrent("a pattern that matches nothing is an error", async () => { + const { dir } = await staleSiblings(); + await expectRejected(dir, 'error: no packages in bun.lock match "zzz-*"', "zzz-*"); +}); + +test.concurrent("a version cannot be combined with a pattern", async () => { + const dir = await staleScoped(); + await expectRejected(dir, "a version cannot be combined with a pattern: @types/*@2", "@types/*@2"); +}); + +test.concurrent("excluding every package is a no-op", async () => { + const dir = await setup({ "package.json": pkgJson({ "no-deps": "^1.0.0" }) }); + await expectNothingToUpdate(dir, "!no-deps"); +}); + +test.concurrent("patterns need a lockfile", async () => { + const { packageDir } = await registry.createTestDir({ + bunfigOpts: { saveTextLockfile: true, linker: "hoisted" }, + files: { "package.json": stringify(pkgJson({ "no-deps": "^1.0.0" })) }, + }); + const { stderr, exitCode } = await run(packageDir, "update", "no-*"); + expect(stderr).toContain("missing lockfile, nothing to update"); + expect(exitCode).toBe(1); +}); + +const DEV_A_DEP: Groups = { "a-dep": "devDependencies" }; + +test.concurrent.each(["--dev", "-D", "-d", "--development"])( + "`bun update %s` only touches devDependencies", + async flag => { + await expectOnlyADepMoved(await staleSiblings(DEV_A_DEP), flag); + }, +); + +test.concurrent.each(["--prod", "-P", "--production", "-p"])( + "`bun update %s` only touches dependencies and still installs devDependencies", + async flag => { + const { dir } = await staleSiblings(DEV_A_DEP); + const { stderr, exitCode } = await run(dir, "update", flag); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(grouped({ "no-deps": "^1.1.0", "a-dep": "^1.0.1" }, DEV_A_DEP)); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + expect(await lockedVersions(dir, "a-dep")).toStrictEqual(["1.0.1"]); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.1"); + await frozen(dir); + expect(exitCode).toBe(0); + }, +); + +test.concurrent("`bun update --prod` includes optionalDependencies", async () => { + const groups: Groups = { "a-dep": "optionalDependencies" }; + const { dir } = await staleSiblings(groups); + const { stderr, exitCode } = await run(dir, "update", "--prod"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(grouped({ "no-deps": "^1.1.0", "a-dep": "^1.0.10" }, groups)); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + expect(await lockedVersions(dir, "a-dep")).toStrictEqual(["1.0.10"]); + await frozen(dir); + expect(exitCode).toBe(0); +}); + +test.concurrent("`bun update --no-optional` leaves optionalDependencies alone", async () => { + const groups: Groups = { "a-dep": "optionalDependencies" }; + const { dir } = await staleSiblings(groups); + const { stderr, exitCode } = await run(dir, "update", "--no-optional"); + expect(stderr).not.toContain("error:"); + expect(await packageJsonOf(dir)).toStrictEqual(grouped({ "no-deps": "^1.1.0", "a-dep": "^1.0.1" }, groups)); + expect(await lockedVersions(dir, "no-deps")).toStrictEqual(["1.1.0"]); + expect(await lockedVersions(dir, "a-dep")).toStrictEqual(["1.0.1"]); + await frozen(dir); + expect(exitCode).toBe(0); +}); + +test.concurrent("a selector with a name outside the selected groups is an error", async () => { + const { dir } = await staleSiblings(DEV_A_DEP); + await expectRejected(dir, 'no dependencies in the selected groups match "no-deps"', "--dev", "no-deps"); +}); + +test.concurrent("a selector combined with `--latest` only rewrites the selected groups", async () => { + await expectOnlyADepMoved(await staleSiblings(DEV_A_DEP), "--dev", "--latest"); +}); + +test.concurrent("a selector matching nothing is a no-op", async () => { + const dir = await setup({ "package.json": pkgJson({ "no-deps": "^1.0.0" }) }); + await expectNothingToUpdate(dir, "--dev"); +}); + +test.concurrent("a version cannot be combined with a selector", async () => { + const { dir } = await staleSiblings(DEV_A_DEP); + await expectRejected( + dir, + "a version cannot be combined with --dev, --prod or --no-optional: a-dep@1", + "--dev", + "a-dep@1", + ); +}); diff --git a/test/cli/install/bun-update.test.ts b/test/cli/install/bun-update.test.ts index 182b93ecf1d6..7dca31ef3f12 100644 --- a/test/cli/install/bun-update.test.ts +++ b/test/cli/install/bun-update.test.ts @@ -2,7 +2,7 @@ import { file, spawn } from "bun"; import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from "bun:test"; import { access, mkdir, readFile, rm, writeFile } from "fs/promises"; import { VerdaccioRegistry, bunExe, bunEnv as env, pack, readdirSorted, toBeValidBin, toHaveBins } from "harness"; -import { basename, join } from "path"; +import { basename, dirname, join } from "path"; import { dummyAfterAll, dummyAfterEach, @@ -27,6 +27,26 @@ expect.extend({ toHaveBins, }); +async function runInstall(cwd = package_dir, ...args: string[]) { + const { stderr, exited } = spawn({ + cmd: [bunExe(), "install", "--linker=hoisted", ...args], + cwd, + stdout: "ignore", + stderr: "pipe", + env, + }); + const [err, code] = await Promise.all([stderr.text(), exited]); + expect(err).not.toContain("error:"); + expect(code).toBe(0); +} + +async function writeTextLockfileBunfig() { + await writeFile( + join(package_dir, "bunfig.toml"), + `[install]\ncache = false\nregistry = "${root_url}/"\nsaveTextLockfile = true\nlinker = "hoisted"\n`, + ); +} + for (const { input } of [{ input: { baz: "~0.0.3", moo: "~0.1.0" } }]) { it(`should update to latest version of dependency (${input.baz[0]})`, async () => { const urls: string[] = []; @@ -373,116 +393,6 @@ it("lockfile should not be modified when there are no version changes, issue#588 expect(urls).toHaveLength(count); }); -it("should support catalog versions in update", async () => { - const urls: string[] = []; - setHandler(dummyRegistry(urls)); - - // Create a monorepo with catalog - await writeFile( - join(package_dir, "package.json"), - JSON.stringify({ - name: "root", - catalog: { - "no-deps": "^1.0.0", - }, - workspaces: ["packages/*"], - }), - ); - - await mkdir(join(package_dir, "packages", "workspace-a"), { recursive: true }); - await writeFile( - join(package_dir, "packages", "workspace-a", "package.json"), - JSON.stringify({ - name: "workspace-a", - dependencies: { - "no-deps": "catalog:", - }, - }), - ); - - // Test that update works with catalog dependencies - const { stdout, stderr, exited } = spawn({ - cmd: [bunExe(), "update", "--dry-run"], - cwd: join(package_dir, "packages", "workspace-a"), - stdout: "pipe", - stderr: "pipe", - env, - }); - - const err = await new Response(stderr).text(); - const out = await new Response(stdout).text(); - - // Should not crash with catalog dependencies - expect(err).not.toContain("panic"); - expect(err).not.toContain("segfault"); - - // Verify catalog reference is preserved in package.json - const pkg = await file(join(package_dir, "packages", "workspace-a", "package.json")).json(); - expect(pkg.dependencies["no-deps"]).toBe("catalog:"); -}); - -it("should support --recursive flag", async () => { - // First verify the flag appears in help - const { - stdout: helpOut, - stderr: helpErr, - exited: helpExited, - } = spawn({ - cmd: [bunExe(), "update", "--help"], - cwd: package_dir, - stdout: "pipe", - stderr: "pipe", - env, - }); - - const help = (await new Response(helpOut).text()) + (await new Response(helpErr).text()); - expect(await helpExited).toBe(0); - expect(help).toContain("--recursive"); - expect(help).toContain("-r"); - - // Now test that --recursive actually works - await writeFile( - join(package_dir, "package.json"), - JSON.stringify({ - name: "root", - workspaces: ["packages/*"], - dependencies: { - "no-deps": "^1.0.0", - }, - }), - ); - - await mkdir(join(package_dir, "packages", "pkg1"), { recursive: true }); - await writeFile( - join(package_dir, "packages", "pkg1", "package.json"), - JSON.stringify({ - name: "pkg1", - dependencies: { - "no-deps": "^1.0.0", - }, - }), - ); - - // Test recursive update (might fail without lockfile, but shouldn't crash) - const { stdout, stderr, exited } = spawn({ - cmd: [bunExe(), "update", "--recursive", "--dry-run"], - cwd: package_dir, - stdout: "pipe", - stderr: "pipe", - env, - }); - - const out = await new Response(stdout).text(); - const err = await new Response(stderr).text(); - - // Should not crash - expect(err).not.toContain("panic"); - expect(err).not.toContain("segfault"); - - // Should recognize the flag (either process workspaces or show error about missing lockfile) - expect(out + err).toMatch(/bun update|missing lockfile|nothing to update/); -}); - // https://github.com/oven-sh/bun/issues/33176 it("--recursive updates dependencies and peerDependencies in workspace members", async () => { const urls: string[] = []; @@ -686,67 +596,105 @@ it("--filter with multiple patterns selects the union of matching workspaces", a }); // https://github.com/oven-sh/bun/issues/33176 -// `bun update ` re-resolves every `` entry in the lockfile, but -// only the cwd package.json is rewritten. Named updates don't fan the -// package.json edit out to members. -// `bun update ` already re-resolves in every workspace, so the workspace selectors are rejected rather than silently ignored. -for (const flags of [["--recursive"], ["--filter", "pkg-a"]]) { - it(`named update rejects ${flags[0]}`, async () => { - const urls: string[] = []; - setHandler(dummyRegistry(urls, { "0.0.3": {}, "0.0.5": {}, latest: "0.0.5" })); +const FAN_OUT_FILES = { + "package.json": { name: "root", private: true, workspaces: ["packages/*"], dependencies: { baz: "0.0.3" } }, + "packages/pkg-a/package.json": { name: "pkg-a", dependencies: { baz: "~0.0.3" } }, + "packages/pkg-b/package.json": { name: "pkg-b", devDependencies: { baz: "^0.0.3" } }, + "packages/pkg-c/package.json": { name: "pkg-c" }, +}; + +async function fanOutTexts() { + const texts: string[] = []; + for (const rel of Object.keys(FAN_OUT_FILES)) texts.push(await file(join(package_dir, rel)).text()); + return texts; +} - await writeFile( - join(package_dir, "package.json"), - JSON.stringify({ - name: "root", - private: true, - workspaces: ["packages/*"], - dependencies: { baz: "~0.0.3" }, - }), - ); - await mkdir(join(package_dir, "packages", "pkg-a"), { recursive: true }); - await writeFile( - join(package_dir, "packages", "pkg-a", "package.json"), - JSON.stringify({ name: "pkg-a", dependencies: { baz: "~0.0.3" } }), - ); +const fanOutJson = (rel: keyof typeof FAN_OUT_FILES) => file(join(package_dir, rel)).json(); - const { stderr, exited } = spawn({ - cmd: [bunExe(), "update", "baz", ...flags, "--linker=hoisted"], - cwd: package_dir, - stdout: "ignore", - stderr: "pipe", - env, - }); - expect(await new Response(stderr).text()).toContain( - "error: --recursive and --filter cannot be combined with package names", - ); - expect(await exited).not.toBe(0); +// Root pins baz exactly, pkg-a uses `~`, pkg-b uses `^` in devDependencies and pkg-c does not depend on it. +async function fanOutRepo( + registryVersions: Record = { "0.0.3": {}, "0.0.5": {}, latest: "0.0.5" }, +) { + setHandler(dummyRegistry([], registryVersions)); + await writeTextLockfileBunfig(); + for (const [rel, json] of Object.entries(FAN_OUT_FILES)) { + await mkdir(dirname(join(package_dir, rel)), { recursive: true }); + await writeFile(join(package_dir, rel), JSON.stringify(json, null, 2) + "\n"); + } + await runInstall(); + return fanOutTexts(); +} - const root = await file(join(package_dir, "package.json")).json(); - const a = await file(join(package_dir, "packages", "pkg-a", "package.json")).json(); - expect(root.dependencies.baz).toBe("~0.0.3"); - expect(a.dependencies.baz).toBe("~0.0.3"); - expect(urls).toStrictEqual([]); +async function spawnUpdate(...args: string[]) { + const { stdout, stderr, exited } = spawn({ + cmd: [bunExe(), "update", ...args, "--linker=hoisted"], + cwd: package_dir, + stdout: "pipe", + stderr: "pipe", + env, }); + const [out, err, exitCode] = await Promise.all([stdout.text(), stderr.text(), exited]); + return { out, err, exitCode }; } -it("--production is rejected", async () => { - const urls: string[] = []; - setHandler(dummyRegistry(urls, { "0.0.3": {}, latest: "0.0.3" })); - await writeFile(join(package_dir, "package.json"), JSON.stringify({ name: "foo", dependencies: { baz: "~0.0.3" } })); +it("named update -r --latest rewrites every workspace that declares the name, keeping each file's style", async () => { + const [, , , pkgC] = await fanOutRepo(); + const { err, exitCode } = await spawnUpdate("baz", "-r", "--latest"); + expect(err).not.toContain("error:"); + expect(exitCode).toBe(0); + expect((await fanOutJson("package.json")).dependencies.baz).toBe("0.0.5"); + expect((await fanOutJson("packages/pkg-a/package.json")).dependencies.baz).toBe("~0.0.5"); + expect((await fanOutJson("packages/pkg-b/package.json")).devDependencies.baz).toBe("^0.0.5"); + expect(await file(join(package_dir, "packages", "pkg-c", "package.json")).text()).toBe(pkgC); +}); - for (const flag of ["--production", "-p", "--prod", "-P"]) { - const { stderr, exited } = spawn({ - cmd: [bunExe(), "update", flag], - cwd: package_dir, - stdout: "ignore", - stderr: "pipe", - env, - }); - expect(await new Response(stderr).text()).toContain("error: --production cannot be used with bun update"); - expect(await exited).not.toBe(0); - } - expect(urls).toStrictEqual([]); +it("named update --filter rewrites only the selected workspace", async () => { + const [root, , pkgB, pkgC] = await fanOutRepo(); + const { err, exitCode } = await spawnUpdate("baz", "--filter", "pkg-a", "--latest"); + expect(err).not.toContain("error:"); + expect(exitCode).toBe(0); + expect((await fanOutJson("packages/pkg-a/package.json")).dependencies.baz).toBe("~0.0.5"); + expect(await fanOutTexts()).toEqual([root, expect.any(String), pkgB, pkgC]); +}); + +it("named update --filter of a workspace that does not depend on the name is an error", async () => { + const before = await fanOutRepo(); + const lockBefore = await file(join(package_dir, "bun.lock")).text(); + const { err, exitCode } = await spawnUpdate("baz", "--filter", "pkg-c"); + expect(err).toContain('"baz" is only a dependency of other workspaces, so there is nothing to update here'); + expect(exitCode).toBe(1); + expect(await fanOutTexts()).toEqual(before); + expect(await file(join(package_dir, "bun.lock")).text()).toBe(lockBefore); +}); + +it("named update -r with a name missing from the lockfile is an error", async () => { + const before = await fanOutRepo(); + const { err, exitCode } = await spawnUpdate("nope", "-r"); + expect(err).toContain('"nope" is not in the lockfile, so there is nothing to update'); + expect(exitCode).toBe(1); + expect(await fanOutTexts()).toEqual(before); +}); + +// Root's exact pin and pkg-b's `^0.0.3` (which excludes 0.0.5) only move with --latest. +it("named update -r moves the ranges and keeps bun.lock in sync", async () => { + const [, , , pkgC] = await fanOutRepo({ "0.0.3": {}, latest: "0.0.3" }); + setHandler(dummyRegistry([], { "0.0.3": {}, "0.0.5": {}, latest: "0.0.5" })); + const { err, exitCode } = await spawnUpdate("baz", "-r"); + expect(err).not.toContain("error:"); + expect(exitCode).toBe(0); + expect((await fanOutJson("packages/pkg-a/package.json")).dependencies.baz).toBe("~0.0.5"); + expect((await fanOutJson("packages/pkg-b/package.json")).devDependencies.baz).toBe("^0.0.3"); + expect((await fanOutJson("package.json")).dependencies.baz).toBe("0.0.3"); + expect(await file(join(package_dir, "packages", "pkg-c", "package.json")).text()).toBe(pkgC); + await runInstall(package_dir, "--frozen-lockfile"); +}); + +it("named update -r --dry-run writes nothing", async () => { + const before = await fanOutRepo(); + const { err, exitCode } = await spawnUpdate("baz", "-r", "--latest", "--dry-run"); + expect(err).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await fanOutTexts()).toEqual(before); }); // https://github.com/oven-sh/bun/issues/33176 @@ -1027,7 +975,7 @@ it("--filter with a negated pattern updates everything except the excluded works async function setupWorkspaces( root: object, members: Record, - versions: Record = { "0.0.3": {}, "0.0.5": {}, latest: "0.0.5" }, + versions: Record = { "0.0.3": {}, "0.0.5": {}, latest: "0.0.5" }, ) { setHandler(dummyRegistry([], versions)); await writeFile( @@ -1038,18 +986,12 @@ async function setupWorkspaces( await mkdir(join(package_dir, "packages", name), { recursive: true }); await writeFile(join(package_dir, "packages", name, "package.json"), JSON.stringify({ name, ...body })); } - const { stderr, exited } = spawn({ - cmd: [bunExe(), "install", "--linker=hoisted"], - cwd: package_dir, - stdout: "ignore", - stderr: "pipe", - env, - }); - const [err, code] = await Promise.all([stderr.text(), exited]); - expect(err).not.toContain("error:"); - expect(code).toBe(0); + await runInstall(); } +const BAZ_0_0_3_ONLY = { "0.0.3": {}, latest: "0.0.3" }; +const bumpBazTo_0_0_5 = () => setHandler(dummyRegistry([], { "0.0.3": {}, "0.0.5": {}, latest: "0.0.5" })); + async function runUpdate(args: string[], cwd = package_dir) { const { stderr, exited } = spawn({ cmd: [bunExe(), "update", ...args, "--linker=hoisted"], @@ -1201,6 +1143,19 @@ it("--recursive --no-save updates node_modules but not any package.json", async }); }); +it("--recursive --latest keeps a member's dist-tag literal and follows the tag", async () => { + await setupWorkspaces({}, { "pkg-a": { dependencies: { baz: "latest" } } }, BAZ_0_0_3_ONLY); + expect(await file(join(package_dir, "node_modules", "baz", "package.json")).json()).toMatchObject({ + version: "0.0.3", + }); + bumpBazTo_0_0_5(); + await runUpdate(["--recursive", "--latest"]); + expect((await pkgJson("pkg-a")).dependencies).toEqual({ baz: "latest" }); + expect(await file(join(package_dir, "node_modules", "baz", "package.json")).json()).toMatchObject({ + version: "0.0.5", + }); +}); + // https://github.com/oven-sh/bun/issues/23507 it("--recursive is idempotent: a second run changes nothing", async () => { await setupWorkspaces({ dependencies: { baz: "~0.0.3" } }, { "pkg-a": { dependencies: { baz: "~0.0.3" } } }); @@ -1343,12 +1298,12 @@ it("should print UTF-8 arrows correctly with colors enabled", async () => { expect(await exited2).toBe(0); }); -// Unlike `dummyRegistry`, this serves a distinct manifest per package name, -// packing a real tarball for each version into `tgzDir`. -async function perNameRegistry( - tgzDir: string, - manifests: Record }>; latest: string }>, -) { +type PerNameManifests = Record< + string, + { versions: Record }>; latest: string } +>; + +async function packPerName(tgzDir: string, manifests: PerNameManifests) { for (const [name, { versions }] of Object.entries(manifests)) { for (const [version, extra] of Object.entries(versions)) { const staging = join(tgzDir, ".staging", `${name}-${version}`); @@ -1357,6 +1312,10 @@ async function perNameRegistry( await pack(staging, env, "--destination", tgzDir); } } +} + +// Unlike `dummyRegistry`, this serves a distinct manifest per package name from tarballs packed by `packPerName`. +function perNameHandler(tgzDir: string, manifests: PerNameManifests) { return (request: Request) => { const url = request.url; if (url.endsWith(".tgz")) return new Response(file(join(tgzDir, basename(url)))); @@ -1371,10 +1330,15 @@ async function perNameRegistry( }; } -async function runInPackageDir(...args: string[]) { +async function perNameRegistry(tgzDir: string, manifests: PerNameManifests) { + await packPerName(tgzDir, manifests); + return perNameHandler(tgzDir, manifests); +} + +async function runIn(cwd: string, ...args: string[]) { const { stdout, stderr, exited } = spawn({ cmd: [bunExe(), ...args], - cwd: package_dir, + cwd, stdout: "pipe", stderr: "pipe", env, @@ -1385,40 +1349,33 @@ async function runInPackageDir(...args: string[]) { return out; } +const runInPackageDir = (...args: string[]) => runIn(package_dir, ...args); + // The set of `shared@` resolutions in the text lockfile. async function lockedSharedResolutions() { const lock = await file(join(package_dir, "bun.lock")).text(); return [...new Set(lock.match(/"shared@[\d.]+"/g))].sort(); } -async function writePerNameBunfig() { - await writeFile( - join(package_dir, "bunfig.toml"), - `[install]\ncache = false\nregistry = "${root_url}/"\nsaveTextLockfile = true\nlinker = "hoisted"\n`, - ); -} - -// `bun update ` must re-resolve every dependency on `` in the -// lockfile, each within its own version range, including a workspace whose -// range resolves to a different version than the current workspace's. -it("should update every resolution of a named package across workspaces", async () => { +// A named update only re-resolves the rows of the workspace it runs in; another workspace's own entry is left alone. +it("bun update from the root leaves a member's own entry alone; running it inside the member moves it", async () => { setHandler( await perNameRegistry(join(package_dir, ".tarballs"), { shared: { versions: { "1.0.0": {}, "1.1.0": {}, "2.0.0": {} }, latest: "2.0.0" }, }), ); - await writePerNameBunfig(); + await writeTextLockfileBunfig(); await writeFile( join(package_dir, "package.json"), JSON.stringify({ name: "root", workspaces: ["packages/*"], dependencies: { shared: "^2.0.0" } }), ); - const pkgOneJson = join(package_dir, "packages", "pkg-one", "package.json"); - await mkdir(join(package_dir, "packages", "pkg-one"), { recursive: true }); + const pkgOneDir = join(package_dir, "packages", "pkg-one"); + const pkgOneJson = join(pkgOneDir, "package.json"); + await mkdir(pkgOneDir, { recursive: true }); await writeFile(pkgOneJson, JSON.stringify({ name: "pkg-one", version: "1.0.0", dependencies: { shared: "1.0.0" } })); await runInPackageDir("install"); - // Widen pkg-one's range. A plain install keeps its stale 1.0.0 because the - // previously resolved package still satisfies the new range. + // Widening the range keeps the stale 1.0.0 on a plain install, since it still satisfies the new range. await writeFile( pkgOneJson, JSON.stringify({ name: "pkg-one", version: "1.0.0", dependencies: { shared: "^1.0.0" } }), @@ -1426,16 +1383,19 @@ it("should update every resolution of a named package across workspaces", async await runInPackageDir("install"); expect(await lockedSharedResolutions()).toEqual(['"shared@1.0.0"', '"shared@2.0.0"']); - // root's ^2.0.0 is already at 2.0.0; pkg-one's ^1.0.0 must move to 1.1.0. + const pkgOneShared = () => file(join(pkgOneDir, "node_modules", "shared", "package.json")).json(); + await runInPackageDir("update", "shared"); + expect(await lockedSharedResolutions()).toEqual(['"shared@1.0.0"', '"shared@2.0.0"']); + expect(await pkgOneShared()).toMatchObject({ version: "1.0.0" }); + + await runIn(pkgOneDir, "update", "shared"); expect(await lockedSharedResolutions()).toEqual(['"shared@1.1.0"', '"shared@2.0.0"']); - expect( - await file(join(package_dir, "packages", "pkg-one", "node_modules", "shared", "package.json")).json(), - ).toMatchObject({ version: "1.1.0" }); + expect(await pkgOneShared()).toMatchObject({ version: "1.1.0" }); - // The update reached a fixpoint: running it again is a no-op. - await runInPackageDir("update", "shared"); + await runIn(pkgOneDir, "update", "shared"); expect(await lockedSharedResolutions()).toEqual(['"shared@1.1.0"', '"shared@2.0.0"']); + expect(await pkgOneShared()).toMatchObject({ version: "1.1.0" }); }); // The same invariant one level deeper: a dependency on `` owned by a @@ -1447,7 +1407,7 @@ it("should update transitive resolutions of a named package", async () => { "dep-x": { versions: { "1.0.0": { dependencies: { shared: "^1.0.0" } } }, latest: "1.0.0" }, }), ); - await writePerNameBunfig(); + await writeTextLockfileBunfig(); // dep-x@1.0.0 depends on shared@^1.0.0, which dedupes onto the root's // exact shared@1.0.0 at install time. await writeFile( @@ -1465,20 +1425,19 @@ it("should update transitive resolutions of a named package", async () => { ).toMatchObject({ version: "1.1.0" }); }); -// `shared` appears only transitively (dep-x@1.0.0 depends on `shared@^1.0.0`); shared@1.1.0 is published after -// the lockfile pinned 1.0.0. +// `shared` is only reachable through dep-x; the install below pins 1.0.0 before the registry starts serving 1.1.0. async function setupTransitiveOnlyShared() { const tgzDir = join(package_dir, ".tarballs"); const depX = { "dep-x": { versions: { "1.0.0": { dependencies: { shared: "^1.0.0" } } }, latest: "1.0.0" } }; - setHandler(await perNameRegistry(tgzDir, { ...depX, shared: { versions: { "1.0.0": {} }, latest: "1.0.0" } })); - await writePerNameBunfig(); + const published = { ...depX, shared: { versions: { "1.0.0": {}, "1.1.0": {} }, latest: "1.1.0" } }; + await packPerName(tgzDir, published); + setHandler(perNameHandler(tgzDir, { ...depX, shared: { versions: { "1.0.0": {} }, latest: "1.0.0" } })); + await writeTextLockfileBunfig(); const packageJson = { name: "root", dependencies: { "dep-x": "^1.0.0" } }; await writeFile(join(package_dir, "package.json"), JSON.stringify(packageJson)); await runInPackageDir("install"); expect(await lockedSharedResolutions()).toStrictEqual(['"shared@1.0.0"']); - setHandler( - await perNameRegistry(tgzDir, { ...depX, shared: { versions: { "1.0.0": {}, "1.1.0": {} }, latest: "1.1.0" } }), - ); + setHandler(perNameHandler(tgzDir, published)); return packageJson; } @@ -1519,7 +1478,7 @@ it("bun update rejects a name that is not in the lockfile", async () => { expect(await file(join(package_dir, "bun.lock")).text()).toBe(lockBefore); }); -// Registry: no-deps 1.0.0/1.0.1/1.1.0/2.0.0 (latest 2.0.0); a-dep 1.0.1..1.0.10; dep-with-tags latest=3.0.0, pre-2=2.0.1. +// Registry: no-deps 1.0.0/1.0.1/1.1.0/2.0.0; a-dep 1.0.1..1.0.10; dep-with-tags latest=3.0.0, pre-2=2.0.1; @types/* 1.0.0/2.0.0. describe("bun update semantics", () => { type Json = Record; const verdaccio = new VerdaccioRegistry(); @@ -1532,10 +1491,16 @@ describe("bun update semantics", () => { verdaccio.stop(); }); - const manifest = (dependencies: Json): Json => ({ name: "foo", dependencies }); + const GROUPS = ["dependencies", "devDependencies", "optionalDependencies", "peerDependencies"]; + // A manifest argument is either a bare dependency map or an object of package.json fields (groups, scripts). + const isFields = (json: Json) => Object.keys(json).some(key => GROUPS.includes(key) || key === "scripts"); + const manifest = (json: Json): Json => + isFields(json) ? { name: "foo", ...json } : { name: "foo", dependencies: json }; + const declared = (json: Json): Record => + isFields(json) ? Object.assign({}, ...GROUPS.map(group => json[group] ?? {})) : json; const stringify = (json: Json) => JSON.stringify(json, null, 2) + "\n"; - const packageJsonOf = (dir: string): Promise => file(join(dir, "package.json")).json(); - const packageJsonText = (dir: string) => file(join(dir, "package.json")).text(); + const packageJsonOf = (dir: string, rel = ""): Promise => file(join(dir, rel, "package.json")).json(); + const packageJsonText = (dir: string, rel = "") => file(join(dir, rel, "package.json")).text(); const lockText = (dir: string) => file(join(dir, "bun.lock")).text(); const lock = async (dir: string): Promise => Bun.JSONC.parse(await lockText(dir)) as Json; const installedVersion = async (dir: string, name: string): Promise => @@ -1568,21 +1533,28 @@ describe("bun update semantics", () => { return result; } - async function setup(dependencies: Json) { + async function createDir(files: Record) { const { packageDir } = await verdaccio.createTestDir({ bunfigOpts: { saveTextLockfile: true, linker: "hoisted" }, - files: { "package.json": stringify(manifest(dependencies)) }, + files: Object.fromEntries( + Object.entries(files).map(([path, json]) => [path, typeof json === "string" ? json : stringify(json)]), + ), }); - await install(packageDir); return packageDir; } + async function setup(json: Json) { + const dir = await createDir({ "package.json": manifest(json) }); + await install(dir); + return dir; + } + // Pin exactly, then widen: the locked versions still satisfy the new ranges, so only `bun update` moves them. async function stale(pinned: Json, widened: Json) { const dir = await setup(pinned); await writeFile(join(dir, "package.json"), stringify(manifest(widened))); expect(await install(dir)).toContain("Saved lockfile"); - for (const [name, literal] of Object.entries(pinned as Record)) { + for (const [name, literal] of Object.entries(declared(pinned))) { expect(await installedVersion(dir, name)).toBe(literal.replace(/^npm:[^@]+@/, "")); } return dir; @@ -1598,17 +1570,31 @@ describe("bun update semantics", () => { return [...new Set(versions)].sort(); } - async function expectInSync(dir: string, dependencies: Json) { - expect((await packageJsonOf(dir)).dependencies).toEqual(dependencies); - expect((await lock(dir)).workspaces[""].dependencies).toEqual(dependencies); + // `expected` is a dependency map, or an object of groups when several groups are asserted at once. + async function expectInSync(dir: string, expected: Json) { + const groups: Json = isFields(expected) ? expected : { dependencies: expected }; + const packageJson = await packageJsonOf(dir); + const root = (await lock(dir)).workspaces[""]; + for (const group of GROUPS) { + if (!(group in groups)) continue; + expect(packageJson[group]).toEqual(groups[group]); + expect(root[group]).toEqual(groups[group]); + } await install(dir, "--frozen-lockfile"); } + async function expectUnchanged(dir: string, before: { packageJson: string; lock: string }) { + expect(await packageJsonText(dir)).toBe(before.packageJson); + expect(await lockText(dir)).toBe(before.lock); + } + + const snapshotFiles = async (dir: string) => ({ packageJson: await packageJsonText(dir), lock: await lockText(dir) }); + const SIBLINGS_PINNED = { "no-deps": "1.0.0", "a-dep": "1.0.1" }; const SIBLINGS_WIDENED = { "no-deps": "^1.0.0", "a-dep": "^1.0.1" }; for (const flags of [[], ["--latest"]]) { - it(`bun update ${["a-dep", ...flags].join(" ")} leaves a stale unnamed sibling alone`, async () => { + it.concurrent(`bun update ${["a-dep", ...flags].join(" ")} leaves a stale unnamed sibling alone`, async () => { const dir = await stale(SIBLINGS_PINNED, SIBLINGS_WIDENED); await update(dir, "a-dep", ...flags); await expectInSync(dir, { "no-deps": "^1.0.0", "a-dep": "^1.0.10" }); @@ -1619,7 +1605,7 @@ describe("bun update semantics", () => { }); } - it("bun update keeps a dist-tag literal as written", async () => { + it.concurrent("bun update keeps a dist-tag literal as written", async () => { const dir = await setup({ "dep-with-tags": "pre-2" }); expect(await installedVersion(dir, "dep-with-tags")).toBe("2.0.1"); await update(dir, "dep-with-tags"); @@ -1627,15 +1613,86 @@ describe("bun update semantics", () => { expect(await lockedVersions(dir, "dep-with-tags")).toEqual(["2.0.1"]); }); - it("bun update --latest replaces a dist-tag literal with a caret range on latest", async () => { - const dir = await setup({ "dep-with-tags": "pre-2" }); - await update(dir, "dep-with-tags", "--latest"); - await expectInSync(dir, { "dep-with-tags": "^3.0.0" }); - expect(await lockedVersions(dir, "dep-with-tags")).toEqual(["3.0.0"]); - expect(await installedVersion(dir, "dep-with-tags")).toBe("3.0.0"); + for (const args of [[], ["dep-with-tags"]]) { + it.concurrent( + `bun update ${[...args, "--latest"].join(" ")} keeps a dist-tag literal and follows the tag`, + async () => { + const dir = await setup({ "dep-with-tags": "pre-2" }); + expect(await installedVersion(dir, "dep-with-tags")).toBe("2.0.1"); + const lockBefore = await lockText(dir); + await update(dir, ...args, "--latest"); + await expectInSync(dir, { "dep-with-tags": "pre-2" }); + expect(await lockedVersions(dir, "dep-with-tags")).toEqual(["2.0.1"]); + expect(await installedVersion(dir, "dep-with-tags")).toBe("2.0.1"); + expect(await lockText(dir)).toBe(lockBefore); + }, + ); + } + + it.concurrent("bun update --latest keeps a dist-tag literal next to a range it does rewrite", async () => { + const dir = await setup({ "dep-with-tags": "pre-2", "no-deps": "~1.0.0" }); + const { stdout } = await update(dir, "--latest"); + await expectInSync(dir, { "dep-with-tags": "pre-2", "no-deps": "~2.0.0" }); + expect(await lockedVersions(dir, "dep-with-tags")).toEqual(["2.0.1"]); + expect(await lockedVersions(dir, "no-deps")).toEqual(["2.0.0"]); + expect(stdout).toContain("no-deps"); + expect(stdout).not.toContain("dep-with-tags"); + }); + + it.concurrent("bun update --latest keeps an aliased dist-tag", async () => { + const dir = await setup({ tagged: "npm:dep-with-tags@pre-2" }); + const before = await snapshotFiles(dir); + await update(dir, "--latest"); + await expectUnchanged(dir, before); + await expectInSync(dir, { tagged: "npm:dep-with-tags@pre-2" }); + expect(await installedVersion(dir, "tagged")).toBe("2.0.1"); + }); + + it.concurrent("bun update --latest is a no-op on `latest` literals whose tag has not moved", async () => { + const dir = await setup({ "no-deps": "latest", aliased: "npm:a-dep@latest" }); + const before = await snapshotFiles(dir); + const { stderr } = await update(dir, "--latest"); + expect(stderr).not.toContain("Saved lockfile"); + await expectUnchanged(dir, before); + expect(await installedVersion(dir, "no-deps")).toBe("2.0.0"); + expect(await installedVersion(dir, "aliased")).toBe("1.0.10"); + }); + + it.concurrent.each<[string, string, string[]]>([ + ["*", "2.0.0", []], + ["1", "1.1.0", []], + ["1.x", "1.1.0", []], + [">=1.0.0 <2", "1.1.0", []], + ["1.0.0 - 1.0.1", "1.0.1", []], + ["npm:no-deps@1.x", "1.1.0", []], + ["*", "2.0.0", ["no-deps"]], + ])( + "a plain update keeps the range %p as written and only moves bun.lock to %p (extra args: %p)", + async (literal, version, names) => { + const pin = literal.startsWith("npm:") ? "npm:no-deps@1.0.0" : "1.0.0"; + const dir = await stale({ "no-deps": pin }, { "no-deps": literal }); + await update(dir, ...names); + await expectInSync(dir, { "no-deps": literal }); + expect(await lockedVersions(dir, "no-deps")).toEqual([version]); + expect(await installedVersion(dir, "no-deps")).toBe(version); + }, + ); + + it.concurrent("--latest still rewrites a non-caret range", async () => { + const dir = await setup({ "no-deps": "1.x" }); + await update(dir, "--latest"); + await expectInSync(dir, { "no-deps": "^2.0.0" }); + expect(await installedVersion(dir, "no-deps")).toBe("2.0.0"); + }); + + it.concurrent("bun update @ on a non-caret entry writes the caret form", async () => { + const dir = await setup({ "no-deps": "*" }); + await update(dir, "no-deps@1"); + await expectInSync(dir, { "no-deps": "^1.1.0" }); + expect(await installedVersion(dir, "no-deps")).toBe("1.1.0"); }); - it("bun update reaches a dependency declared behind an npm: alias", async () => { + it.concurrent("bun update reaches a dependency declared behind an npm: alias", async () => { const dir = await stale({ aliased: "npm:no-deps@1.0.0" }, { aliased: "npm:no-deps@~1.0.0" }); expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.0"]); await update(dir, "no-deps"); @@ -1644,38 +1701,449 @@ describe("bun update semantics", () => { expect(await installedVersion(dir, "aliased")).toBe("1.0.1"); }); - it("bun update @ --latest is an error and writes nothing", async () => { - const dir = await setup({ "no-deps": "~1.0.0" }); - const packageJsonBefore = await packageJsonText(dir); - const lockBefore = await lockText(dir); - const { stderr, exitCode } = await run(dir, "update", "no-deps@1", "--latest"); - expect(stderr).toMatch(/error: .*--latest/); - expect(stderr).not.toContain("Saved lockfile"); - expect(await packageJsonText(dir)).toBe(packageJsonBefore); - expect(await lockText(dir)).toBe(lockBefore); - expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); - expect(exitCode).not.toBe(0); + it.concurrent( + "bun update moves the plain entry and the aliased entry, each within its own range", + async () => { + const dir = await stale( + { "no-deps": "1.0.0", aliased: "npm:no-deps@1.0.0" }, + { "no-deps": "^1.0.0", aliased: "npm:no-deps@~1.0.0" }, + ); + await update(dir, "no-deps"); + await expectInSync(dir, { "no-deps": "^1.1.0", aliased: "npm:no-deps@~1.0.1" }); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.1", "1.1.0"]); + expect(await installedVersion(dir, "no-deps")).toBe("1.1.0"); + expect(await installedVersion(dir, "aliased")).toBe("1.0.1"); + }, + ); + + it.concurrent("bun update --latest moves an aliased entry to latest in its pin style", async () => { + const dir = await setup({ aliased: "npm:no-deps@~1.0.0" }); + expect(await installedVersion(dir, "aliased")).toBe("1.0.1"); + await update(dir, "no-deps", "--latest"); + await expectInSync(dir, { aliased: "npm:no-deps@~2.0.0" }); + expect(await lockedVersions(dir, "no-deps")).toEqual(["2.0.0"]); + expect(await installedVersion(dir, "aliased")).toBe("2.0.0"); }); - it("bun update --dry-run writes nothing", async () => { + for (const flag of ["--latest", "-L"]) { + it.concurrent(`bun update @ ${flag} is an error and writes nothing`, async () => { + const dir = await setup({ "no-deps": "~1.0.0" }); + const before = await snapshotFiles(dir); + const { stderr, exitCode } = await run(dir, "update", "no-deps@1", flag); + expect(stderr).toContain("error: --latest cannot be combined with a version"); + expect(stderr).not.toContain("Saved lockfile"); + await expectUnchanged(dir, before); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); + expect(exitCode).not.toBe(0); + }); + } + + it.concurrent("bun update --dry-run writes nothing", async () => { const dir = await stale({ "no-deps": "1.0.0" }, { "no-deps": "^1.0.0" }); - const packageJsonBefore = await packageJsonText(dir); - const lockBefore = await lockText(dir); + const before = await snapshotFiles(dir); const { stderr } = await update(dir, "no-deps", "--dry-run"); expect(stderr).not.toContain("Saved lockfile"); - expect(await packageJsonText(dir)).toBe(packageJsonBefore); - expect(await lockText(dir)).toBe(lockBefore); + await expectUnchanged(dir, before); expect(await installedVersion(dir, "no-deps")).toBe("1.0.0"); }); - it("bun update leaves an =x.y.z pin untouched", async () => { + it.concurrent("bun update leaves an =x.y.z pin untouched", async () => { const dir = await setup({ "no-deps": "=1.0.0" }); - const packageJsonBefore = await packageJsonText(dir); - const lockBefore = await lockText(dir); + const before = await snapshotFiles(dir); await update(dir); - expect(await packageJsonText(dir)).toBe(packageJsonBefore); - expect(await lockText(dir)).toBe(lockBefore); + await expectUnchanged(dir, before); await expectInSync(dir, { "no-deps": "=1.0.0" }); expect(await installedVersion(dir, "no-deps")).toBe("1.0.0"); }); + + it.concurrent("-L is an alias of --latest, bare and named", async () => { + const bare = await setup({ "no-deps": "~1.0.0", "a-dep": "~1.0.1" }); + await update(bare, "-L"); + await expectInSync(bare, { "no-deps": "~2.0.0", "a-dep": "~1.0.10" }); + + const named = await setup({ "no-deps": "~1.0.0", "a-dep": "~1.0.1" }); + await update(named, "no-deps", "-L"); + await expectInSync(named, { "no-deps": "~2.0.0", "a-dep": "~1.0.1" }); + + const { stdout, exitCode } = await run(named, "update", "--help"); + expect(stdout).toContain("-L, --latest"); + expect(stdout).toContain("-d, --dev"); + expect(exitCode).toBe(0); + }); + + it.concurrent("bun up is an alias of bun update", async () => { + const dir = await stale({ "no-deps": "1.0.0" }, { "no-deps": "^1.0.0" }); + const up = await run(dir, "up"); + expect(up.stderr).not.toContain("error:"); + expect(up.exitCode).toBe(0); + await expectInSync(dir, { "no-deps": "^1.1.0" }); + + const upLatest = await run(dir, "up", "no-deps", "-L"); + expect(upLatest.stderr).not.toContain("error:"); + expect(upLatest.exitCode).toBe(0); + await expectInSync(dir, { "no-deps": "^2.0.0" }); + + const help = await run(dir, "up", "--help"); + expect(help.stdout).toContain("bun update"); + expect(help.exitCode).toBe(0); + }); + + it.concurrent("bun up wins over a package.json script named up; bun run up still runs the script", async () => { + const scripts = { up: "echo SCRIPT_RAN" }; + const dir = await stale( + { dependencies: { "no-deps": "1.0.0" }, scripts }, + { dependencies: { "no-deps": "^1.0.0" }, scripts }, + ); + const up = await run(dir, "up"); + expect(up.stdout).not.toContain("SCRIPT_RAN"); + expect(up.exitCode).toBe(0); + await expectInSync(dir, { "no-deps": "^1.1.0" }); + + const script = await run(dir, "run", "up"); + expect(script.stdout).toContain("SCRIPT_RAN"); + expect(script.exitCode).toBe(0); + }); + + describe("patterns", () => { + const TRIO_PINNED = { "no-deps": "1.0.0", "a-dep": "1.0.1", "dep-with-tags": "1.0.0" }; + const TRIO_WIDENED = { "no-deps": "^1.0.0", "a-dep": "^1.0.1", "dep-with-tags": "^1.0.0" }; + + it.concurrent("bun update '@types/*' --latest updates the matching packages and nothing else", async () => { + const dir = await stale( + { "@types/no-deps": "1.0.0", "@types/is-number": "1.0.0", "no-deps": "1.0.0" }, + { "@types/no-deps": "^1.0.0", "@types/is-number": "^1.0.0", "no-deps": "^1.0.0" }, + ); + await update(dir, "@types/*", "--latest"); + await expectInSync(dir, { "@types/no-deps": "^2.0.0", "@types/is-number": "^2.0.0", "no-deps": "^1.0.0" }); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.0"]); + expect(await installedVersion(dir, "@types/no-deps")).toBe("2.0.0"); + expect(await installedVersion(dir, "@types/is-number")).toBe("2.0.0"); + }); + + it.concurrent("two patterns update the union within their ranges", async () => { + const dir = await stale(TRIO_PINNED, TRIO_WIDENED); + await update(dir, "a-*", "dep-*"); + await expectInSync(dir, { "no-deps": "^1.0.0", "a-dep": "^1.0.10", "dep-with-tags": "^1.0.1" }); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.0"]); + expect(await lockedVersions(dir, "a-dep")).toEqual(["1.0.10"]); + expect(await lockedVersions(dir, "dep-with-tags")).toEqual(["1.0.1"]); + }); + + it.concurrent("--dry-run with a pattern writes nothing", async () => { + const dir = await stale(TRIO_PINNED, TRIO_WIDENED); + const before = await snapshotFiles(dir); + const { stderr } = await update(dir, "a-*", "--dry-run"); + expect(stderr).not.toContain("Saved lockfile"); + await expectUnchanged(dir, before); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.1"); + }); + + it.concurrent.each([ + [[], { "no-deps": "^1.0.0", "a-dep": "^1.0.10", "dep-with-tags": "^1.0.1" }], + [["--latest"], { "no-deps": "^1.0.0", "a-dep": "^1.0.10", "dep-with-tags": "^3.0.0" }], + ])("a negation pattern updates everything except the match (flags: %p)", async (flags, expected) => { + const dir = await stale(TRIO_PINNED, TRIO_WIDENED); + await update(dir, "!no-deps", ...flags); + await expectInSync(dir, expected); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.0"]); + }); + + it.concurrent("a positive pattern minus a negation updates the matched set minus the exclusion", async () => { + const dir = await stale(TRIO_PINNED, TRIO_WIDENED); + await update(dir, "*-*", "!a-dep"); + await expectInSync(dir, { "no-deps": "^1.1.0", "a-dep": "^1.0.1", "dep-with-tags": "^1.0.1" }); + expect(await lockedVersions(dir, "a-dep")).toEqual(["1.0.1"]); + }); + + it.concurrent.each([ + ["zzz-*", 'error: no packages in bun.lock match "zzz-*"'], + ["@types/*@2", "error: a version cannot be combined with a pattern: @types/*@2"], + ])("bun update %p is an error that writes nothing", async (arg, message) => { + const dir = await setup({ "no-deps": "~1.0.0" }); + const before = await snapshotFiles(dir); + const { stderr, exitCode } = await run(dir, "update", arg); + expect(stderr).toContain(message); + expect(stderr).not.toContain("unrecognised dependency format"); + await expectUnchanged(dir, before); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.1"); + expect(exitCode).toBe(1); + }); + }); + + describe("group selectors", () => { + const THREE_GROUPS_PINNED = { + dependencies: { "no-deps": "1.0.0" }, + devDependencies: { "a-dep": "1.0.1" }, + optionalDependencies: { "dep-with-tags": "1.0.0" }, + }; + const THREE_GROUPS_WIDENED = { + dependencies: { "no-deps": "^1.0.0" }, + devDependencies: { "a-dep": "^1.0.1" }, + optionalDependencies: { "dep-with-tags": "^1.0.0" }, + }; + + it.concurrent.each(["--dev", "-D"])("%s updates only devDependencies", async flag => { + const dir = await stale(THREE_GROUPS_PINNED, THREE_GROUPS_WIDENED); + await update(dir, flag); + await expectInSync(dir, { + dependencies: { "no-deps": "^1.0.0" }, + devDependencies: { "a-dep": "^1.0.10" }, + optionalDependencies: { "dep-with-tags": "^1.0.0" }, + }); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.0"]); + expect(await lockedVersions(dir, "dep-with-tags")).toEqual(["1.0.0"]); + expect(await lockedVersions(dir, "a-dep")).toEqual(["1.0.10"]); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.0"); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.10"); + }); + + it.concurrent.each(["--prod", "-P", "--production", "-p"])( + "%s updates dependencies and optionalDependencies and keeps devDependencies installed", + async flag => { + const dir = await stale(THREE_GROUPS_PINNED, THREE_GROUPS_WIDENED); + await update(dir, flag); + await expectInSync(dir, { + dependencies: { "no-deps": "^1.1.0" }, + devDependencies: { "a-dep": "^1.0.1" }, + optionalDependencies: { "dep-with-tags": "^1.0.1" }, + }); + expect(await lockedVersions(dir, "a-dep")).toEqual(["1.0.1"]); + expect(await installedVersion(dir, "a-dep")).toBe("1.0.1"); + expect(await installedVersion(dir, "no-deps")).toBe("1.1.0"); + expect(await installedVersion(dir, "dep-with-tags")).toBe("1.0.1"); + }, + ); + + it.concurrent("--dev composes with --latest", async () => { + const dir = await setup({ + dependencies: { "dep-with-tags": "~1.0.0" }, + devDependencies: { "no-deps": "~1.0.0" }, + }); + await update(dir, "--dev", "--latest"); + await expectInSync(dir, { + dependencies: { "dep-with-tags": "~1.0.0" }, + devDependencies: { "no-deps": "~2.0.0" }, + }); + expect(await lockedVersions(dir, "dep-with-tags")).toEqual(["1.0.1"]); + }); + + it.concurrent("--dev with a name declared in another group updates nothing", async () => { + const dir = await stale( + { dependencies: { "no-deps": "1.0.0" }, devDependencies: { "a-dep": "1.0.1" } }, + { dependencies: { "no-deps": "^1.0.0" }, devDependencies: { "a-dep": "^1.0.1" } }, + ); + const before = await snapshotFiles(dir); + const { stderr, exitCode } = await run(dir, "update", "no-deps", "--dev"); + expect(stderr).toContain('error: no dependencies in the selected groups match "no-deps"'); + await expectUnchanged(dir, before); + expect(await installedVersion(dir, "no-deps")).toBe("1.0.0"); + expect(exitCode).toBe(1); + }); + }); + + // Nothing is stale after `install` (no-deps ^ -> 1.1.0, ~ -> 1.0.1); what varies is which package.json files get rewritten. + describe("bun update -r / --filter", () => { + const FILES: Record = { + "package.json": { name: "root", workspaces: ["packages/*"], dependencies: { "no-deps": "^1.0.0" } }, + "packages/api/package.json": { + name: "api", + version: "1.0.0", + dependencies: { "no-deps": "^1.0.0", "a-dep": "^1.0.1", aliased: "npm:no-deps@~1.0.0" }, + }, + "packages/web/package.json": '{"name":"web","peerDependencies":{"no-deps":"~1.0.0"}}', + "packages/pkg-a/package.json": { + name: "pkg-a", + devDependencies: { "no-deps": "^1.0.0" }, + dependencies: { api: "workspace:*" }, + }, + "packages/pkg-b/package.json": '{"name":"pkg-b"}', + }; + const MEMBERS = ["", "packages/api", "packages/web", "packages/pkg-a", "packages/pkg-b"] as const; + type Texts = Record<(typeof MEMBERS)[number], string>; + + async function texts(dir: string): Promise { + const out = {} as Texts; + for (const rel of MEMBERS) out[rel] = await packageJsonText(dir, rel); + return out; + } + + async function fanOut() { + const dir = await createDir(FILES); + await install(dir); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.1", "1.1.0"]); + return { dir, before: await texts(dir), lockBefore: await lockText(dir) }; + } + + const API_UPDATED = { "no-deps": "^1.1.0", "a-dep": "^1.0.1", aliased: "npm:no-deps@~1.0.1" }; + + it.concurrent("--filter rewrites the selected workspace only, reaching an alias by its real name", async () => { + const { dir, before } = await fanOut(); + await update(dir, "no-deps", "--filter", "api"); + expect((await packageJsonOf(dir, "packages/api")).dependencies).toEqual(API_UPDATED); + const after = await texts(dir); + expect(after).toEqual({ ...before, "packages/api": after["packages/api"] }); + const { workspaces } = await lock(dir); + expect(workspaces["packages/api"].dependencies).toEqual(API_UPDATED); + expect(workspaces[""].dependencies).toEqual({ "no-deps": "^1.0.0" }); + await install(dir, "--frozen-lockfile"); + }); + + it.concurrent.each([[["-r"]], [["--filter", "*"]]])( + "%p rewrites every workspace declaring the name, in whichever group, and leaves the rest byte-identical", + async flags => { + const { dir, before } = await fanOut(); + await update(dir, "no-deps", ...flags); + expect((await packageJsonOf(dir)).dependencies).toEqual({ "no-deps": "^1.1.0" }); + expect((await packageJsonOf(dir, "packages/api")).dependencies).toEqual(API_UPDATED); + expect(await packageJsonOf(dir, "packages/web")).toEqual({ + name: "web", + peerDependencies: { "no-deps": "~1.0.1" }, + }); + expect(await packageJsonOf(dir, "packages/pkg-a")).toEqual({ + name: "pkg-a", + devDependencies: { "no-deps": "^1.1.0" }, + dependencies: { api: "workspace:*" }, + }); + expect(await packageJsonText(dir, "packages/pkg-b")).toBe(before["packages/pkg-b"]); + const { workspaces } = await lock(dir); + expect(workspaces[""].dependencies).toEqual({ "no-deps": "^1.1.0" }); + expect(workspaces["packages/api"].dependencies).toEqual(API_UPDATED); + expect(workspaces["packages/web"].peerDependencies).toEqual({ "no-deps": "~1.0.1" }); + expect(workspaces["packages/pkg-a"].devDependencies).toEqual({ "no-deps": "^1.1.0" }); + await install(dir, "--frozen-lockfile"); + }, + ); + + it.concurrent( + "--latest with two filters leaves unselected workspaces' ranges in package.json and bun.lock", + async () => { + const { dir, before } = await fanOut(); + await update(dir, "no-deps", "--latest", "--filter", "api", "--filter", "pkg-a"); + expect((await packageJsonOf(dir, "packages/api")).dependencies).toEqual({ + "no-deps": "^2.0.0", + "a-dep": "^1.0.1", + aliased: "npm:no-deps@~2.0.0", + }); + expect((await packageJsonOf(dir, "packages/pkg-a")).devDependencies).toEqual({ "no-deps": "^2.0.0" }); + expect(await packageJsonText(dir)).toBe(before[""]); + expect(await packageJsonText(dir, "packages/web")).toBe(before["packages/web"]); + expect(await packageJsonText(dir, "packages/pkg-b")).toBe(before["packages/pkg-b"]); + expect((await lock(dir)).workspaces[""].dependencies).toEqual({ "no-deps": "^1.0.0" }); + expect(await lockedVersions(dir, "no-deps")).toEqual(["1.0.1", "1.1.0", "2.0.0"]); + await install(dir, "--frozen-lockfile"); + }, + ); + + it.concurrent("several names never add a name to a workspace that does not declare it", async () => { + const { dir, before } = await fanOut(); + await update(dir, "no-deps", "a-dep", "--filter", "api", "--filter", "web"); + expect((await packageJsonOf(dir, "packages/api")).dependencies).toEqual({ ...API_UPDATED, "a-dep": "^1.0.10" }); + expect(await packageJsonOf(dir, "packages/web")).toEqual({ + name: "web", + peerDependencies: { "no-deps": "~1.0.1" }, + }); + const after = await texts(dir); + expect(after).toEqual({ + ...before, + "packages/api": after["packages/api"], + "packages/web": after["packages/web"], + }); + expect(JSON.stringify((await lock(dir)).workspaces["packages/web"])).not.toContain("a-dep"); + await install(dir, "--frozen-lockfile"); + }); + + it.concurrent.each([ + [ + ["no-deps", "--filter", "pkg-b"], + '"no-deps" is only a dependency of other workspaces, so there is nothing to update here', + ], + [["is-number", "--filter", "api"], '"is-number" is not in the lockfile, so there is nothing to update'], + [["no-deps", "--filter", "nope"], 'No workspace packages matched the filter "nope"'], + ])("bun update %p is an error that writes nothing", async (args, message) => { + const { dir, before, lockBefore } = await fanOut(); + const { stderr, exitCode } = await run(dir, "update", ...args); + expect(stderr).toContain(message); + expect(await texts(dir)).toEqual(before); + expect(await lockText(dir)).toBe(lockBefore); + expect(exitCode).toBe(1); + }); + + it.concurrent("--dry-run writes nothing", async () => { + const { dir, before, lockBefore } = await fanOut(); + const { stderr } = await update(dir, "no-deps", "--filter", "api", "--dry-run"); + expect(stderr).not.toContain("Saved lockfile"); + expect(await texts(dir)).toEqual(before); + expect(await lockText(dir)).toBe(lockBefore); + }); + + it.concurrent("--filter decides the target, not the cwd", async () => { + const { dir, before } = await fanOut(); + const { stderr, exitCode } = await run(join(dir, "packages", "web"), "update", "no-deps", "--filter", "api"); + expect(stderr).not.toContain("error:"); + expect(exitCode).toBe(0); + expect((await packageJsonOf(dir, "packages/api")).dependencies).toEqual(API_UPDATED); + expect(await packageJsonText(dir, "packages/web")).toBe(before["packages/web"]); + expect(await packageJsonText(dir)).toBe(before[""]); + }); + + it.concurrent("a catalog reference keeps the member's literal and moves the root catalog entry", async () => { + const dir = await createDir({ + "package.json": { name: "root", workspaces: { packages: ["packages/*"], catalog: { "no-deps": "^1.0.0" } } }, + "packages/api/package.json": { name: "api", dependencies: { "no-deps": "catalog:" } }, + "packages/web/package.json": '{"name":"web","dependencies":{"no-deps":"catalog:"}}', + }); + await install(dir); + const webBefore = await packageJsonText(dir, "packages/web"); + await update(dir, "no-deps", "--latest", "--filter", "api"); + expect((await packageJsonOf(dir, "packages/api")).dependencies).toEqual({ "no-deps": "catalog:" }); + expect((await packageJsonOf(dir)).workspaces.catalog).toEqual({ "no-deps": "^2.0.0" }); + expect((await lock(dir)).catalog).toEqual({ "no-deps": "^2.0.0" }); + expect(await packageJsonText(dir, "packages/web")).toBe(webBefore); + expect(await lockedVersions(dir, "no-deps")).toEqual(["2.0.0"]); + await install(dir, "--frozen-lockfile"); + }); + + // root -> app -> lib -> util; tool -> lib (devDependency); lone has no workspace edges. + it.concurrent("an unnamed update accepts relation selectors", async () => { + const dir = await createDir({ + "package.json": { + name: "root", + workspaces: ["packages/*"], + dependencies: { app: "workspace:*", "dep-with-tags": "1.0.0" }, + }, + "packages/app/package.json": { + name: "app", + version: "1.0.0", + dependencies: { lib: "workspace:*", "is-number": "1.0.0" }, + }, + "packages/lib/package.json": { + name: "lib", + version: "1.0.0", + dependencies: { util: "workspace:*", "no-deps": "1.0.0" }, + }, + "packages/util/package.json": { name: "util", version: "1.0.0", dependencies: { "a-dep": "1.0.1" } }, + "packages/tool/package.json": { + name: "tool", + version: "1.0.0", + devDependencies: { lib: "workspace:*" }, + dependencies: { "@types/is-number": "1.0.0" }, + }, + "packages/lone/package.json": { name: "lone", version: "1.0.0", dependencies: { "@types/no-deps": "1.0.0" } }, + }); + await install(dir); + const untouched = ["", "packages/app", "packages/tool", "packages/lone"]; + const before = await Promise.all(untouched.map(rel => packageJsonText(dir, rel))); + + await update(dir, "--latest", "--filter", "lib..."); + const libDeps = { util: "workspace:*", "no-deps": "2.0.0" }; + const utilDeps = { "a-dep": "1.0.10" }; + expect((await packageJsonOf(dir, "packages/lib")).dependencies).toEqual(libDeps); + expect((await packageJsonOf(dir, "packages/util")).dependencies).toEqual(utilDeps); + expect(await Promise.all(untouched.map(rel => packageJsonText(dir, rel)))).toEqual(before); + const { workspaces } = await lock(dir); + expect(workspaces["packages/lib"].dependencies).toEqual(libDeps); + expect(workspaces["packages/util"].dependencies).toEqual(utilDeps); + await install(dir, "--frozen-lockfile"); + }); + }); }); diff --git a/test/cli/install/bun-workspaces.test.ts b/test/cli/install/bun-workspaces.test.ts index f6d72798485a..40ff2981f5fa 100644 --- a/test/cli/install/bun-workspaces.test.ts +++ b/test/cli/install/bun-workspaces.test.ts @@ -1853,6 +1853,57 @@ describe("install --filter", () => { expect(await exited).toBe(0); await checkWorkspace(); }); + + test.concurrent("relation selectors walk the workspace graph", async () => { + using ctx = await setupTest(); + const { packageDir, packageJson, env } = ctx; + const pkg = (name: string, deps: Record) => + write(join(packageDir, "packages", name, "package.json"), JSON.stringify({ name, version: "1.0.0", ...deps })); + await Promise.all([ + write( + packageJson, + JSON.stringify({ + name: "root", + workspaces: ["packages/*"], + dependencies: { app: "workspace:*", "left-pad": "1.0.0" }, + }), + ), + pkg("app", { dependencies: { lib: "workspace:*", "is-number": "1.0.0" } }), + pkg("lib", { dependencies: { util: "workspace:*", "no-deps": "1.0.0" } }), + pkg("util", { dependencies: { "a-dep": "1.0.1" } }), + pkg("tool", { devDependencies: { lib: "1.0.0" }, dependencies: { "no-deps-bins": "1.0.0" } }), + pkg("lone", { dependencies: { "peer-no-deps": "1.0.0" } }), + ]); + + const externals = ["a-dep", "no-deps", "is-number", "no-deps-bins", "left-pad", "peer-no-deps"]; + const installed = () => Promise.all(externals.map(name => exists(join(packageDir, "node_modules", name)))); + + async function installWithFilter(filter: string) { + await using proc = spawn({ + cmd: [bunExe(), "install", "--filter", filter], + cwd: packageDir, + stdout: "ignore", + stderr: "pipe", + env, + }); + const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); + expect(stderr).not.toContain("error:"); + return exitCode; + } + + // util and its dependents: lib, app, root (via app), tool (via its devDependency on lib); not lone + const dependentsExit = await installWithFilter("...util"); + expect(await installed()).toEqual([true, true, true, true, true, false]); + expect(await file(join(packageDir, "bun.lock")).text()).toContain('"peer-no-deps": "1.0.0"'); + expect(dependentsExit).toBe(0); + + await rm(join(packageDir, "node_modules"), { recursive: true, force: true }); + + // only app's dependencies: lib and util + const dependenciesExit = await installWithFilter("app^..."); + expect(await installed()).toEqual([true, true, false, false, false, false]); + expect(dependenciesExit).toBe(0); + }); }); test.concurrent("can override npm package with workspace package under a different name", async () => { diff --git a/test/cli/install/catalogs.test.ts b/test/cli/install/catalogs.test.ts index b93888673d93..c4bb46dae808 100644 --- a/test/cli/install/catalogs.test.ts +++ b/test/cli/install/catalogs.test.ts @@ -2,18 +2,75 @@ import { file, spawn, write } from "bun"; import { readTarball } from "bun:internal-for-testing"; import { afterAll, beforeAll, describe, expect, test } from "bun:test"; import { existsSync, lstatSync, readlinkSync } from "fs"; -import { exists, readdir, rm } from "fs/promises"; -import { VerdaccioRegistry, bunEnv, bunExe, pack, runBunInstall } from "harness"; +import { exists, readdir, realpath, rm } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe, pack, runBunInstall, tempDir } from "harness"; import { join } from "path"; var registry = new VerdaccioRegistry(); +type Manifests = Record>>>; + +// Registry packages that ship a raw `catalog:` specifier; verdaccio has none. +const catalogManifests: Manifests = { + "leaf": { "1.0.0": {}, "2.0.0": {} }, + "wants-leaf-peer": { "1.0.0": { peerDependencies: { leaf: "catalog:" } } }, + "wants-leaf-dep": { "1.0.0": { dependencies: { leaf: "catalog:" } } }, + "no-deps": { "1.0.0": {}, "2.0.0": {} }, + "catalog-peer": { + "1.0.0": { peerDependencies: { "no-deps": "catalog:" } }, + "2.0.0": { peerDependencies: { "no-deps": "catalog:peers" } }, + }, +}; + +async function serveRegistry(manifests: Manifests) { + const tarballs = new Map(); + for (const [name, versions] of Object.entries(manifests)) { + for (const [version, extra] of Object.entries(versions)) { + const archive = new Bun.Archive( + { "package/package.json": JSON.stringify({ name, version, ...extra }) }, + { compress: "gzip" }, + ); + tarballs.set(`/${name}-${version}.tgz`, await archive.bytes()); + } + } + return Bun.serve({ + port: 0, + fetch(request) { + const { origin, pathname } = new URL(request.url); + const tarball = tarballs.get(pathname); + if (tarball) return new Response(tarball); + const name = pathname.slice(1); + const entry = manifests[name]; + if (!entry) return new Response("not found", { status: 404 }); + const versions: Record = {}; + for (const [version, extra] of Object.entries(entry)) { + versions[version] = { name, version, dist: { tarball: `${origin}/${name}-${version}.tgz` }, ...extra }; + } + const latest = Object.keys(entry).sort(Bun.semver.order).at(-1); + return Response.json({ name, versions, "dist-tags": { latest } }); + }, + }); +} + +let catalogRegistry: Awaited>; + +function writeRegistryProject(files: Record, registryUrl: string) { + return tempDir("catalog-registry-", { + ...files, + "bunfig.toml": ({ root }) => + Bun.TOML.stringify({ + install: { cache: join(root, ".bun-cache"), registry: registryUrl, saveTextLockfile: true }, + }), + }); +} + beforeAll(async () => { - await registry.start(); + [catalogRegistry] = await Promise.all([serveRegistry(catalogManifests), registry.start()]); }); afterAll(() => { registry.stop(); + catalogRegistry.stop(true); }); describe("basic", () => { @@ -548,6 +605,18 @@ describe("update", () => { }); describe("errors", () => { + async function failingInstall(cwd: string) { + await using proc = spawn({ + cmd: [bunExe(), "install"], + cwd, + stdout: "pipe", + stderr: "pipe", + env: bunEnv, + }); + const [, err, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); + return { err, exitCode }; + } + test("fails gracefully when no catalog is found for a package", async () => { const { packageDir, packageJson } = await registry.createTestDir(); @@ -569,19 +638,10 @@ describe("errors", () => { }), ); - const { stdout, stderr, exited } = spawn({ - cmd: [bunExe(), "install"], - cwd: packageDir, - stdout: "pipe", - stderr: "pipe", - env: bunEnv, - }); - - const out = await stdout.text(); - const err = await stderr.text(); - + const { err, exitCode } = await failingInstall(packageDir); expect(err).toContain("no-deps@catalog: failed to resolve"); expect(err).toContain("a-dep@catalog:aaaaaaaaaaaaaaaaa failed to resolve"); + expect(exitCode).not.toBe(0); }); test("invalid dependency version", async () => { @@ -601,25 +661,73 @@ describe("errors", () => { }), ); - const { stdout, stderr, exited } = spawn({ - cmd: [bunExe(), "install"], - cwd: packageDir, - stdout: "pipe", - stderr: "pipe", - env: bunEnv, - }); + const { err, exitCode } = await failingInstall(packageDir); + expect(err).toContain("no-deps@catalog: failed to resolve"); + expect(exitCode).not.toBe(0); + }); - const out = await stdout.text(); - const err = await stderr.text(); + // pnpm: deps-installer/test/catalogs.ts "external dependency using catalog protocol errors" + test.concurrent("a catalog: dependency inside a registry package fails to resolve", async () => { + using dir = writeRegistryProject( + { + "package.json": JSON.stringify({ + name: "root", + workspaces: { catalog: { leaf: "^2.0.0" } }, + dependencies: { "wants-leaf-dep": "1.0.0" }, + }), + }, + catalogRegistry.url.href, + ); - expect(err).toContain("no-deps@catalog: failed to resolve"); + const { err, exitCode } = await failingInstall(String(dir)); + expect(err).toContain("leaf@catalog: failed to resolve"); + expect(exitCode).not.toBe(0); + expect(await exists(join(String(dir), "bun.lock"))).toBeFalse(); }); + + test.concurrent( + "a catalog: dependency inside a file: folder dependency fails to resolve, the same package as a workspace works", + async () => { + const localLib = JSON.stringify({ name: "local-lib", dependencies: { "no-deps": "catalog:" } }); + const [folder, workspace] = await Promise.all([ + registry.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "root", + workspaces: { catalog: { "no-deps": "^1.0.0" } }, + dependencies: { "local-lib": "file:./local-lib" }, + }), + "local-lib/package.json": localLib, + }, + }), + registry.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "root", + workspaces: { packages: ["packages/*"], catalog: { "no-deps": "^1.0.0" } }, + }), + "packages/local-lib/package.json": localLib, + }, + }), + ]); + + const { err, exitCode } = await failingInstall(folder.packageDir); + expect(err).toContain("no-deps@catalog: failed to resolve"); + expect(exitCode).not.toBe(0); + + await runBunInstall(bunEnv, workspace.packageDir); + expect((await file(join(workspace.packageDir, "node_modules", "no-deps", "package.json")).json()).version).toBe( + "1.1.0", + ); + }, + ); }); describe("peer dependencies", () => { type Linker = "hoisted" | "isolated"; - // `--linker` in addition to bunfig: a user-level ~/.npmrc `install-strategy` would otherwise override bunfig's linker. async function spawnInstall(dir: string, linker: Linker, ...args: string[]) { await using proc = Bun.spawn({ cmd: [bunExe(), "install", "--linker", linker, ...args], @@ -685,6 +793,7 @@ describe("peer dependencies", () => { libVersion?: string; linker: Linker; saveTextLockfile?: boolean; + registry?: string; }; async function makeRepo(opts: RepoOpts): Promise { @@ -715,6 +824,19 @@ describe("peer dependencies", () => { ...extraFiles, }, }); + if (opts.registry) { + await Bun.write( + join(packageDir, "bunfig.toml"), + Bun.TOML.stringify({ + install: { + cache: join(packageDir, ".bun-cache"), + registry: opts.registry, + linker: opts.linker, + saveTextLockfile: opts.saveTextLockfile, + }, + }), + ); + } return packageDir; } @@ -997,7 +1119,7 @@ describe("peer dependencies", () => { }); }); - // pnpm errors here; Bun never fails an install over an unresolved peer, so the peer must simply not get a nested copy. + // pnpm errors here; Bun never fails an install over an unresolved peer, so the peer must simply not get a nested copy (a registry package's own `catalog:` peer takes this exact path). describe.each([ ["catalog:", { catalog: {} }], ["catalog:peers", { catalogs: { other: { "no-deps": ">=1.0.0" } } }], @@ -1016,6 +1138,174 @@ describe("peer dependencies", () => { }); }); + const registryPeerKeys = ["app", "catalog-peer", "lib", "no-deps"]; + + describe.each(linkers)("linker=%s", linker => { + test.concurrent("a catalog: peer inside a registry package never reads the consumer's catalog", async () => { + using project = writeRegistryProject( + { + "package.json": JSON.stringify({ + name: "root", + workspaces: { packages: ["packages/*"], catalog: { leaf: "^2.0.0" } }, + dependencies: { "leaf": "1.0.0", "wants-leaf-peer": "1.0.0" }, + }), + "packages/lib/package.json": JSON.stringify({ name: "lib", peerDependencies: { leaf: "catalog:" } }), + }, + catalogRegistry.url.href, + ); + const dir = String(project); + const keys = ["leaf", "lib", "lib/leaf", "wants-leaf-peer"]; + + await install(dir, linker); + expect(await packageKeys(dir)).toEqual(keys); + const lockfile = await Bun.file(join(dir, "bun.lock")).text(); + + await rmNodeModules(dir); + const frozen = await install(dir, linker, "--frozen-lockfile"); + expect(frozen.err).not.toContain("lockfile had changes"); + expect(await Bun.file(join(dir, "bun.lock")).text()).toBe(lockfile); + expect(await packageKeys(dir)).toEqual(keys); + }); + + // pnpm: resolving-deps-resolver walk.rs resolves_children_through_catalogs — only importers substitute catalogs. + test.concurrent( + "a registry package's catalog: peer ignores the root catalog and binds to the workspace's copy", + async () => { + const dir = await makeRepo({ + rootDependencies: {}, + catalog: { "no-deps": "^2.0.0" }, + appDependencies: { "no-deps": "1.0.0", "catalog-peer": "1.0.0" }, + peerSpec: "*", + linker, + registry: catalogRegistry.url.href, + }); + const result = await record(dir, linker); + expect(result.keys).toEqual(registryPeerKeys); + expect(result.keysAfterReload).toEqual(registryPeerKeys); + expect(result.reloadSavedLockfile).toBeFalse(); + expect(await Bun.file(join(dir, "bun.lock")).text()).not.toContain("no-deps@2.0.0"); + if (linker === "isolated") { + const storeEntry = await realpath(join(dir, "packages", "app", "node_modules", "catalog-peer")); + expect((await Bun.file(join(storeEntry, "..", "no-deps", "package.json")).json()).version).toBe("1.0.0"); + } else { + expect(existsSync(join(dir, "node_modules", "catalog-peer", "node_modules"))).toBeFalse(); + } + }, + ); + + // Overrides are root-owned: an override VALUE of `catalog:` still applies to a registry package's peer. + test.concurrent("an override valued catalog: still applies to a registry package's peer", async () => { + const dir = await makeRepo({ + rootDependencies: {}, + overrides: { "no-deps": "catalog:" }, + catalog: { "no-deps": "1.0.0" }, + appDependencies: { "no-deps": "2.0.0", "peer-deps-fixed": "1.0.0" }, + peerSpec: "*", + linker, + }); + const result = await record(dir, linker); + const keys = ["app", "lib", "no-deps", "peer-deps-fixed"]; + expect(result.keys).toEqual(keys); + expect(result.keysAfterReload).toEqual(keys); + expect(result.reloadSavedLockfile).toBeFalse(); + const lockfile = await Bun.file(join(dir, "bun.lock")).text(); + expect(lockfile).not.toContain("no-deps@2.0.0"); + const { packages } = Bun.JSONC.parse(lockfile) as { packages: Record }; + expect(packages["no-deps"][0]).toBe("no-deps@1.0.0"); + }); + }); + + test.concurrent("a registry package's catalog: peer with no provider anywhere installs nothing for it", async () => { + const dir = await makeRepo({ + rootDependencies: {}, + catalog: { "no-deps": "^2.0.0" }, + appDependencies: { "catalog-peer": "1.0.0" }, + peerSpec: "*", + optionalPeer: true, + linker: "hoisted", + registry: catalogRegistry.url.href, + }); + const keys = ["app", "catalog-peer", "lib"]; + const result = await record(dir, "hoisted"); + expect(result.keys).toEqual(keys); + expect(result.keysAfterReload).toEqual(keys); + expect(result.reloadSavedLockfile).toBeFalse(); + expect(await Bun.file(join(dir, "bun.lock")).text()).not.toContain("no-deps@"); + expect(existsSync(join(dir, "node_modules", "no-deps"))).toBeFalse(); + }); + + test.concurrent("a registry package's named catalog:peers peer is scoped the same way", async () => { + const dir = await makeRepo({ + rootDependencies: {}, + catalogs: { peers: { "no-deps": "^2.0.0" } }, + appDependencies: { "no-deps": "1.0.0", "catalog-peer": "2.0.0" }, + peerSpec: "*", + linker: "hoisted", + registry: catalogRegistry.url.href, + }); + const result = await record(dir, "hoisted"); + expect(result.keys).toEqual(registryPeerKeys); + expect(result.keysAfterReload).toEqual(registryPeerKeys); + expect(result.reloadSavedLockfile).toBeFalse(); + expect(await Bun.file(join(dir, "bun.lock")).text()).not.toContain("no-deps@2.0.0"); + expect(existsSync(join(dir, "node_modules", "catalog-peer", "node_modules"))).toBeFalse(); + }); + + test.concurrent("changing the root catalog does not re-resolve a registry package's catalog: peer", async () => { + const dir = await makeRepo({ + rootDependencies: {}, + catalog: { "no-deps": "^1.0.0" }, + appDependencies: { "no-deps": "1.0.0", "catalog-peer": "1.0.0" }, + peerSpec: "*", + linker: "hoisted", + registry: catalogRegistry.url.href, + }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(registryPeerKeys); + + await rewriteRootPackageJson(dir, { rootDependencies: {}, catalog: { "no-deps": "^2.0.0" } }); + await install(dir, "hoisted"); + expect(await packageKeys(dir)).toEqual(registryPeerKeys); + expect(await Bun.file(join(dir, "bun.lock")).text()).not.toContain("no-deps@2.0.0"); + + const { err } = await install(dir, "hoisted"); + expect(err).not.toContain("Saved lockfile"); + }); + + test.concurrent("a file: folder dependency does not see the catalog either", async () => { + const { packageDir: dir } = await registry.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "root", + workspaces: ["packages/*"], + catalog: { "no-deps": "^2.0.0" }, + }), + "packages/app/package.json": JSON.stringify({ + name: "app", + dependencies: { "no-deps": "1.0.0", "vendored": "file:../../vendor/vendored" }, + }), + "vendor/vendored/package.json": JSON.stringify({ + name: "vendored", + version: "1.0.0", + peerDependencies: { "no-deps": "catalog:" }, + }), + }, + }); + const expectNoNestedCopy = async () => { + const lockfile = await Bun.file(join(dir, "bun.lock")).text(); + expect(lockfile).not.toContain("no-deps@2.0.0"); + expect((await packageKeys(dir)).filter(key => key.endsWith("vendored/no-deps"))).toEqual([]); + }; + + await install(dir, "hoisted"); + await expectNoNestedCopy(); + await rmNodeModules(dir); + const { err } = await install(dir, "hoisted"); + expect(err).not.toContain("Saved lockfile"); + await expectNoNestedCopy(); + }); + // pnpm #12159 shape: an override whose value is a catalog reference wins over the peer's own range, fresh and on reload. describe.each(linkers)("linker=%s", linker => { describe.each([ diff --git a/test/cli/install/config-precedence.test.ts b/test/cli/install/config-precedence.test.ts index 383830dd961f..bb339cb3cebb 100644 --- a/test/cli/install/config-precedence.test.ts +++ b/test/cli/install/config-precedence.test.ts @@ -5,8 +5,12 @@ import { join } from "path"; // Own config dir: other install files' VerdaccioRegistry start()/stop() delete the shared htpasswd, invalidating our token. const sharedRegistryDir = join(import.meta.dir, "registry"); +const sharedVerdaccioConfig = readFileSync(join(sharedRegistryDir, "verdaccio.yaml"), "utf8"); +if (!sharedVerdaccioConfig.includes("storage: ./packages")) { + throw new Error("registry/verdaccio.yaml no longer has a 'storage: ./packages' line to redirect"); +} const registryDir = tempDir("config-precedence-registry", { - "verdaccio.yaml": readFileSync(join(sharedRegistryDir, "verdaccio.yaml"), "utf8").replace( + "verdaccio.yaml": sharedVerdaccioConfig.replace( "storage: ./packages", `storage: ${JSON.stringify(join(sharedRegistryDir, "packages"))}`, ), @@ -62,8 +66,6 @@ async function install(root: string, args: string[] = []) { USERPROFILE: home, XDG_CONFIG_HOME: home, BUN_INSTALL_CACHE_DIR: join(root, "cache"), - // Authenticated requests leak their header buffer into the HTTP client (NetworkTask.rs); LSan would abort the install. - ASAN_OPTIONS: [bunEnv.ASAN_OPTIONS, "detect_leaks=0"].filter(Boolean).join(":"), }, stdout: "pipe", stderr: "pipe", @@ -83,8 +85,8 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(isIsolated(String(dir))).toBe(true); + expect(exitCode).toBe(0); }); test("project bunfig registry beats project .npmrc registry", async () => { @@ -96,9 +98,9 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(dead.hits).toBe(0); expect(existsSync(join(String(dir), "project", "node_modules", "no-deps", "package.json"))).toBe(true); + expect(exitCode).toBe(0); }); test("project bunfig registry beats ~/.npmrc registry", async () => { @@ -110,8 +112,9 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(dead.hits).toBe(0); + expect(existsSync(join(String(dir), "project", "node_modules", "no-deps", "package.json"))).toBe(true); + expect(exitCode).toBe(0); }); test("project .npmrc registry beats ~/.npmrc registry", async () => { @@ -123,9 +126,9 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(dead.hits).toBe(0); expect(existsSync(join(String(dir), "project", "node_modules", "no-deps", "package.json"))).toBe(true); + expect(exitCode).toBe(0); }); test("~/.npmrc _authToken applies to the registry set in project bunfig", async () => { @@ -136,10 +139,10 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(existsSync(join(String(dir), "project", "node_modules", "@needs-auth", "test-pkg", "package.json"))).toBe( true, ); + expect(exitCode).toBe(0); }); test("project .npmrc scoped registry and token apply alongside a bunfig registry", async () => { @@ -151,11 +154,11 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(dead.hits).toBe(0); expect(existsSync(join(String(dir), "project", "node_modules", "@needs-auth", "test-pkg", "package.json"))).toBe( true, ); + expect(exitCode).toBe(0); }); test("bunfig scoped registry keeps credentials from ~/.npmrc", async () => { @@ -167,8 +170,11 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(dead.hits).toBe(0); + expect(existsSync(join(String(dir), "project", "node_modules", "@needs-auth", "test-pkg", "package.json"))).toBe( + true, + ); + expect(exitCode).toBe(0); }); test("bunfig scoped registry beats the same scope in .npmrc", async () => { @@ -180,9 +186,9 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(dead.hits).toBe(0); expect(existsSync(join(String(dir), "project", "node_modules", "@types", "no-deps", "package.json"))).toBe(true); + expect(exitCode).toBe(0); }); test("bunfig registry credentials survive a same-URL registry= line in project .npmrc", async () => { @@ -193,10 +199,10 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(existsSync(join(String(dir), "project", "node_modules", "@needs-auth", "test-pkg", "package.json"))).toBe( true, ); + expect(exitCode).toBe(0); }); test("bunfig scoped registry credentials survive the same scope URL in project .npmrc", async () => { @@ -211,11 +217,11 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(dead.hits).toBe(0); expect(existsSync(join(String(dir), "project", "node_modules", "@needs-auth", "test-pkg", "package.json"))).toBe( true, ); + expect(exitCode).toBe(0); }); test("--linker beats ~/.npmrc, project .npmrc and bunfig", async () => { @@ -227,8 +233,8 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir), ["--linker", "isolated"]); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(isIsolated(String(dir))).toBe(true); + expect(exitCode).toBe(0); }); test("~/.npmrc install-strategy applies when bunfig does not set a linker", async () => { @@ -239,7 +245,7 @@ describe.concurrent("bun install config precedence", () => { }); const { stderr, exitCode } = await install(String(dir)); expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); expect(isIsolated(String(dir))).toBe(true); + expect(exitCode).toBe(0); }); }); diff --git a/test/cli/install/frozen-lockfile-missing-workspace.test.ts b/test/cli/install/frozen-lockfile-missing-workspace.test.ts index bd263a35aebd..30ee8404fdf7 100644 --- a/test/cli/install/frozen-lockfile-missing-workspace.test.ts +++ b/test/cli/install/frozen-lockfile-missing-workspace.test.ts @@ -1,5 +1,5 @@ import { file, write } from "bun"; -import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { afterAll, beforeAll, expect, test } from "bun:test"; import { exists, rm } from "fs/promises"; import { VerdaccioRegistry, bunEnv, bunExe } from "harness"; import { join } from "path"; @@ -17,14 +17,11 @@ afterAll(() => { const app = { name: "app", version: "1.0.0", dependencies: { "no-deps": "1.0.0" } }; const shared = { name: "shared", version: "1.0.0" }; -const notFound = 'Workspace not found "packages/api"'; - -// `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. async function bun(dir: string, args: string[]) { await using proc = Bun.spawn({ cmd: [bunExe(), ...args, "--linker", "hoisted"], cwd: dir, - env: bunEnv, + env: { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }, stdin: "ignore", stdout: "pipe", stderr: "pipe", @@ -33,80 +30,42 @@ async function bun(dir: string, args: string[]) { return { stdout, stderr, exitCode }; } -// bun.lock ends up describing `onDisk`; package.json lists `listed`; `remove` folders and node_modules are gone. -async function tree(onDisk: Record, listed: string[], remove: string[] = []) { - const { packageDir: dir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); - await Promise.all([ - write(join(dir, "package.json"), JSON.stringify({ name: "mono", workspaces: Object.keys(onDisk) })), - ...Object.entries(onDisk).map(([path, pkg]) => write(join(dir, path, "package.json"), JSON.stringify(pkg))), - ]); - - const { stderr, exitCode } = await bun(dir, ["install"]); - expect(stderr).toContain("Saved lockfile"); - expect(exitCode).toBe(0); - const lock = await file(join(dir, "bun.lock")).text(); - - await Promise.all([ - write(join(dir, "package.json"), JSON.stringify({ name: "mono", workspaces: listed })), - rm(join(dir, "node_modules"), { recursive: true }), - ...remove.map(path => rm(join(dir, path), { recursive: true })), - ]); - return { dir, lock }; -} - -async function expectRejected(dir: string, lock: string, args: string[]) { - const { stderr, exitCode } = await bun(dir, args); - expect(stderr).toContain(notFound); - expect(stderr).not.toContain("lockfile had changes"); - expect(await file(join(dir, "bun.lock")).text()).toBe(lock); - expect(await exists(join(dir, "node_modules"))).toBeFalse(); - expect(exitCode).toBe(1); - return { stderr }; -} - -const notFoundLine = (stderr: string) => stderr.split("\n").find(line => line.includes("Workspace not found")); - -describe("a listed workspace that is on neither disk nor in bun.lock", () => { - test.concurrent("--frozen-lockfile rejects it like a plain install", async () => { - const { dir, lock } = await tree({ "packages/app": app }, ["packages/app", "packages/api"]); - - const plain = await bun(dir, ["install"]); - expect(plain.stderr).toContain(notFound); - expect(plain.exitCode).toBe(1); +test.concurrent( + "--frozen-lockfile tolerates a workspace pruned from disk but still rejects an unknown one listed next to it", + async () => { + const { packageDir: dir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); + await Promise.all([ + write( + join(dir, "package.json"), + JSON.stringify({ name: "mono", workspaces: ["packages/app", "packages/shared"] }), + ), + write(join(dir, "packages", "app", "package.json"), JSON.stringify(app)), + write(join(dir, "packages", "shared", "package.json"), JSON.stringify(shared)), + ]); + + const full = await bun(dir, ["install"]); + expect(full.stderr).toContain("Saved lockfile"); + expect(full.exitCode).toBe(0); + const lock = await file(join(dir, "bun.lock")).text(); + expect(lock).toContain('"packages/shared"'); + expect(lock).not.toContain('"packages/api"'); + + await Promise.all([ + write( + join(dir, "package.json"), + JSON.stringify({ name: "mono", workspaces: ["packages/app", "packages/shared", "packages/api"] }), + ), + rm(join(dir, "node_modules"), { recursive: true }), + rm(join(dir, "packages", "shared"), { recursive: true }), + ]); + + const { stderr, exitCode } = await bun(dir, ["install", "--frozen-lockfile"]); + + expect(stderr).toContain('Workspace not found "packages/api"'); + expect(stderr).not.toContain('Workspace not found "packages/shared"'); + expect(stderr).not.toContain("lockfile had changes"); + expect(await file(join(dir, "bun.lock")).text()).toBe(lock); expect(await exists(join(dir, "node_modules"))).toBeFalse(); - - const { stderr } = await expectRejected(dir, lock, ["install", "--frozen-lockfile"]); - - expect(notFoundLine(stderr)).toBeDefined(); - expect(notFoundLine(stderr)).toBe(notFoundLine(plain.stderr)); - }); - - test.concurrent("--production rejects it too", async () => { - const { dir, lock } = await tree({ "packages/app": app }, ["packages/app", "packages/api"]); - - await expectRejected(dir, lock, ["install", "--production"]); - }); - - test.concurrent("bun ci rejects it too", async () => { - const { dir, lock } = await tree({ "packages/app": app }, ["packages/app", "packages/api"]); - - await expectRejected(dir, lock, ["ci"]); - }); - - test.concurrent( - "a workspace pruned from disk but still in bun.lock is tolerated while an unknown one next to it is still rejected", - async () => { - const { dir, lock } = await tree( - { "packages/app": app, "packages/shared": shared }, - ["packages/app", "packages/shared", "packages/api"], - ["packages/shared"], - ); - expect(lock).toContain('"packages/shared"'); - expect(lock).not.toContain('"packages/api"'); - - const { stderr } = await expectRejected(dir, lock, ["install", "--frozen-lockfile"]); - - expect(stderr).not.toContain('Workspace not found "packages/shared"'); - }, - ); -}); + expect(exitCode).toBe(1); + }, +); diff --git a/test/cli/install/frozen-lockfile-pruned.test.ts b/test/cli/install/frozen-lockfile-pruned.test.ts index ecf42de44872..27134c5ffec8 100644 --- a/test/cli/install/frozen-lockfile-pruned.test.ts +++ b/test/cli/install/frozen-lockfile-pruned.test.ts @@ -1,7 +1,7 @@ import { file, write } from "bun"; import { afterAll, beforeAll, describe, expect, test } from "bun:test"; -import { exists, lstat, readlink } from "fs/promises"; -import { VerdaccioRegistry, bunEnv, bunExe } from "harness"; +import { exists, lstat } from "fs/promises"; +import { VerdaccioRegistry, bunEnv, bunExe, isWindows } from "harness"; import { join } from "path"; type Linker = "hoisted" | "isolated"; @@ -67,7 +67,63 @@ const explicitMonorepo: Tree = { root: { ...rootPackageJson, workspaces: ["packages/app", "packages/shared", "packages/other"] }, }; -// `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. +const withApp = (extra: PackageJson): Tree => ({ + ...monorepo, + packages: { ...monorepo.packages, "packages/app": { ...appPackageJson, ...extra } }, +}); + +const survivorTree = withApp({ dependencies: { ...(appPackageJson.dependencies as object), other: "workspace:*" } }); + +const survivorError = (dependent: string, ws = "other") => + `workspace "${dependent}" depends on workspace "${ws}" (packages/${ws}), which is listed in bun.lock but not on disk`; +const rootSurvivorError = + 'the root package depends on workspace "other" (packages/other), which is listed in bun.lock but not on disk'; +const survivorNote = "note: a pruned checkout must keep every workspace that its remaining workspaces depend on"; + +const catalogTree: Tree = { + root: { name: "mono", workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.1", "left-pad": "1.0.0" } } }, + packages: { + "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "catalog:" } }, + "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "catalog:" } }, + }, +}; + +const namedCatalogTree: Tree = { + root: { + name: "mono", + workspaces: { packages: ["packages/*"], catalogs: { build: { "a-dep": "1.0.1", "left-pad": "1.0.0" } } }, + }, + packages: { + "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "catalog:build" } }, + "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "catalog:build" } }, + }, +}; + +// Scoped to the top-level catalog/catalogs block so importer rows, package rows and overrides are out of reach. +function trimCatalogLine(lock: string, name: string, spec: string): string { + const line = new RegExp(`^ +"${name}": "${spec}",\\n`, "m"); + let trimmed = lock; + let removed = 0; + for (const header of ['\n "catalog": {\n', '\n "catalogs": {\n']) { + const start = lock.indexOf(header); + if (start === -1) continue; + const end = lock.indexOf("\n },", start); + expect(end).toBeGreaterThan(start); + const block = lock.slice(start, end); + const matches = block.match(new RegExp(line.source, "gm")) ?? []; + removed += matches.length; + trimmed = trimmed.replace(block, block.replace(line, "")); + } + expect(removed).toBe(1); + expect(trimmed).not.toBe(lock); + return trimmed; +} + +const binFiles = (dir: string, name: string) => + isWindows + ? [join(dir, "node_modules", ".bin", `${name}.exe`), join(dir, "node_modules", ".bin", `${name}.bunx`)] + : [join(dir, "node_modules", ".bin", name)]; + async function raw(dir: string, linker: Linker, args: string[], cwd = dir) { await using proc = Bun.spawn({ cmd: [bunExe(), ...args, "--linker", linker], @@ -79,7 +135,6 @@ async function raw(dir: string, linker: Linker, args: string[], cwd = dir) { stderr: "pipe", }); const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); - expect(stderr).not.toContain("panic:"); return { stdout, stderr, exitCode }; } @@ -423,9 +478,133 @@ describe.each(["hoisted", "isolated"] as Linker[])("linker: %s", linker => { expect(await lockText(packageDir)).toBe(pruned); }); + + test.concurrent("a survivor depending on a pruned workspace fails with the workspace error", async () => { + const { packageDir, full } = await verbatimScenario(linker, survivorTree, survivors); + + const { stderr, exitCode } = await raw(packageDir, linker, ["install", "--frozen-lockfile"]); + + expect(stderr).toContain(survivorError("app")); + expect(stderr).toContain(survivorNote); + expect(stderr).not.toContain('Workspace dependency "other" not found'); + expect(stderr).not.toContain("failed to resolve"); + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); + expect(exitCode).toBe(1); + }); + + test.concurrent("a catalog entry only the pruned workspace used may be missing from bun.lock", async () => { + const { packageDir, full } = await verbatimScenario(linker, catalogTree, ["packages/app"]); + const trimmed = trimCatalogLine(full, "left-pad", "1.0.0"); + await write(join(packageDir, "bun.lock"), trimmed); + + const { stderr } = await frozen(packageDir, linker, 0); + + expect(stderr).toContain(prunedNote); + expect(await lockText(packageDir)).toBe(trimmed); + expect(await exists(installedPath(packageDir, linker, "a-dep", "1.0.1"))).toBeTrue(); + expect(await exists(join(packageDir, "node_modules", "left-pad"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "left-pad", "1.0.0"))).toBeFalse(); + }); + + test.concurrent("verbatim lockfile still passes when a surviving workspace has a lifecycle hook", async () => { + const tree = withApp({ scripts: { postinstall: "echo ok" } }); + const { packageDir, full } = await verbatimScenario(linker, tree, survivors); + + await frozen(packageDir, linker, 0); + + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "left-pad", "1.0.0"))).toBeFalse(); + expect(await exists(installedPath(packageDir, linker, "a-dep", "1.0.1"))).toBeTrue(); + }); }); describe("hoisted", () => { + test.concurrent("the root package depending on a pruned workspace is reported", async () => { + const tree: Tree = { ...monorepo, root: { ...rootPackageJson, dependencies: { other: "workspace:*" } } }; + const { packageDir, full } = await verbatimScenario("hoisted", tree, survivors); + + const { stderr, exitCode } = await raw(packageDir, "hoisted", ["install", "--frozen-lockfile"]); + + expect(stderr).toContain(rootSurvivorError); + expect(stderr).toContain(survivorNote); + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules", "other"))).toBeFalse(); + expect(exitCode).toBe(1); + }); + + test.concurrent("--production still reports a devDependencies edge to a pruned workspace", async () => { + const tree = withApp({ devDependencies: { other: "workspace:*" } }); + const { packageDir, full } = await verbatimScenario("hoisted", tree, survivors); + + const { stderr, exitCode } = await raw(packageDir, "hoisted", ["install", "--frozen-lockfile", "--production"]); + + expect(stderr).toContain(survivorError("app")); + expect(await lockText(packageDir)).toBe(full); + expect(exitCode).toBe(1); + }); + + test.concurrent.each(["optionalDependencies", "peerDependencies"])( + "a %s edge to a pruned workspace is reported too", + async group => { + const tree = withApp({ [group]: { other: "workspace:*" } }); + const { packageDir, full } = await verbatimScenario("hoisted", tree, survivors); + + const { stderr, exitCode } = await raw(packageDir, "hoisted", ["install", "--frozen-lockfile"]); + + expect(stderr).toContain(survivorError("app")); + expect(await lockText(packageDir)).toBe(full); + expect(exitCode).toBe(1); + }, + ); + + test.concurrent("every offending survivor is listed", async () => { + const tree: Tree = { + ...survivorTree, + packages: { + ...survivorTree.packages, + "packages/shared": { ...sharedPackageJson, dependencies: { other: "workspace:*" } }, + }, + }; + const { packageDir, full } = await verbatimScenario("hoisted", tree, survivors); + + const { stderr, exitCode } = await raw(packageDir, "hoisted", ["install", "--frozen-lockfile"]); + + expect(stderr).toContain(survivorError("app")); + expect(stderr).toContain(survivorError("shared")); + expect(stderr.split(survivorNote)).toHaveLength(2); + expect(await lockText(packageDir)).toBe(full); + expect(exitCode).toBe(1); + }); + + test.concurrent("--silent suppresses the survivor error but still fails", async () => { + const { packageDir, full } = await verbatimScenario("hoisted", survivorTree, survivors); + + const { stdout, stderr, exitCode } = await raw(packageDir, "hoisted", ["install", "--frozen-lockfile", "--silent"]); + + expect(stderr).not.toContain("depends on workspace"); + expect(stderr).not.toContain("error:"); + expect(stdout).toBe(""); + expect(await lockText(packageDir)).toBe(full); + expect(await exists(join(packageDir, "node_modules"))).toBeFalse(); + expect(exitCode).toBe(1); + }); + + test.concurrent("named catalog group: unreferenced entry may be missing", async () => { + const { packageDir, full } = await verbatimScenario("hoisted", namedCatalogTree, ["packages/app"]); + const trimmed = trimCatalogLine(full, "left-pad", "1.0.0"); + await write(join(packageDir, "bun.lock"), trimmed); + + await frozen(packageDir, "hoisted", 0); + + expect(await lockText(packageDir)).toBe(trimmed); + expect(await exists(join(packageDir, "node_modules", "left-pad"))).toBeFalse(); + expect(await file(join(packageDir, "node_modules", "a-dep", "package.json")).json()).toMatchObject({ + version: "1.0.1", + }); + }); + test.concurrent("bun ci behaves the same on turbo output", async () => { const { packageDir, pruned } = await prunedTree("hoisted"); @@ -506,6 +685,9 @@ describe("hoisted", () => { const { stderr } = await frozen(packageDir, "hoisted", 0); expect(stderr).not.toContain("not on disk"); + expect(await file(join(packageDir, "node_modules", "a-dep", "package.json")).json()).toMatchObject({ + version: "1.0.1", + }); }); test.concurrent("skipped-workspace note is pluralized", async () => { @@ -560,17 +742,13 @@ describe("hoisted", () => { // pnpm#8795: a catalog change is caught by the frozen check before `--lockfile-only` gets a chance to write. test.concurrent("--frozen-lockfile --lockfile-only still fails on a catalog change in a pruned tree", async () => { - const tree: Tree = { - root: { name: "mono", workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.1" } } }, - packages: { - "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "catalog:" } }, - "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "1.0.0" } }, - }, - }; - const { packageDir, full } = await verbatimScenario("hoisted", tree, ["packages/app"]); + const { packageDir, full } = await verbatimScenario("hoisted", catalogTree, ["packages/app"]); await write( join(packageDir, "package.json"), - JSON.stringify({ name: "mono", workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.2" } } }), + JSON.stringify({ + name: "mono", + workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.2", "left-pad": "1.0.0" } }, + }), ); await frozen(packageDir, "hoisted", 1, ["install", "--frozen-lockfile", "--lockfile-only"]); @@ -598,16 +776,21 @@ describe("hoisted", () => { // pnpm#6312 / pnpm#9741: bun.lock's shape does not depend on the linker or on --production/--omit. test.concurrent("pruned bun.lock written with the hoisted linker passes frozen under other settings", async () => { const pruned = turboPrune(await fullLockfile("hoisted")); - const { packageDir } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); - await writeTree(packageDir, turboOutput, survivors); - await write(join(packageDir, "bun.lock"), pruned); + const { packageDir: isolatedDir } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); + await writeTree(isolatedDir, turboOutput, survivors); + await write(join(isolatedDir, "bun.lock"), pruned); - await frozen(packageDir, "isolated", 0); - expect(await lockText(packageDir)).toBe(pruned); - expect(await exists(installedPath(packageDir, "isolated", "a-dep", "1.0.1"))).toBeTrue(); + await frozen(isolatedDir, "isolated", 0); + expect(await lockText(isolatedDir)).toBe(pruned); + expect(await exists(installedPath(isolatedDir, "isolated", "a-dep", "1.0.1"))).toBeTrue(); - await frozen(packageDir, "hoisted", 0, ["install", "--frozen-lockfile", "--production", "--omit=optional"]); - expect(await lockText(packageDir)).toBe(pruned); + const { packageDir: productionDir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" } }); + await writeTree(productionDir, turboOutput, survivors); + await write(join(productionDir, "bun.lock"), pruned); + + await frozen(productionDir, "hoisted", 0, ["install", "--frozen-lockfile", "--production", "--omit=optional"]); + expect(await lockText(productionDir)).toBe(pruned); + expect(await exists(installedPath(productionDir, "hoisted", "a-dep", "1.0.1"))).toBeTrue(); }); // pnpm#5794: a pruned bun.lock a plain install would rewrite also fails frozen (the two checks are the same comparison). @@ -669,15 +852,15 @@ describe("hoisted", () => { }; const { packageDir, fullDir } = await verbatimScenario("hoisted", tree, survivors); expect(await exists(join(fullDir, "other-postinstall.txt"))).toBeTrue(); - expect(await exists(join(fullDir, "node_modules", ".bin", "other-cli"))).toBeTrue(); + for (const bin of binFiles(fullDir, "other-cli")) expect(await exists(bin)).toBeTrue(); const { stdout, stderr } = await frozen(packageDir, "hoisted", 0); expect(stdout + stderr).not.toContain("other-postinstall"); expect(stdout + stderr).not.toContain("ENOENT"); expect(await exists(join(packageDir, "other-postinstall.txt"))).toBeFalse(); - expect(await exists(join(packageDir, "node_modules", ".bin", "other-cli"))).toBeFalse(); - expect(await readlink(join(packageDir, "node_modules", ".bin", "app-cli"))).toContain("app"); + for (const bin of binFiles(packageDir, "other-cli")) expect(await exists(bin)).toBeFalse(); + for (const bin of binFiles(packageDir, "app-cli")) expect(await exists(bin)).toBeTrue(); }); test.concurrent("--production composes with the pruned-workspace filter", async () => { @@ -748,17 +931,7 @@ describe("hoisted", () => { }); test.concurrent("pruned tree using catalogs passes --frozen-lockfile when the catalog is left intact", async () => { - const tree: Tree = { - root: { - name: "mono", - workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.1", "left-pad": "1.0.0" } }, - }, - packages: { - "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "catalog:" } }, - "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "catalog:" } }, - }, - }; - const { packageDir, full } = await verbatimScenario("hoisted", tree, ["packages/app"]); + const { packageDir, full } = await verbatimScenario("hoisted", catalogTree, ["packages/app"]); expect(full).toContain('"left-pad": "1.0.0"'); await frozen(packageDir, "hoisted", 0); @@ -770,29 +943,7 @@ describe("hoisted", () => { }); }); - // Pins the boundary: unlike pnpm, a catalog entry trimmed from bun.lock is a real package.json/lockfile disagreement. - test.concurrent("pruned tree whose bun.lock catalog was trimmed still fails --frozen-lockfile", async () => { - const tree: Tree = { - root: { - name: "mono", - workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.1", "left-pad": "1.0.0" } }, - }, - packages: { - "packages/app": { name: "app", version: "1.0.0", dependencies: { "a-dep": "catalog:" } }, - "packages/other": { name: "other", version: "1.0.0", dependencies: { "left-pad": "catalog:" } }, - }, - }; - const { packageDir, full } = await verbatimScenario("hoisted", tree, ["packages/app"]); - const trimmed = full.replace(/\n +"left-pad": "1\.0\.0",?\n/, "\n"); - expect(trimmed).not.toBe(full); - await write(join(packageDir, "bun.lock"), trimmed); - - await frozen(packageDir, "hoisted", 1); - - expect(await lockText(packageDir)).toBe(trimmed); - }); - - // The guards below pass on main too; they pin the boundaries of the frozen-lockfile relaxation. + // The guards below pass on main too; they pin the boundaries of the pruned-workspace relaxation. test.concurrent("a pruned workspace does not mask a workspace added on disk", async () => { const { packageDir, full } = await verbatimTree("hoisted"); await write(join(packageDir, "packages", "extra", "package.json"), JSON.stringify({ name: "extra" })); @@ -898,26 +1049,90 @@ describe("hoisted", () => { }, ); - // An invalid prune (turbo always keeps transitive workspace deps): the survivor's `workspace:` edge is reported. - test.concurrent("a survivor depending on a pruned workspace fails naming the missing workspace", async () => { + // Fences of the catalog-subset relaxation: only entries no surviving importer or override references may be missing. + test.concurrent("a catalog entry a surviving workspace references must stay in bun.lock", async () => { + const { packageDir, full } = await verbatimScenario("hoisted", catalogTree, ["packages/app"]); + const trimmed = trimCatalogLine(full, "a-dep", "1.0.1"); + await write(join(packageDir, "bun.lock"), trimmed); + + await frozen(packageDir, "hoisted", 1); + + expect(await lockText(packageDir)).toBe(trimmed); + }); + + test.concurrent("a catalog entry an override references must stay in bun.lock", async () => { const tree: Tree = { - root: rootPackageJson, + root: { + name: "mono", + workspaces: { packages: ["packages/*"], catalog: { "no-deps": "1.0.0" } }, + overrides: { "no-deps": "catalog:" }, + }, packages: { - ...monorepo.packages, - "packages/app": { - ...appPackageJson, - dependencies: { ...(appPackageJson.dependencies as object), other: "workspace:*" }, - }, + "packages/app": { name: "app", version: "1.0.0", dependencies: { "one-dep": "1.0.0" } }, + "packages/other": { name: "other", version: "1.0.0" }, }, }; - const { packageDir, full } = await verbatimScenario("hoisted", tree, survivors); + const { packageDir, full } = await verbatimScenario("hoisted", tree, ["packages/app"]); + const trimmed = trimCatalogLine(full, "no-deps", "1.0.0"); + await write(join(packageDir, "bun.lock"), trimmed); const { stderr, exitCode } = await raw(packageDir, "hoisted", ["install", "--frozen-lockfile"]); - expect(stderr).toContain('Workspace dependency "other" not found'); - expect(stderr).toContain("other@workspace:* failed to resolve"); + expect(stderr).toContain("error:"); + expect(await lockText(packageDir)).toBe(trimmed); expect(exitCode).toBe(1); + }); + + test.concurrent("bun.lock may not carry catalog entries package.json lacks", async () => { + const { packageDir, full } = await verbatimScenario("hoisted", catalogTree, ["packages/app"]); + await write( + join(packageDir, "package.json"), + JSON.stringify({ name: "mono", workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.1" } } }), + ); + + await frozen(packageDir, "hoisted", 1); + + expect(await lockText(packageDir)).toBe(full); + }); + + test.concurrent("a differing specifier is still a change even when the lockfile is a subset", async () => { + const { packageDir, full } = await verbatimScenario("hoisted", catalogTree, ["packages/app"]); + const trimmed = trimCatalogLine(full, "left-pad", "1.0.0"); + await write(join(packageDir, "bun.lock"), trimmed); + await write( + join(packageDir, "package.json"), + JSON.stringify({ + name: "mono", + workspaces: { packages: ["packages/*"], catalog: { "a-dep": "1.0.2", "left-pad": "1.0.0" } }, + }), + ); + + const { stderr, exitCode } = await raw(packageDir, "hoisted", ["install", "--frozen-lockfile"]); + + expect(stderr).toContain("error:"); + expect(await lockText(packageDir)).toBe(trimmed); + expect(exitCode).toBe(1); + }); + + test.concurrent("a plain install writes the full catalog back", async () => { + const { packageDir, full } = await verbatimScenario("hoisted", catalogTree, ["packages/app"]); + await write(join(packageDir, "bun.lock"), trimCatalogLine(full, "left-pad", "1.0.0")); + + const { stderr } = await install(packageDir, "hoisted"); + + expect(stderr).toContain("Saved lockfile"); + const lock = await lockText(packageDir); + expect(lock).toContain('"left-pad": "1.0.0"'); + expect(lock).not.toContain('"packages/other"'); + }); + + test.concurrent("verbatim lockfile with a hook still fails on a real change", async () => { + const tree = withApp({ scripts: { postinstall: "echo ok" } }); + const { packageDir, full } = await verbatimScenario("hoisted", tree, survivors); + await editApp(packageDir, app => (app.dependencies["left-pad"] = "1.0.0")); + + await frozen(packageDir, "hoisted", 1); + expect(await lockText(packageDir)).toBe(full); - expect(await lstat(join(packageDir, "node_modules", "other")).catch(e => e.code)).toBe("ENOENT"); }); }); diff --git a/test/cli/install/isolated-relink.test.ts b/test/cli/install/isolated-relink.test.ts index 39e12f1d4766..b44b49d15678 100644 --- a/test/cli/install/isolated-relink.test.ts +++ b/test/cli/install/isolated-relink.test.ts @@ -1,7 +1,7 @@ import { file, write } from "bun"; import { afterAll, beforeAll, expect, test } from "bun:test"; import { lstat, realpath } from "fs/promises"; -import { VerdaccioRegistry, bunEnv, bunExe, normalizeBunSnapshot } from "harness"; +import { VerdaccioRegistry, bunEnv, bunExe } from "harness"; import { dirname, join } from "path"; const registry = new VerdaccioRegistry(); @@ -14,19 +14,15 @@ afterAll(() => { registry.stop(); }); -// `--linker` is passed explicitly: an `install-strategy` in the user's ~/.npmrc overrides the bunfig linker. -async function install(dir: string, ...args: string[]) { +async function install(dir: string): Promise<[stdout: string, stderr: string, exitCode: number]> { await using proc = Bun.spawn({ - cmd: [bunExe(), "install", ...args, "--linker", "isolated"], - env: bunEnv, + cmd: [bunExe(), "install", "--linker", "isolated"], + env: { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(dir, ".bun-cache") }, cwd: dir, stdout: "pipe", stderr: "pipe", }); - const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); - expect(stderr).not.toContain("error:"); - expect(exitCode).toBe(0); - return stdout; + return Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]); } // Store entry names may carry hash suffixes, so reach one-range-dep's store node_modules through the top-level link. @@ -39,16 +35,15 @@ function nestedNoDepsPackageJson(link: string) { return file(join(link, "package.json")).json(); } -test.concurrent("an existing store entry is re-linked when an override re-resolves its dependency", async () => { +test("an existing store entry is re-linked when an override re-resolves its dependency", async () => { const { packageDir, packageJson } = await registry.createTestDir({ bunfigOpts: { linker: "isolated" } }); - const storeHashPath = join(packageDir, "node_modules", ".bun", ".store-hash"); - const storeHash = file(storeHashPath); await write(packageJson, JSON.stringify({ name: "foo", dependencies: { "one-range-dep": "1.0.0" } })); - await install(packageDir); + const [, firstErr, firstExitCode] = await install(packageDir); + expect(firstErr).not.toContain("error:"); + expect(firstExitCode).toBe(0); const link = await nestedNoDeps(packageDir); expect(await nestedNoDepsPackageJson(link)).toEqual({ name: "no-deps", version: "1.1.0" }); - const storeHashAfterFirstInstall = await storeHash.text(); await write( packageJson, @@ -58,18 +53,18 @@ test.concurrent("an existing store entry is re-linked when an override re-resolv overrides: { "no-deps": "1.0.0" }, }), ); - const out = await install(packageDir); + const [out, err, exitCode] = await install(packageDir); expect(await nestedNoDepsPackageJson(link)).toEqual({ name: "no-deps", version: "1.0.0" }); expect(out).toMatch(/\d+ packages? installed/); expect(out).not.toContain("(no changes)"); - expect(await storeHash.text()).not.toBe(storeHashAfterFirstInstall); + expect(err).not.toContain("error:"); + expect(exitCode).toBe(0); const linkMtime = (await lstat(link)).mtimeMs; - const storeHashMtime = (await lstat(storeHashPath)).mtimeMs; - const again = await install(packageDir); - expect(normalizeBunSnapshot(again)).toContain("(no changes)"); + const [again, againErr, againExitCode] = await install(packageDir); + expect(again).toContain("(no changes)"); + expect(againErr).not.toContain("error:"); + expect(againExitCode).toBe(0); expect((await lstat(link)).mtimeMs).toBe(linkMtime); expect(await nestedNoDepsPackageJson(link)).toEqual({ name: "no-deps", version: "1.0.0" }); - expect(await storeHash.exists()).toBe(true); - expect((await lstat(storeHashPath)).mtimeMs).toBe(storeHashMtime); }); diff --git a/test/cli/install/lockfile-only.test.ts b/test/cli/install/lockfile-only.test.ts index 4e52ddd6f180..516d0d2da78c 100644 --- a/test/cli/install/lockfile-only.test.ts +++ b/test/cli/install/lockfile-only.test.ts @@ -136,7 +136,9 @@ describe("--lockfile-only under --frozen-lockfile", () => { async flag => { using tmp = tempDir("lockfile-only-frozen", project); const dir = String(tmp); - expect((await run(dir, "--lockfile-only")).exitCode).toBe(0); + const setup = await run(dir, "--lockfile-only"); + expect(setup.stderr).not.toContain("error:"); + expect(setup.exitCode).toBe(0); const canary = readFileSync(lock(dir), "utf8") + "\n"; writeFileSync(lock(dir), canary); @@ -165,7 +167,9 @@ describe("--lockfile-only under --frozen-lockfile", () => { it.concurrent("--lockfile-only rewrites an up-to-date bun.lock", async () => { using tmp = tempDir("lockfile-only-rewrite", project); const dir = String(tmp); - expect((await run(dir, "--lockfile-only")).exitCode).toBe(0); + const setup = await run(dir, "--lockfile-only"); + expect(setup.stderr).not.toContain("error:"); + expect(setup.exitCode).toBe(0); const original = readFileSync(lock(dir), "utf8"); writeFileSync(lock(dir), original + "\n"); diff --git a/test/cli/install/migration/__snapshots__/pnpm-comprehensive.test.ts.snap b/test/cli/install/migration/__snapshots__/pnpm-comprehensive.test.ts.snap index 03f769bbb384..af8a1be1963e 100644 --- a/test/cli/install/migration/__snapshots__/pnpm-comprehensive.test.ts.snap +++ b/test/cli/install/migration/__snapshots__/pnpm-comprehensive.test.ts.snap @@ -33,16 +33,22 @@ exports[`pnpm comprehensive migration tests large single package with many depen "optionalDependencies": { "fsevents": "^2.3.3", }, + "peerDependencies": { + "react-native": ">=0.72.0", + }, + "optionalPeers": [ + "react-native", + ], }, }, "packages": { - "@emotion/react": ["@emotion/react@11.11.3", "", { "peerDependencies": { "react": "18.2.0" } }, ""], + "@emotion/react": ["@emotion/react@11.11.3", "", { "peerDependencies": { "react": ">=16.8.0" } }, ""], - "@emotion/styled": ["@emotion/styled@11.11.0", "", { "peerDependencies": { "@emotion/react": "11.11.3", "react": "18.2.0" } }, ""], + "@emotion/styled": ["@emotion/styled@11.11.0", "", { "peerDependencies": { "@emotion/react": "^11.0.0", "react": ">=16.8.0" } }, ""], "@experimental/super-long-scoped-package-name-with-many-words": ["@experimental/super-long-scoped-package-name-with-many-words@0.0.0-experimental-abcdef123456-20250812-build.9876543210", "", {}, "sha512-experimentalAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], - "@tanstack/react-query": ["@tanstack/react-query@5.17.9", "", { "peerDependencies": { "react": "18.2.0" } }, "sha512-tanstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], + "@tanstack/react-query": ["@tanstack/react-query@5.17.9", "", { "peerDependencies": { "react": ">=18.0.0" } }, "sha512-tanstackAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], "@types/node": ["@types/node@20.10.8", "", {}, ""], @@ -70,13 +76,13 @@ exports[`pnpm comprehensive migration tests large single package with many depen "negotiator": ["negotiator@0.6.3", "", {}, ""], - "next": ["next@14.0.4", "", { "peerDependencies": { "react": "18.2.0", "react-dom": "18.2.0" }, "bin": { "next": "dist/bin/next" } }, "sha512-nextAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], + "next": ["next@14.0.4", "", { "peerDependencies": { "react": "^18.2.0", "react-dom": "^18.2.0" }, "bin": { "next": "dist/bin/next" } }, "sha512-nextAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], "prettier": ["prettier@3.1.1", "", { "bin": { "prettier": "bin/prettier.cjs" } }, "sha512-prettierAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], "react": ["react@18.2.0", "", { "dependencies": { "loose-envify": "1.4.0" } }, ""], - "react-dom": ["react-dom@18.2.0", "", { "dependencies": { "scheduler": "0.23.0" }, "peerDependencies": { "react": "18.2.0" } }, ""], + "react-dom": ["react-dom@18.2.0", "", { "dependencies": { "scheduler": "0.23.0" }, "peerDependencies": { "react": "^18.2.0" } }, ""], "scheduler": ["scheduler@0.23.0", "", { "dependencies": { "loose-envify": "1.4.0" } }, ""], @@ -194,7 +200,7 @@ exports[`pnpm comprehensive migration tests complex monorepo with cross-dependen "@company/web": ["@company/web@workspace:apps/web"], - "@radix-ui/react-dialog": ["@radix-ui/react-dialog@1.0.5", "", { "peerDependencies": { "react": "18.2.0", "react-dom": "18.2.0" } }, ""], + "@radix-ui/react-dialog": ["@radix-ui/react-dialog@1.0.5", "", { "peerDependencies": { "react": "^16.8 || ^17.0 || ^18.0", "react-dom": "^16.8 || ^17.0 || ^18.0" } }, ""], "@types/cors": ["@types/cors@2.8.17", "", {}, ""], @@ -224,7 +230,7 @@ exports[`pnpm comprehensive migration tests complex monorepo with cross-dependen "loose-envify": ["loose-envify@1.4.0", "", { "dependencies": { "js-tokens": "4.0.0" }, "bin": { "loose-envify": "cli.js" } }, ""], - "next": ["next@14.0.4", "", { "peerDependencies": { "react": "18.2.0", "react-dom": "18.2.0" }, "bin": { "next": "dist/bin/next" } }, "sha512-nextAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], + "next": ["next@14.0.4", "", { "peerDependencies": { "react": "^18.2.0", "react-dom": "^18.2.0" }, "bin": { "next": "dist/bin/next" } }, "sha512-nextAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], "nodemon": ["nodemon@3.0.2", "", { "bin": { "nodemon": "bin/nodemon.js" } }, ""], @@ -232,7 +238,7 @@ exports[`pnpm comprehensive migration tests complex monorepo with cross-dependen "react": ["react@18.2.0", "", { "dependencies": { "loose-envify": "1.4.0" } }, ""], - "react-dom": ["react-dom@18.2.0", "", { "dependencies": { "scheduler": "0.23.0" }, "peerDependencies": { "react": "18.2.0" } }, ""], + "react-dom": ["react-dom@18.2.0", "", { "dependencies": { "scheduler": "0.23.0" }, "peerDependencies": { "react": "^18.2.0" } }, ""], "scheduler": ["scheduler@0.23.0", "", { "dependencies": { "loose-envify": "1.4.0" } }, ""], @@ -248,7 +254,7 @@ exports[`pnpm comprehensive migration tests complex monorepo with cross-dependen exports[`pnpm comprehensive migration tests pnpm with patches and overrides: patches-overrides 1`] = ` "{ - "lockfileVersion": 2, + "lockfileVersion": 3, "configVersion": 1, "workspaces": { "": { @@ -264,7 +270,9 @@ exports[`pnpm comprehensive migration tests pnpm with patches and overrides: pat }, "overrides": { "mime-types": "2.1.33", - "negotiator@>0.6.0": "0.6.2", + "negotiator@>0.6.0": { + ".": "0.6.2", + }, }, "packages": { "accepts": ["accepts@1.3.8", "", { "dependencies": { "mime-types": "2.1.33", "negotiator": "0.6.2" } }, "sha512-acceptsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], @@ -307,9 +315,9 @@ exports[`pnpm comprehensive migration tests pnpm with peer dependencies and auto "packages": { "@angular/animations": ["@angular/animations@17.0.8", "", { "dependencies": { "tslib": "2.6.2" }, "peerDependencies": { "@angular/core": "17.0.8" } }, ""], - "@angular/common": ["@angular/common@17.0.8", "", { "dependencies": { "tslib": "2.6.2" }, "peerDependencies": { "@angular/core": "17.0.8", "rxjs": "7.8.1" } }, ""], + "@angular/common": ["@angular/common@17.0.8", "", { "dependencies": { "tslib": "2.6.2" }, "peerDependencies": { "@angular/core": "17.0.8", "rxjs": "^6.5.3 || ^7.4.0" } }, ""], - "@angular/core": ["@angular/core@17.0.8", "", { "dependencies": { "tslib": "2.6.2" }, "peerDependencies": { "rxjs": "7.8.1", "zone.js": "0.14.2" } }, ""], + "@angular/core": ["@angular/core@17.0.8", "", { "dependencies": { "tslib": "2.6.2" }, "peerDependencies": { "rxjs": "^6.5.3 || ^7.4.0", "zone.js": "~0.14.0" } }, ""], "rxjs": ["rxjs@7.8.1", "", { "dependencies": { "tslib": "2.6.2" } }, "sha512-rxjsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], diff --git a/test/cli/install/migration/__snapshots__/pnpm-migration-complete.test.ts.snap b/test/cli/install/migration/__snapshots__/pnpm-migration-complete.test.ts.snap index 50b5e5571231..58de333615b4 100644 --- a/test/cli/install/migration/__snapshots__/pnpm-migration-complete.test.ts.snap +++ b/test/cli/install/migration/__snapshots__/pnpm-migration-complete.test.ts.snap @@ -194,15 +194,15 @@ exports[`PNPM Migration Complete Test Suite comprehensive PNPM migration with al }, }, "packages": { - "@emotion/react": ["@emotion/react@11.11.3", "", { "peerDependencies": { "react": "18.2.0" } }, ""], + "@emotion/react": ["@emotion/react@11.11.3", "", { "peerDependencies": { "react": ">=16.8.0" } }, ""], - "@emotion/styled": ["@emotion/styled@11.11.0", "", { "peerDependencies": { "@emotion/react": "11.11.3", "react": "18.2.0" } }, ""], + "@emotion/styled": ["@emotion/styled@11.11.0", "", { "peerDependencies": { "@emotion/react": "^11.0.0", "react": ">=16.8.0" } }, ""], - "@mui/material": ["@mui/material@5.15.0", "", { "optionalDependencies": { "@emotion/react": "11.11.3", "@emotion/styled": "11.11.0" }, "peerDependencies": { "react": "18.2.0", "react-dom": "18.2.0" } }, ""], + "@mui/material": ["@mui/material@5.15.0", "", { "peerDependencies": { "@emotion/react": "^11.5.0", "@emotion/styled": "^11.3.0", "react": "^17.0.0 || ^18.0.0", "react-dom": "^17.0.0 || ^18.0.0" }, "optionalPeers": ["@emotion/react", "@emotion/styled"] }, ""], "react": ["react@18.2.0", "", {}, ""], - "react-dom": ["react-dom@18.2.0", "", { "peerDependencies": { "react": "18.2.0" } }, "sha512-reactdomAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], + "react-dom": ["react-dom@18.2.0", "", { "peerDependencies": { "react": "^18.2.0" } }, "sha512-reactdomAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="], } } " @@ -332,6 +332,9 @@ exports[`PNPM Migration Complete Test Suite comprehensive PNPM migration with al "optionalDependencies": { "fsevents": "^2.3.3", }, + "peerDependencies": { + "react": ">=16.0.0", + }, }, }, "packages": { diff --git a/test/cli/install/migration/pnpm-lock-v9.test.ts b/test/cli/install/migration/pnpm-lock-v9.test.ts index 6c61d2a24de7..4a7fd37e6456 100644 --- a/test/cli/install/migration/pnpm-lock-v9.test.ts +++ b/test/cli/install/migration/pnpm-lock-v9.test.ts @@ -13,11 +13,11 @@ afterAll(() => { verdaccio.stop(); }); -async function migrate(cwd: string) { +async function run(cwd: string, ...args: string[]) { await using proc = Bun.spawn({ - cmd: [bunExe(), "pm", "migrate"], + cmd: [bunExe(), ...args], cwd, - env: bunEnv, + env: { ...bunEnv, BUN_INSTALL_CACHE_DIR: join(cwd, ".bun-cache") }, stdout: "pipe", stderr: "pipe", }); @@ -27,6 +27,10 @@ async function migrate(cwd: string) { return { stdout, stderr, exitCode }; } +function migrate(cwd: string) { + return run(cwd, "pm", "migrate"); +} + function fixture(name: string) { return tempDir(`pnpm-${name}`, join(import.meta.dir, "pnpm", name)); } @@ -35,9 +39,48 @@ async function bunLockOf(dir: string) { return await Bun.file(join(dir, "bun.lock")).text(); } +function workspacesSection(bunLock: string) { + const start = bunLock.indexOf(` "workspaces": {`); + const end = bunLock.indexOf(` "packages": {`); + expect(start).not.toBe(-1); + expect(end).not.toBe(-1); + return bunLock.slice(start, end); +} + +function workspaceBlock(bunLock: string, key: string) { + const start = bunLock.indexOf(` "${key}": {\n`); + expect(start).not.toBe(-1); + const end = bunLock.indexOf("\n },", start); + expect(end).not.toBe(-1); + return bunLock.slice(start, end + "\n },".length); +} + +async function installedPackageJson(root: string, workspace: string, name: string) { + const nested = Bun.file(join(root, workspace, "node_modules", name, "package.json")); + return await ((await nested.exists()) ? nested : Bun.file(join(root, "node_modules", name, "package.json"))).json(); +} + const PKG_A_GIT = "pkg-a@git+ssh://git@example.com/org/pkg-a.git#0123456789abcdef0123456789abcdef01234567"; +const NO_DEPS_1_0_0_INTEGRITY = + "sha512-v4w12JRjUGvfHDUP8vFDwu0gUWu04j0cv9hLb1Abf9VdaXu4XcrddYFTMVBVvmldKViGWH7jrb6xPJRF0wq6gw=="; const NO_DEPS_1_0_1_INTEGRITY = "sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A=="; +const NO_DEPS_2_0_0_INTEGRITY = + "sha512-W3duJKZPcMIG5rA1io5cSK/bhW9rWFz+jFxZsKS/3suK4qHDkQNxUTEXee9/hTaAoDCeHWQqogukWYKzfr6X4g=="; +const ONE_DEP_1_0_0_INTEGRITY = + "sha512-qG6lZjwM1vFmRCHwP+XpOKu6FkrBmwr20+54+qaHGdjZlw/wz8aJrhFqX4dZksqmBLZtj2mzL77Yf04WKs1+Kg=="; +const A_DEP_1_0_1_INTEGRITY = + "sha512-6nmTaPgO2U/uOODqOhbjbnaB4xHuZ+UB7AjKUA3g2dT4WRWeNxgp0dC8Db4swXSnO5/uLLUdFmUJKINNBO/3wg=="; +const PEER_DEPS_1_0_0_INTEGRITY = + "sha512-CHQ5sQXwUo38G++dkzJ/rJ9Ge98MeMTQjjC9UK2t0frp8Lrhm3zNooOLakFyHW4UcyD3vuTS3Qv324Bj6B5Tjw=="; +const PEER_DEPS_FIXED_1_0_0_INTEGRITY = + "sha512-gVs9cSdy6TAQIEWu1tVEK1mAspCQxYziTGQlv4a2XQpzOBZvoQ/y6lOeu3tqNNrNQnLwdvwAQTlvazV5+HfV7g=="; +const PEER_DEPS_TOO_1_0_0_INTEGRITY = + "sha512-sBx0TKrsB8FkRN2lzkDjMuctPGEKn1TmNUBv3dJOtnZM8nd255o5ZAPRpAI2XFLHZAavBlK/e73cZNwnUxlRog=="; +const ONE_OPTIONAL_PEER_DEP_1_0_2_INTEGRITY = + "sha512-S25U8/QXGIKfn/AWtsce1aVMnDjDL+ykFtAufpsuKGad32NlsCpi9TDuXvzoTQ+MdaZpGV3c4xghUZUsNeMp4A=="; +const LOCAL_TARBALL_INTEGRITY = + "sha512-HP/5Rgt3pVFLzjmN9qJJ6vZMgCwoCIl/m2bPndYT283CUqnmFiMx0GeeIJ7SyK6TYoJM78SEvFEOQie++caHqw=="; function registryLockfileWithTarball(tarball: string) { return `lockfileVersion: '9.0' @@ -61,6 +104,28 @@ snapshots: `; } +function registryQualifiedNoDepsLockfile(registry: string) { + return `lockfileVersion: '9.0' + +importers: + + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: ${registry}:1.0.1 + +packages: + + no-deps@${registry}:1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + +snapshots: + + no-deps@${registry}:1.0.1: {} +`; +} + describe("pnpm-lock.yaml v9", () => { // Cases using toMatchSnapshot are sequential: snapshot matchers are unsupported inside a concurrent group. test("v9 git and userinfo-tarball references migrate", async () => { @@ -106,23 +171,11 @@ describe("pnpm-lock.yaml v9", () => { expect(bunLock).toContain(`"aliased-no-deps": "npm:no-deps@2.0.0"`); expect(bunLock).toContain(`"aliased-no-deps": ["no-deps@2.0.0"`); - await using install = Bun.spawn({ - cmd: [bunExe(), "install", "--frozen-lockfile"], - cwd: packageDir, - env: bunEnv, - stdout: "pipe", - stderr: "pipe", - }); - - const [installStdout, installStderr, installExitCode] = await Promise.all([ - install.stdout.text(), - install.stderr.text(), - install.exited, - ]); + const install = await run(packageDir, "install", "--frozen-lockfile"); - expect(installStderr).not.toContain("error:"); - expect(installStdout).toContain("packages installed"); - expect(installExitCode).toBe(0); + expect(install.stderr).not.toContain("error:"); + expect(install.stdout).toContain("packages installed"); + expect(install.exitCode).toBe(0); expect(nodeModulesPackages(packageDir)).toMatchInlineSnapshot(` "node_modules/aliased-no-deps/no-deps@2.0.0 @@ -187,7 +240,90 @@ describe("pnpm-lock.yaml v9", () => { bunfigOpts: { linker: "hoisted" }, files: { "package.json": JSON.stringify({ name: "registry-qualified", dependencies: { "no-deps": "^1.0.0" } }), - "pnpm-lock.yaml": `lockfileVersion: '9.0' + "pnpm-lock.yaml": registryQualifiedNoDepsLockfile("work"), + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain( + "pnpm-lock.yaml packages from pnpm registry 'work' are not in namedRegistries of pnpm-workspace.yaml, resolving them from the configured registry instead", + ); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"no-deps@1.0.1"`); + expect(bunLock).not.toContain("work:"); + expect(bunLock).not.toContain("git+ssh"); + }); + + describe("named registries", () => { + // pnpm11/lockfile/utils/src/pkgSnapshotToResolution.ts: named registry -> scope registry -> default + test("built-in npmjs: entries record the npmjs registry", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ name: "npmjs-qualified", dependencies: { "no-deps": "^1.0.0" } }), + "pnpm-lock.yaml": registryQualifiedNoDepsLockfile("npmjs"), + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain( + "pnpm-lock.yaml packages from pnpm registry 'npmjs' will be fetched from https://registry.npmjs.org/; add that registry to bunfig.toml or .npmrc if it needs authentication", + ); + expect(stderr).not.toContain("not in namedRegistries"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + // bun.lock spells the default registry as "" (bun.lock.rs url_is_under_registry(DEFAULT_URL)). + expect(bunLock).toContain(`["no-deps@1.0.1", "", {}, "${NO_DEPS_1_0_1_INTEGRITY}"]`); + expect(bunLock).not.toContain(verdaccio.registryUrl()); + expect(bunLock).not.toContain("npmjs:"); + }); + + test("namedRegistries entry pointing at the configured registry needs no warning", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ name: "named-registry-same", dependencies: { "no-deps": "^1.0.0" } }), + "pnpm-workspace.yaml": `namedRegistries:\n work: ${verdaccio.registryUrl()}\n`, + "pnpm-lock.yaml": registryQualifiedNoDepsLockfile("work"), + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).not.toContain("pnpm registry"); + expect(stderr).not.toContain("warn:"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"no-deps@1.0.1"`); + expect(bunLock).not.toContain("work:"); + + const install = await run(packageDir, "install", "--frozen-lockfile"); + + expect(install.stderr).not.toContain("error:"); + expect(install.exitCode).toBe(0); + expect(nodeModulesPackages(packageDir)).toMatchInlineSnapshot(`"node_modules/no-deps/no-deps@1.0.1"`); + }); + + test("namedRegistries entry pointing at another registry is used for the tarballs", async () => { + const named = `http://127.0.0.1:${verdaccio.port}/`; + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "named-registry-other", + dependencies: { "no-deps": "^1.0.0", "peer-deps-fixed": "^1.0.0" }, + }), + "pnpm-workspace.yaml": `namedRegistries:\n work: ${named}\n`, + "pnpm-lock.yaml": `lockfileVersion: '9.0' importers: @@ -196,31 +332,112 @@ importers: no-deps: specifier: ^1.0.0 version: work:1.0.1 + peer-deps-fixed: + specifier: ^1.0.0 + version: work:1.0.0(no-deps@work:1.0.1) packages: no-deps@work:1.0.1: - resolution: {integrity: sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A==} + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + + peer-deps-fixed@work:1.0.0: + resolution: {integrity: ${PEER_DEPS_FIXED_1_0_0_INTEGRITY}} + peerDependencies: + no-deps: ^1.0.0 snapshots: no-deps@work:1.0.1: {} + + peer-deps-fixed@work:1.0.0(no-deps@work:1.0.1): + dependencies: + no-deps: work:1.0.1 `, - }, + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain( + `pnpm-lock.yaml packages from pnpm registry 'work' will be fetched from ${named}; add that registry to bunfig.toml or .npmrc if it needs authentication`, + ); + expect(stderr.split("pnpm registry 'work'").length - 1).toBe(1); + expect(stderr).not.toContain("not in namedRegistries"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain( + `["no-deps@1.0.1", "${named}no-deps/-/no-deps-1.0.1.tgz", {}, "${NO_DEPS_1_0_1_INTEGRITY}"]`, + ); + expect(bunLock).toContain( + `["peer-deps-fixed@1.0.0", "${named}peer-deps-fixed/-/peer-deps-fixed-1.0.0.tgz", { "peerDependencies": { "no-deps": "^1.0.0" } }, "${PEER_DEPS_FIXED_1_0_0_INTEGRITY}"]`, + ); + expect(bunLock).not.toContain(verdaccio.registryUrl()); + expect(bunLock).not.toContain("work:"); + + const install = await run(packageDir, "install", "--frozen-lockfile"); + + expect(install.stderr).not.toContain("error:"); + expect(install.exitCode).toBe(0); + expect(nodeModulesPackages(packageDir)).toMatchInlineSnapshot(` + "node_modules/no-deps/no-deps@1.0.1 + node_modules/peer-deps-fixed/peer-deps-fixed@1.0.0" + `); }); - const { stderr, exitCode } = await migrate(packageDir); + test("two packages from one unknown registry warn once", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "unknown-registry-twice", + dependencies: { "no-deps": "^1.0.0", "one-dep": "^1.0.0" }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' - expect(stderr).toContain( - "pnpm-lock.yaml package 'no-deps@work:1.0.1' is from pnpm registry 'work', resolving it from the configured registry instead", - ); - expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); - expect(exitCode).toBe(0); +importers: - const bunLock = await bunLockOf(packageDir); - expect(bunLock).toContain(`"no-deps@1.0.1"`); - expect(bunLock).not.toContain("work:"); - expect(bunLock).not.toContain("git+ssh"); + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: work:1.0.1 + one-dep: + specifier: ^1.0.0 + version: work:1.0.0 + +packages: + + no-deps@work:1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + + one-dep@work:1.0.0: + resolution: {integrity: ${ONE_DEP_1_0_0_INTEGRITY}} + +snapshots: + + no-deps@work:1.0.1: {} + + one-dep@work:1.0.0: + dependencies: + no-deps: work:1.0.1 +`, + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr.split("pnpm registry 'work'").length - 1).toBe(1); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"no-deps@1.0.1"`); + expect(bunLock).toContain(`"one-dep@1.0.0"`); + expect(bunLock).not.toContain("work:"); + }); }); test.concurrent("reports a package whose resolution cannot be parsed", async () => { @@ -440,21 +657,64 @@ importers: expect(bunLock).toMatchSnapshot("bun.lock"); }); - test.concurrent("local file: tarballs with tarball+integrity and integrity-only .tar.gz resolutions", async () => { - // tar-pkg entry ported from pnpm11/installing/deps-restorer/test/fixtures/has-local-dep/pkg/pnpm-lock.yaml - using dir = fixture("v9-local-tarballs"); + describe("local file: tarballs", () => { + test.concurrent("tarball+integrity, integrity-only .tar.gz, and upper-case / .tar spellings", async () => { + // tar-pkg entry ported from pnpm11/installing/deps-restorer/test/fixtures/has-local-dep/pkg/pnpm-lock.yaml + using dir = fixture("v9-local-tarballs"); - const { stderr, exitCode } = await migrate(String(dir)); + const { stderr, exitCode } = await migrate(String(dir)); - expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); - expect(exitCode).toBe(0); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); - const bunLock = await bunLockOf(String(dir)); - expect(bunLock).toContain( - `["tar-pkg@../tar-pkg-1.0.0.tgz", {}, "sha512-HP/5Rgt3pVFLzjmN9qJJ6vZMgCwoCIl/m2bPndYT283CUqnmFiMx0GeeIJ7SyK6TYoJM78SEvFEOQie++caHqw=="]`, + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`["tar-pkg@../tar-pkg-1.0.0.tgz", {}, "${LOCAL_TARBALL_INTEGRITY}"]`); + expect(bunLock).toContain(`["tar-gz-pkg@../tar-gz-pkg-1.0.0.tar.gz", {}, "sha512-`); + expect(bunLock).toContain(`["plain@../plain-1.0.0.tar", {}, "sha512-`); + expect(bunLock).toContain(`["upper@../UPPER-1.0.0.TGZ", {}, "sha512-`); + expect(bunLock).not.toContain("tar-gz-pkg@file:"); + expect(bunLock).not.toContain("plain@file:"); + expect(bunLock).not.toContain("upper@file:"); + }); + + // pnpm11/lockfile/utils/src/refIsLocalTarball.ts is case-insensitive and accepts .tar + test.concurrent.each(["up-1.0.0.TGZ", "mixed-1.0.0.Tar.Gz", "plain-1.0.0.tar"])( + "integrity-only file: entry ending in %s is a tarball, not a folder", + async file => { + using dir = tempDir("pnpm-v9-local-tarball-spelling", { + "package.json": JSON.stringify({ name: "tarball-spelling", dependencies: { pkg: `file:vendor/${file}` } }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +importers: + + .: + dependencies: + pkg: + specifier: file:vendor/${file} + version: file:vendor/${file} + +packages: + + pkg@file:vendor/${file}: + resolution: {integrity: ${LOCAL_TARBALL_INTEGRITY}} + version: 1.0.0 + +snapshots: + + pkg@file:vendor/${file}: {} +`, + }); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`["pkg@vendor/${file}", {}, "${LOCAL_TARBALL_INTEGRITY}"]`); + expect(bunLock).not.toContain(`pkg@file:vendor/${file}`); + }, ); - expect(bunLock).toContain(`["tar-gz-pkg@../tar-gz-pkg-1.0.0.tar.gz", {}, "sha512-`); - expect(bunLock).not.toContain("tar-gz-pkg@file:"); }); test.concurrent("file: directory with type: directory and a nested file: dependency", async () => { @@ -516,16 +776,11 @@ importers: expect(bunLock).toContain(`"foo": ["foo@workspace:packages/foo"]`); expect(bunLock).not.toContain("foo@file:"); - await using install = Bun.spawn({ - cmd: [bunExe(), "install", "--frozen-lockfile", "--linker", "hoisted"], - cwd: String(dir), - env: bunEnv, - stdout: "pipe", - stderr: "pipe", - }); - const [installStderr, installExitCode] = await Promise.all([install.stderr.text(), install.exited]); - expect(installStderr).not.toContain("error:"); - expect(installExitCode).toBe(0); + const install = await run(String(dir), "install", "--frozen-lockfile", "--linker", "hoisted"); + + expect(install.stderr).not.toContain("error:"); + expect(await installedPackageJson(String(dir), "", "foo")).toEqual({ name: "foo", version: "1.0.0" }); + expect(install.exitCode).toBe(0); }); test.concurrent( @@ -548,104 +803,971 @@ importers: ); }); - describe("registry tarball: urls", () => { - // pnpm/pnpm#13534: GitHub Packages / npm Enterprise tarballs are not on the canonical `/-/` path - test("recorded under the configured registry is kept", async () => { - const registry = verdaccio.registryUrl(); - const tarball = `${registry}download/no-deps/1.0.1/0123456789abcdef`; + describe("pruned snapshots", () => { + // pnpm11/lockfile/fs convertToLockfileObject rebuilds `file:` directories whose packages: entry turbo prune dropped + test("file: variants without a packages entry are rebuilt", async () => { const { packageDir } = await verdaccio.createTestDir({ bunfigOpts: { linker: "hoisted" }, - files: { - "package.json": JSON.stringify({ name: "kept-tarball", dependencies: { "no-deps": "^1.0.0" } }), - "pnpm-lock.yaml": registryLockfileWithTarball(tarball), - }, + files: join(import.meta.dir, "pnpm/v9-snapshot-only-file-variants"), }); const { stderr, exitCode } = await migrate(packageDir); + expect(stderr).not.toContain("no package entry"); expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); expect(exitCode).toBe(0); const bunLock = await bunLockOf(packageDir); - expect(bunLock).toContain(`["no-deps@1.0.1", "${tarball}", {}, "${NO_DEPS_1_0_1_INTEGRITY}"]`); + expect(bunLock).toContain(`"dir": ["dir@workspace:packages/dir"]`); + expect(bunLock).toContain(`["local@file:vendor/local", { "dependencies": { "no-deps": "1.0.0" } }]`); + expect(bunLock).not.toContain("dir@file:"); + expect(bunLock).not.toContain("tb@"); }); - // pnpm/pnpm#5920 / #4361: a stale or injected off-registry tarball is rebuilt from the configured registry - test("recorded on a foreign host is rebuilt from the configured registry", async () => { - const registry = verdaccio.registryUrl(); - const { packageDir } = await verdaccio.createTestDir({ - bunfigOpts: { linker: "hoisted" }, - files: { - "package.json": JSON.stringify({ name: "foreign-tarball", dependencies: { "no-deps": "^1.0.0" } }), - "pnpm-lock.yaml": registryLockfileWithTarball("https://evil.example.com/no-deps/-/no-deps-1.0.1.tgz"), - }, - }); + test.concurrent("a lockfile with snapshots but no packages section migrates", async () => { + using dir = tempDir("pnpm-v9-snapshots-only", { + "package.json": JSON.stringify({ name: "snapshots-only", dependencies: { local: "file:vendor/local" } }), + "vendor/local/package.json": JSON.stringify({ name: "local", version: "1.0.0" }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' - const { stderr, exitCode } = await migrate(packageDir); +importers: - expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); - expect(exitCode).toBe(0); + .: + dependencies: + local: + specifier: file:vendor/local + version: file:vendor/local(x@1.0.0) - const bunLock = await bunLockOf(packageDir); - expect(bunLock).toContain( - `["no-deps@1.0.1", "${registry}no-deps/-/no-deps-1.0.1.tgz", {}, "${NO_DEPS_1_0_1_INTEGRITY}"]`, - ); - expect(bunLock).not.toContain("evil.example.com"); - }); +snapshots: - test("under a registry whose hostname is a prefix of the recorded url's is rebuilt", async () => { - const registry = verdaccio.registryUrl(); - const lookalike = `${registry.slice(0, -1)}.evil.example.com/no-deps/-/no-deps-1.0.1.tgz`; - const { packageDir } = await verdaccio.createTestDir({ - bunfigOpts: { linker: "hoisted" }, - files: { - "package.json": JSON.stringify({ name: "lookalike-tarball", dependencies: { "no-deps": "^1.0.0" } }), - "pnpm-lock.yaml": registryLockfileWithTarball(lookalike), - }, + local@file:vendor/local(x@1.0.0): {} +`, }); - const { stderr, exitCode } = await migrate(packageDir); + const { stderr, exitCode } = await migrate(String(dir)); + expect(stderr).not.toContain("no package entry"); expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); expect(exitCode).toBe(0); - const bunLock = await bunLockOf(packageDir); - expect(bunLock).toContain(`"${registry}no-deps/-/no-deps-1.0.1.tgz"`); - expect(bunLock).not.toContain("evil.example.com"); - }); - }); - - describe("git sub-directory dependencies", () => { - // pnpm/pnpm#8243: `repo#commit&path:sub/dir` has no bun equivalent; refuse instead of installing the repo root - test.concurrent("type: git resolution with path: is rejected naming the package", async () => { - using dir = fixture("v9-git-subdirectory"); - - const { stderr, exitCode } = await migrate(String(dir)); - - expect(stderr).toContain( - "pnpm-lock.yaml package 'pkg@git+ssh://git@example.com/org/monorepo.git#cba04669e621b85fbdb33371604de1a2898e68e9&path:packages/pkg' is a git sub-directory dependency", - ); - expect(exitCode).toBe(1); - expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + const bunLock = await bunLockOf(String(dir)); + expect(bunLock).toContain(`["local@file:vendor/local", {}]`); }); - test.concurrent("git-hosted tarball resolution with path: is rejected naming the package", async () => { - // shape from pnpm11/resolving/git-resolver/test/index.ts "with both sub folder and branch" - const id = - "https://codeload.github.com/o/mono/tar.gz/777e8a3e78cc89bbf41fb3fd9f6cf922d5463313#path:/packages/pkg"; - using dir = tempDir("pnpm-v9-git-hosted-subdirectory", { - "package.json": JSON.stringify({ - name: "git-hosted-subdirectory", - dependencies: { pkg: "github:o/mono#beta&path:/packages/pkg" }, - }), - "pnpm-lock.yaml": `lockfileVersion: '9.0' + // guard: only directories are rebuilt; a tarball needs the integrity its packages: entry carried + test.concurrent.each(["tb-1.0.0.tgz", "tb-1.0.0.TGZ", "tb-1.0.0.Tar.Gz"])( + "a referenced snapshot-only tarball variant (%s) is still reported", + async file => { + using dir = tempDir("pnpm-v9-snapshot-only-tarball", { + "package.json": JSON.stringify({ + name: "snapshot-only-tarball", + dependencies: { tb: `file:vendor/${file}` }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' importers: .: dependencies: - pkg: - specifier: github:o/mono#beta&path:/packages/pkg + tb: + specifier: file:vendor/${file} + version: file:vendor/${file}(x@1.0.0) + +snapshots: + + tb@file:vendor/${file}(x@1.0.0): {} +`, + }); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain( + `pnpm-lock.yaml has no package entry 'tb@file:vendor/${file}' for dependency 'tb' of importer '.'`, + ); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }, + ); + }); + + describe("peer dependencies", () => { + test("packages keep their declared peer ranges and optional peers", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "declared-peers", + dependencies: { + "a-dep": "1.0.1", + "no-deps": "^1.0.0", + "peer-deps": "^1.0.0", + "peer-deps-fixed": "^1.0.0", + }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + +importers: + + .: + dependencies: + a-dep: + specifier: 1.0.1 + version: 1.0.1 + no-deps: + specifier: ^1.0.0 + version: 1.0.1 + peer-deps: + specifier: ^1.0.0 + version: 1.0.0(a-dep@1.0.1)(no-deps@1.0.1) + peer-deps-fixed: + specifier: ^1.0.0 + version: 1.0.0 + +packages: + + a-dep@1.0.1: + resolution: {integrity: ${A_DEP_1_0_1_INTEGRITY}} + + no-deps@1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + + peer-deps@1.0.0: + resolution: {integrity: ${PEER_DEPS_1_0_0_INTEGRITY}} + peerDependencies: + no-deps: ^1.0.0 + a-dep: '*' + d: '>=2' + peerDependenciesMeta: + a-dep: + optional: true + d: + optional: true + e: + optional: true + + peer-deps-fixed@1.0.0: + resolution: {integrity: ${PEER_DEPS_FIXED_1_0_0_INTEGRITY}} + peerDependencies: + g: ^1 + +snapshots: + + a-dep@1.0.1: {} + + no-deps@1.0.1: {} + + peer-deps@1.0.0(a-dep@1.0.1)(no-deps@1.0.1): + dependencies: + no-deps: 1.0.1 + optionalDependencies: + a-dep: 1.0.1 + + peer-deps-fixed@1.0.0: {} +`, + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain( + `{ "peerDependencies": { "a-dep": "*", "d": ">=2", "e": "*", "no-deps": "^1.0.0" }, "optionalPeers": ["a-dep", "d", "e"] }`, + ); + expect(bunLock).toContain(`{ "peerDependencies": { "g": "^1" }, "optionalPeers": ["g"] }`); + expect(bunLock).not.toContain(`"optionalDependencies"`); + expect(bunLock).not.toContain(`"no-deps": "1.0.1"`); + }); + + // pnpm11/__fixtures__/with-peer: the packages entry declares `ajv: ^6.9.1`, the snapshot resolves 6.10.2 + test("declared ranges win over the snapshot's resolved versions; peers pnpm left out are still emitted", async () => { + const registry = verdaccio.registryUrl(); + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "declared-ranges", + dependencies: { + "no-deps": "^1.0.0", + "one-optional-peer-dep": "1.0.2", + "peer-deps-fixed": "^1.0.0", + "peer-deps-too": "^1.0.0", + }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +settings: + autoInstallPeers: false + +importers: + + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.1 + one-optional-peer-dep: + specifier: 1.0.2 + version: 1.0.2 + peer-deps-fixed: + specifier: ^1.0.0 + version: 1.0.0(no-deps@1.0.1) + peer-deps-too: + specifier: ^1.0.0 + version: 1.0.0 + +packages: + + no-deps@1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + + one-optional-peer-dep@1.0.2: + resolution: {integrity: ${ONE_OPTIONAL_PEER_DEP_1_0_2_INTEGRITY}} + peerDependencies: + no-deps: ^1.0.0 + peerDependenciesMeta: + no-deps: + optional: true + + peer-deps-fixed@1.0.0: + resolution: {integrity: ${PEER_DEPS_FIXED_1_0_0_INTEGRITY}} + peerDependencies: + no-deps: ^1.0.0 + + peer-deps-too@1.0.0: + resolution: {integrity: ${PEER_DEPS_TOO_1_0_0_INTEGRITY}} + peerDependencies: + no-deps: '*' + +snapshots: + + no-deps@1.0.1: {} + + one-optional-peer-dep@1.0.2: {} + + peer-deps-fixed@1.0.0(no-deps@1.0.1): + dependencies: + no-deps: 1.0.1 + + peer-deps-too@1.0.0: {} +`, + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain( + `["peer-deps-fixed@1.0.0", "${registry}peer-deps-fixed/-/peer-deps-fixed-1.0.0.tgz", { "peerDependencies": { "no-deps": "^1.0.0" } }, "${PEER_DEPS_FIXED_1_0_0_INTEGRITY}"]`, + ); + expect(bunLock).toContain( + `["one-optional-peer-dep@1.0.2", "${registry}one-optional-peer-dep/-/one-optional-peer-dep-1.0.2.tgz", { "peerDependencies": { "no-deps": "^1.0.0" }, "optionalPeers": ["no-deps"] }, "${ONE_OPTIONAL_PEER_DEP_1_0_2_INTEGRITY}"]`, + ); + expect(bunLock).toContain( + `["peer-deps-too@1.0.0", "${registry}peer-deps-too/-/peer-deps-too-1.0.0.tgz", { "peerDependencies": { "no-deps": "*" } }, "${PEER_DEPS_TOO_1_0_0_INTEGRITY}"]`, + ); + expect(bunLock).not.toContain(`"no-deps": "1.0.1"`); + + const install = await run(packageDir, "install"); + + expect(install.stderr).not.toContain("error:"); + expect(install.exitCode).toBe(0); + expect(nodeModulesPackages(packageDir)).toMatchInlineSnapshot(` + "node_modules/no-deps/no-deps@1.0.1 + node_modules/one-optional-peer-dep/one-optional-peer-dep@1.0.2 + node_modules/peer-deps-fixed/peer-deps-fixed@1.0.0 + node_modules/peer-deps-too/peer-deps-too@1.0.0" + `); + }); + + test("a peer range dedupes onto the hoisted version like a fresh install", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: join(import.meta.dir, "pnpm/v9-peer-range-dedupe"), + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"peer-deps": ["peer-deps@1.0.0"`); + expect(bunLock).toContain(`{ "peerDependencies": { "no-deps": "*" } }`); + expect(bunLock).toContain(`"provides-peer-deps-1-0-0/no-deps": ["no-deps@1.0.0"`); + expect(bunLock).not.toContain(`"peer-deps/no-deps"`); + + const install = await run(packageDir, "install", "--frozen-lockfile"); + + expect(install.stderr).not.toContain("error:"); + expect(install.exitCode).toBe(0); + expect(nodeModulesPackages(packageDir)).toMatchInlineSnapshot(` + "node_modules/no-deps/no-deps@1.0.1 + node_modules/one-dep/one-dep@1.0.0 + node_modules/peer-deps/peer-deps@1.0.0 + node_modules/provides-peer-deps-1-0-0/node_modules/no-deps/no-deps@1.0.0 + node_modules/provides-peer-deps-1-0-0/provides-peer-deps-1-0-0@1.0.0" + `); + }); + + test("peer variants of one package migrate identically regardless of snapshot order", async () => { + // pnpm11/lockfile/fs convertToLockfileObject: every variant joins packages[removeSuffix(key)] + const packageJsons = { + "package.json": JSON.stringify({ name: "v9-peer-variants", workspaces: ["apps/*"] }), + "apps/a/package.json": JSON.stringify({ + name: "a", + dependencies: { "no-deps": "1.0.1", "peer-deps": "^1.0.0" }, + }), + "apps/b/package.json": JSON.stringify({ + name: "b", + dependencies: { "no-deps": "2.0.0", "peer-deps": "^1.0.0" }, + }), + }; + const variant101 = ` peer-deps@1.0.0(no-deps@1.0.1): + dependencies: + no-deps: 1.0.1 +`; + const variant200 = ` peer-deps@1.0.0(no-deps@2.0.0): + dependencies: + no-deps: 2.0.0 +`; + const lockfile = (variants: string) => `lockfileVersion: '9.0' + +importers: + + .: {} + + apps/a: + dependencies: + no-deps: + specifier: 1.0.1 + version: 1.0.1 + peer-deps: + specifier: ^1.0.0 + version: 1.0.0(no-deps@1.0.1) + + apps/b: + dependencies: + no-deps: + specifier: 2.0.0 + version: 2.0.0 + peer-deps: + specifier: ^1.0.0 + version: 1.0.0(no-deps@2.0.0) + +packages: + + no-deps@1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + + no-deps@2.0.0: + resolution: {integrity: ${NO_DEPS_2_0_0_INTEGRITY}} + + peer-deps@1.0.0: + resolution: {integrity: ${PEER_DEPS_1_0_0_INTEGRITY}} + peerDependencies: + no-deps: '*' + +snapshots: + + no-deps@1.0.1: {} + + no-deps@2.0.0: {} + +${variants}`; + + const { packageDir: forward } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { ...packageJsons, "pnpm-lock.yaml": lockfile(`${variant101}\n${variant200}`) }, + }); + const { packageDir: swapped } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { ...packageJsons, "pnpm-lock.yaml": lockfile(`${variant200}\n${variant101}`) }, + }); + + const [forwardResult, swappedResult] = await Promise.all([migrate(forward), migrate(swapped)]); + + expect(forwardResult.stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(swappedResult.stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(forwardResult.exitCode).toBe(0); + expect(swappedResult.exitCode).toBe(0); + + const bunLock = await bunLockOf(forward); + expect(bunLock).toContain(`{ "peerDependencies": { "no-deps": "*" } }`); + expect(bunLock).toContain(`"no-deps@1.0.1"`); + expect(bunLock).toContain(`"no-deps@2.0.0"`); + expect(await bunLockOf(swapped)).toBe(bunLock); + + const install = await run(forward, "install", "--frozen-lockfile"); + + expect(install.stderr).not.toContain("error:"); + expect(install.exitCode).toBe(0); + expect((await installedPackageJson(forward, "apps/a", "no-deps")).version).toBe("1.0.1"); + expect((await installedPackageJson(forward, "apps/b", "no-deps")).version).toBe("2.0.0"); + }); + + test("root and workspace peerDependencies come from package.json", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: join(import.meta.dir, "pnpm/v9-importer-peers"), + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).not.toContain("does not record peer dependency"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(workspaceBlock(bunLock, "")).toBe( + [ + ` "": {`, + ` "name": "v9-importer-peers",`, + ` "dependencies": {`, + ` "no-deps": "^1.0.0",`, + ` },`, + ` "peerDependencies": {`, + ` "@types/is-number": "*",`, + ` "@types/no-deps": "*",`, + ` "no-deps": "^1.0.0",`, + ` "peer-deps-fixed": "^1.0.0",`, + ` },`, + ` "optionalPeers": [`, + ` "@types/is-number",`, + ` "@types/no-deps",`, + ` ],`, + ` },`, + ].join("\n"), + ); + expect(workspaceBlock(bunLock, "packages/a")).toBe( + [ + ` "packages/a": {`, + ` "name": "a",`, + ` "peerDependencies": {`, + ` "no-deps": "^1.0.0",`, + ` },`, + ` },`, + ].join("\n"), + ); + }); + + test("a name in both devDependencies and peerDependencies gets both entries, like a fresh install", async () => { + const packageJson = JSON.stringify({ + name: "dev-and-peer", + devDependencies: { "no-deps": "^1.0.0" }, + peerDependencies: { "no-deps": "^1.0.0" }, + }); + const { packageDir: migrated } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": packageJson, + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + +importers: + + .: + devDependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.1 + +packages: + + no-deps@1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + +snapshots: + + no-deps@1.0.1: {} +`, + }, + }); + + const { stderr, exitCode } = await migrate(migrated); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const migratedRoot = workspaceBlock(await bunLockOf(migrated), ""); + expect(migratedRoot).toContain(` "devDependencies": {\n "no-deps": "^1.0.0",\n },`); + expect(migratedRoot).toContain(` "peerDependencies": {\n "no-deps": "^1.0.0",\n },`); + + const { packageDir: fresh } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { "package.json": packageJson }, + }); + + const install = await run(fresh, "install"); + + expect(install.stderr).toContain("Saved lockfile"); + expect(install.exitCode).toBe(0); + expect(migratedRoot).toBe(workspaceBlock(await bunLockOf(fresh), "")); + }); + + test("workspace peers pnpm did not auto-install are merged from the member's package.json", async () => { + // port of pnpm11/installing/deps-installer/test/install/injectLocalPackages.ts 'inject local packages' + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "isolated" }, + files: { + "package.json": JSON.stringify({ name: "v9-workspace-peers", workspaces: ["packages/*"] }), + "packages/project-1/package.json": JSON.stringify({ + name: "project-1", + version: "1.0.0", + dependencies: { "a-dep": "1.0.1" }, + peerDependencies: { "no-deps": ">=1.0.0" }, + }), + "packages/project-2/package.json": JSON.stringify({ + name: "project-2", + version: "1.0.0", + dependencies: { "project-1": "workspace:*" }, + devDependencies: { "no-deps": "1.0.1" }, + dependenciesMeta: { "project-1": { injected: true } }, + }), + "packages/project-3/package.json": JSON.stringify({ + name: "project-3", + version: "1.0.0", + dependencies: { "project-2": "workspace:*" }, + devDependencies: { "no-deps": "2.0.0" }, + dependenciesMeta: { "project-2": { injected: true } }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +settings: + autoInstallPeers: false + injectWorkspacePackages: true + +importers: + + .: {} + + packages/project-1: + dependencies: + a-dep: + specifier: 1.0.1 + version: 1.0.1 + + packages/project-2: + dependencies: + project-1: + specifier: workspace:* + version: file:packages/project-1(no-deps@1.0.1) + devDependencies: + no-deps: + specifier: 1.0.1 + version: 1.0.1 + + packages/project-3: + dependencies: + project-2: + specifier: workspace:* + version: file:packages/project-2(no-deps@2.0.0) + devDependencies: + no-deps: + specifier: 2.0.0 + version: 2.0.0 + +packages: + + a-dep@1.0.1: + resolution: {integrity: ${A_DEP_1_0_1_INTEGRITY}} + + no-deps@1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + + no-deps@2.0.0: + resolution: {integrity: ${NO_DEPS_2_0_0_INTEGRITY}} + + project-1@file:packages/project-1: + resolution: {directory: packages/project-1, type: directory} + version: 1.0.0 + peerDependencies: + no-deps: '>=1.0.0' + + project-2@file:packages/project-2: + resolution: {directory: packages/project-2, type: directory} + version: 1.0.0 + +snapshots: + + a-dep@1.0.1: {} + + no-deps@1.0.1: {} + + no-deps@2.0.0: {} + + project-1@file:packages/project-1(no-deps@1.0.1): + dependencies: + a-dep: 1.0.1 + no-deps: 1.0.1 + + project-1@file:packages/project-1(no-deps@2.0.0): + dependencies: + a-dep: 1.0.1 + no-deps: 2.0.0 + + project-2@file:packages/project-2(no-deps@2.0.0): + dependencies: + project-1: file:packages/project-1(no-deps@2.0.0) + transitivePeerDependencies: + - no-deps +`, + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain( + "pnpm-lock.yaml does not record peer dependency 'no-deps' of importer 'packages/project-1'; the next bun install will resolve it", + ); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(workspaceBlock(bunLock, "packages/project-1")).toContain( + ` "peerDependencies": {\n "no-deps": ">=1.0.0",\n },`, + ); + expect(bunLock).toContain(`"project-1": ["project-1@workspace:packages/project-1"]`); + expect(bunLock).toContain(`"project-2": ["project-2@workspace:packages/project-2"]`); + expect(bunLock).not.toContain("@file:packages/"); + // the injected packages: entry declares the same peer; it must not become package-level metadata too + expect(bunLock.split(`"no-deps": ">=1.0.0"`).length - 1).toBe(1); + + const install = await run(packageDir, "install"); + + expect(install.stderr).not.toContain("error:"); + expect(install.exitCode).toBe(0); + expect(await Bun.file(join(packageDir, "packages/project-2/node_modules/project-1/package.json")).json()).toEqual( + expect.objectContaining({ name: "project-1" }), + ); + expect(await Bun.file(join(packageDir, "packages/project-3/node_modules/project-2/package.json")).json()).toEqual( + expect.objectContaining({ name: "project-2" }), + ); + }); + + test("an unrecorded required peer is reported and left for bun install", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "unrecorded-peer", + dependencies: { "no-deps": "^1.0.0" }, + peerDependencies: { "has-peer": "^1.0.0" }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +settings: + autoInstallPeers: false + +importers: + + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.1 + +packages: + + no-deps@1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + +snapshots: + + no-deps@1.0.1: {} +`, + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain( + "pnpm-lock.yaml does not record peer dependency 'has-peer' of importer '.'; the next bun install will resolve it", + ); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + expect(workspaceBlock(await bunLockOf(packageDir), "")).toContain( + [ + ` "peerDependencies": {`, + ` "has-peer": "^1.0.0",`, + ` },`, + ` "optionalPeers": [`, + ` "has-peer",`, + ` ],`, + ].join("\n"), + ); + }); + + test("peers declared with a catalog: range keep the catalog reference", async () => { + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "catalog-peer", + workspaces: { catalog: { "no-deps": "^1.0.0" } }, + peerDependencies: { "no-deps": "catalog:" }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + +catalogs: + default: + no-deps: + specifier: ^1.0.0 + version: 1.0.1 + +importers: + + .: + dependencies: + no-deps: + specifier: 'catalog:' + version: 1.0.1 + +packages: + + no-deps@1.0.1: + resolution: {integrity: ${NO_DEPS_1_0_1_INTEGRITY}} + +snapshots: + + no-deps@1.0.1: {} +`, + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).not.toContain("missing entry"); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + const root = workspaceBlock(bunLock, ""); + expect(root).toContain(` "peerDependencies": {\n "no-deps": "catalog:",\n },`); + expect(root).not.toContain(`"dependencies"`); + expect(bunLock).toContain(`"catalog": {\n "no-deps": "^1.0.0",\n }`); + + const install = await run(packageDir, "install"); + + expect(install.stderr).not.toContain("Saved lockfile"); + expect(install.exitCode).toBe(0); + expect(workspaceBlock(await bunLockOf(packageDir), "")).toBe(root); + }); + + test("bun install after bun pm migrate does not rewrite bun.lock", async () => { + // real pnpm output: the root's peer-only `a-dep` sits under the importer's dependencies + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: join(import.meta.dir, "pnpm/basic"), + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const migrated = await bunLockOf(packageDir); + const root = workspaceBlock(migrated, ""); + expect(root).toContain(` "peerDependencies": {\n "a-dep": "1.0.1",\n },`); + expect(root).toContain(` "dependencies": {\n "no-deps": "~1.0.0",\n },`); + expect(root.split(`"a-dep"`).length - 1).toBe(1); + + const install = await run(packageDir, "install"); + + expect(install.stderr).not.toContain("Saved lockfile"); + expect(install.stderr).not.toContain("error:"); + expect(install.exitCode).toBe(0); + expect(await bunLockOf(packageDir)).toBe(migrated); + expect(await installedPackageJson(packageDir, "", "a-dep")).toEqual( + expect.objectContaining({ name: "a-dep", version: "1.0.1" }), + ); + }); + + test("bun install straight from pnpm-lock.yaml writes the same importers as bun pm migrate", async () => { + const files = join(import.meta.dir, "pnpm/basic"); + const { packageDir: viaMigrate } = await verdaccio.createTestDir({ bunfigOpts: { linker: "hoisted" }, files }); + const { packageDir: viaInstall } = await verdaccio.createTestDir({ bunfigOpts: { linker: "hoisted" }, files }); + + const migrated = await migrate(viaMigrate); + expect(migrated.stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(migrated.exitCode).toBe(0); + + const [installAfterMigrate, directInstall] = await Promise.all([ + run(viaMigrate, "install"), + run(viaInstall, "install"), + ]); + + expect(installAfterMigrate.stderr).not.toContain("Saved lockfile"); + expect(installAfterMigrate.exitCode).toBe(0); + expect(directInstall.stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(directInstall.stderr.split("Saved lockfile").length - 1).toBe(1); + expect(directInstall.exitCode).toBe(0); + + const importers = workspacesSection(await bunLockOf(viaMigrate)); + expect(importers).toContain(` "peerDependencies": {\n "a-dep": "1.0.1",\n },`); + expect(workspacesSection(await bunLockOf(viaInstall))).toBe(importers); + }); + }); + + test("excludeLinksFromLockfile omissions are reported", async () => { + // pnpm11/installing/deps-installer/test/install/excludeLinksFromLockfile.ts + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ + name: "exclude-links", + dependencies: { "no-deps": "^1.0.0", linked: "link:../linked" }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +settings: + excludeLinksFromLockfile: true + +importers: + + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.0 + +packages: + + no-deps@1.0.0: + resolution: {integrity: ${NO_DEPS_1_0_0_INTEGRITY}} + +snapshots: + + no-deps@1.0.0: {} +`, + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain( + "pnpm-lock.yaml omits linked dependency 'linked' of importer '.' (excludeLinksFromLockfile); it is not migrated", + ); + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"no-deps@1.0.0"`); + expect(bunLock).not.toContain("linked"); + }); + + describe("registry tarball: urls", () => { + // pnpm/pnpm#13534: GitHub Packages / npm Enterprise tarballs are not on the canonical `/-/` path + test("recorded under the configured registry is kept", async () => { + const registry = verdaccio.registryUrl(); + const tarball = `${registry}download/no-deps/1.0.1/0123456789abcdef`; + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ name: "kept-tarball", dependencies: { "no-deps": "^1.0.0" } }), + "pnpm-lock.yaml": registryLockfileWithTarball(tarball), + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`["no-deps@1.0.1", "${tarball}", {}, "${NO_DEPS_1_0_1_INTEGRITY}"]`); + }); + + // guards for the keep-tarball path above (pnpm/pnpm#5920 / #4361): off-registry urls are rebuilt + test("recorded on a foreign host is rebuilt from the configured registry", async () => { + const registry = verdaccio.registryUrl(); + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ name: "foreign-tarball", dependencies: { "no-deps": "^1.0.0" } }), + "pnpm-lock.yaml": registryLockfileWithTarball("https://evil.example.com/no-deps/-/no-deps-1.0.1.tgz"), + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain( + `["no-deps@1.0.1", "${registry}no-deps/-/no-deps-1.0.1.tgz", {}, "${NO_DEPS_1_0_1_INTEGRITY}"]`, + ); + expect(bunLock).not.toContain("evil.example.com"); + }); + + test("under a registry whose hostname is a prefix of the recorded url's is rebuilt", async () => { + const registry = verdaccio.registryUrl(); + const lookalike = `${registry.slice(0, -1)}.evil.example.com/no-deps/-/no-deps-1.0.1.tgz`; + const { packageDir } = await verdaccio.createTestDir({ + bunfigOpts: { linker: "hoisted" }, + files: { + "package.json": JSON.stringify({ name: "lookalike-tarball", dependencies: { "no-deps": "^1.0.0" } }), + "pnpm-lock.yaml": registryLockfileWithTarball(lookalike), + }, + }); + + const { stderr, exitCode } = await migrate(packageDir); + + expect(stderr).toContain("migrated lockfile from pnpm-lock.yaml"); + expect(exitCode).toBe(0); + + const bunLock = await bunLockOf(packageDir); + expect(bunLock).toContain(`"${registry}no-deps/-/no-deps-1.0.1.tgz"`); + expect(bunLock).not.toContain("evil.example.com"); + }); + }); + + describe("git sub-directory dependencies", () => { + // pnpm/pnpm#8243: `repo#commit&path:sub/dir` has no bun equivalent; refuse instead of installing the repo root + test.concurrent("type: git resolution with path: is rejected naming the package", async () => { + using dir = fixture("v9-git-subdirectory"); + + const { stderr, exitCode } = await migrate(String(dir)); + + expect(stderr).toContain( + "pnpm-lock.yaml package 'pkg@git+ssh://git@example.com/org/monorepo.git#cba04669e621b85fbdb33371604de1a2898e68e9&path:packages/pkg' is a git sub-directory dependency", + ); + expect(exitCode).toBe(1); + expect(existsSync(join(String(dir), "bun.lock"))).toBe(false); + }); + + test.concurrent("git-hosted tarball resolution with path: is rejected naming the package", async () => { + // shape from pnpm11/resolving/git-resolver/test/index.ts "with both sub folder and branch" + const id = + "https://codeload.github.com/o/mono/tar.gz/777e8a3e78cc89bbf41fb3fd9f6cf922d5463313#path:/packages/pkg"; + using dir = tempDir("pnpm-v9-git-hosted-subdirectory", { + "package.json": JSON.stringify({ + name: "git-hosted-subdirectory", + dependencies: { pkg: "github:o/mono#beta&path:/packages/pkg" }, + }), + "pnpm-lock.yaml": `lockfileVersion: '9.0' + +importers: + + .: + dependencies: + pkg: + specifier: github:o/mono#beta&path:/packages/pkg version: ${id} packages: @@ -714,14 +1836,22 @@ importers: const bunLock = await bunLockOf(String(dir)); expect(bunLock).toContain(`"lockfileVersion": 3`); - expect(bunLock).toContain(`"semver@<7.5.2": {`); - expect(bunLock).toContain(`".": "7.5.2"`); - expect(bunLock).toContain(`"@scope/pkg@^1": {`); - expect(bunLock).toContain(`".": "1.9.0"`); - expect(bunLock).toContain(`"foo": {`); - expect(bunLock).toContain(`"bar": "2.0.0"`); - expect(bunLock).toContain(`"plain": "1.0.0"`); - expect(bunLock).toContain(`"@scope/plain": "3.0.0"`); + expect(bunLock).not.toContain("left-pad"); + expect(overridesSection(bunLock)).toMatchInlineSnapshot(` + " "overrides": { + "@scope/pkg@^1": { + ".": "1.9.0", + }, + "@scope/plain": "3.0.0", + "foo": { + "bar": "2.0.0", + }, + "plain": "1.0.0", + "semver@<7.5.2": { + ".": "7.5.2", + }, + }," + `); }); test.concurrent("parent selectors become nested rules", async () => { @@ -795,19 +1925,14 @@ importers: expect(bunLock).toContain(`"common": ["common@workspace:shared/common"]`); expect(bunLock).not.toContain("link:"); - await using install = Bun.spawn({ - cmd: [bunExe(), "install", "--frozen-lockfile", "--linker", "hoisted"], - cwd: String(dir), - env: bunEnv, - stdout: "pipe", - stderr: "pipe", - }); - const [installStderr, installExitCode] = await Promise.all([install.stderr.text(), install.exited]); - expect(installStderr).not.toContain("error:"); - expect(installExitCode).toBe(0); + const install = await run(String(dir), "install", "--frozen-lockfile", "--linker", "hoisted"); + + expect(install.stderr).not.toContain("error:"); + expect(await installedPackageJson(String(dir), "apps/web", "common")).toEqual({ name: "common", version: "1.2.0" }); + expect(install.exitCode).toBe(0); }); - test.concurrent("prettier-style multi-line resolution mappings migrate identically (pnpm/pnpm#4084)", async () => { + test.concurrent("prettier-style multi-line resolution mappings migrate identically", async () => { using plain = fixture("v9-git-references"); using formatted = fixture("v9-git-references"); @@ -831,6 +1956,7 @@ importers: expect(formattedResult.stderr).toContain("migrated lockfile from pnpm-lock.yaml"); expect(formattedResult.exitCode).toBe(0); + expect(plainResult.stderr).toContain("migrated lockfile from pnpm-lock.yaml"); expect(plainResult.exitCode).toBe(0); expect(await bunLockOf(String(formatted))).toBe(await bunLockOf(String(plain))); }); diff --git a/test/cli/install/migration/pnpm/v9-importer-peers/package.json b/test/cli/install/migration/pnpm/v9-importer-peers/package.json new file mode 100644 index 000000000000..9c1b9a157ae5 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-importer-peers/package.json @@ -0,0 +1,22 @@ +{ + "name": "v9-importer-peers", + "workspaces": [ + "packages/*" + ], + "dependencies": { + "no-deps": "^1.0.0" + }, + "peerDependencies": { + "no-deps": "^1.0.0", + "peer-deps-fixed": "^1.0.0", + "@types/no-deps": "*" + }, + "peerDependenciesMeta": { + "@types/no-deps": { + "optional": true + }, + "@types/is-number": { + "optional": true + } + } +} diff --git a/test/cli/install/migration/pnpm/v9-importer-peers/packages/a/package.json b/test/cli/install/migration/pnpm/v9-importer-peers/packages/a/package.json new file mode 100644 index 000000000000..28f9fac31100 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-importer-peers/packages/a/package.json @@ -0,0 +1,6 @@ +{ + "name": "a", + "peerDependencies": { + "no-deps": "^1.0.0" + } +} diff --git a/test/cli/install/migration/pnpm/v9-importer-peers/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-importer-peers/pnpm-lock.yaml new file mode 100644 index 000000000000..48fcf97b0861 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-importer-peers/pnpm-lock.yaml @@ -0,0 +1,40 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.1 + peer-deps-fixed: + specifier: ^1.0.0 + version: 1.0.0(no-deps@1.0.1) + + packages/a: + dependencies: + no-deps: + specifier: ^1.0.0 + version: 1.0.1 + +packages: + + no-deps@1.0.1: + resolution: {integrity: sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A==} + + peer-deps-fixed@1.0.0: + resolution: {integrity: sha512-gVs9cSdy6TAQIEWu1tVEK1mAspCQxYziTGQlv4a2XQpzOBZvoQ/y6lOeu3tqNNrNQnLwdvwAQTlvazV5+HfV7g==} + peerDependencies: + no-deps: ^1.0.0 + +snapshots: + + no-deps@1.0.1: {} + + peer-deps-fixed@1.0.0(no-deps@1.0.1): + dependencies: + no-deps: 1.0.1 diff --git a/test/cli/install/migration/pnpm/v9-local-tarballs/package.json b/test/cli/install/migration/pnpm/v9-local-tarballs/package.json index 494743727fa0..d70d23097682 100644 --- a/test/cli/install/migration/pnpm/v9-local-tarballs/package.json +++ b/test/cli/install/migration/pnpm/v9-local-tarballs/package.json @@ -2,7 +2,9 @@ "name": "v9-local-tarballs", "version": "1.0.0", "dependencies": { + "plain": "file:../plain-1.0.0.tar", "tar-gz-pkg": "file:../tar-gz-pkg-1.0.0.tar.gz", - "tar-pkg": "file:../tar-pkg-1.0.0.tgz" + "tar-pkg": "file:../tar-pkg-1.0.0.tgz", + "upper": "file:../UPPER-1.0.0.TGZ" } } diff --git a/test/cli/install/migration/pnpm/v9-local-tarballs/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-local-tarballs/pnpm-lock.yaml index ebc8fe861962..dfc1f07b5b30 100644 --- a/test/cli/install/migration/pnpm/v9-local-tarballs/pnpm-lock.yaml +++ b/test/cli/install/migration/pnpm/v9-local-tarballs/pnpm-lock.yaml @@ -8,15 +8,25 @@ importers: .: dependencies: + plain: + specifier: file:../plain-1.0.0.tar + version: file:../plain-1.0.0.tar tar-gz-pkg: specifier: file:../tar-gz-pkg-1.0.0.tar.gz version: file:../tar-gz-pkg-1.0.0.tar.gz tar-pkg: specifier: file:../tar-pkg-1.0.0.tgz version: file:../tar-pkg-1.0.0.tgz + upper: + specifier: file:../UPPER-1.0.0.TGZ + version: file:../UPPER-1.0.0.TGZ packages: + plain@file:../plain-1.0.0.tar: + resolution: {integrity: sha512-HP/5Rgt3pVFLzjmN9qJJ6vZMgCwoCIl/m2bPndYT283CUqnmFiMx0GeeIJ7SyK6TYoJM78SEvFEOQie++caHqw==} + version: 1.0.0 + tar-gz-pkg@file:../tar-gz-pkg-1.0.0.tar.gz: resolution: {integrity: sha512-HP/5Rgt3pVFLzjmN9qJJ6vZMgCwoCIl/m2bPndYT283CUqnmFiMx0GeeIJ7SyK6TYoJM78SEvFEOQie++caHqw==} version: 1.0.0 @@ -25,8 +35,16 @@ packages: resolution: {integrity: sha512-HP/5Rgt3pVFLzjmN9qJJ6vZMgCwoCIl/m2bPndYT283CUqnmFiMx0GeeIJ7SyK6TYoJM78SEvFEOQie++caHqw==, tarball: file:../tar-pkg-1.0.0.tgz} version: 1.0.0 + upper@file:../UPPER-1.0.0.TGZ: + resolution: {integrity: sha512-HP/5Rgt3pVFLzjmN9qJJ6vZMgCwoCIl/m2bPndYT283CUqnmFiMx0GeeIJ7SyK6TYoJM78SEvFEOQie++caHqw==} + version: 1.0.0 + snapshots: + plain@file:../plain-1.0.0.tar: {} + tar-gz-pkg@file:../tar-gz-pkg-1.0.0.tar.gz: {} tar-pkg@file:../tar-pkg-1.0.0.tgz: {} + + upper@file:../UPPER-1.0.0.TGZ: {} diff --git a/test/cli/install/migration/pnpm/v9-peer-range-dedupe/package.json b/test/cli/install/migration/pnpm/v9-peer-range-dedupe/package.json new file mode 100644 index 000000000000..da922bf7d9b4 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-peer-range-dedupe/package.json @@ -0,0 +1,8 @@ +{ + "name": "v9-peer-range-dedupe", + "version": "1.0.0", + "dependencies": { + "one-dep": "1.0.0", + "provides-peer-deps-1-0-0": "1.0.0" + } +} diff --git a/test/cli/install/migration/pnpm/v9-peer-range-dedupe/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-peer-range-dedupe/pnpm-lock.yaml new file mode 100644 index 000000000000..66aaaa097fff --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-peer-range-dedupe/pnpm-lock.yaml @@ -0,0 +1,54 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + dependencies: + one-dep: + specifier: 1.0.0 + version: 1.0.0 + provides-peer-deps-1-0-0: + specifier: 1.0.0 + version: 1.0.0 + +packages: + + no-deps@1.0.0: + resolution: {integrity: sha512-v4w12JRjUGvfHDUP8vFDwu0gUWu04j0cv9hLb1Abf9VdaXu4XcrddYFTMVBVvmldKViGWH7jrb6xPJRF0wq6gw==} + + no-deps@1.0.1: + resolution: {integrity: sha512-3X6cn4+UJdXJuLPu11v8i/fGLe2PdI6v1yKTELam04lY5esCAFdG/qQts6N6rLrL6g1YRq+MKBAwxbmUQk355A==} + + one-dep@1.0.0: + resolution: {integrity: sha512-qG6lZjwM1vFmRCHwP+XpOKu6FkrBmwr20+54+qaHGdjZlw/wz8aJrhFqX4dZksqmBLZtj2mzL77Yf04WKs1+Kg==} + + peer-deps@1.0.0: + resolution: {integrity: sha512-CHQ5sQXwUo38G++dkzJ/rJ9Ge98MeMTQjjC9UK2t0frp8Lrhm3zNooOLakFyHW4UcyD3vuTS3Qv324Bj6B5Tjw==} + peerDependencies: + no-deps: '*' + + provides-peer-deps-1-0-0@1.0.0: + resolution: {integrity: sha512-DSOgqUXTkw06FqE/14D5KvaGbl3e3Rri71F9UeSRGV1CtQE84mO69ZXE1QhSed5zM0EQy1n/zkwDdtwsmOaFsA==} + +snapshots: + + no-deps@1.0.0: {} + + no-deps@1.0.1: {} + + one-dep@1.0.0: + dependencies: + no-deps: 1.0.1 + + peer-deps@1.0.0(no-deps@1.0.0): + dependencies: + no-deps: 1.0.0 + + provides-peer-deps-1-0-0@1.0.0: + dependencies: + no-deps: 1.0.0 + peer-deps: 1.0.0(no-deps@1.0.0) diff --git a/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/package.json b/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/package.json new file mode 100644 index 000000000000..6ab6baac63f3 --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/package.json @@ -0,0 +1,10 @@ +{ + "name": "v9-snapshot-only-file-variants", + "workspaces": [ + "packages/*" + ], + "dependencies": { + "dir": "workspace:*", + "local": "file:vendor/local" + } +} diff --git a/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/packages/dir/package.json b/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/packages/dir/package.json new file mode 100644 index 000000000000..35568d7c38df --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/packages/dir/package.json @@ -0,0 +1,4 @@ +{ + "name": "dir", + "version": "1.0.0" +} diff --git a/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/pnpm-lock.yaml b/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/pnpm-lock.yaml new file mode 100644 index 000000000000..7a5777d1158c --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/pnpm-lock.yaml @@ -0,0 +1,36 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + injectWorkspacePackages: true + +importers: + + .: + dependencies: + dir: + specifier: workspace:* + version: file:packages/dir(no-deps@1.0.0) + local: + specifier: file:vendor/local + version: file:vendor/local(no-deps@1.0.0) + + packages/dir: {} + +packages: + + no-deps@1.0.0: + resolution: {integrity: sha512-v4w12JRjUGvfHDUP8vFDwu0gUWu04j0cv9hLb1Abf9VdaXu4XcrddYFTMVBVvmldKViGWH7jrb6xPJRF0wq6gw==} + +snapshots: + + dir@file:packages/dir(no-deps@1.0.0): {} + + local@file:vendor/local(no-deps@1.0.0): + dependencies: + no-deps: 1.0.0 + + no-deps@1.0.0: {} + + tb@file:vendor/tb-1.0.0.tgz(no-deps@1.0.0): {} diff --git a/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/vendor/local/package.json b/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/vendor/local/package.json new file mode 100644 index 000000000000..090e7a15f95e --- /dev/null +++ b/test/cli/install/migration/pnpm/v9-snapshot-only-file-variants/vendor/local/package.json @@ -0,0 +1,7 @@ +{ + "name": "local", + "version": "1.0.0", + "dependencies": { + "no-deps": "1.0.0" + } +} diff --git a/test/cli/install/nested-overrides.test.ts b/test/cli/install/nested-overrides.test.ts index 58340e740804..4e3580152557 100644 --- a/test/cli/install/nested-overrides.test.ts +++ b/test/cli/install/nested-overrides.test.ts @@ -33,7 +33,7 @@ async function run(dir: string, ...cmd: string[]) { } const install = (dir: string, ...args: string[]) => run(dir, "install", ...args); -const migrate = (dir: string) => run(dir, "pm", "migrate", "-f"); +const migrate = (dir: string) => run(dir, "pm", "migrate"); async function installOk(dir: string, ...args: string[]) { const result = await install(dir, ...args); @@ -42,7 +42,8 @@ async function installOk(dir: string, ...args: string[]) { return result; } -async function versionSeenBy(packageDir: string, from: string | undefined, name: string): Promise { +// `from` is a dependency name (resolved through node_modules) or a workspace path; returns what that package sees as `name@version`. +async function packageSeenBy(packageDir: string, from: string | undefined, name: string): Promise { const cwd = from === undefined ? packageDir @@ -50,7 +51,11 @@ async function versionSeenBy(packageDir: string, from: string | undefined, name: ? join(packageDir, from) : realpathSync(join(packageDir, "node_modules", from)); await using proc = Bun.spawn({ - cmd: [bunExe(), "-e", `console.log(require(${JSON.stringify(name + "/package.json")}).version)`], + cmd: [ + bunExe(), + "-e", + `const p = require(${JSON.stringify(name + "/package.json")}); console.log(p.name + "@" + p.version)`, + ], cwd, env: bunEnv, stdout: "pipe", @@ -62,6 +67,11 @@ async function versionSeenBy(packageDir: string, from: string | undefined, name: return out.trim(); } +async function versionSeenBy(packageDir: string, from: string | undefined, name: string): Promise { + const seen = await packageSeenBy(packageDir, from, name); + return seen.slice(seen.lastIndexOf("@") + 1); +} + const lock = (dir: string) => file(join(dir, "bun.lock")).text(); function overridesSection(text: string) { @@ -214,16 +224,25 @@ describe.concurrent("syntax", () => { }); test("yarn resolutions path with scoped parent and child", async () => { - const dir = await project({ - dependencies: { "@scoped/has-bin-entry": "1.0.0" }, - resolutions: { "@scoped/has-bin-entry/@types/no-deps": "1.0.0" }, - }); - const { err, exitCode } = await install(dir); + const dir = await project( + { workspaces: ["packages/*"], resolutions: { "@scoped/app/@types/no-deps": "1.0.0" } }, + "hoisted", + { + "packages/app/package.json": JSON.stringify({ name: "@scoped/app", dependencies: { "@types/no-deps": "*" } }), + "packages/other/package.json": JSON.stringify({ name: "other", dependencies: { "@types/no-deps": "*" } }), + }, + ); + const { err } = await installOk(dir); expect(err).not.toContain("warn:"); - expect(exitCode).toBe(0); - const text = await lock(dir); - expect(text).toContain('"@scoped/has-bin-entry": {'); - expect(text).toContain('"@types/no-deps": "1.0.0"'); + expect(await versionSeenBy(dir, "packages/app", "@types/no-deps")).toBe("1.0.0"); + expect(await versionSeenBy(dir, "packages/other", "@types/no-deps")).toBe("2.0.0"); + expect(overridesSection(await lock(dir))).toMatchInlineSnapshot(` + ""overrides": { + "@scoped/app": { + "@types/no-deps": "1.0.0", + }, + }," + `); }); test("pnpm parent>child selector", async () => { @@ -332,6 +351,25 @@ describe.concurrent("syntax", () => { expect(await lock(dir)).not.toContain('"lockfileVersion": 3'); }); }); + + // pnpm's `-` value deletes the dependency; Bun warns and installs as if the rule were absent. + describe.concurrent.each([ + { rules: { overrides: { "no-deps": "-" } }, label: "override" }, + { rules: { overrides: { "one-dep": { "no-deps": "-" } } }, label: "override" }, + { rules: { resolutions: { "no-deps": "-" } }, label: "resolution" }, + { rules: { resolutions: { "one-dep/no-deps": "-" } }, label: "resolution" }, + ])('a "-" value warns and is ignored %j', ({ rules, label }) => { + test("install still succeeds", async () => { + const dir = await project({ dependencies: { "one-dep": "1.0.0", "one-range-dep": "1.0.0" }, ...rules }); + const { err, exitCode } = await install(dir); + expect(err).toContain(`${label} "no-deps" removes the dependency ('-'), which bun does not support`); + expect(err).not.toContain("error:"); + expect(exitCode).toBe(0); + expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + expect(await lock(dir)).not.toContain('"overrides"'); + }); + }); }); // The selector range is matched against the range the dependent declares; a rule applies when the two intersect. @@ -526,7 +564,7 @@ describe.concurrent("version-scoped targets", () => { describe.concurrent.each(["hoisted", "isolated"] as const)("linker=%s", linker => { test("two dependents of one name see different versions", async () => { const dir = await project(npmObjectProject, linker); - await installOk(dir); + await installOk(dir, "--linker", linker); expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); }); @@ -543,10 +581,10 @@ describe.concurrent.each(["hoisted", "isolated"] as const)("linker=%s", linker = "packages/app-b/package.json": JSON.stringify({ name: "app-b", dependencies: { "no-deps": "*" } }), }, ); - await installOk(dir); + await installOk(dir, "--linker", linker); expect(await versionSeenBy(dir, "packages/app-a", "no-deps")).toBe("1.0.0"); expect(await versionSeenBy(dir, "packages/app-b", "no-deps")).toBe("2.0.0"); - await installOk(dir, "--frozen-lockfile"); + await installOk(dir, "--linker", linker, "--frozen-lockfile"); }); }); @@ -640,7 +678,34 @@ describe.concurrent("lockfile", () => { expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); }); - // `1` and `^1.0.1` differ as ranges even though Version::eql treats `1` and `^1.0.0` as equal. + async function expectFrozenOverridesFailure(dir: string) { + const frozen = await install(dir, "--frozen-lockfile"); + expect(frozen.err).toContain("overrides"); + expect(frozen.err).toContain("frozen"); + expect(frozen.exitCode).toBe(1); + } + + // Every rewrite below resolves to the same packages as before, so only the overrides section of bun.lock differs. + test("changing a flat rule's range text is a frozen-lockfile change", async () => { + const deps = { "one-range-dep": "1.0.0" }; + const dir = await project({ dependencies: deps, overrides: { "no-deps": "1.1.0" } }); + await installOk(dir); + await write( + join(dir, "package.json"), + JSON.stringify({ name: "nested-overrides", dependencies: deps, overrides: { "no-deps": "^1.1.0" } }), + ); + await expectFrozenOverridesFailure(dir); + const { err } = await installOk(dir); + expect(err).toContain("Saved lockfile"); + expect(overridesSection(await lock(dir))).toMatchInlineSnapshot(` + ""overrides": { + "no-deps": "^1.1.0", + }," + `); + expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + await installOk(dir, "--frozen-lockfile"); + }); + test("changing a flat target range is a frozen-lockfile change", async () => { const dir = await project(rangedProject); await installOk(dir); @@ -652,14 +717,14 @@ describe.concurrent("lockfile", () => { overrides: { "no-deps@^1.0.1": "1.0.0", "one-dep": rangedProject.overrides["one-dep"] }, }), ); - const frozen = await install(dir, "--frozen-lockfile"); - expect(frozen.err).toContain("lockfile had changes, but lockfile is frozen"); - expect(frozen.exitCode).toBe(1); - await installOk(dir); + await expectFrozenOverridesFailure(dir); + const { err } = await installOk(dir); + expect(err).toContain("Saved lockfile"); const section = overridesSection(await lock(dir)); expect(section).toContain('"no-deps@^1.0.1": {'); expect(section).not.toContain('"no-deps@1": {'); expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + await installOk(dir, "--frozen-lockfile"); }); test("changing a nested target range is a frozen-lockfile change", async () => { @@ -673,12 +738,14 @@ describe.concurrent("lockfile", () => { overrides: { "no-deps@1": "1.0.0", "one-dep": { "no-deps@^1.0.1": "2.0.0" } }, }), ); - const frozen = await install(dir, "--frozen-lockfile"); - expect(frozen.err).toContain("lockfile had changes, but lockfile is frozen"); - expect(frozen.exitCode).toBe(1); - await installOk(dir); - expect(overridesSection(await lock(dir))).toContain('"no-deps@^1.0.1": "2.0.0"'); + await expectFrozenOverridesFailure(dir); + const { err } = await installOk(dir); + expect(err).toContain("Saved lockfile"); + const section = overridesSection(await lock(dir)); + expect(section).toContain('"no-deps@^1.0.1": "2.0.0"'); + expect(section).not.toContain('"no-deps@1": "2.0.0"'); expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("2.0.0"); + await installOk(dir, "--frozen-lockfile"); }); // Released Bun wrote a `name@range` key as a top-level string row that never applied; that row is not a ranged rule. @@ -885,7 +952,8 @@ describe.concurrent("flat override fixes", () => { overrides: { "no-deps": "$one-fixed-dep" }, }); await installOk(dir); - expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.0.0"); + // one-fixed-dep@1.0.0 installed into the no-deps slot would also report version 1.0.0, so the name is checked too. + expect(await packageSeenBy(dir, "one-range-dep", "no-deps")).toBe("no-deps@1.0.0"); const first = await lock(dir); const section = overridesSection(first); expect(section).toContain('"no-deps": "1.0.0"'); @@ -931,6 +999,7 @@ describe.concurrent("flat override fixes", () => { expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); }); + // Precedence guard for the member lookup above; this case also passes without it. test("the root's own declaration wins over members", async () => { const dir = await project(root({ "no-deps": "$no-deps" }, { "no-deps": "1.0.1" }), "hoisted", { "packages/app/package.json": member("app", "1.0.0"), @@ -967,8 +1036,9 @@ describe.concurrent("flat override fixes", () => { expect(await lock(dir)).not.toContain("no-deps@1.0.0"); }); + // ofd2 declares no-deps@2.0.0 exactly; a `^1.0.0` dependent could legitimately dedupe onto the catalog's 1.0.x depending on manifest arrival order. test("nested rule", async () => { - const deps = { "one-dep": "1.0.0", "one-range-dep": "1.0.0" }; + const deps = { "one-dep": "1.0.0", ofd2: twoParents.ofd2 }; const overrides = { "one-dep": { "no-deps": "catalog:" } }; const { packageDir: dir } = await registry.createTestDir({ bunfigOpts: { linker: "hoisted" }, @@ -976,12 +1046,12 @@ describe.concurrent("flat override fixes", () => { }); await installOk(dir); expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.0"); - expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("2.0.0"); await write(join(dir, "package.json"), withCatalog("1.0.1", deps, overrides)); await installOk(dir); expect(await versionSeenBy(dir, "one-dep", "no-deps")).toBe("1.0.1"); - expect(await versionSeenBy(dir, "one-range-dep", "no-deps")).toBe("1.1.0"); + expect(await versionSeenBy(dir, "ofd2", "no-deps")).toBe("2.0.0"); }); }); }); diff --git a/test/cli/install/registry/packages/@scoped/pkg-1/package.json b/test/cli/install/registry/packages/@scoped/pkg-1/package.json index 73ad985dca84..a7773c5bd58d 100644 --- a/test/cli/install/registry/packages/@scoped/pkg-1/package.json +++ b/test/cli/install/registry/packages/@scoped/pkg-1/package.json @@ -16,15 +16,15 @@ "dist": { "integrity": "sha512-ZukE1diuSo8MITnpi03IfVcMXL9cUr4oxvGuuVnIga3ju9ctyntuWqJSi6BKdzyvvV1qpeRkduDDNZ6ZDtly2A==", "shasum": "096b75abed5582b7c73a93578f891e588b82987a", - "tarball": "http://localhost:45907/@scoped/pkg-1/-/@scoped/pkg-1-1.1.1.tgz" + "tarball": "http://localhost:28551/@scoped/pkg-1/-/@scoped/pkg-1-1.1.1.tgz" }, "contributors": [] } }, "time": { - "modified": "2026-08-14T04:24:23.926Z", - "created": "2026-08-14T04:24:23.926Z", - "1.1.1": "2026-08-14T04:24:23.926Z" + "modified": "2026-08-14T05:25:07.737Z", + "created": "2026-08-14T05:25:07.737Z", + "1.1.1": "2026-08-14T05:25:07.737Z" }, "users": {}, "dist-tags": { diff --git a/test/cli/install/registry/packages/catalog-dep/catalog-dep-1.0.0.tgz b/test/cli/install/registry/packages/catalog-dep/catalog-dep-1.0.0.tgz new file mode 100644 index 0000000000000000000000000000000000000000..ca6e1c773865c7aa8fc4b8f0288ab535497430e3 GIT binary patch literal 192 zcmb2|=3oGW|8LKq?qV_&XnnZmvxw{3WhUbe(^iA_dvY$ zlD(VTcfUKnw{x-h#YuIW`4%NxEty}rdq&Y!e*H!NEAN*TUvxAnxpwob)zew0r<~q# q+Lzt3N>EVubJD4;cmGIU>7MiFBqPL;P;%Zoo*&oF@H1#IFaQ8=Rb58_ literal 0 HcmV?d00001 diff --git a/test/cli/install/registry/packages/catalog-dep/package.json b/test/cli/install/registry/packages/catalog-dep/package.json new file mode 100644 index 000000000000..910d86b7123f --- /dev/null +++ b/test/cli/install/registry/packages/catalog-dep/package.json @@ -0,0 +1,22 @@ +{ + "_id": "catalog-dep", + "name": "catalog-dep", + "dist-tags": { + "latest": "1.0.0" + }, + "versions": { + "1.0.0": { + "name": "catalog-dep", + "version": "1.0.0", + "dependencies": { + "no-deps": "catalog:" + }, + "_id": "catalog-dep@1.0.0", + "dist": { + "integrity": "sha512-jlVqcHEjM+7gYpduoLJj3a5s2QQxerO/50jL1F7AtV10v7nASoEgFN3EHsQAHp/8vZdfL/lqLtGYzFKRCldlqw==", + "shasum": "f5a80faf78828f7ceecb2554a1aa78edb8895130", + "tarball": "http://localhost:4873/catalog-dep/-/catalog-dep-1.0.0.tgz" + } + } + } +} \ No newline at end of file diff --git a/test/cli/install/registry/packages/catalog-peer/catalog-peer-1.0.0.tgz b/test/cli/install/registry/packages/catalog-peer/catalog-peer-1.0.0.tgz new file mode 100644 index 0000000000000000000000000000000000000000..f12e0614df710ade36e284d4cc8a4f8fc26b37bd GIT binary patch literal 197 zcmb2|=3oGW|8LLl&N}QM(E4!gcg?ny$*I?-+sm->n?2AY&!yZJTh?sW0> uao^&WEjE{ZD&y+1<;?26*T22Kxgzk1kUk^CrBJfK|32%(MPC>+7#IKzm|%JU literal 0 HcmV?d00001 diff --git a/test/cli/install/registry/packages/catalog-peer/catalog-peer-2.0.0.tgz b/test/cli/install/registry/packages/catalog-peer/catalog-peer-2.0.0.tgz new file mode 100644 index 0000000000000000000000000000000000000000..9a40494617bfeb2a8a089b29225d624663a38c4e GIT binary patch literal 201 zcmb2|=3oGW|8LLl&SEwcXidCY)#X^aXXe!M>l;erT-LBZj@jZ=bV*`=wL)Ig?HLUZ z|IcC$KJ#$OmOV?GZKr3KtTmJ&w>#JizbPp$=jYzsqB{4|@2VRjo2BOM@%XXnR$n&X=9U|o%dck}uGSCJ xd9SPQo}D;tv$!B<<=r=Hx5S@+mJ}BKK}o#65aL)U*; + peerDependencies?: Record; +}; + +const packages: Record = { + "catalog-peer": [ + { version: "1.0.0", peerDependencies: { "no-deps": "catalog:" } }, + { version: "2.0.0", peerDependencies: { "no-deps": "catalog:peers" } }, + ], + "catalog-dep": [{ version: "1.0.0", dependencies: { "no-deps": "catalog:" } }], +}; + +for (const [name, manifests] of Object.entries(packages)) { + const dir = join(packagesDir, name); + await mkdir(dir, { recursive: true }); + + const versions: Record = {}; + let latest = ""; + for (const manifest of manifests) { + const pkgJson = { name, ...manifest }; + const tarball = join(dir, `${name}-${manifest.version}.tgz`); + await Bun.Archive.write( + tarball, + { "package/package.json": JSON.stringify(pkgJson, null, 2) }, + { compress: "gzip" }, + ); + + const bytes = await Bun.file(tarball).bytes(); + versions[manifest.version] = { + ...pkgJson, + _id: `${name}@${manifest.version}`, + dist: { + integrity: `sha512-${Buffer.from(new Bun.CryptoHasher("sha512").update(bytes).digest()).toString("base64")}`, + shasum: new Bun.CryptoHasher("sha1").update(bytes).digest("hex"), + tarball: `http://localhost:4873/${name}/-/${name}-${manifest.version}.tgz`, + }, + }; + latest = manifest.version; + } + + await writeFile( + join(dir, "package.json"), + JSON.stringify({ _id: name, name, "dist-tags": { latest }, versions }, null, 2), + ); +} + +console.log("Created catalog-peer and catalog-dep test packages"); diff --git a/test/cli/install/registry/packages/publish-version-update/package.json b/test/cli/install/registry/packages/publish-version-update/package.json new file mode 100644 index 000000000000..5b9cbd1eb818 --- /dev/null +++ b/test/cli/install/registry/packages/publish-version-update/package.json @@ -0,0 +1,47 @@ +{ + "name": "publish-version-update", + "versions": { + "9.9.9": { + "name": "publish-version-update", + "version": "9.9.9", + "scripts": { + "prepublishOnly": "/Users/jarred/code/bun/build/debug/bun-debug update-version.js" + }, + "dependencies": { + "publish-version-update": "9.9.9" + }, + "_id": "publish-version-update@9.9.9", + "_integrity": "sha512-wBSHrAugp24Z+LD/QGYrofqDYGSbrrHSmB4o8HPwIXkbPGZtHndTcNkhSr+kYtU0d/otW4e6a1heF02+OE2vFg==", + "_nodeVersion": "26.3.0", + "_npmVersion": "10.8.3", + "integrity": "sha512-wBSHrAugp24Z+LD/QGYrofqDYGSbrrHSmB4o8HPwIXkbPGZtHndTcNkhSr+kYtU0d/otW4e6a1heF02+OE2vFg==", + "shasum": "7dfba4fef2ca45810a7bd6d2631c69ad8f838046", + "dist": { + "integrity": "sha512-wBSHrAugp24Z+LD/QGYrofqDYGSbrrHSmB4o8HPwIXkbPGZtHndTcNkhSr+kYtU0d/otW4e6a1heF02+OE2vFg==", + "shasum": "7dfba4fef2ca45810a7bd6d2631c69ad8f838046", + "tarball": "http://localhost:6108/publish-version-update/-/publish-version-update-9.9.9.tgz" + }, + "contributors": [] + } + }, + "time": { + "modified": "2026-08-14T05:15:52.536Z", + "created": "2026-08-14T05:15:52.536Z", + "9.9.9": "2026-08-14T05:15:52.536Z" + }, + "users": {}, + "dist-tags": { + "latest": "9.9.9" + }, + "_uplinks": {}, + "_distfiles": {}, + "_attachments": { + "publish-version-update-9.9.9.tgz": { + "shasum": "7dfba4fef2ca45810a7bd6d2631c69ad8f838046", + "version": "9.9.9" + } + }, + "_rev": "", + "_id": "publish-version-update", + "readme": "" +} \ No newline at end of file diff --git a/test/cli/install/registry/packages/publish-version-update/publish-version-update-9.9.9.tgz b/test/cli/install/registry/packages/publish-version-update/publish-version-update-9.9.9.tgz new file mode 100644 index 0000000000000000000000000000000000000000..c048964c1aeddfca1c13a44f21e8386868a61b0c GIT binary patch literal 367 zcmV-#0g(P5iwFP!00002|Lv8_Zi6roMOo)7EU$<}z$Aq9p>DhAqFGe^00$4$2{o>b zTSZm=y~Cqbnl_s%id2b%kYUEYGe*8b^K8Qlh;%oGYg;Mr#4yIvBq7a+9FC05l9aI7 zJY{j5rYs>Wp2czO5$2r%XKQY-l@o08+vffW?;}E};?F>@2-R*?O8Yc<1!HBUCQb`p z0}YxveXN;w8RFGNZDypdEk+w53}^$r%2JivW_I*wacQ*X#(;>jN4+-tR4cb=fQ*?Yw~_^uCXNeE2i^f4Z0-_&YBFn#y1ybm=u7W~|X2DrGDB|L0Z=F|PZ z2Mq{yb$&s|y$CK3`*Vddy)U~I3cVRw!%?NSsI84uh0M1;N(Dr@vJA)+lod|C zo%<0FPTt-e?FB^_rv5slQZl(H5^KxqZT1lSi$xwte>S>WW` zxgT-o6x`KQFDUsq?Jh$;cE#t?X Date: Fri, 14 Aug 2026 05:29:31 +0000 Subject: [PATCH 04/25] [autofix.ci] apply automated fixes --- docs/pm/filter.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/pm/filter.mdx b/docs/pm/filter.mdx index e326502aa80c..91948ac2acd4 100644 --- a/docs/pm/filter.mdx +++ b/docs/pm/filter.mdx @@ -29,12 +29,12 @@ A directory in braces selects every workspace located in that directory or anywh Adding `...` to a pattern also selects the workspaces related to it through workspace dependencies: -| Pattern | Selects | -| ---------- | --------------------------------------------------------------------------- | -| `foo...` | `foo` and the workspaces it depends on, directly or transitively | -| `foo^...` | only the workspaces `foo` depends on, not `foo` itself | -| `...foo` | `foo` and the workspaces that depend on it, directly or transitively | -| `...^foo` | only the workspaces that depend on `foo`, not `foo` itself | +| Pattern | Selects | +| --------- | -------------------------------------------------------------------- | +| `foo...` | `foo` and the workspaces it depends on, directly or transitively | +| `foo^...` | only the workspaces `foo` depends on, not `foo` itself | +| `...foo` | `foo` and the workspaces that depend on it, directly or transitively | +| `...^foo` | only the workspaces that depend on `foo`, not `foo` itself | The middle part is a name glob or a directory selector (`...{./packages/api}`), and `!` still goes first: `--filter '!...foo'` drops `foo` and everything that depends on it. From a5f2855185d9926442fd7829b0995211213f1841 Mon Sep 17 00:00:00 2001 From: Jarred Sumner Date: Thu, 13 Aug 2026 22:38:12 -0700 Subject: [PATCH 05/25] install: subset installs under --filter and remaining follow-ups add/remove/update --filter now link only the selected workspaces, like install --filter; every filtered command warns about patterns that match nothing. bun run --filter accepts the same {dir} and relation selectors, and --filter may precede the subcommand. prune plans against the workspaces present on a pruned checkout and the frozen catalog-subset tolerance prints a note. A package defined in both catalog and catalogs.default is an error; add --catalog decides per target, prints a note when an entry is reused with a different range, and catalogs nameless git/tarball positionals after resolution. --latest keeps a transitive dist-tag row on its own tag; named updates are looked up through a hash index; -i honors --dev/--prod. pm licenses --json reports paths; pnpm-lock.yaml peer variants are chosen per importer; dedupe keeps versions needed to reach a patched package and says so. No-Verification-Needed: user asked to skip --- docs/pm/catalogs.mdx | 4 +- docs/pm/cli/add.mdx | 7 +- docs/pm/cli/dedupe.mdx | 3 +- docs/pm/cli/install.mdx | 4 +- docs/pm/cli/pm.mdx | 5 +- docs/pm/cli/prune.mdx | 2 +- docs/pm/cli/update.mdx | 11 +- docs/pm/filter.mdx | 24 +- docs/runtime/index.mdx | 2 +- src/install/PackageManager.rs | 27 +- .../PackageManager/CommandLineArguments.rs | 5 +- .../PackageManager/PackageManagerEnqueue.rs | 42 +- .../PackageManager/PackageManagerOptions.rs | 2 +- src/install/PackageManager/UpdateRequest.rs | 56 +- src/install/PackageManager/add_catalog.rs | 593 ++++++++++++++---- .../PackageManager/add_remove_with_filter.rs | 39 +- .../PackageManager/install_with_manager.rs | 35 +- .../PackageManager/security_scanner.rs | 30 +- .../updatePackageJSONAndInstall.rs | 14 +- .../PackageManager/workspace_selection.rs | 147 ++++- src/install/dedupe.rs | 191 ++++-- src/install/lib.rs | 1 + src/install/lockfile.rs | 15 + src/install/lockfile/CatalogMap.rs | 35 +- src/install/lockfile/Package.rs | 113 ++-- src/install/lockfile/Tree.rs | 10 +- src/install/lockfile/pruned_workspaces.rs | 103 +-- src/install/pnpm.rs | 58 +- src/install/prune.rs | 87 ++- src/install/update_scope.rs | 23 +- src/runtime/cli/filter_arg.rs | 220 +++---- src/runtime/cli/filter_run.rs | 80 +-- src/runtime/cli/multi_run.rs | 76 +-- src/runtime/cli/pm_licenses_command.rs | 129 ++-- src/runtime/cli/update_interactive_command.rs | 22 +- test/cli/install/bun-add-catalog.test.ts | 384 +++++++++++- test/cli/install/bun-add-filter.test.ts | 143 ++++- test/cli/install/bun-dedupe.test.ts | 77 +++ test/cli/install/bun-pm-licenses.test.ts | 122 +++- test/cli/install/bun-prune.test.ts | 111 +++- .../bun-security-scanner-workspaces.test.ts | 80 +++ .../cli/install/bun-update-transitive.test.ts | 119 +++- test/cli/install/bun-update.test.ts | 143 ++++- test/cli/install/catalogs.test.ts | 72 +++ .../install/frozen-lockfile-pruned.test.ts | 101 ++- .../install/migration/pnpm-lock-v9.test.ts | 136 +++- .../pnpm/v9-peer-variant-merge/package.json | 10 + .../packages/with/package.json | 7 + .../pnpm/v9-peer-variant-merge/pnpm-lock.yaml | 43 ++ .../vendor/has-peer/package.json | 7 + .../vendor/peer/package.json | 4 + test/cli/run/filter-workspace.test.ts | 208 ++++++ 52 files changed, 3201 insertions(+), 781 deletions(-) create mode 100644 test/cli/install/migration/pnpm/v9-peer-variant-merge/package.json create mode 100644 test/cli/install/migration/pnpm/v9-peer-variant-merge/packages/with/package.json create mode 100644 test/cli/install/migration/pnpm/v9-peer-variant-merge/pnpm-lock.yaml create mode 100644 test/cli/install/migration/pnpm/v9-peer-variant-merge/vendor/has-peer/package.json create mode 100644 test/cli/install/migration/pnpm/v9-peer-variant-merge/vendor/peer/package.json diff --git a/docs/pm/catalogs.mdx b/docs/pm/catalogs.mdx index bed797b5ee2f..31c85b99b208 100644 --- a/docs/pm/catalogs.mdx +++ b/docs/pm/catalogs.mdx @@ -246,6 +246,8 @@ bun add react --catalog Bun writes the entry to the catalog in the root `package.json` (the range the current `package.json` already declares, otherwise the resolved version) and `"catalog:"` to the `package.json` you ran the command in. See [`bun add --catalog`](/pm/cli/add#--catalog). +When the entry already existed and differs from the range the package declared, `bun add` says so on stderr, e.g. `note: react in app now follows the catalog entry "^19.0.0" instead of "^18.2.0"`. With `--filter`, the first selected package that declares a range seeds a missing entry; a later package whose declared range does not fit that entry keeps its range and gets a `keeps` note instead. + `bun add react` without the flag also uses the default catalog when it lists `react`: the package gets `"catalog:"`, and an existing `"catalog:"` reference is kept. `bun add react@latest` (or any explicit version) writes a concrete range instead. Named catalogs are only used through `--catalog=` or an existing reference, and a version you type is never checked against the catalog (there is no equivalent of pnpm's `catalogMode` setting). ## Lockfile Integration @@ -294,7 +296,7 @@ Bun's lockfile tracks catalog versions, so installs are consistent across enviro - Catalog references must match a dependency defined in either `catalog` or one of the named `catalogs` - Empty strings and whitespace in catalog names are ignored (treated as default catalog) -- `catalog:default` is the same as `catalog:`; both read the `catalog` field, or `catalogs.default` if that is where the entry is defined (as pnpm names it) +- `catalog:default` is the same as `catalog:`; both read the `catalog` field, or `catalogs.default` if that is where the entry is defined (as pnpm names it). A package may be defined in only one of the two: if both `catalog` and `catalogs.default` list it, `bun install` stops with `error: "react" is defined in both "catalog" and "catalogs.default"` (`bun add --catalog` always edits the one that already defines the package) - Invalid dependency versions in catalogs fail to resolve during `bun install` - `catalog:` is only understood in the root `package.json` and in workspace packages. A `catalog:` specifier inside an installed package (from the registry, git, a tarball or a `file:` folder) never reads your catalogs: as a regular dependency it fails to resolve (`x@catalog: failed to resolve`), and as a peer dependency it is treated as unsatisfiable and no copy is installed for it. Publish with `bun publish` or `bun pm pack`, which substitute the ranges (see [Publishing](#publishing)) diff --git a/docs/pm/cli/add.mdx b/docs/pm/cli/add.mdx index 341462852ff5..2ed7e3d61559 100644 --- a/docs/pm/cli/add.mdx +++ b/docs/pm/cli/add.mdx @@ -114,12 +114,13 @@ A few things to know, some of which differ from pnpm's `--save-catalog`: - If the catalog already has an entry for the package, `bun add react --catalog` leaves it alone and only writes `"react": "catalog:"` to the current package. `bun add react@19.1.0 --catalog` keeps an entry such as `^19.0.0` that the version satisfies and only changes which version is installed; any other version or range replaces the entry, which moves every package that depends on `"catalog:"` along with it. - Without a version, a range the current `package.json` already has (`"react": "^18.2.0"`) is what gets cataloged. The package is only resolved to its latest version when neither the catalog nor the `package.json` mentions it. -- A package that already depends on `"react": "catalog:legacy"` stays in `legacy`, whatever name the flag carries: the `legacy` entry is reused, or replaced if you gave a version. With `--filter`, this is decided for each selected package. +- A package that already depends on `"react": "catalog:legacy"` stays in `legacy`, whatever name the flag carries: the `legacy` entry is reused, or replaced if you gave a version. +- With `--filter`, the entry is decided per selected package. An entry the root already has is used by all of them. When there is none, the first selected package that declares a range seeds it; a later package that declares the same range, or an exact version inside it, is switched to `"catalog:"`, and one that declares anything else keeps its range. Bun prints a `note:` for every package whose range was changed or kept. - `--catalog` and `--catalog=default` are the same catalog: the entry is written to whichever of `catalog` or `catalogs.default` the root `package.json` already defines (`catalog` is created when there is neither), so a repository migrated from pnpm never ends up with the package in both. - The recorded version is the one that was installed, so an entry in [`overrides`](/pm/overrides) for the package ends up in the catalog too. - The name must be attached with `=` (`--catalog=testing`); `--catalog testing` adds a package called `testing`. - pnpm's `--save-catalog` is spelled `--catalog` and `--save-catalog-name testing` is `--catalog=testing`; Bun silently skips flags it does not know, so the pnpm spellings add the package as an ordinary dependency (and `--save-catalog-name testing` also tries to add a package called `testing`) instead of failing. -- Packages must be added by name (`react`, `react@^18`, `alias@npm:react`, `alias@https://example.com/react.tgz`). Bare URLs and paths, relative `file:`/`link:`/folder specs, and workspace packages (by name or via `workspace:`) are refused before anything is written. +- A positional without a name (`bun add https://example.com/react.tgz --catalog`, `bun add github:facebook/react --catalog`) is installed first and then cataloged under the package's real name, with the same text a plain `bun add` would have written. If the catalog already has a different entry for that name, or the current package already declared that name itself, that declaration is left as it is and Bun prints a note; `bun add --catalog` converts it. Relative paths (`./vendor/foo`, `foo@file:../foo`, `link:`) and workspace packages (by name or via `workspace:`) are still refused before anything is written. - The root `package.json` must have a `workspaces` field. Catalogs are also used without the flag: in a workspace whose default catalog lists `react`, `bun add react` with no version writes `"react": "catalog:"`, and a package that already depends on `"catalog:"` keeps that reference. Pass a version (`bun add react@latest`) to write a concrete range instead. @@ -145,7 +146,7 @@ A few things to know, some of which differ from pnpm: - Name patterns are globs, and `*` does not cross `/`: `*-utils` does not match `@acme/date-utils`; use `*/*-utils` or `@acme/*`. - Path patterns must match a workspace directory exactly, relative to the current directory: `--filter ./packages` selects nothing, `--filter ./packages/*` or `--filter '{packages}'` selects every package in it, and `--filter .` only works from a workspace's own directory. - Local paths (`./vendor/logger`, `file:../logger`, `./logger.tgz`) are relative to the current directory and are re-spelled relative to each selected package (`"logger": "../../vendor/logger"` in `packages/api/package.json`). -- The flag chooses which `package.json` files are edited, not what is installed: a single install of the whole workspace runs afterwards. The `package.json` files are written as soon as the lockfile is saved, so they agree with `bun.lock` even if a root `postinstall` script then fails. +- The flag chooses both which `package.json` files are edited and which workspaces are installed. Everything is still resolved and `bun.lock` is updated for the whole repository, but, as with `bun install --filter`, only the selected workspaces' dependencies are linked into `node_modules`; the other workspaces' `node_modules` (and the root's own dependencies and lifecycle scripts, unless a pattern selected the root) are left exactly as they were, and a plain `bun install` brings them up to date. The `package.json` files are written as soon as the lockfile is saved, so they agree with `bun.lock` even if a root `postinstall` script then fails. - The install summary is printed from the point of view of one selected workspace (the first one that received every requested package); the other selected `package.json` files get the same edits even though they are not listed in the output. - `--dry-run` skips writing; `--filter` cannot be combined with `--global`. diff --git a/docs/pm/cli/dedupe.mdx b/docs/pm/cli/dedupe.mdx index 7d42930f3872..71e56303593c 100644 --- a/docs/pm/cli/dedupe.mdx +++ b/docs/pm/cli/dedupe.mdx @@ -38,7 +38,8 @@ bun dedupe --check - Overrides and catalogs are honoured: a dependency is re-pointed using its effective range. - A dependency listed directly in the root or a workspace `package.json` is only moved to an older version when that is the only way to remove a version: if a transitive dependency pins an older version exactly, the direct range is collapsed onto it (pnpm keeps both versions in that case). Use `bun update` or an [override](/pm/overrides) if you want the newer version to win. - A range whose version survives is never moved. -- Dependencies pointing at a version listed in `patchedDependencies` are never moved, bundled dependencies stay on the copy inside their tarball, and dependencies specified as a dist-tag (such as `latest`), a git URL, or a tarball keep the version they resolved to. A dist-tag keeps its version because only the registry knows what the tag points at now; other ranges are still collapsed onto that version. +- A version listed in `patchedDependencies` is never moved away from and never removed. If removing some other version would leave a patched package unreachable, that version is kept as well, and both `bun dedupe` and `bun dedupe --check` say so on stderr, e.g. `note: kept one-fixed-dep@1.0.0 (needed to reach patched no-deps@1.0.0)`. Delete the patch entry (or `bun update` the dependent) if you want the version removed. +- Bundled dependencies stay on the copy inside their tarball, and dependencies specified as a dist-tag (such as `latest`), a git URL, or a tarball keep the version they resolved to. A dist-tag keeps its version because only the registry knows what the tag points at now; other ranges are still collapsed onto that version. - Only the ranges of packages that remain installed are counted; a version that the same run removes does not influence where its own dependencies end up, so running the command twice never changes anything the second time. - It works from the ranges recorded in `bun.lock`. If the dependencies, overrides, or catalogs in any `package.json` changed since the last install, it stops with `error: the lockfile is out of date with package.json, nothing was deduplicated` and exits with code `1` (also with `--check`); run `bun install` first. Edits that do not affect resolution (`scripts`, `trustedDependencies`) do not block it. - The command needs an existing `bun.lock` to work from: when there is none, `bun dedupe` (and `bun dedupe --check`) reports `missing lockfile, nothing to dedupe` and exits with code `1`, so run `bun install` first. diff --git a/docs/pm/cli/install.mdx b/docs/pm/cli/install.mdx index 9925ebd572a6..afff5728f01d 100644 --- a/docs/pm/cli/install.mdx +++ b/docs/pm/cli/install.mdx @@ -174,7 +174,7 @@ bun install --frozen-lockfile Bun does not enable `--frozen-lockfile` on its own when it detects a CI environment (`CI=1` only turns off the progress bar); pass the flag or run `bun ci` if a stale `bun.lock` should fail the build. When there is no `bun.lock` at all, `bun install --frozen-lockfile` (and `bun ci`) resolves and installs from `package.json` without writing a lockfile; the error is only raised when a lockfile exists and does not match `package.json`. -`--frozen-lockfile` also works on a pruned copy of a monorepo (for example the output of `turbo prune`, or a Docker context that copies `bun.lock` with only some workspace folders): workspaces listed in `bun.lock` whose `package.json` is not on disk are skipped rather than treated as a lockfile change, and packages the pruned lockfile still lists are installed as written. This works whether the root `workspaces` field uses globs or lists each folder explicitly, and Bun prints a `note:` with the number of skipped workspaces (`--verbose` names them), so a `bun.lock` that is stale because a workspace was deleted without re-running `bun install` is still visible in CI logs. Only workspaces that `bun.lock` knows about are skipped: a `workspaces` entry whose folder is missing and which `bun.lock` does not list (for example a typo) still fails with `Workspace not found`, exactly as it does without `--frozen-lockfile`. If a remaining workspace (or the root `package.json`) depends on a skipped workspace, the install fails with `workspace "app" depends on workspace "other" (packages/other), which is listed in bun.lock but not on disk`: whatever prunes the checkout has to keep the workspaces the survivors depend on (`turbo prune` does). This is stricter than pnpm, whose `--frozen-lockfile` also ignores a workspace that is still on disk but was removed from the `workspaces` list — Bun only skips a workspace whose `package.json` is gone; un-listing one that is still present is a lockfile change. Changing any `package.json` that is on disk or trimming `overrides` from `bun.lock` still fails the install; `catalog`/`catalogs` entries, on the other hand, may be missing from a pruned `bun.lock` as long as only the skipped workspaces used them — every entry `bun.lock` does keep must still match `package.json`, and an entry that a remaining workspace or an `overrides` rule refers to must be present. The skipped workspaces' exclusive dependencies are not downloaded or installed, but they remain in `bun.lock`, so `bun pm ls` and `bun audit` still report them. A package that a surviving workspace lists as an optional peer dependency is installed if the pruned `bun.lock` still contains it, even when only a skipped workspace depended on it. This also holds when only `trustedDependencies` (which `turbo prune` leaves out of the pruned `bun.lock`) or `patchedDependencies` differ between `package.json` and `bun.lock`; a package is only dropped from the lockfile when a dependency, `overrides` or `catalog` entry actually changed, or when `bun audit fix` upgrades it. [`bun prune`](/pm/cli/prune) accepts such a checkout as well. +`--frozen-lockfile` also works on a pruned copy of a monorepo (for example the output of `turbo prune`, or a Docker context that copies `bun.lock` with only some workspace folders): workspaces listed in `bun.lock` whose `package.json` is not on disk are skipped rather than treated as a lockfile change, and packages the pruned lockfile still lists are installed as written. This works whether the root `workspaces` field uses globs or lists each folder explicitly, and Bun prints a `note:` with the number of skipped workspaces (`--verbose` names them), so a `bun.lock` that is stale because a workspace was deleted without re-running `bun install` is still visible in CI logs. Only workspaces that `bun.lock` knows about are skipped: a `workspaces` entry whose folder is missing and which `bun.lock` does not list (for example a typo) still fails with `Workspace not found`, exactly as it does without `--frozen-lockfile`. If a remaining workspace (or the root `package.json`) depends on a skipped workspace, the install fails with `workspace "app" depends on workspace "other" (packages/other), which is listed in bun.lock but not on disk`: whatever prunes the checkout has to keep the workspaces the survivors depend on (`turbo prune` does). A plain `bun install` reports the same error in that situation; it looks at what the remaining `package.json` files declare, so deleting a workspace together with the entries that pointed at it installs normally and drops it from `bun.lock`. This is stricter than pnpm, whose `--frozen-lockfile` also ignores a workspace that is still on disk but was removed from the `workspaces` list — Bun only skips a workspace whose `package.json` is gone; un-listing one that is still present is a lockfile change. Changing any `package.json` that is on disk or trimming `overrides` from `bun.lock` still fails the install; `catalog`/`catalogs` entries, on the other hand, may be missing from a pruned `bun.lock` as long as only the skipped workspaces used them — every entry `bun.lock` does keep must still match `package.json`, and an entry that a remaining workspace or an `overrides` rule refers to must be present. When entries were left out this way, the install prints `note: skipped 1 catalog entry missing from bun.lock that no remaining workspace uses` alongside the skipped-workspace note. The skipped workspaces' exclusive dependencies are not downloaded or installed, but they remain in `bun.lock`, so `bun pm ls` and `bun audit` still report them. A package that a surviving workspace lists as an optional peer dependency is installed if the pruned `bun.lock` still contains it, even when only a skipped workspace depended on it. This also holds when only `trustedDependencies` (which `turbo prune` leaves out of the pruned `bun.lock`) or `patchedDependencies` differ between `package.json` and `bun.lock`; a package is only dropped from the lockfile when a dependency, `overrides` or `catalog` entry actually changed, or when `bun audit fix` upgrades it. [`bun prune`](/pm/cli/prune) accepts such a checkout as well and prunes it against the workspaces that are on disk. `--frozen-lockfile` never writes `bun.lock`, including with `--lockfile-only`. The one exception is a lockfile format migration: converting `bun.lockb` with `--save-text-lockfile`, or migrating from `package-lock.json`, `yarn.lock` or `pnpm-lock.yaml`, still writes `bun.lock`. To check a lockfile without installing anything, use `bun install --frozen-lockfile --dry-run`. @@ -526,7 +526,7 @@ The migration process handles: - `name@registry:version` entries written for pnpm's named registries use the URL from `namedRegistries` in `pnpm-workspace.yaml` (`gh` and `npmjs` are built in); an unknown name falls back to Bun's registry with a warning - Peer dependencies keep the ranges each package declares, together with `peerDependenciesMeta`, and the `peerDependencies` of the root and of every workspace are read from their `package.json`, so the migrated `bun.lock` is the file `bun install` would have written and the next install leaves it unchanged - A peer dependency that pnpm did not record is reported and left for the next `bun install` to resolve -- When a package appears in `snapshots:` under several peer suffixes, the first one in the file is used +- A package that pnpm recorded under several peer suffixes becomes a single `bun.lock` entry: which peers each importer gets is decided again by `bun install` from the declared ranges (the isolated linker creates one `node_modules/.bun` entry per peer set, as pnpm's store does), and a peer that pnpm resolved to a local folder or workspace package in some importers keeps that resolution even though other importers left it unmet - A `file:` directory whose `packages:` entry was removed by a pruning tool such as `turbo prune` is rebuilt from its snapshot, and injected workspace packages (`dependenciesMeta.*.injected`) become ordinary workspace dependencies - Migrates git (`git+ssh://`, `git+https://`), GitHub, tarball URL and `file:` dependencies, including transitive ones and npm aliases (`npm:`) recorded in the lockfile; a local tarball is recognised by a `.tgz`, `.tar.gz` or `.tar` extension in any case - Handles patched dependencies with integrity hashes; pnpm's current hash-only `patchedDependencies` entries are matched to the patch files listed in `package.json` or `pnpm-workspace.yaml` diff --git a/docs/pm/cli/pm.mdx b/docs/pm/cli/pm.mdx index 5f3e0daf62d9..879d28f27b11 100644 --- a/docs/pm/cli/pm.mdx +++ b/docs/pm/cli/pm.mdx @@ -189,7 +189,7 @@ Unknown (4) License groups are printed in name order with `Unknown` always last, and packages within a group are sorted by name and then version. Packages that are only reached through `devDependencies` are marked `(dev)`. When nothing is listed, the text output is the single line `No packages found` and `--json` prints `{}`. -Pass `--prod` (or `-P`, `-p`, `--production`) to skip `devDependencies`, and `--json` to get a machine-readable object keyed by license, where each entry has `name`, `versions` (in semver order), `license` (the same string as the key, `Unknown` included) and, when the newest listed version declares them, its `homepage`, `author` and `description`: +Pass `--prod` (or `-P`, `-p`, `--production`) to skip `devDependencies`, and `--json` to get a machine-readable object keyed by license, where each entry has `name`, `versions` (in semver order), `paths` (the directory each of those versions was read from, in the same order — under `node_modules` for hoisted installs, inside `node_modules/.bun` for isolated installs), `license` (the same string as the key, `Unknown` included) and, when the newest listed version declares them, its `homepage`, `author` and `description`: ```bash terminal icon="terminal" bun pm licenses --json --prod @@ -201,6 +201,7 @@ bun pm licenses --json --prod { "name": "path-parse", "versions": ["1.0.6"], + "paths": ["/home/me/app/node_modules/path-parse"], "license": "MIT", "homepage": "https://github.com/jbgutierrez/path-parse#readme", "author": "Javier Blanco ", @@ -210,7 +211,7 @@ bun pm licenses --json --prod } ``` -`--long` prints each package's `author`, `description` and `homepage` (whichever it declares) on indented lines under its entry in the text output; it does not change `--json`. +`--long` prints each package's `author`, `description` and `homepage` (whichever it declares) on indented lines under its entry in the text output; it does not change `--json`, and `paths` are only part of the `--json` output. `--prod` drops every `devDependencies` edge — the same packages `bun install --production` leaves out, including the `devDependencies` of `file:` dependencies; `optionalDependencies`, `peerDependencies` and everything a production dependency pulls in are still listed, and from a workspace root every workspace's production dependencies are included. Run it inside a workspace package to list only that package's dependencies, and use [`bun why`](/pm/cli/why) to see which dependency pulls in an unexpected package. diff --git a/docs/pm/cli/prune.mdx b/docs/pm/cli/prune.mdx index 3ed3de9068b7..e27857cc28e2 100644 --- a/docs/pm/cli/prune.mdx +++ b/docs/pm/cli/prune.mdx @@ -70,7 +70,7 @@ Removed 1 package - Nothing outside `node_modules` is ever deleted. A package folder that you replaced with a symlink is left alone, and the `node_modules` folder inside it is not pruned. With `install.globalStore`, only the project's links into the store are removed. - Packages disabled for the current `os`/`cpu` are removed, just as `bun install` would skip them. Pass `--os` / `--cpu` to prune for another platform. - What stays is decided by `bun.lock` alone, and packages are matched by name: a package that is installed under the right name at the wrong version is left for `bun install` to replace, and a dependency you removed from `package.json` is pruned after the `bun install` that updates `bun.lock` (until then `bun prune` refuses to run). A copy nested inside another package (`node_modules/a/node_modules/b`) is only removed once the copy that replaces it higher up is installed at the version `bun.lock` expects (checked from its `package.json`, or `.bun-tag` for git dependencies); until then it is kept and `bun prune` prints a `warn:` line naming both folders — typically after `bun prune --production` when a `devDependency` pinned a different version of `b` at the root. Run `bun install` (with the same flags), then `bun prune` again. -- A `bun.lock` that still lists workspaces whose folders were removed (a pruned checkout that `bun install --frozen-lockfile` accepts) is accepted too. +- A `bun.lock` that still lists workspaces whose folders were removed (a pruned checkout that `bun install --frozen-lockfile` accepts) is pruned the way that install would install it: the missing workspaces' links and the packages only they needed are removed, `--filter` does not keep them, and the same `note:` lines `bun install --frozen-lockfile` prints for such a checkout (the skipped workspaces, catalog entries left out of `bun.lock`) are printed. A remaining workspace that still depends on a missing one is an error, exactly as it is for `bun install`. - If an entry cannot be deleted, the others are still removed, `error: failed to remove ...` is printed for each failure and the command exits with code 1 (`pnpm prune` only warns) — in a Dockerfile, a package that could not be removed should fail the build. - Lifecycle scripts are never run. - Equivalent to `pnpm prune` and `npm prune`. Unlike `pnpm prune`, it works in monorepos, and `--filter` narrows it to some workspaces. It does not touch the global cache; the counterpart of `pnpm store prune` is [`bun pm cache rm`](/pm/cli/pm#cache). Not related to `turbo prune`, which copies a subset of a monorepo's workspaces, `package.json` files and `bun.lock` into an output directory: `bun prune` only deletes from `node_modules`, so a Dockerfile can use `turbo prune`, then `bun install`, then `bun prune --production` after building. diff --git a/docs/pm/cli/update.mdx b/docs/pm/cli/update.mdx index 349411a167c3..1fbfc757080a 100644 --- a/docs/pm/cli/update.mdx +++ b/docs/pm/cli/update.mdx @@ -42,7 +42,7 @@ bun update --interactive bun update -i ``` -The flag opens a terminal interface that lists every outdated direct dependency with its current and target versions. Once you confirm, the selection is applied like `bun update ` with the selected names: those entries are rewritten in `package.json` and updated everywhere they occur in `bun.lock`, while the packages you left unselected, and every other transitive package, keep their locked versions. Confirming with nothing selected changes nothing. +The flag opens a terminal interface that lists every outdated direct dependency with its current and target versions (`--dev`, `--prod` and `--no-optional` narrow that list, see [below](#--dev---prod---no-optional)). Once you confirm, the selection is applied like `bun update ` with the selected names: those entries are rewritten in `package.json` and updated everywhere they occur in `bun.lock`, while the packages you left unselected, and every other transitive package, keep their locked versions. Confirming with nothing selected changes nothing. ### Interactive Interface @@ -115,9 +115,9 @@ Within each section, individual packages may have a suffix (` dev`, ` peer`, ` o ## `--recursive` and `--filter` -In a monorepo, a plain `bun update` rewrites only the `package.json` of the workspace you run it in (transitive packages are shared, so they update either way). `bun update --recursive` (`-r`) also updates the direct dependencies of every workspace and rewrites each workspace's `package.json`; `bun update --filter ` limits that to the matching workspaces; the pattern syntax, including `{dir}` and `...` relations, is described on the [filtering](/pm/filter) page. Both need an existing `bun.lock`. They also work with `--interactive`, which then adds a "Workspace" column showing which workspace each dependency belongs to. +In a monorepo, a plain `bun update` rewrites only the `package.json` of the workspace you run it in (transitive packages are shared, so they update either way). `bun update --recursive` (`-r`) also updates the direct dependencies of every workspace and rewrites each workspace's `package.json`; `bun update --filter ` (`-F`) limits that to the matching workspaces; the pattern syntax, including `{dir}` and `...` relations, is described on the [filtering](/pm/filter) page. Both need an existing `bun.lock`. They also work with `--interactive`, which then adds a "Workspace" column showing which workspace each dependency belongs to. -`bun update -r` and `bun update --filter ` rewrite the entry for `` in every selected workspace whose `package.json` declares it, each keeping its own `^`/`~`/exact style; the `package.json` of a workspace that was not selected is left alone, and nested packages that depend on `` still move in `bun.lock`. A name that none of the selected workspaces uses, directly or through their dependencies, is an error. `--latest` and `--dry-run` apply the same way: +`bun update -r` and `bun update --filter ` rewrite the entry for `` in every selected workspace whose `package.json` declares it, each keeping its own `^`/`~`/exact style; the `package.json` of a workspace that was not selected is left alone, and nested packages that depend on `` still move in `bun.lock`. A name that none of the selected workspaces uses, directly or through their dependencies, is an error. `--latest` and `--dry-run` apply the same way. With `--filter`, with or without names, the install that follows links only the selected workspaces, exactly like [`bun install --filter`](/pm/filter), and a pattern that matches nothing prints a warning; `-r` installs every workspace: ```sh terminal icon="terminal" bun update --recursive @@ -129,12 +129,13 @@ bun update zod --filter '...^ui' ## `--dev`, `--prod`, `--no-optional` -`bun update --dev` (`-D`) only updates the entries in `devDependencies`; `--prod` (`-P`, also `--production`) only those in `dependencies` and `optionalDependencies`; `--no-optional` skips `optionalDependencies`. These select which entries of the current workspace's `package.json` (or of the workspaces chosen with `-r`/`--filter`) are updated, and combine with names, patterns and `--latest`. They do not change what gets installed: `bun update --production` still installs `devDependencies`, unlike `bun install --production`. They need an existing `bun.lock`, and print `No packages to update` when no entry qualifies: +`bun update --dev` (`-D`) only updates the entries in `devDependencies`; `--prod` (`-P`, also `--production`) only those in `dependencies` and `optionalDependencies`; `--no-optional` skips `optionalDependencies`. These select which entries of the current workspace's `package.json` (or of the workspaces chosen with `-r`/`--filter`) are updated, and combine with names, patterns and `--latest`. They do not change what gets installed: `bun update --production` still installs `devDependencies`, unlike `bun install --production`. They need an existing `bun.lock`, and print `No packages to update` when no entry qualifies. With `--interactive` the same flags decide which entries the picker lists (`bun update -i --dev`): ```sh terminal icon="terminal" bun update --dev bun update --prod --latest bun update -D '@types/*' +bun update -i --prod ``` ## `--global` @@ -150,7 +151,7 @@ bun update -g typescript By default, `bun update` updates each dependency to the latest version that satisfies the version range in your `package.json`. -To update the dependencies declared in `package.json` to the latest version regardless of whether it satisfies that range, use the `--latest` flag (`-L`). Transitive packages still move within the ranges their dependents declare, and a dependency that is already ahead of the `latest` tag (for example a prerelease) is left where it is rather than downgraded: +To update the dependencies declared in `package.json` to the latest version regardless of whether it satisfies that range, use the `--latest` flag (`-L`). Transitive packages still move within the ranges their dependents declare, and an entry of the workspace you run in that is already ahead of the `latest` tag (for example a prerelease) is left where it is rather than downgraded. That only applies to the entries `--latest` rewrites: an entry you declared as a dist-tag, or a `"foo": "next"`-style entry declared by another package or workspace, still resolves through its own tag: ```sh terminal icon="terminal" bun update --latest diff --git a/docs/pm/filter.mdx b/docs/pm/filter.mdx index 91948ac2acd4..ac86b497c2ab 100644 --- a/docs/pm/filter.mdx +++ b/docs/pm/filter.mdx @@ -5,7 +5,7 @@ description: "Select packages by pattern in a monorepo using the --filter flag" The `--filter` (or `-F`) flag selects packages in a monorepo by pattern. A pattern is a package name glob, a `./path` glob, a `{dir}` directory selector, or a `...` dependency relation. -`bun install`, `bun add`, `bun remove`, `bun update` and `bun outdated` support `--filter` (pass it after the subcommand, or as `--filter=` before it); for `bun add`/`bun remove` it selects which workspace `package.json` files are edited. You can also use it to run scripts in multiple packages at once. The `{dir}` and `...` forms are only understood by these package-manager commands, not by `bun run --filter`. +The same patterns are understood by `bun install`, `bun add`, `bun remove`, `bun update` and `bun outdated`, and by `bun run --filter` (including `--parallel` and `--sequential`), which runs a script in every selected package. For the package-manager commands, pass the flag after the subcommand or write it as `--filter=` before it, because `bun --filter ` always runs the script ``. For `bun add`, `bun remove` and `bun update` the filter selects which workspace `package.json` files are edited and, like `bun install --filter`, which workspaces get installed; the `node_modules` of an unselected workspace is left as it was. --- @@ -38,7 +38,7 @@ Adding `...` to a pattern also selects the workspaces related to it through work The middle part is a name glob or a directory selector (`...{./packages/api}`), and `!` still goes first: `--filter '!...foo'` drops `foo` and everything that depends on it. -Relations follow the workspace links recorded in `bun.lock`, from any dependency group. `bun install --filter 'web...'` works on a fresh clone, but `bun add`, `bun remove` and `bun update ` with a relation need an existing `bun.lock` and ask you to run `bun install` first when it is missing. +For the package-manager commands, relations follow the workspace links recorded in `bun.lock`, from any dependency group. `bun run --filter` instead follows the `dependencies`, `devDependencies` and `optionalDependencies` entries of the workspace `package.json` files themselves (the same links Bun uses to order the scripts), so it needs no `bun.lock` and works before the first install. `bun install --filter 'web...'` works on a fresh clone, but `bun add`, `bun remove` and `bun update ` with a relation need an existing `bun.lock` and ask you to run `bun install` first when it is missing. ```bash terminal icon="terminal" bun install --filter 'web...' @@ -52,7 +52,7 @@ bun outdated --filter '{./packages/apps}' By default, `bun install` installs dependencies for every package in the monorepo. To install dependencies for specific packages, use `--filter`. -A set of patterns selects everything matched by a positive pattern, minus everything matched by a `!` pattern, so `bun install --filter api --filter '!web'` installs only `api`. If nothing matches, `bun install` installs nothing and exits 0. +A set of patterns selects everything matched by a positive pattern, minus everything matched by a `!` pattern, so `bun install --filter api --filter '!web'` installs only `api`. If nothing matches, `bun install` installs nothing and exits 0. A positive pattern that matches nothing prints `warn: No workspace packages matched the filter ""` (for `bun install`, `bun outdated` and `bun update` as well as `bun add`/`bun remove`); `!` patterns never warn. Given a monorepo with workspaces `pkg-a`, `pkg-b`, and `pkg-c` under `./packages`: @@ -110,6 +110,24 @@ Both scripts run in parallel, and a terminal UI shows their respective outputs: ![Terminal Output](https://github.com/oven-sh/bun/assets/48869301/2a103e42-9921-4c33-948f-a1ad6e6bac71) +The other pattern forms work the same way: + +```bash terminal icon="terminal" +# build web and everything it depends on, in dependency order +bun --filter 'web...' build + +# test the packages that depend on ui +bun --filter '...^ui' test + +# run dev in every package under packages/apps +bun --filter '{./packages/apps}' dev + +# lint every package except docs +bun --filter '*' --filter '!docs' lint +``` + +A `!` pattern removes packages matched by the other patterns, and a `package.json` without a `name` is only selected by a `./path` pattern. + ### Running scripts in workspaces Filters respect your [workspace configuration](/pm/workspaces): if your `package.json` specifies which packages are part of the workspace, diff --git a/docs/runtime/index.mdx b/docs/runtime/index.mdx index 36924689a424..5333c55d7e2d 100644 --- a/docs/runtime/index.mdx +++ b/docs/runtime/index.mdx @@ -144,7 +144,7 @@ bun run --bun vite In a monorepo, the `--filter` argument runs a script in many packages at once. -`bun run --filter