refactor: Separate package graph node assembly - #13441
Merged
Merged
Conversation
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
This was referenced Jul 22, 2026
anthonyshew
force-pushed
the
shew/package-node-assembly
branch
from
July 23, 2026 15:11
b161e71 to
bc19f26
Compare
anthonyshew
force-pushed
the
shew/package-node-assembly
branch
from
July 23, 2026 16:54
bc19f26 to
6ef0d38
Compare
This was referenced Jul 23, 2026
tknickman
approved these changes
Jul 23, 2026
anthonyshew
added a commit
that referenced
this pull request
Jul 23, 2026
### Description Part 1 of the package/scope knowledge migration stack. - Characterize canonical package identities, directories, native definition paths, root scope behavior, unnamed workspaces, duplicate diagnostics, and single-package mode. - Assert workspace discovery parity across npm, pnpm, Yarn, Berry, Bun, Nub, and Aube. - Preserve malformed-manifest source paths in observable diagnostics. - Make no production behavior changes. Current producers are JavaScript package discovery, `DiscoveryResponse`, `WorkspaceData`, `JavaScriptToolchain::discover_packages`, and package graph `BuildState`. Package/scope consumers include graph assembly, filtering, configuration loading, boundaries, devtools, query, and microfrontends. Phase 1 deletion checkpoint: package identity/path consumers no longer read `PackageJson`; native definition paths are not universally named `package_json_path`; repository root, root execution scope, aggregate scopes, and graph sentinel are distinct; the old package/scope producer and temporary projections are deleted. Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13448 -> #13443 ### Testing Instructions - `cargo test -p turborepo-repository` - `cargo test -p turbo --test invalid_package_json_test` - `cargo test -p turbo --test unnamed_packages_test --test single_package_graph_test` - `pnpm run format`
anthonyshew
added a commit
that referenced
this pull request
Jul 23, 2026
### Why this PR exists Package identity and path ownership currently remain entangled with `PackageJson` compatibility data. That makes JavaScript manifest structure an accidental core package model and prevents Cargo or future ecosystems from contributing equivalent package facts without synthesizing JavaScript descriptors. This PR introduces one immutable, parser-neutral package/scope generation and makes it authoritative for graph assembly. The package graph retains the exact generation used to create its nodes, so consumers cannot observe package facts from a different repository snapshot. ### What changes - Add crate-private `RepositoryKnowledge` for repository root, root JavaScript execution scope, package and aggregate identities, directories, native definition paths, and open `ToolchainId` provenance. - Build package/scope knowledge once from existing JavaScript discovery and the current Cargo compatibility output. - Use that knowledge as the authority for package graph node assembly. - Retain `PackageInfo` only as temporary relationship/task compatibility data. Its package identity, path, scope kind, and provenance are projected from knowledge. - Keep one identity between discovery facts and temporary descriptors. - Reject native definition paths outside the repository, including symlink escapes. - Expose narrow knowledge-backed `PackageGraph` queries for later consumer migrations. - Update architecture documentation for the package/scope cutover. ### What does not change This PR does not introduce workspace-root uniqueness, nested workspace detection, relationship projections, external resolution, task knowledge, watch generations, or prune behavior. Those concerns move in their own stack layers or later phases. Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13448 -> #13443 ### Testing Instructions - `cargo test -p turborepo-repository` - `cargo test -p turborepo-engine` - `cargo test -p turborepo-lib` - `cargo test -p turbo --test cargo_workspace_test` - `cargo test -p turbo --test final_hash_contract` - `cargo lint` - `pnpm run format`
anthonyshew
added a commit
that referenced
this pull request
Jul 23, 2026
### Why this PR exists The repository model now has authoritative packages and scopes, but it still has no universal way to say which native workspace authority produced them. Without that fact, core cannot enforce the repository constraint that one native kind contributes at most one physical workspace root. Putting this behind another behavioral `Toolchain` callback would retain the architecture we are replacing. This PR extends the existing discovery result into a data envelope: every toolchain contributes packages/scopes and parser-neutral workspace-root kind/path observations together. Core binds provenance to the registry entry that produced the envelope and owns validation. ### What changes - Add open `WorkspaceRoot` kind/path observations and a `DiscoveredPackages` envelope. - Change the existing `Toolchain::discover_packages` result rather than adding another trait method. - Make JavaScript report its repository root using the authoritative package-manager command family. - Make Cargo report the workspace root used by its current metadata invocation. - Bind root provenance in core, preventing adapters from spoofing another `ToolchainId`. - Require every toolchain that contributes packages to own an accepted root. - Reject different physical roots with the same native kind. - Deduplicate aliases of one physical root while retaining all producing owners. - Reject root paths outside the repository, including unresolved paths beneath symlink escapes. - Reject JavaScript discovery responses whose manager family disagrees with the authoritative manager; pnpm versions and Yarn/Berry remain canonical families. - Replace release-only duplicate registry assertions with typed errors. ### What does not change This layer reports only each producer’s base active root. It does not inspect discovered JavaScript package manifests for nested workspace authorities; #13448 adds that native behavior. It also does not validate lockfile/resolution domains or change watch invalidation. Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13448 -> #13443 ### Testing Instructions - `cargo test -p turborepo-repository` - `cargo test -p turborepo-engine -p turborepo-lib` - `cargo test -p turbo --test cargo_workspace_test --test single_package_graph_test` - `cargo lint` - `pnpm run format`
anthonyshew
added a commit
that referenced
this pull request
Jul 23, 2026
### Description Part 4 of the package/scope knowledge migration stack. Move the first consumer family, `turborepo-scope`, to knowledge-backed package enumeration and directory ownership: - Package inference uses real package names and directories, preserving longest-prefix nested-package selection. - Default, exclude-only, parent-directory, and changed-package candidate sets use authoritative package/scope knowledge. - Cargo aggregate scopes remain selectable by name while remaining distinct from real packages and the root scope. - Test aggregate toolchains report their native workspace root through the shared discovery envelope. - Reserved root identity is not reconstructed from user-facing package names. Dependency/dependent traversal and affectedness propagation remain on the existing package graph relationships. Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13443 ### Testing Instructions - `cargo test -p turborepo-scope` - `cargo test -p turbo --test filter_run_test` - `cargo test -p turbo --test cargo_workspace_test` - `cargo lint` - `pnpm run format`
anthonyshew
pushed a commit
that referenced
this pull request
Jul 25, 2026
## Release v2.10.7-canary.1 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/30132657821) ### Changes - release(turborepo): 2.10.6-canary.5 (#13424) (`ceee06b`) - chore: Upgrade brace-expansion (#13429) (`d45bf2a`) - chore: Update tar to 7.5.18 (#13428) (`d0dbd5a`) - chore: Upgrade js-yaml (#13427) (`c8e36ad`) - fix: Match JIT inputs for affected tasks (#13426) (`1b95fcb`) - fix: Preserve project Yarn package extensions (#13425) (`f08f35c`) - release(turborepo): 2.10.6 (#13431) (`56bf418`) - chore: Refine bug report template help section (#13433) (`48d609c`) - fix: Verify version after attempting codemod upgrade (#13446) (`7ce7c1c`) - test: Characterize JavaScript package discovery (#13440) (`51a8a03`) - refactor: Separate package graph node assembly (#13441) (`e9b0a29`) - fix: Upgrade @vercel/connect (#13450) (`92db24c`) - refactor: Add authoritative package knowledge (#13442) (`0729f3b`) - feat: Select runnable task entrypoints (#13452) (`0a9cb5b`) - refactor: Add workspace root contribution contract (#13447) (`a66bf45`) - refactor: Use package knowledge for scope selection (#13443) (`0e00d89`) - fix: Prevent mobile homepage overflow (#13453) (`4778467`) - docs: Document service readiness probes (#13472) (`f925b44`) - fix: Preserve pnpm root dependencies during prune (#13476) (`0746adc`) - fix: Preserve pnpm aliased dependencies during prune (#13477) (`96dc00c`) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
anthonyshew
added a commit
that referenced
this pull request
Jul 26, 2026
## Release v2.10.7 > [!CAUTION] > Versioned docs aliasing failed during the original release run. [View logs](https://github.com/vercel/turborepo/actions/runs/30162780405). The npm packages are already published. This PR restores the failed release PR and advances `version.txt` to `2.10.8-canary.0`. ### Changes - release(turborepo): 2.10.6 (#13431) (`56bf418`) - chore: Refine bug report template help section (#13433) (`48d609c`) - fix: Verify version after attempting codemod upgrade (#13446) (`7ce7c1c`) - test: Characterize JavaScript package discovery (#13440) (`51a8a03`) - refactor: Separate package graph node assembly (#13441) (`e9b0a29`) - fix: Upgrade @vercel/connect (#13450) (`92db24c`) - refactor: Add authoritative package knowledge (#13442) (`0729f3b`) - feat: Select runnable task entrypoints (#13452) (`0a9cb5b`) - refactor: Add workspace root contribution contract (#13447) (`a66bf45`) - refactor: Use package knowledge for scope selection (#13443) (`0e00d89`) - fix: Prevent mobile homepage overflow (#13453) (`4778467`) - docs: Document service readiness probes (#13472) (`f925b44`) - fix: Preserve pnpm root dependencies during prune (#13476) (`0746adc`) - fix: Preserve pnpm aliased dependencies during prune (#13477) (`96dc00c`) - release(turborepo): 2.10.7-canary.1 (#13478) (`2a02c34`) - test: Add Terminal UI sanity tests (#13475) (`07ebd1d`) - chore: Upgrade Next.js (#13482) (`0a329d6`) - chore: Upgrade brace-expansion (#13483) (`549fc99`) - fix: Support multiple macOS file watch roots (#13484) (`2f8a962`) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this PR exists
PackageGraphBuildercurrently owns two different jobs at the same time:That coupling is manageable while
PackageJsonis the source of package identity and paths. It becomes a blocker for the next PR, where parser-neutral repository knowledge must become authoritative. If we introduce that knowledge directly into the existing builder, the same change would simultaneously alter discovery, package identity, path ownership, duplicate handling, and graph mutation. A regression would be difficult to localize, and reviewers could not tell whether changed graph output came from the new observation model or from rewritten graph assembly.We need a narrow boundary where already-observed package facts can be handed to graph-owned code:
The next PR can therefore replace the source of package facts without also redesigning how graph nodes are assembled. This is the strangler seam: new repository knowledge will feed the assembler, while graph topology remains core-owned and behaviorally unchanged.
Why these fields move together
The package map, petgraph graph, root indexes, and node lookup are one invariant set. Adding a package must update all of them together; otherwise graph traversal and package lookup can disagree.
PackageGraphAssemblergives that invariant one owner instead of leaving the fields directly mutable throughoutBuildState.It also makes the distinction between these two existing concepts explicit in one place:
PackageNode::Root: the graph implementation sentinel.PackageNode::Workspace(PackageName::Root): the root execution scope.That distinction matters once repository roots, package roots, and aggregate scopes become separate knowledge concepts.
What changes
This PR extracts a private, graph-owned assembler responsible for:
BuildStateremains the caller, so construction order and graph output remain unchanged.What deliberately does not change
This is a mechanical prerequisite, not the package-knowledge cutover. It does not change:
PackageJsoncompatibility data.The temporary
PackageInfopayload is still present here. The following PR changes package identity/path authority to parser-neutral repository knowledge; doing that here would defeat the purpose of separating the two independently reviewable moves.Review guidance
The expected result is structural equivalence. Useful review questions are:
finish()transfer the same graph-owned state intoPackageGraph?Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13448 -> #13443
Testing Instructions
cargo test -p turborepo-repositorycargo lintpnpm run format