refactor: Add authoritative package knowledge - #13442
Merged
Merged
Conversation
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
anthonyshew
changed the title
This was referenced Jul 22, 2026
anthonyshew
changed the title
anthonyshew
force-pushed
the
shew/package-scope-knowledge
branch
from
July 23, 2026 15:11
8363552 to
ce91983
Compare
anthonyshew
force-pushed
the
shew/package-scope-knowledge
branch
from
July 23, 2026 16:54
ce91983 to
d284b06
Compare
anthonyshew
changed the title
This was referenced Jul 23, 2026
tknickman
approved these changes
Jul 23, 2026
tknickman
approved these changes
Jul 23, 2026
anthonyshew
added a commit
that referenced
this pull request
Jul 23, 2026
### Description Part 1 of the package/scope knowledge migration stack. - Characterize canonical package identities, directories, native definition paths, root scope behavior, unnamed workspaces, duplicate diagnostics, and single-package mode. - Assert workspace discovery parity across npm, pnpm, Yarn, Berry, Bun, Nub, and Aube. - Preserve malformed-manifest source paths in observable diagnostics. - Make no production behavior changes. Current producers are JavaScript package discovery, `DiscoveryResponse`, `WorkspaceData`, `JavaScriptToolchain::discover_packages`, and package graph `BuildState`. Package/scope consumers include graph assembly, filtering, configuration loading, boundaries, devtools, query, and microfrontends. Phase 1 deletion checkpoint: package identity/path consumers no longer read `PackageJson`; native definition paths are not universally named `package_json_path`; repository root, root execution scope, aggregate scopes, and graph sentinel are distinct; the old package/scope producer and temporary projections are deleted. Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13448 -> #13443 ### Testing Instructions - `cargo test -p turborepo-repository` - `cargo test -p turbo --test invalid_package_json_test` - `cargo test -p turbo --test unnamed_packages_test --test single_package_graph_test` - `pnpm run format`
anthonyshew
added a commit
that referenced
this pull request
Jul 23, 2026
### Why this PR exists
`PackageGraphBuilder` currently owns two different jobs at the same
time:
1. **Observe native repository state**: discover workspaces, parse
manifests, resolve package-manager behavior, and prepare package
metadata.
2. **Maintain graph structure**: create the root/sentinel nodes, insert
package nodes, reject duplicate names, and keep the package map,
petgraph nodes, and lookup indexes synchronized.
That coupling is manageable while `PackageJson` is the source of package
identity and paths. It becomes a blocker for the next PR, where
parser-neutral repository knowledge must become authoritative. If we
introduce that knowledge directly into the existing builder, the same
change would simultaneously alter discovery, package identity, path
ownership, duplicate handling, and graph mutation. A regression would be
difficult to localize, and reviewers could not tell whether changed
graph output came from the new observation model or from rewritten graph
assembly.
We need a narrow boundary where already-observed package facts can be
handed to graph-owned code:
```text
Before
native discovery + manifest parsing + package facts + graph invariants
all inside BuildState
After this PR
native discovery + manifest parsing
|
v
package facts / compatibility payload
|
v
PackageGraphAssembler
- root and sentinel setup
- package node insertion
- duplicate validation
- lookup/index maintenance
```
The next PR can therefore replace the source of package facts without
also redesigning how graph nodes are assembled. This is the strangler
seam: new repository knowledge will feed the assembler, while graph
topology remains core-owned and behaviorally unchanged.
### Why these fields move together
The package map, petgraph graph, root indexes, and node lookup are one
invariant set. Adding a package must update all of them together;
otherwise graph traversal and package lookup can disagree.
`PackageGraphAssembler` gives that invariant one owner instead of
leaving the fields directly mutable throughout `BuildState`.
It also makes the distinction between these two existing concepts
explicit in one place:
- `PackageNode::Root`: the graph implementation sentinel.
- `PackageNode::Workspace(PackageName::Root)`: the root execution scope.
That distinction matters once repository roots, package roots, and
aggregate scopes become separate knowledge concepts.
### What changes
This PR extracts a private, graph-owned assembler responsible for:
- Root and root-workspace node initialization.
- Structural root edge creation.
- Package node insertion.
- Duplicate package-name diagnostics.
- Capacity reservation and lookup-table maintenance.
- Returning the assembled graph state when construction finishes.
`BuildState` remains the caller, so construction order and graph output
remain unchanged.
### What deliberately does not change
This is a mechanical prerequisite, not the package-knowledge cutover. It
does not change:
- Workspace discovery.
- Manifest parsing or `PackageJson` compatibility data.
- Package identity or path semantics.
- Dependency splitting or relationship edges.
- Package-manager or lockfile behavior.
- External dependency closure calculation.
- Filtering, task graph, hashing, caching, watch, or prune behavior.
The temporary `PackageInfo` payload is still present here. The following
PR changes package identity/path authority to parser-neutral repository
knowledge; doing that here would defeat the purpose of separating the
two independently reviewable moves.
### Review guidance
The expected result is structural equivalence. Useful review questions
are:
- Are root/sentinel nodes and their structural edge initialized exactly
once?
- Does every package insertion update the package map, graph, and node
lookup together?
- Are duplicate diagnostics preserved?
- Did relationship and lockfile logic remain in its previous phase?
- Does `finish()` transfer the same graph-owned state into
`PackageGraph`?
Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13448 -> #13443
### Testing Instructions
- `cargo test -p turborepo-repository`
- `cargo lint`
- `pnpm run format`
anthonyshew
force-pushed
the
shew/package-scope-knowledge
branch
from
July 23, 2026 19:41
d284b06 to
a925dae
Compare
anthonyshew
added a commit
that referenced
this pull request
Jul 23, 2026
### Why this PR exists The repository model now has authoritative packages and scopes, but it still has no universal way to say which native workspace authority produced them. Without that fact, core cannot enforce the repository constraint that one native kind contributes at most one physical workspace root. Putting this behind another behavioral `Toolchain` callback would retain the architecture we are replacing. This PR extends the existing discovery result into a data envelope: every toolchain contributes packages/scopes and parser-neutral workspace-root kind/path observations together. Core binds provenance to the registry entry that produced the envelope and owns validation. ### What changes - Add open `WorkspaceRoot` kind/path observations and a `DiscoveredPackages` envelope. - Change the existing `Toolchain::discover_packages` result rather than adding another trait method. - Make JavaScript report its repository root using the authoritative package-manager command family. - Make Cargo report the workspace root used by its current metadata invocation. - Bind root provenance in core, preventing adapters from spoofing another `ToolchainId`. - Require every toolchain that contributes packages to own an accepted root. - Reject different physical roots with the same native kind. - Deduplicate aliases of one physical root while retaining all producing owners. - Reject root paths outside the repository, including unresolved paths beneath symlink escapes. - Reject JavaScript discovery responses whose manager family disagrees with the authoritative manager; pnpm versions and Yarn/Berry remain canonical families. - Replace release-only duplicate registry assertions with typed errors. ### What does not change This layer reports only each producer’s base active root. It does not inspect discovered JavaScript package manifests for nested workspace authorities; #13448 adds that native behavior. It also does not validate lockfile/resolution domains or change watch invalidation. Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13448 -> #13443 ### Testing Instructions - `cargo test -p turborepo-repository` - `cargo test -p turborepo-engine -p turborepo-lib` - `cargo test -p turbo --test cargo_workspace_test --test single_package_graph_test` - `cargo lint` - `pnpm run format`
anthonyshew
added a commit
that referenced
this pull request
Jul 23, 2026
### Description Part 4 of the package/scope knowledge migration stack. Move the first consumer family, `turborepo-scope`, to knowledge-backed package enumeration and directory ownership: - Package inference uses real package names and directories, preserving longest-prefix nested-package selection. - Default, exclude-only, parent-directory, and changed-package candidate sets use authoritative package/scope knowledge. - Cargo aggregate scopes remain selectable by name while remaining distinct from real packages and the root scope. - Test aggregate toolchains report their native workspace root through the shared discovery envelope. - Reserved root identity is not reconstructed from user-facing package names. Dependency/dependent traversal and affectedness propagation remain on the existing package graph relationships. Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13443 ### Testing Instructions - `cargo test -p turborepo-scope` - `cargo test -p turbo --test filter_run_test` - `cargo test -p turbo --test cargo_workspace_test` - `cargo lint` - `pnpm run format`
anthonyshew
pushed a commit
that referenced
this pull request
Jul 25, 2026
## Release v2.10.7-canary.1 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/30132657821) ### Changes - release(turborepo): 2.10.6-canary.5 (#13424) (`ceee06b`) - chore: Upgrade brace-expansion (#13429) (`d45bf2a`) - chore: Update tar to 7.5.18 (#13428) (`d0dbd5a`) - chore: Upgrade js-yaml (#13427) (`c8e36ad`) - fix: Match JIT inputs for affected tasks (#13426) (`1b95fcb`) - fix: Preserve project Yarn package extensions (#13425) (`f08f35c`) - release(turborepo): 2.10.6 (#13431) (`56bf418`) - chore: Refine bug report template help section (#13433) (`48d609c`) - fix: Verify version after attempting codemod upgrade (#13446) (`7ce7c1c`) - test: Characterize JavaScript package discovery (#13440) (`51a8a03`) - refactor: Separate package graph node assembly (#13441) (`e9b0a29`) - fix: Upgrade @vercel/connect (#13450) (`92db24c`) - refactor: Add authoritative package knowledge (#13442) (`0729f3b`) - feat: Select runnable task entrypoints (#13452) (`0a9cb5b`) - refactor: Add workspace root contribution contract (#13447) (`a66bf45`) - refactor: Use package knowledge for scope selection (#13443) (`0e00d89`) - fix: Prevent mobile homepage overflow (#13453) (`4778467`) - docs: Document service readiness probes (#13472) (`f925b44`) - fix: Preserve pnpm root dependencies during prune (#13476) (`0746adc`) - fix: Preserve pnpm aliased dependencies during prune (#13477) (`96dc00c`) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
anthonyshew
added a commit
that referenced
this pull request
Jul 26, 2026
## Release v2.10.7 > [!CAUTION] > Versioned docs aliasing failed during the original release run. [View logs](https://github.com/vercel/turborepo/actions/runs/30162780405). The npm packages are already published. This PR restores the failed release PR and advances `version.txt` to `2.10.8-canary.0`. ### Changes - release(turborepo): 2.10.6 (#13431) (`56bf418`) - chore: Refine bug report template help section (#13433) (`48d609c`) - fix: Verify version after attempting codemod upgrade (#13446) (`7ce7c1c`) - test: Characterize JavaScript package discovery (#13440) (`51a8a03`) - refactor: Separate package graph node assembly (#13441) (`e9b0a29`) - fix: Upgrade @vercel/connect (#13450) (`92db24c`) - refactor: Add authoritative package knowledge (#13442) (`0729f3b`) - feat: Select runnable task entrypoints (#13452) (`0a9cb5b`) - refactor: Add workspace root contribution contract (#13447) (`a66bf45`) - refactor: Use package knowledge for scope selection (#13443) (`0e00d89`) - fix: Prevent mobile homepage overflow (#13453) (`4778467`) - docs: Document service readiness probes (#13472) (`f925b44`) - fix: Preserve pnpm root dependencies during prune (#13476) (`0746adc`) - fix: Preserve pnpm aliased dependencies during prune (#13477) (`96dc00c`) - release(turborepo): 2.10.7-canary.1 (#13478) (`2a02c34`) - test: Add Terminal UI sanity tests (#13475) (`07ebd1d`) - chore: Upgrade Next.js (#13482) (`0a329d6`) - chore: Upgrade brace-expansion (#13483) (`549fc99`) - fix: Support multiple macOS file watch roots (#13484) (`2f8a962`) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this PR exists
Package identity and path ownership currently remain entangled with
PackageJsoncompatibility data. That makes JavaScript manifest structure an accidental core package model and prevents Cargo or future ecosystems from contributing equivalent package facts without synthesizing JavaScript descriptors.This PR introduces one immutable, parser-neutral package/scope generation and makes it authoritative for graph assembly. The package graph retains the exact generation used to create its nodes, so consumers cannot observe package facts from a different repository snapshot.
What changes
RepositoryKnowledgefor repository root, root JavaScript execution scope, package and aggregate identities, directories, native definition paths, and openToolchainIdprovenance.PackageInfoonly as temporary relationship/task compatibility data. Its package identity, path, scope kind, and provenance are projected from knowledge.PackageGraphqueries for later consumer migrations.What does not change
This PR does not introduce workspace-root uniqueness, nested workspace detection, relationship projections, external resolution, task knowledge, watch generations, or prune behavior. Those concerns move in their own stack layers or later phases.
Stack: #13440 -> #13441 -> #13442 -> #13447 -> #13448 -> #13443
Testing Instructions
cargo test -p turborepo-repositorycargo test -p turborepo-enginecargo test -p turborepo-libcargo test -p turbo --test cargo_workspace_testcargo test -p turbo --test final_hash_contractcargo lintpnpm run format