-
Notifications
You must be signed in to change notification settings - Fork 3k
perf(dev): add pnpm worktree bootstrap foundation #10449
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
yiliang114
wants to merge
28
commits into
main
Choose a base branch
from
codex/pnpm-worktree-bootstrap
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+22,697
−2
Open
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
d4ecf3d
perf(dev): add pnpm worktree bootstrap foundation
yiliang114 74bad44
fix(dev): harden worktree bootstrap process handling
yiliang114 fcb27a4
perf(dev): prefer cache-only worktree installs
yiliang114 605d205
fix(dev): preserve cached install cancellation
yiliang114 f7161c7
fix(build): harden pnpm worktree bootstrap
yiliang114 f2ab686
merge: sync main and narrow pnpm bootstrap stage
yiliang114 90cbf2d
chore: record merge attribution
yiliang114 e7f7a87
fix(dev): keep worktree bootstrap clean
yiliang114 a51bcac
ci: track pnpm smoke workflow size
yiliang114 b2ec83d
Merge branch 'main' into codex/pnpm-worktree-bootstrap
yiliang114 5535f7b
fix(ci): drop stale importers from the new pnpm lockfile
yiliang114 b01d415
Merge remote-tracking branch 'origin/main' into codex/pnpm-worktree-b…
yiliang114 9fe920a
Merge branch 'main' into codex/pnpm-worktree-bootstrap
yiliang114 9300088
fix(ci): refresh pnpm lockfile after merging main
yiliang114 b2dbbe7
Merge remote-tracking branch 'origin/codex/pnpm-worktree-bootstrap' i…
yiliang114 0cb3aec
fix(scripts): resolve the path variable case-insensitively on Windows
yiliang114 fd43387
test(scripts): pin the worktree bootstrap guards against mutation
yiliang114 eaf0da0
Merge branch 'main' into codex/pnpm-worktree-bootstrap
yiliang114 5ef1884
Merge branch 'main' into codex/pnpm-worktree-bootstrap
yiliang114 0b1de75
chore(deps): sync pnpm-lock.yaml with qwen-live ACP backend deps
yiliang114 def9ec3
Merge remote-tracking branch 'origin/codex/pnpm-worktree-bootstrap' i…
yiliang114 eb6d0c2
fix(dev): stop patch-package from corrupting the pnpm store
154570d
Merge branch 'main' into codex/pnpm-worktree-bootstrap
yiliang114 4191008
Merge remote-tracking branch 'origin/main' into prmerge-10449
yiliang114 b106462
fix(ci): refresh pnpm-lock.yaml for qwen-live prompts deps
yiliang114 35171c8
Merge branch 'codex/pnpm-worktree-bootstrap' of https://github.com/Qw…
yiliang114 6ed4ea3
Merge origin/main into codex/pnpm-worktree-bootstrap
yiliang114 d401674
Merge upstream codex/pnpm-worktree-bootstrap updates
yiliang114 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| name: 'pnpm Worktree Smoke' | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: | ||
| - 'main' | ||
| - 'release/**' | ||
| paths: | ||
| - '.github/workflows/pnpm-worktree-smoke.yml' | ||
| - '.npmrc' | ||
| - '.pnpmfile.mjs' | ||
| - 'package.json' | ||
| - 'packages/*/package.json' | ||
| - '!packages/desktop-shell/package.json' | ||
| - '!packages/live-host/package.json' | ||
| - 'packages/channels/*/package.json' | ||
| - 'integrations/*/package.json' | ||
| - 'patches/**' | ||
| - 'packages/audio-capture/install.js' | ||
| - 'packages/core/scripts/postinstall.js' | ||
| - 'packages/vscode-ide-companion/scripts/generate-notices.js' | ||
| - 'pnpm-lock.yaml' | ||
| - 'pnpm-workspace.yaml' | ||
| - 'scripts/generate-git-commit-info.js' | ||
| - 'scripts/prepare.js' | ||
| - 'scripts/pnpm-package.js' | ||
| - 'scripts/setup-worktree.js' | ||
| push: | ||
| branches: | ||
| - 'main' | ||
| paths: | ||
| - '.github/workflows/pnpm-worktree-smoke.yml' | ||
| - '.npmrc' | ||
| - '.pnpmfile.mjs' | ||
| - 'package.json' | ||
| - 'packages/*/package.json' | ||
| - '!packages/desktop-shell/package.json' | ||
| - '!packages/live-host/package.json' | ||
| - 'packages/channels/*/package.json' | ||
| - 'integrations/*/package.json' | ||
| - 'patches/**' | ||
| - 'packages/audio-capture/install.js' | ||
| - 'packages/core/scripts/postinstall.js' | ||
| - 'packages/vscode-ide-companion/scripts/generate-notices.js' | ||
| - 'pnpm-lock.yaml' | ||
| - 'pnpm-workspace.yaml' | ||
| - 'scripts/generate-git-commit-info.js' | ||
| - 'scripts/prepare.js' | ||
| - 'scripts/pnpm-package.js' | ||
| - 'scripts/setup-worktree.js' | ||
| workflow_dispatch: | ||
|
|
||
| concurrency: | ||
| group: '${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}' | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: 'read' | ||
|
|
||
| jobs: | ||
| install: | ||
| name: 'Install (${{ matrix.os }})' | ||
| runs-on: '${{ matrix.os }}' | ||
| timeout-minutes: 20 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| os: | ||
| - 'ubuntu-latest' | ||
| - 'macos-latest' | ||
| - 'windows-latest' | ||
| steps: | ||
| - name: 'Checkout' | ||
| uses: 'actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10' # v6.0.3 | ||
|
|
||
| - name: 'Set up Node.js 22.x' | ||
| uses: 'actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e' # v6.4.0 | ||
| with: | ||
| node-version: '22.x' | ||
|
|
||
| - name: 'Install frozen pnpm worktree dependencies' | ||
| run: 'node scripts/setup-worktree.js' | ||
|
|
||
| - name: 'Ensure bootstrap keeps the worktree clean' | ||
| run: 'git diff --exit-code' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| /** | ||
| * @license | ||
| * Copyright 2026 Qwen Team | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| const workspacePackages = new Set([ | ||
| '@qwen-code/acp-bridge', | ||
| '@qwen-code/audio-capture', | ||
| '@qwen-code/channel-base', | ||
| '@qwen-code/channel-dingtalk', | ||
| '@qwen-code/channel-dws', | ||
| '@qwen-code/channel-feishu', | ||
| '@qwen-code/channel-github', | ||
| '@qwen-code/channel-gitlab', | ||
| '@qwen-code/channel-qqbot', | ||
| '@qwen-code/channel-telegram', | ||
| '@qwen-code/channel-wecom', | ||
| '@qwen-code/channel-weixin', | ||
| '@qwen-code/qwen-code-core', | ||
| '@qwen-code/sdk', | ||
| '@qwen-code/web-templates', | ||
| '@qwen-code/webui', | ||
| ]); | ||
|
|
||
| const dependencyFields = [ | ||
| 'dependencies', | ||
| 'devDependencies', | ||
| 'optionalDependencies', | ||
| ]; | ||
|
|
||
| export const hooks = { | ||
| readPackage(packageJson) { | ||
| for (const field of dependencyFields) { | ||
| const dependencies = packageJson[field]; | ||
| if (!dependencies) continue; | ||
|
|
||
| for (const name of Object.keys(dependencies)) { | ||
| if (workspacePackages.has(name)) { | ||
| dependencies[name] = 'workspace:*'; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return packageJson; | ||
| }, | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -83,6 +83,7 @@ rules: | |
| check-keys: true | ||
|
|
||
| ignore: | ||
| - 'pnpm-lock.yaml' | ||
| - 'thirdparty/' | ||
| - 'third_party/' | ||
| - 'vendor/' | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| # pnpm worktree bootstrap | ||
|
|
||
| ## Problem | ||
|
|
||
| Every npm-backed Git worktree materializes another full dependency tree. The | ||
| root `prepare` lifecycle also builds and bundles the repository unless the | ||
| caller knows to set `QWEN_SKIP_PREPARE`, so a worktree pays for generated | ||
| artifacts before source-based development needs them. | ||
|
|
||
| On the same commit and APFS volume, a warm-cache npm install added about 1.44 | ||
| GiB while a warm-store pnpm install added about 99 MiB. The dependency-only | ||
| installs took 27 and 22 seconds respectively; a separate full build took about | ||
| 129 seconds. | ||
|
|
||
| ## Design | ||
|
|
||
| The repository declares pnpm as its package manager and mirrors the existing | ||
| npm workspace boundaries in `pnpm-workspace.yaml`. A hoisted linker is used | ||
| for the initial migration because current build and packaging scripts still | ||
| contain assumptions inherited from npm's layout. | ||
|
|
||
| The committed pnpm lockfile is the source of dependency resolution for this | ||
| bootstrap only. pnpm-specific overrides preserve the dependency versions used | ||
| by the current npm installation without changing npm's manifest or lockfile. | ||
| Dependency install scripts are denied unless they are explicitly listed in | ||
| `allowBuilds`; the allowlist contains only packages whose scripts run in the | ||
| current npm installation. | ||
|
|
||
| During the dual-lock transition, `.pnpmfile.mjs` rewrites known internal | ||
| dependencies to pnpm's `workspace:*` protocol in memory. This covers both the | ||
| existing `file:` dependencies and the exact channel dependency versions that | ||
| the release script updates. As a result, release version bumps do not stale the | ||
| pnpm lockfile, while the checked-in manifests and npm lockfile remain untouched. | ||
| The compatibility hook can be removed when the manifests adopt `workspace:` | ||
| during the final cutover. | ||
|
|
||
| New worktrees use `node scripts/setup-worktree.js`. The script prefers Corepack | ||
| so an existing pnpm cache can stay fully offline, and falls back to npm's | ||
| bundled `npx` on Node versions that no longer include Corepack. Both paths use | ||
| the exact pnpm package declared by `packageManager`. The script freezes the | ||
| lockfile and first attempts an offline install from the shared local store. It | ||
| retries with registry access only when that cache-only attempt is incomplete. | ||
| This avoids waiting for pnpm to prefetch optional binaries for other platforms | ||
| on the common warm-store path. The script sets `QWEN_SKIP_PREPARE=1` plus a | ||
| bootstrap-private notice-generation guard, keeping dependency install scripts | ||
| enabled while skipping repository build, bundle, Husky setup, and npm-layout | ||
| notice generation. Script execution does not | ||
| implicitly install stale dependencies; the bootstrap command is the explicit | ||
| installation boundary. Building from this pnpm layout is deferred to Stage 2. | ||
|
|
||
| ## Migration boundary | ||
|
|
||
| Stage 1 applies only to dependency installation in additional Git worktrees. | ||
| It does not change repository build commands, CI build orchestration, release | ||
| versioning, packaging, or publishing. `package-lock.json` remains authoritative | ||
| for every existing npm path. A path-filtered workflow exercises the real frozen | ||
| bootstrap on Linux, macOS, and Windows whenever a pnpm installation input | ||
| changes. The generated pnpm lockfile is excluded from the repository's | ||
| human-authored YAML style rules. | ||
|
|
||
| Stage 2 can separately make the pnpm layout a supported build and development | ||
| path, then migrate CI installation and caches. Stage 3 can address release | ||
| installation and build orchestration. npm remains the product boundary for | ||
| package creation, registry publication, and clean artifact installation until | ||
| the scripts that intentionally read `package-lock.json` are migrated. | ||
|
|
||
| ## Verification | ||
|
|
||
| The Stage 1 pnpm path must complete a frozen install without modifying tracked | ||
| files. Script tests cover the bootstrap command, its prepare-skip environment, | ||
| the version-independent workspace rewrite, and process failure behavior. The | ||
| cross-platform workflow provides the real install gate. Building from the pnpm | ||
| layout is intentionally deferred to Stage 2; existing npm build and release | ||
| validation remain unchanged. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.