fix(grammars): load vendored tree-sitter grammars from vendor/ by absolute path (#2111)

[magyargergo]

Summary

Fixes the recurring Windows npm error code EPERM … syscall symlink (errno -4048) reported in #2111 when adding the MCP server to Antigravity. This is not the #2101/#2110 module-load crash — it is an install-time arborist failure during the _npx reify that the MCP client triggers on every npx gitnexus launch (note the …\_npx\<hash>\… path in the report).

Root cause

The postinstall materialize step copied each vendored grammar (vendor/tree-sitter-{c,dart,proto,swift,kotlin}) into node_modules/gitnexus/node_modules/tree-sitter-* as a real package, so runtime require('tree-sitter-dart') would resolve.

Those copies are in no dependency graph, so every subsequent npm/npx arborist reify treats them as extraneous and prunes/relocates them:

• On Windows, the relocation goes through @npmcli/move-file's symlink path → EPERM: operation not permitted, symlink (symlinks require Developer Mode/admin). Install/setup aborts.
• On every OS, the 2nd run silently deletes the grammars (Dart/Swift/Kotlin/Proto/C parsing breaks).

This is the same class as #1728, which the materialize step itself claimed to have fixed (it only moved the symlink trigger from file: deps to extraneous-node cleanup).

Reproduced

npx -p gitnexus@1.6.8-rc.6 gitnexus --version   # run 1: materializes node_modules/.../tree-sitter-*
npx -p gitnexus@1.6.8-rc.6 gitnexus --version   # run 2 (reuses _npx cache): "removed 5 packages" → EPERM symlink on Windows

Fix

Adopt the ecosystem-canonical prebuildify + node-gyp-build pattern: never copy grammars into node_modules. Load each by absolute path from vendor/<name> via a new requireVendoredGrammar helper — the grammar's own bindings/node runs node-gyp-build(<dir>) and loads the committed vendor/<name>/prebuilds/<platform>-<arch>/… directly (all 5 grammars ship all 6 tuples). vendor/ is inside the package but not a node_modules subtree, so arborist never sees the grammars and the reify is idempotent — no EPERM, no silent deletion.

• New src/core/tree-sitter/vendored-grammars.ts (requireVendoredGrammar / vendoredGrammarDir / VENDORED_GRAMMAR_PACKAGES; VENDOR_ROOT stable in dev and dist/).
• Route every consumer through it: parser-loader, parse-worker, gRPC proto, include-extractor (C), http-patterns kotlin, and the CLI optional-grammars probe.
• postinstall drops the materialize step; build-tree-sitter-grammars.cjs now source-builds in place under vendor/ (gitignored) only when a platform lacks a prebuild; deleted scripts/materialize-vendor-grammars.cjs.
• Tests + the grammar-introspection helper load grammars from vendor/ too (single source of truth).

Testing

• New test/unit/vendored-grammars.test.ts — regression guard: fails if anyone reintroduces a bare require('tree-sitter-<vendored>'), and asserts every grammar resolves under vendor/ (never node_modules) and loads.
• Validated the post-fix consumer state: removed node_modules/tree-sitter-*, then all 5 grammars load from vendor/ by absolute path (cold), and confirmed the published rc tarball ships all 6 prebuilds + node-types.json per grammar.
• Green: parser-loader ABI smoke, build-probe, grammar introspection/literal-validation, dart/kotlin/swift resolvers, gRPC, include-extractor, worker-pool, CLI, and all 8 converted unit suites (~1,200+ tests). Pre-commit eslint + prettier + typecheck passed.

Refs #2111, #1728.

🤖 Generated with Claude Code

[vercel]

@magyargergo is attempting to deploy a commit to the NexusCore Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions]

CI Report

✅ All checks passed

Pipeline Status

Stage	Status	Details
✅ Typecheck	success	tsc --noEmit
✅ Tests	success	unit tests, 3 platforms
✅ E2E	success	gitnexus-web changes only

Test Results

Tests	Passed	Failed	Skipped	Duration
11041	11025	0	16	579s

✅ All 11025 tests passed

16 test(s) skipped — expand for details

• COBOL pipeline benchmark > scales with file count
• C++ ADL emit benchmark > emit phase scales sub-quadratically with co-scaled files and sites
• C++ pipeline benchmark > scales with file count
• C# pipeline benchmark > scales with file count — namespaces spread across the solution
• C# pipeline benchmark > scales with file count — all types in one (global) namespace bucket
• C# pipeline benchmark > scales with file count — all types in one (named) namespace bucket
• Go pipeline benchmark > scales with file count (workers enabled)
• Go pipeline benchmark — worker pool (issue Worker idle timeout kills long Go scope extraction and surfaces as Napi::Error during analyze #1848) > does not quarantine the large generated Go file on sub-batch idle timeout
• Go structural interface detection benchmark > scales linearly with interface × struct count
• Go structural interface detection split-phase benchmark > separates index-build and detection time
• PHP pipeline benchmark > scales with file count (workers enabled)
• Ruby pipeline benchmark > scales with file count (workers enabled)
• Rust pipeline benchmark > scales with file count (workers enabled)
• Vue pipeline benchmark > scales with component count
• run.cjs direct-exec entrypoint (fix(cli): steer docs, skills, and hooks through a CLI-neutral project-local runner (#1939) #1945) > resolves a .cmd shim via the Windows shell branch, passing args and exit code
• buildTypeEnv > known limitations (documented skip tests) > Ruby block parameter: users.each { |user| } — closure param inference, different feature

Code Coverage

Tests

Metric	Coverage	Covered	Base	Delta	Status
Statements	75.35%	36134/47950	N/A%	—	🟢 ███████████████░░░░░
Branches	63.15%	22389/35453	N/A%	—	🟢 ████████████░░░░░░░░
Functions	80.97%	3887/4800	N/A%	—	🟢 ████████████████░░░░
Lines	79.13%	32660/41273	N/A%	—	🟢 ███████████████░░░░░

---

_{📋 View full run · Generated by CI}

[magyargergo]

Tri-review digest — PR #2144 (load vendored tree-sitter grammars from vendor/)

Methods & independence: Compound-Engineering personas (correctness, adversarial, maintainability) + GitNexus risk/test lanes — all Claude. Codex was unavailable (usage limit), so this is a two-method, both-Claude review — cross-persona agreement here is "consistent across personas," not three independent engines. Every finding was coordinator-verified and fixed in d1cb5e0c before posting.

Validated (credit where due)

• Correctness lane: clean tsc --noEmit, live-loaded all 5 vendored grammars, 933 unit tests green. Refuted with evidence: VENDOR_ROOT (src/core/tree-sitter/vendored-grammars.ts:16) resolves correctly in dev/dist/worker (worker imports the helper relatively; tsc doesn't bundle); every consumer converted (grep: only comment mentions remain); no dangling/unused createRequire; unknown-vs-any types flow clean; the optional-grammar missing-vs-broken probe is preserved (node-gyp-build's "No native build was found" matches the existing regex in optional-grammars.ts).
• CI: tree-sitter ABI green on windows/ubuntu/macos; packaged install smoke (ubuntu) green; quality lint/typecheck/format green. (Vercel fail = deploy-auth, not code.)
• Structural soundness: the fix removes the node_modules participation that caused the extraneous-prune/EPERM bug; loading by absolute path twice returns the same require-cache object (Node caches by realpath); node-gyp-build resolves from vendor/ (direct dep — package.json:76 — survives --omit=optional/pnpm-strict).

Findings — all addressed in d1cb5e0c

1. [P2 · adversarial · reproduced] Publish hygiene. Retargeting the source-build into vendor/<name>/build/ means a stray build dir would ship (files:["vendor"] overrides .gitignore/.npmignore) and shadow the committed prebuild — node-gyp-build (node-gyp-build.js:33-41) resolves build/Release before prebuilds/. → Added a prepack guard in scripts/assert-publish-grammar-coverage.cjs (findStrayBuildArtifacts) that fails npm pack if any vendor/*/build exists.
2. [P2 · adversarial + maintainability · reproduced] Regression-guard holes. The new guard's regex missed dynamic import(), side-effect import 'x', /subpath, and backtick loads, and only scanned src/. → Broadened the regex, now scans test/ (excl. fixtures), dropped the //-substring false-negative, and added a self-test asserting every load form is caught (prose/tree-sitter-cpp ignored) — test/unit/vendored-grammars.test.ts.
3. [P2/P3 · maintainability] Drift guard. requireVendoredGrammar accepted any string (three grammar lists could silently drift). → It now throws on a non-vendored name — src/core/tree-sitter/vendored-grammars.ts.
4. [P3 · maintainability] Stale comments. kotlin/query.ts:3 called the grammar an "optionalDependency" (now vendored); proto.ts:26 _require. → Corrected.

Refuted / not issues

node-gyp-build resolution under --omit=optional/pnpm-strict (direct dep); double-load ABI (same cache object); tarball completeness (verified via npm pack of 1.6.8-rc.6 — all 6 prebuilds + bindings/node + src/node-types.json ship per grammar); anything else recreating node_modules/tree-sitter-* (only the now-deleted materialize did).

Coverage limit

No automated test reproduces the original Windows-only 2nd-npx-reify EPERM (install-level). The packaged install smoke CI job and the no-bare-require guard are the proxies.

---

Automated multi-tool digest — 2 Claude methods; Codex unavailable (not three independent engines). Verify before acting. Note: this is a self-review (PR author == reviewing identity); findings were applied directly rather than left as change-requests.