feat(ingestion): tree-sitter node-type/field validation gate + remove dead literals by magyargergo · Pull Request #1937 · abhigyanpatwari/GitNexus

magyargergo · 2026-05-30T20:17:45Z

Summary

Adds a CI gate that validates every tree-sitter node-type and field-name literal used across the ingestion code layer against each grammar's actual node-types.json, and removes the existence-dead literals it surfaces. This kills the systemic "dead branches keyed on node types the grammar never emits" class (the AST audit's dominant finding) and prevents new grammar drift on a bump.

Closes #1920 (enabler in the AST coverage epic #1919).

What the gate covers

test/integration/grammar-literal-validation.test.ts (+ helpers test/helpers/{grammar-introspection,literal-collectors}.ts) validates literals across all three resolution surfaces:

Legacy Call-Resolution DAG + the ungated structure phase (type-extractors/*, *-extractors/configs/*, export-detection.ts) — static AST scan by consumption site (node.type === '…', childForFieldName('…'), Set.has(node.type) members).
Registry scope-resolution captures + scope queries — via query compilation (complements query-compilation.test.ts, which only covers the legacy *_QUERIES).
Registry resolution layer (scope-resolver / type-binding / receiver-binding / interpret / arity / import-decomposer …) — the production path for migrated languages — via a TS-TypeChecker discriminator that collects a literal only when its .type receiver resolves to a tree-sitter SyntaxNode, so resolved-symbol .type kinds (e.g. 'Class') are never mistaken for grammar nodes.

Oracle: node-types.json membership (union of types + subtypes + field/child types, including anonymous tokens & supertypes) with a live new Parser.Query probe fallback for literals the static JSON under-reports. Optional/uninstalled grammars are skipped (never a build failure). Ships with an empty allowlist.

Dead literals removed (behavior-neutral)

Each removal deletes a dead branch/fallback verified absent from the installed grammar (the correct node is already handled elsewhere):

C/C++: reference_type, pointer_type, scoped_identifier, scoped_type_identifier, rvalue_reference_declarator, variadic_parameter
C#: equals_value_clause, identifier_name, simple_identifier, record_struct_declaration, record_class_declaration
Dart: generic_type, type field · PHP: nullable_type, alias_clause, return_value field · Ruby: method_call, symbol
Rust: method_call_expression, slice_type, shorthand_field_pattern · Swift: struct_declaration, internal_name · Java: comment · Go: parameter
cross-language: parameterized_type, and dead childForFieldName('pattern'|'modifiers'|'formal_parameters'|'declaration'|'default') / class_expression fallbacks

One behavior fix (scope-resolution)

PHP findEnclosingTypeDeclaration omitted anonymous_class, so a method inside new class {…} mis-bound $this to the enclosing named class. Added anonymous_class so anonymous-class methods are correctly skipped.

Verification

tsc --noEmit clean
Gate green with an empty allowlist (including the TypeChecker resolution-layer coverage)
Scope-resolution parity 26/26 on both REGISTRY_PRIMARY_*=0 and =1
Full resolver suite: no new failures
net +60 / −209 lines across the ingestion source (deletions dominate — dead-code removal)

Note: full npm test could not be run green inside the dev worktree because scripts/build.js cannot produce parse-worker.js through the symlinked node_modules (status 127); ~11 worker-dependent tests fail environmentally there, identical with or without this branch. CI builds normally, so they pass on CI.

🤖 Generated with Claude Code

Refs #1923

Refs #1926

Refs #1935

Refs #1930

Refs #1934

… dead literals Add a CI gate (test/integration/grammar-literal-validation.test.ts) validating every node-type and field-name literal in the ingestion code layer against each grammar's node-types.json, with a live `new Parser.Query` probe fallback for literals the static JSON under-reports. Covers all three surfaces: - legacy Call-Resolution DAG (type-extractors, *-extractors/configs) + the ungated structure phase (field/method extractors, export-detection) — AST scan; - registry scope-resolution captures + scope queries (Mode 3 compile); - the registry RESOLUTION layer (scope-resolver/type-binding/receiver-binding/ interpret/arity/import-decomposer …) via a TS-TypeChecker discriminator that collects a literal ONLY when its `.type` receiver is a tree-sitter SyntaxNode (so resolved-symbol `.type` kinds like 'Class' are never mistaken for nodes). Helpers: test/helpers/{grammar-introspection,literal-collectors}.ts. Remove every existence-dead literal the gate surfaces (behavior-neutral dead-branch/fallback deletions verified absent from the installed grammar), spanning the legacy, structure-phase, and registry production paths: reference_type/pointer_type/scoped_identifier/scoped_type_identifier/ rvalue_reference_declarator/variadic_parameter (C/C++), equals_value_clause/ identifier_name/simple_identifier/record_struct_declaration/record_class_declaration (C#), generic_type/`type` field (Dart), nullable_type (PHP), method_call/symbol (Ruby), method_call_expression/slice_type/shorthand_field_pattern (Rust), struct_declaration/internal_name (Swift), comment (Java), parameter/ parameterized_type and dead childForFieldName('pattern'|'modifiers'| 'formal_parameters'|'declaration'|'default'|'return_value'|'alias_clause') / class_expression fallbacks. Gate ships with an empty allowlist. One behavior FIX (scope-resolution): PHP `findEnclosingTypeDeclaration` omitted `anonymous_class`, so a method inside an anonymous class mis-bound `$this` to the enclosing named class; add `anonymous_class` so it is correctly skipped. Verified: tsc clean; gate green (empty allowlist); scope-resolution parity 26/26 on both REGISTRY_PRIMARY_*=0 and =1; resolver suite no new failures. Issue #1920 (epic #1919). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

vercel · 2026-05-30T20:17:50Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
gitnexus	Ready	Preview, Comment	May 31, 2026 9:09am

github-actions · 2026-05-30T20:37:26Z

CI Report

✅ All checks passed

Pipeline Status

Stage	Status	Details
✅ Typecheck	`success`	tsc --noEmit
✅ Tests	`success`	unit tests, 3 platforms
✅ E2E	`success`	gitnexus-web changes only

Test Results

Tests	Passed	Failed	Skipped	Duration
10397	10388	0	9	640s

✅ All 10388 tests passed

9 test(s) skipped — expand for details

COBOL pipeline benchmark > scales with file count
C# pipeline benchmark > scales with file count — namespaces spread across the solution
C# pipeline benchmark > scales with file count — all types in one (global) namespace bucket
Go pipeline benchmark > scales with file count (workers enabled)
Go pipeline benchmark — worker pool (issue Worker idle timeout kills long Go scope extraction and surfaces as Napi::Error during analyze #1848) > does not quarantine the large generated Go file on sub-batch idle timeout
PHP pipeline benchmark > scales with file count (workers enabled)
Ruby pipeline benchmark > scales with file count (workers enabled)
Rust pipeline benchmark > scales with file count (workers enabled)
buildTypeEnv > known limitations (documented skip tests) > Ruby block parameter: users.each { |user| } — closure param inference, different feature

Code Coverage

Tests

Metric	Coverage	Covered	Base	Delta	Status
Statements	80.22%	36075/44969	79.84%	📈 +0.4	🟢 ████████████████░░░░
Branches	68.9%	23001/33381	68.5%	📈 +0.4	🟢 █████████████░░░░░░░
Functions	85.19%	3717/4363	84.94%	📈 +0.2	🟢 █████████████████░░░
Lines	83.76%	32485/38782	83.36%	📈 +0.4	🟢 ████████████████░░░░

_{📋 View full run · Generated by CI}

… tests Three tests asserted defensive handling of node types the installed grammars never emit (verified via real tree-sitter parse), so they broke once the dead literals were removed in af9d709: - parsing.test.ts isNodeExported / csharp: `record struct` and `record class` both parse to `record_declaration` (kept in CSHARP_DECL_TYPES) — tree-sitter-c-sharp emits no `record_struct_declaration` / `record_class_declaration` node. Switch the two mock nodes to `record_declaration`. - extract-generic-type-args.test.ts: Java emits `generic_type` and Kotlin `user_type`+`type_projection`; `parameterized_type` is produced by no installed grammar, so the shared extractor returns [] for it. Convert the case to a documented negative assertion (real paths already covered by the generic_type cases). No source behavior change: production export detection (record_declaration) and generic type-arg extraction (generic_type / type_projection) were already correct. Fixes the 3 CI failures on PR #1937. Issue #1920 (epic #1919). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…wlisted) Restore the `parameterized_type` branch in extractSimpleTypeName / extractGenericTypeArgs (type-extractors/shared.ts) so a parameterized_type node still yields its type arguments (List<User> -> [User]). Current tree-sitter-java emits `generic_type` and tree-sitter-kotlin `user_type`+`type_projection`, so this is a defensive alternate node kept for grammar-version resilience; it is allowlisted in the node-type validation gate with a documented justification rather than removed. extract-generic-type-args.test.ts now asserts the User type argument is captured from a parameterized_type node. Issue #1920 (epic #1919). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…rameterized_type guess extractGenericTypeArgs / extractSimpleTypeName special-cased `parameterized_type`, a node type NO installed grammar emits (real parse: Java/TypeScript/Rust -> generic_type, C# -> generic_name, Kotlin -> user_type). It was a guess masking a real gap; remove it. The genuine 'Kotlin alternate node type' is `user_type` (`List<User>` parses to user_type > [type_identifier, type_arguments]), which the extractor returned [] for. Handle it: read a user_type's own type_arguments, else recurse into its wrapped child (preserving the existing user_type > generic_type unwrap). No production caller passes user_type today (Kotlin generics resolve via jvm.ts), so this only makes the function's documented Kotlin contract correct — zero behaviour change for current callers (Java/TS/C#/Rust pass generic_type/name). Replace the mock parameterized_type test with REAL-PARSE coverage across Java/TypeScript/C#/Rust/Kotlin (+ Java Map<String,User>) so a wrong node-type guess can't silently pass again. Gate allowlist returns to empty. Issue #1920 (epic #1919). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

github-actions · 2026-05-31T05:51:26Z

✨ PR Autofix

Found fixable formatting / unused-import issues across 70 changed lines. Comment /autofix on this PR to apply them, or run npm run lint:fix && npm run format locally.

{"schema":"gitnexus.pr-autofix/v2","state":"fixes-available","pr_number":1937,"changed_lines":70,"head_sha":"da48f1845e0f2c602a95c96f477620ee05da81d4","run_id":"26708227353","apply_command":"/autofix"}

…gate) CI runs `prettier --check .` from the repo root (printWidth 100) and flagged the new real-parse cases array's long single-line object literals. Wrap them. Format-only; no behaviour change. Issue #1920 (epic #1919). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

magyargergo

PR Tri-Review — node-type/field validation gate + dead-literal removal

Verdict: production-ready with minor follow-ups. Merge state: MERGEABLE / CLEAN (CI green — run 26705126928). Branch hygiene: 5 focused commits on origin/main@d1d2a64d, no merge-from-main, no unrelated churn. The production changes are correct and validated; every finding below concerns the new gate's completeness and none blocks merge.

Methods: 5 Claude reviewer lanes (risk, correctness, adversarial, maintainability, testing) + a synthesis-critic gate. Codex was dispatched but returned no structured findings, so this is a multi-Claude-lane review — agreement here is consistent across personas, not independent cross-engine confirmation.

✅ What's solid (validated, not assumed)

The central dead-literal claim held under empirical real-parse testing by the adversarial and correctness lanes. Every removed node-type literal — C# record_struct_declaration/equals_value_clause, C++ scoped_identifier/variadic_parameter, Rust slice_type/method_call_expression/shorthand_field_pattern, PHP nullable_type/alias_clause, Swift struct_declaration, Ruby symbol/method_call, Dart generic_type, TS class_expression, … — was confirmed not emitted by the grammar for the language(s) that code path serves. No silent regression.
type-extractors/shared.ts: dropping parameterized_type and handling Kotlin's real user_type is a net improvement — the old code returned [] for Kotlin generics. Depth-guarded; no infinite recursion.
languages/php/receiver-binding.ts: the anonymous_class change is a genuine correctness fix — it closes a latent $this mis-bind (a method in an anonymous class nested in a named class previously bound $this to the outer class). anonymous_class is the real grammar node; the old anonymous_class_declaration literal never matched.

🔧 Findings — gate completeness (none block merge)

Two inline comments (P2). In short:

Node-agnostic field validation lets node-scoped-dead childForFieldName lookups through — the kept ?? childForFieldName('pattern') fallbacks at type-extractors/csharp.ts:101 and type-extractors/jvm.ts:90,419 (the same dead-field pattern this PR removes from go/php/python/swift) survive the gate. Reproduced.
Mode-4 (resolution layer) can silently contribute zero literals — resolutionLayerProgramOk is set but never asserted and the gate floors nothing, so a TS-build failure on a CI host would let resolution-layer files go unvalidated while the gate stays green.

📋 Lower-priority (body only)

P3 — a dead parameterized_type literal still lives at type-env.ts:980 on a real SyntaxNode (child.type === 'parameterized_type'), outside all four scan surfaces; the collector doc comment incorrectly claims type-env.ts uses only resolved-symbol kinds. Behavior-neutral, but a blind spot in the PR's own remove-and-gate thesis (type-env.ts isn't in this diff — follow-up).
P3 nits — (a) literal-collectors.ts doc says "THREE modes" but four are implemented (in-file order 1,2,4,3); (b) collectResolutionLayerLiterals (Mode 4) returns type Mode2Result (misnamed); (c) the two knownFailures.has() checks at :71/:87 run against the empty Set at :27 (dead conditionals — consider expect(knownFailures.size).toBe(0)); (d) the real-parse generics matrix lacks Kotlin/C# multi-arg cases (only mock-tested); (e) the probeNodeType "unavailable" test asserts only not-throw, not the return value.

🧪 Refuted (actively checked, found false — validation is a result)

Every removed-literal regression hypothesis; user_type infinite-recursion / wrong-args; the PHP anonymous_class change breaking a previously-correct binding.

⏭️ Pre-existing (not caused here — follow-up)

Java argument-arity overcounts interleaved block_comment/line_comment (the removed comment literal was already dead, so removal is behavior-neutral, but it surfaces a real overcount that can skew arity-based overload resolution).

Automated multi-tool digest (5 Claude lanes; Codex unavailable). Verify before acting.

…(U1) Add probeField(language, nodeType, field) — the node-scoped analogue of probeNodeType: compiles `(<nodeType> <field>: (_)) @_` against the live grammar and classifies TSQueryErrorStructure/Field -> dead, TSQueryErrorNodeType (node absent here) -> unavailable, compile -> valid. Conservative-toward-valid (supertype-typed fields make some wrong fields compile), so it never produces a false positive. Add isFieldError classifier; make validateField's node-scoped path membership-then-probe so node-types.json field under-reporting can't yield a false `dead`. Foundation for the node-scoped field validation gate (no gate behavior change yet). Issue #1920 (epic #1919).

…nv.ts (U2) CollectedField gains receiverNodeType, captured conservatively by receiverNodeTypeOf: only when a childForFieldName receiver is unambiguously narrowed by a single enclosing positive guard (if (recv.type==='X') then-branch, or switch case 'X') with no reassignment/shadowing of the receiver in the enclosing function. Any uncertainty -> undefined (sound global fallback); fail-safe (benign false negative, never a false positive). Extend Mode-4's resolutionLayerFiles to include shared resolution files directly under ingestion/ (type-env.ts), tagged with the full gated language set via fileLanguages (valid-if-any). Entries now carry a language SET. Rename Mode2Result -> ScanResult; fix the header doc (THREE -> FOUR modes). Gate behavior unchanged until U3 consumes receiverNodeType. Issue #1920.

…terals (U3, U4) U3: the gate validates childForFieldName lookups node-scoped (validateField with the captured receiverNodeType) and fails loudly on a degraded/vacuous run (asserts resolutionLayerProgramOk, floors collected counts, requires knownFailures empty). U4: remove every dead field/literal the hardened gate flags — all behavior-neutral (the dead disjunct never fired on reachable nodes; verified by real parse + the type-extractor/resolution unit suites, 484 passing): - type-env.ts: parameterized_type (emitted by no grammar) and switch_block_label (real Java enhanced switch is switch_label/switch_rule) from the SyntaxNode .type sets - languages/csharp/captures.ts: generic_name has no `name` field -> firstNamedChild - type-extractors/jvm.ts: Kotlin property_declaration has no name/type fields (positional children) -> findChild; drop the else-branch `pattern` fallbacks x2 - type-extractors/csharp.ts: drop the else-branch `pattern` fallback (parity with go/php/python/swift) Gate green with node-scoped validation on; tsc clean. Closes the Mode-4 type-env coverage opened in U2. Latent follow-up: Java enhanced-switch arms (switch_rule) are absent from NARROWING_BRANCH_TYPES — a separate behavior fix. Issue #1920 (epic #1919).

tree-sitter-java emits block_comment/line_comment as named children of argument_list; counting them inflated @reference.arity / @reference.parameter- types / @reference.arg-names for any Java call with an inline comment, which skews arity-based overload resolution (arity feeds call-processor symbol-ID generation). Filter them at the single arg-list site (also corrects the downstream args.map). The previously-removed `comment` literal never matched — the real nodes are block_comment/line_comment (the #1920 gate lesson). Isolated from the behavior-neutral gate units (U1-U4) since this changes production graph output. Java resolver suite 178/178; new java-call-arity test covers block/line comments, leading comment, constructor calls, and the no-comment regression. Issue #1920 (epic #1919).

…ssertions (U6) - extract-generic-type-args: add real-parse Kotlin Map<String,User> (user_type > type_arguments > type_projection) and C# Dictionary<string,User> (generic_name > type_argument_list) multi-arg cases. - grammar-introspection: the probeNodeType test now asserts 'dead' for a bogus node on installed grammars (not merely not-throw), and documents the null-model split (validateField -> unavailable; validateNodeType -> still probes the live grammar). Issue #1920 (epic #1919).

CI runs `prettier --check .` from the repo root (printWidth 100); the gitnexus/ pre-commit hook formatted these two files differently. Format-only, no behavior change. Issue #1920.

…to validation gate The grammar-literal-validation gate (abhigyanpatwari#1937) added to main exposed latent bugs in GDScript extractor code that used incorrect tree-sitter field names: - for_statement: 'name'/'iterable' → 'left'/'right' (GDScript grammar uses different field names) - base_call/attribute_call: removed 'name' field lookup (identifier is child, not field) - assignment: removed 'type' field check (doesn't exist in GDScript grammar) - parameter: removed (GDScript uses typed_parameter/default_parameter/typed_default_parameter) Also added GDScript to test helper mappings (GRAMMAR_PACKAGES, DIR_LANG, BASENAME_LANGS, PREFIX_LANGS) so the validation gate can properly validate GDScript literals.

vercel Bot deployed to Preview May 30, 2026 20:18 View deployment

vercel Bot deployed to Preview May 31, 2026 05:24 View deployment

vercel Bot deployed to Preview May 31, 2026 05:31 View deployment

vercel Bot deployed to Preview May 31, 2026 05:50 View deployment

vercel Bot deployed to Preview May 31, 2026 06:15 View deployment

magyargergo commented May 31, 2026

View reviewed changes

Comment thread gitnexus/test/integration/grammar-literal-validation.test.ts Outdated

Comment thread gitnexus/test/integration/grammar-literal-validation.test.ts

magyargergo added 5 commits May 31, 2026 08:22

vercel Bot deployed to Preview May 31, 2026 08:53 View deployment

style(test): apply root prettier formatting (CI format gate)

ba8cc62

CI runs `prettier --check .` from the repo root (printWidth 100); the gitnexus/ pre-commit hook formatted these two files differently. Format-only, no behavior change. Issue #1920.

vercel Bot deployed to Preview May 31, 2026 09:09 View deployment

magyargergo merged commit b43aa10 into main May 31, 2026
28 checks passed

magyargergo deleted the feat/1920-node-type-validator branch May 31, 2026 09:29

magyargergo mentioned this pull request May 31, 2026

fix(cli): steer docs, skills, and hooks through a CLI-neutral project-local runner (#1939) #1945

Merged

5 tasks

This was referenced Jun 1, 2026

C#: parsing-layer coverage gaps (7 findings) #1924

Closed

Kotlin: parsing-layer coverage gaps (8 findings) #1930

Closed

PHP: parsing-layer coverage gaps (4 findings) #1931

Closed

Ruby: parsing-layer coverage gaps (4 findings) #1933

Closed

magyargergo mentioned this pull request Jun 1, 2026

Rust: parsing-layer coverage gaps (9 findings) #1934

Closed

9 tasks

magyargergo mentioned this pull request Jun 4, 2026

Parsing-layer infra hardening: pin tree-sitter ABI (infra-1) + safe-parse timeout/ERROR handling (infra-2) #1922

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(ingestion): tree-sitter node-type/field validation gate + remove dead literals#1937

feat(ingestion): tree-sitter node-type/field validation gate + remove dead literals#1937
magyargergo merged 11 commits into
mainfrom
feat/1920-node-type-validator

magyargergo commented May 30, 2026 •

edited

Loading

Uh oh!

vercel Bot commented May 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 31, 2026 •

edited

Loading

Uh oh!

magyargergo left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

magyargergo commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What the gate covers

Dead literals removed (behavior-neutral)

One behavior fix (scope-resolution)

Verification

Uh oh!

vercel Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI Report

Pipeline Status

Test Results

Code Coverage

Tests

Uh oh!

github-actions Bot commented May 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✨ PR Autofix

Uh oh!

magyargergo left a comment

Choose a reason for hiding this comment

PR Tri-Review — node-type/field validation gate + dead-literal removal

✅ What's solid (validated, not assumed)

🔧 Findings — gate completeness (none block merge)

📋 Lower-priority (body only)

🧪 Refuted (actively checked, found false — validation is a result)

⏭️ Pre-existing (not caused here — follow-up)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

magyargergo commented May 30, 2026 •

edited

Loading

vercel Bot commented May 30, 2026 •

edited

Loading

github-actions Bot commented May 30, 2026 •

edited

Loading

github-actions Bot commented May 31, 2026 •

edited

Loading