feat(ingestion): Java Spring route annotation → Route node extraction#2078
Conversation
…tion Previously, GitNexus only supported Route node generation for JS/TS ecosystems (Express, Next.js, Fastify, etc.) and Python (FastAPI, Flask). Java Spring's annotation-based routing (@RequestMapping, @GetMapping, @PostMapping, etc.) was only supported at the group contract layer (http-patterns/java.ts) for cross-repo matching, but NOT at the ingestion layer for generating graph Route nodes. This commit adds ingestion-layer support: 1. JAVA_QUERIES (tree-sitter-queries.ts): - Added method-level annotation captures (@GetMapping, @PostMapping, @PutMapping, @DeleteMapping, @PatchMapping) → @decorator captures - Added class-level @RequestMapping → @decorator capture (prefix) - Supports both positional ("/path") and named (path="/path", value="/path") annotation argument forms 2. parse-worker.ts: - Java class-level @RequestMapping is detected and stored as a prefix (not pushed as a standalone Route) - After per-file capture processing, the prefix is applied to all method-level routes in the same file via the existing ExtractedDecoratorRoute.prefix field - The routes phase (normalizeExtractedRoutePath) handles the prefix joining, producing final URLs like /api/users/list 3. Tests: - Unit test (worker-backed): 4 cases covering prefix joining, bare routes, class-level exclusion, multi-file isolation - Integration test (full pipeline): 6 cases covering end-to-end Route node + HANDLES_ROUTE edge generation Closes the feature gap where `route_map`, `shape_check`, and `api_impact` MCP tools returned empty results for Java Spring projects.
|
Someone is attempting to deploy a commit to the NexusCore Team on Vercel. A member of the Team first needs to authorize it. |
✨ PR AutofixFound fixable formatting / unused-import issues across 15 changed lines. Comment |
|
/autofix |
|
✅ Applied autofix and pushed a commit. (apply run) |
CI Report✅ All checks passed Pipeline Status
Test Results
✅ All 10893 tests passed 16 test(s) skipped — expand for details
Code CoverageTests
📋 View full run · Generated by CI |
magyargergo
left a comment
There was a problem hiding this comment.
Tri-review: Java Spring route ingestion (PR #2078)
Verdict: changes requested. One failing CI check caused by this PR, plus three corroborated P2 issues — the headline being the language-specific Java logic added to the shared parse worker.
Methods & engine independence. Reviewed by 6 Claude reviewer lanes (correctness, adversarial, maintainability, testing returned full findings; the two GitNexus risk / test-CI lanes ended mid-investigation — their domains were covered by the CE lanes and by my own verification) plus Codex, the only genuinely independent engine here. The six Claude lanes share one underlying model, so their agreement is "consistent across personas," not independent confirmation — the strong signal below is Codex + a Claude lane agreeing, and several findings were verified by actually parsing real tree-sitter-java. Codex was live (foreground re-run after the first background pass returned empty).
What's solid (validated). The feature works end-to-end for the common one-controller-per-file case. It cleanly reuses the existing ExtractedDecoratorRoute.prefix plumbing (routes.ts joins prefix+path via normalizeExtractedRoutePath); javaClassPrefix is correctly per-file (no cross-file leak — refuted); a suspected mixed FastAPI+Spring prefix-clobber in parse-impl.ts was refuted by both Codex and correctness (the expansion is guarded on .py + decoratorReceiver==='router', neither of which Java sets); and normalizeExtractedRoutePath handles the slash-join edge cases. Positional + named-arg method annotations and GET/POST/PUT/DELETE all work. Good reuse of existing infrastructure.
Headline (the change requested) — [P2] Language-specific Java logic in shared parse-worker.ts
The PR adds the first language === SupportedLanguages.Java special-cases inside the previously language-agnostic decorator-route block (parse-worker.ts:1322, and the post-loop apply at :2279). AGENTS.md (§RFC #909) and CLAUDE.md both state: "Shared code in gitnexus/src/core/ingestion/ must not name languages — plug language behavior in via LanguageProvider/ScopeResolver hooks."
Honest framing: this extends existing tech-debt rather than uniquely breaking a rule — the file already names TS/Vue/Python/Go, and even the FastAPI/Laravel route extractors are reached via a one-line if (language === X) dispatch. So the achievable bar isn't "zero language names"; it's isolate the language logic into a module and leave only a thin dispatch line, the way the repo already does it: route-extractors/laravel.ts (extractLaravelRoutes, parse-worker:2255) and route-extractors/fastapi-router-bindings.ts (extractFastAPIRouterBindings, parse-worker:2266).
Recommended: extract the Spring logic to route-extractors/spring.ts (e.g. extractSpringRoutes(tree, filePath) returning routes with the class prefix already joined internally). This keeps the shared worker language-agnostic for this concern and dissolves the multi-class bug below (the extractor can hold a per-class prefix map). (Codex + ce-maintainability + gitnexus-risk.)
Inline findings (anchored on the diff)
- [P2 · CI blocker]
quality / formatis red because of this PR — rootnpx prettier --check .flags the newspring-route-pipeline.test.ts. Fix:prettier --writeit. - [P2 · bug]
@PatchMappingroutes silently dropped —ROUTE_DECORATOR_NAMESomits'PatchMapping'. (Codex + adversarial + testing + verified.) - [P2 · bug] Multiple
@RequestMappingclasses in one file → singlejavaClassPrefixscalar last-wins → wrong prefix on the earlier class's routes. (Codex + 4 Claude lanes; empirically parsed.) - [P2 · test hygiene] The unit suite is a silent no-op in unit-only runs (runs zero production code); the integration test is what actually covers the feature. (Coordinator-verified.)
Lower-priority / coverage gaps (not blocking)
- [P3] Method-level
@RequestMapping(method=RequestMethod.POST)is recorded ashttpMethod:'GET'. Latent only —httpMethodis not consumed for decorator routes (routes.ts:221-228reads onlyroutePath/prefix; the Route node is keyed on the URL atroutes.ts:247). Becomes a real bug only ifhttpMethodis ever persisted/keyed. (correctness + Codex.) - [P3] Array-value
@GetMapping({"/a","/b"}), constant-ref@GetMapping(PATH), and bare@GetMappingproduce no query match and are silently dropped. (adversarial.) - [P3] Coverage gaps: named-arg class-level
@RequestMapping(value=...)is untested (query added, fixtures only positional); the unit test asserts intermediateroutePath/prefixrather than the joined URL (the integration test does assert/api/users/list); the HANDLES_ROUTE assertion uses>=1. (testing.)
Refuted (validation is a feature)
Cross-file prefix leak; mixed FastAPI+Spring prefix clobbering; route-path slash-join errors — all investigated and found sound.
CI
quality / format FAIL (this PR — finding 1) · Vercel FAIL (deploy authorization, unrelated) · CodeQL / lint / typecheck / Analyze pass · several platform-sensitive + coverage jobs still pending at review time.
Automated multi-tool digest (6 Claude reviewer lanes + Codex, the one independent engine). Inline anchors mark verified findings; please verify before acting.
| @@ -0,0 +1,92 @@ | |||
| /** | |||
There was a problem hiding this comment.
[P2 · CI blocker] This file fails quality / format. CI runs npx prettier --check . from the repo root and flags this file (job log: [warn] gitnexus/test/integration/spring-route-pipeline.test.ts → exit 1). A local prettier --check with the gitnexus-package config passes, but the root config is what gates CI. Fix: npx prettier --write gitnexus/test/integration/spring-route-pipeline.test.ts and commit. [reproduced — CI log]
…ping, multi-class support Addresses all P2 findings from tri-review: 1. **Architecture**: Extracted Spring route logic from parse-worker.ts into a dedicated `route-extractors/spring.ts` module (matching the pattern of `laravel.ts` and `fastapi-router-bindings.ts`). parse-worker now has a single dispatch line — no language-specific logic inline. 2. **PatchMapping bug**: Added `'PatchMapping'` to `ROUTE_DECORATOR_NAMES` (was silently dropped before). 3. **Multi-class bug**: The new `extractSpringRoutes` walks each class declaration independently with its own prefix — no more single-scalar `javaClassPrefix` last-wins issue. 4. **Test hygiene**: Unit tests now import `extractSpringRoutes` directly (no dist build / worker pool dependency). Tests run in all tiers. 5. **Removed JAVA_QUERIES decorator patterns**: The Spring extractor does its own AST walk, so the tree-sitter query captures for Java annotations are no longer needed (avoids duplicate route emission). Additional test coverage: - Multi-class in one file with independent prefixes - @PatchMapping support - Named annotation args (path= and value=) on class-level @RequestMapping
|
Thanks for the thorough review! All P2 findings addressed in e932605:
P3 items (array values, |
Addresses the second review comment: instead of an inline `if (language === SupportedLanguages.Java)` dispatch in parse-worker, the Spring route extraction is now wired through a new optional `extractDecoratorRoutes` hook on LanguageProviderConfig. - Added `extractDecoratorRoutes` to LanguageProviderConfig interface - Java provider registers `extractSpringRoutes` as its implementation - parse-worker calls `provider.extractDecoratorRoutes?.()` generically - Removed direct import of spring.ts from parse-worker This keeps parse-worker fully language-agnostic — no language names appear in the dispatch path for route extraction.
|
Good point — agreed that this should go through the provider rather than an inline language check. Fixed in 33e7849: added an optional extractDecoratorRoutes hook to LanguageProviderConfig, wired it up in the Java provider, and parse-worker now calls provider.extractDecoratorRoutes?.() generically. No language names in the dispatch path anymore. Let me know if there's anything else I should adjust! |
…orts
Addresses all 4 inline review comments:
1. Rewrote spring.ts to use a single predicate-free Parser.Query
(same pattern as group-layer JAVA_ROUTE_ANNOTATION_PATTERNS).
Two-phase loop: first pass collects class prefixes by node.id,
second pass resolves method routes via findEnclosingClass.
No more manual DFS / recursion.
2-3. Moved inline import(...) type references in language-provider.ts
to proper top-level imports (Parser, ExtractedDecoratorRoute).
4. Covered by #1 — recursive helpers removed entirely.
Added 3 extra test cases: non-route named args filtering,
prefix isolation across mixed classes, line number accuracy.
|
Thanks for the feedback! All four comments addressed:
Also added 3 extra test cases: non-route named args filtering ( Thanks again for the careful review — much cleaner this way! |
|
@claude can you compare the route extrator in java.ts and spring.ts? What's the difference and how could we migrate spring.ts into java.ts and kotlin.ts? |
|
Claude finished @magyargergo's task in 1m 34s —— View job Comparing
|
group/extractors/http-patterns/java.ts |
route-extractors/spring.ts |
|
|---|---|---|
| Layer | Group layer (cross-repo HTTP contract matching) | Ingestion layer (graph Route node generation) |
| Output type | HttpDetection[] (role, path, method, confidence) |
ExtractedDecoratorRoute[] (filePath, routePath, prefix, lineNumber) |
| Consumer | api_impact, route_map, cross-repo contract diffing |
parse-worker → Route graph nodes + HANDLES_ROUTE edges |
| Scope | Providers and consumers (RestTemplate, WebClient, OkHttp, Feign, Apache HttpClient) | Providers only (@GetMapping etc.) |
| Query infra | compilePatterns / runCompiledPatterns from tree-sitter-scanner.js |
new Parser.Query(Java, ...) directly |
Key structural differences
1. What they do with the path
group/java.ts joins the class prefix and method path inline before returning:
// java.ts (group layer)
path: joinPath(prefix, route.rawPath), // → "/api/users/list" fully resolvedroute-extractors/spring.ts returns them separately — joining happens later in routes.ts via normalizeExtractedRoutePath:
// spring.ts (ingestion layer)
routePath: '/list',
prefix: '/api/users', // joined downstream2. Query coverage
group/java.ts also covers interface_declaration (for OpenFeign @FeignClient / @RequestLine contracts), and captures @member (the method name). spring.ts omits both — no interface support, no method name capture.
3. unquote vs unquoteLiteral
group/java.ts uses unquoteLiteral (from the scanner utils), which returns null for non-string nodes. spring.ts has its own unquote() that assumes the input is always a quoted string. The group-layer version is safer.
4. Cross-file inheritance
group/java.ts has scanSpringProject / collectSpringTypes which resolves interface→controller inheritance across files (a controller implementing a @RequestMapping interface inherits its routes). spring.ts has no equivalent — it's per-file only.
Migration path: spring.ts → java.ts + kotlin.ts
The spring.ts ingestion extractor is already wired correctly into java.ts (the language provider) via extractDecoratorRoutes: extractSpringRoutes. The question is whether the logic should live as a standalone spring.ts file or be absorbed.
Why spring.ts should stay separate (current approach is correct):
- The group-layer
java.tsis ~880 lines covering 6+ HTTP frameworks. Merging ingestion logic in would violate the layer separation this codebase enforces. - The query structure, output types, and path-joining semantics differ enough that sharing code would require awkward adapters.
What can be improved — sharing the query and helpers:
group/java.ts and route-extractors/spring.ts both define nearly identical:
METHOD_ANNOTATION_TO_HTTP(same map, duplicated)findEnclosingClass(identical function, duplicated — seegroup/java.ts:368vsspring.ts:101)- The tree-sitter query pattern (same 4-branch shape, minor differences in
@membercapture) isRouteMemberKey/isRouteKey(same logic, different names)
These could be extracted to a shared route-extractors/spring-shared.ts:
// route-extractors/spring-shared.ts
export const METHOD_ANNOTATION_TO_HTTP: Record<string, string> = { ... };
export function findEnclosingClass(...) { ... }
export function isRouteMemberKey(...): boolean { ... }Adding Kotlin support (kotlin.ts):
Spring annotations work identically in Kotlin, but the AST grammar differs:
- Kotlin uses
function_declarationinstead ofmethod_declaration - Annotations use
multi_annotationorannotationnodes undermodifiers(same field name, different grammar)
The migration would be:
-
Parameterize
spring.tsto accept the language grammar and method node type name:// spring.ts — generalized export function extractSpringRoutesGeneric( tree: Parser.Tree, filePath: string, language: Language, // tree-sitter-java or tree-sitter-kotlin methodNodeType: string, // 'method_declaration' or 'function_declaration' lineOffset = 0, ): ExtractedDecoratorRoute[] { ... } // Java convenience wrapper (keeps current API) export const extractSpringRoutes = (tree, filePath, lineOffset) => extractSpringRoutesGeneric(tree, filePath, Java, 'method_declaration', lineOffset);
-
Add a Kotlin wrapper (either in
spring.tsor a newspring-kotlin.ts):import Kotlin from 'tree-sitter-kotlin'; export const extractSpringKotlinRoutes = (tree, filePath, lineOffset) => extractSpringRoutesGeneric(tree, filePath, Kotlin, 'function_declaration', lineOffset);
-
Wire into
kotlin.ts:// languages/kotlin.ts import { extractSpringKotlinRoutes } from '../route-extractors/spring.js'; export const kotlinProvider = defineLanguage({ ... extractDecoratorRoutes: extractSpringKotlinRoutes, });
The main validation needed before doing this: verify the Kotlin tree-sitter grammar uses the same annotation / annotation_argument_list / element_value_pair node type names as Java. If the annotation subtree differs, the query branches would need separate variants.
|
@henry201605 let me think about our approach a little bit. I have a feeling we are doing this processing twice. |
|
@claude following upon the previous comment, analyze whether we are doing duplicated processes and wasting cpu time. |
|
Claude finished @magyargergo's task in 2m 37s —— View job Analysis: CPU duplication between the two Spring route extractors
Short answerYes, the same Spring annotations are extracted twice — but across two different pipeline phases, not within a single command. There is also a subtler within- Two separate phases, two separate parses
Both run the same predicate-free tree-sitter query against the same Java files: The parse results are never shared: The subtler duplication inside
|
|
@henry201605 did you know about the group command? |
|
Thanks for thinking through this carefully — and the Claude analysis above was really helpful in framing it. To your question: yes, I'm familiar with On the "processing twice" point — you're right, and the analysis captures it well. There are really two kinds of duplication:
I think this PR is what surfaces that runtime redundancy (before it, Java had no Route nodes, so the source scan was the only path). The cleaner fix really sits in What I'd be glad to clean up directly in this PR is the source-level duplication:
And I agree with the analysis that the two extractors should otherwise stay separate — different output types ( That's my read, but you know this codebase far better than I do — if you have a cleaner approach in mind for either the source-level sharing or the runtime path, I'm very happy to go with whatever direction you prefer. Just let me know. |
|
Thanks, that makes sense, and I agree the two extractors should remain separate given that they serve different layers and produce different output types. For this PR, could you please:
The broader Please create and link a follow-up issue for that optimization. Once the items above are addressed, ping me and I’ll take another pass. |
Addresses review follow-up on abhigyanpatwari#2078: - Extract the primitives shared by the ingestion (route-extractors/spring.ts) and group (http-patterns/java.ts) Spring extractors into a new route-extractors/spring-shared.ts: METHOD_ANNOTATION_TO_HTTP, findEnclosingClass, isRouteMemberKey, and a safe unquoteSpringLiteral. Both extractors now import from it (group -> ingestion, the layer-correct direction) so the shared semantics can't drift apart. - Replace spring.ts's local unquote() with the safer unquoteSpringLiteral (returns null for non-string nodes instead of assuming a quoted string). - Add test/unit/spring-route-extractor-parity.test.ts: runs one shared Spring fixture through both extractors and asserts they surface the same provider method/path combinations. The broader HttpRouteExtractor source-scan optimization is tracked in abhigyanpatwari#2138.
|
Thanks again for the detailed guidance on this — all four items are addressed now:
Everything's green locally. Whenever you have a moment, I'd really appreciate another pass — and thanks again for taking the time to review this so carefully. |
magyargergo
left a comment
There was a problem hiding this comment.
Thanks for tagging along!
Summary
Adds ingestion-layer support for Java Spring annotation-based routing, enabling
route_map,shape_check, andapi_impactMCP tools to work on Java Spring projects.Before this PR: Route node count for all Java projects = 0. The group contract layer (
http-patterns/java.ts) supported Spring annotations for cross-repo HTTP contract matching, but the ingestion pipeline did not generate graph Route nodes from them.After this PR: Spring
@GetMapping,@PostMapping,@PutMapping,@DeleteMapping,@PatchMappingannotations on methods generate Route nodes. Class-level@RequestMappingis correctly joined as a URL prefix, with per-class resolution for multiple controllers in one file.Implementation
Route extraction is wired through the
LanguageProvidercontract rather than language-specific branches in the shared parse worker:route-extractors/spring.ts(new)The ingestion-layer Spring extractor. Uses a single predicate-free tree-sitter query to capture all route annotations, then a two-phase loop: pass 1 collects class-level
@RequestMappingprefixes keyed byclass_declarationnode id; pass 2 resolves each method route's prefix viafindEnclosingClass. Handles positional and named (path=/value=) arguments, and multiple classes per file with independent prefixes.language-provider.ts+languages/java.tsAdded an optional
extractDecoratorRouteshook toLanguageProviderConfig. The Java provider registersextractSpringRoutes. The parse worker callsprovider.extractDecoratorRoutes?.()generically — no language names in the shared dispatch path.route-extractors/spring-shared.ts(new)Shared primitives used by both the ingestion extractor (
spring.ts) and the group extractor (http-patterns/java.ts) so the two can't drift apart:METHOD_ANNOTATION_TO_HTTP,findEnclosingClass,isRouteMemberKey, and a safeunquoteSpringLiteral. Lives iningestion/(lower layer); group imports from it, matching the existinggroup → ingestiondirection.Tests
test/unit/java-spring-route-ingestion.test.ts(9 cases): prefix joining, bare routes, all five HTTP verbs,@PatchMapping, named args, multi-class isolation,produces/consumesfiltering, line numbers.test/unit/spring-route-extractor-parity.test.ts(2 cases): parses one shared fixture through both the ingestion and group extractors and asserts they surface the same provider method/path combinations.test/integration/spring-route-pipeline.test.ts(8 cases): full pipeline Route node + HANDLES_ROUTE edge generation.Follow-up
The group-layer
HttpRouteExtractornow runs both its graph path and source-scan path for Java and dedupes the result, making the source scan redundant for languages with full ingestion Route-node coverage. That optimization is tracked separately in #2138 (it also affects FastAPI / Laravel and needs to consider partial graph coverage).