From 747937cca1061e84e2b8afebcfbb5ca1f2d21c65 Mon Sep 17 00:00:00 2001 From: ivkond Date: Sat, 11 Apr 2026 18:52:45 +0000 Subject: [PATCH 01/10] feat(group): extractor expansion + manifest extractor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Part 2 of 4 in the split of #606 (ticket: #792). Follows #795 (bridge.lbug storage foundation, already merged), but this PR has no code-level dependency on #795 — it only imports types and the ContractExtractor interface that existed on upstream main before either PR. It could have been reviewed in parallel with #795. ## What changed Expands the 3 existing contract extractors with substantially more language/framework coverage, and adds a new `manifest-extractor` that resolves `group.yaml`-declared cross-links against the per-repo graph via exact-name lookups. ### New file (228 LOC) - `gitnexus/src/core/group/extractors/manifest-extractor.ts` — exact graph lookup for `group.yaml`-declared cross-links. HTTP paths are canonicalized before Route.name matching; gRPC is resolved by service/method name (NO `.proto`-filename fallback); topic and lib use exact-name match. Falls back to a synthetic `manifest::::` uid when the graph has no matching symbol, so cross-impact traversal still has a stable anchor for the contract. ### Modified extractors (+958 LOC prod) - `extractors/grpc-extractor.ts` (+522) — `.proto` parser with comment and string-literal sanitization (braces inside strings no longer truncate service bodies); package/service/method canonical IDs; server/client detection across Go (`grpc.NewServer`, `RegisterXxxServer`, `XxxGrpc.XxxImplBase`), Java (`@GrpcService`, `BlockingStub`), Python (`servicer_to_server`, `XxxStub`), and TypeScript/Node (`@GrpcMethod`, `ClientGrpc`, `loadPackageDefinition`). - `extractors/http-route-extractor.ts` (+174) — Go gin/echo/stdlib `HandleFunc`, NestJS `@Controller`+`@Get`/etc, Python FastAPI decorators, Java Spring `@RequestMapping`/`@GetMapping`, restTemplate / WebClient / OkHttp consumers. - `extractors/topic-extractor.ts` (+98) — sarama `ProducerMessage{}` struct literal detection (replaces a constructor-anchored regex that missed topics inside producer loops), kafka-go Writer/Reader, Python NATS (`await nc.subscribe`/`await nc.publish`), JetStream helpers. ### Modified and new tests (+1264 LOC) - `grpc-extractor.test.ts` (+539) — full coverage of the new proto parser (strings-with-braces regression, comments-with-braces regression), per-language server/client detection - `http-route-extractor.test.ts` (+240) — per-framework route extraction + normalization edge cases - `topic-extractor.test.ts` (+177) — the sarama in-loop regression, JetStream, Python NATS, kafka-go Writer/Reader - `manifest-extractor.test.ts` (+308 NEW) — HTTP path normalization, gRPC exact lookup with proto-fallback regression, lib and topic exact matching, synthetic-uid fallback behavior ### Self-review fixes folded in Carried forward from the #606 self-review (commit `d15b8cb`): - **HIGH #1** — `manifest-extractor.resolveSymbol` was too fuzzy. Previously used `CONTAINS` on route/name fields plus an unconditional `filePath ENDS WITH '.proto'` fallback for gRPC. Consequences: `/orders` matched `/suborders`, and any repo with any `.proto` file returned a random proto symbol for a gRPC manifest entry. Replaced with exact equality + deterministic `ORDER BY` + synthetic-uid fallback for unresolved manifests. Regression tests included. - **MED #3** — gRPC proto parser brace-depth counting now sanitizes strings and comments first (`stripProtoCommentsAndStrings`). A valid proto with `option deprecated_reason = "use NewService { instead"` used to have its service body closed early by the `"{"` inside the literal, silently dropping methods after the offending string. Regression tests for both string-with-brace and comment-with-brace cases. - **MED #4** — sarama Kafka regex changed from `sarama.NewSyncProducer[\s\S]{0,300}?Topic:` (anchored on constructor, caught only first topic in a loop) to `sarama.ProducerMessage{...Topic:}` (matches every struct literal directly). Regression test with a for-loop that constructs multiple `ProducerMessage`s. - **MED #7** — `manifest-extractor.resolveSymbol` no longer has a silent `catch { /* fall through */ }`. Errors from the graph executor are logged via `console.warn` with link type, contract name, repo key, and error message before falling through to the synthetic-uid path. ## Why Reviewer focus here is pure regex / parser correctness — no storage, no Cypher queries, no algorithmic changes to the cross-link algorithm. Separating this from the bridge foundation PR (#795) meant reviewers could stay in a single mental mode (parsing logic) instead of context-switching between DDL, Cypher, and regex. ## How to verify - `cd gitnexus && npx tsc --noEmit` - `cd gitnexus && npx vitest run test/unit/group/grpc-extractor.test.ts --pool=forks` - `cd gitnexus && npx vitest run test/unit/group/http-route-extractor.test.ts --pool=forks` - `cd gitnexus && npx vitest run test/unit/group/topic-extractor.test.ts --pool=forks` - `cd gitnexus && npx vitest run test/unit/group/manifest-extractor.test.ts --pool=forks` Local pre-push: typecheck clean, all 99 extractor unit tests pass (grpc 43, http 18, topic 30, manifest 8). ## Risk / rollback **Low.** Extractors have no user-facing surface in this PR — they produce `ExtractedContract[]` that is consumed by `sync.ts` in the next split (#793). No existing behavior changes for users who don't run a `group sync`. Rollback = `git revert` of the merge commit; the modifications to `grpc-extractor.ts` / `http-route-extractor.ts` / `topic-extractor.ts` revert to the pre-PR versions that still work (they're subsets of the new functionality). ## Scope discipline (per GUARDRAILS.md) - Only the 8 files above are touched; no drive-by refactors - No CI/release/security config changes - No secrets or machine-specific paths - Content lifted from #606 (CI 11/11 green on `d15b8cb`) ## Dependencies - **Base:** `main` (upstream already includes #795 as `1ff324c`) - **Blocks:** sync pipeline (#793) and the cross-impact feature (#794) - **Tracker issue:** #792 - **Parent PR:** #606 Co-authored-by: Claude --- .../core/group/extractors/grpc-extractor.ts | 522 ++++++++++++++--- .../group/extractors/http-route-extractor.ts | 174 +++++- .../group/extractors/manifest-extractor.ts | 228 ++++++++ .../core/group/extractors/topic-extractor.ts | 98 ++++ .../test/unit/group/grpc-extractor.test.ts | 539 +++++++++++++++++- .../unit/group/http-route-extractor.test.ts | 240 +++++--- .../unit/group/manifest-extractor.test.ts | 308 ++++++++++ .../test/unit/group/topic-extractor.test.ts | 177 +++++- 8 files changed, 2113 insertions(+), 173 deletions(-) create mode 100644 gitnexus/src/core/group/extractors/manifest-extractor.ts create mode 100644 gitnexus/test/unit/group/manifest-extractor.test.ts diff --git a/gitnexus/src/core/group/extractors/grpc-extractor.ts b/gitnexus/src/core/group/extractors/grpc-extractor.ts index b4cefadc51..6e52c0b746 100644 --- a/gitnexus/src/core/group/extractors/grpc-extractor.ts +++ b/gitnexus/src/core/group/extractors/grpc-extractor.ts @@ -25,20 +25,110 @@ function serviceOnlyContractId(serviceName: string): string { return `grpc::${serviceName}/*`; } +/** + * Replace all .proto comments and string literals with spaces, preserving the + * original length and character offsets of the input. This lets downstream + * regex / brace-depth parsers run on a "sanitized" copy without having to + * understand proto syntax, while any RegExp.exec/index-based lookups that + * were already positional against `content` continue to work against the + * original string. + * + * Supported comment forms: `// line comment`, `/* block comment * /`. + * Supported strings: double-quoted ("…") and single-quoted ('…') with `\` + * escape handling. Raw/unterminated strings are not supported — we stop + * on a line break for line-style comments and on EOF for unterminated + * strings/blocks, which matches how most real proto files parse. + */ +function stripProtoCommentsAndStrings(content: string): string { + const out = new Array(content.length); + let i = 0; + while (i < content.length) { + const ch = content[i]; + const next = content[i + 1]; + + // Line comment: // ... \n + if (ch === '/' && next === '/') { + out[i] = ' '; + out[i + 1] = ' '; + i += 2; + while (i < content.length && content[i] !== '\n') { + out[i] = content[i] === '\r' ? '\r' : ' '; + i++; + } + continue; + } + + // Block comment: /* ... */ + if (ch === '/' && next === '*') { + out[i] = ' '; + out[i + 1] = ' '; + i += 2; + while (i < content.length) { + if (content[i] === '*' && content[i + 1] === '/') { + out[i] = ' '; + out[i + 1] = ' '; + i += 2; + break; + } + // Preserve newlines so line numbers stay stable for downstream code. + out[i] = content[i] === '\n' || content[i] === '\r' ? content[i] : ' '; + i++; + } + continue; + } + + // String literal: "..." or '...' + if (ch === '"' || ch === "'") { + const quote = ch; + out[i] = ' '; // replace opening quote + i++; + while (i < content.length) { + const c = content[i]; + if (c === '\\' && i + 1 < content.length) { + // Skip escaped pair (e.g. \" \n \\) + out[i] = ' '; + out[i + 1] = ' '; + i += 2; + continue; + } + if (c === quote) { + out[i] = ' '; + i++; + break; + } + // Preserve newlines; proto technically disallows unescaped newlines + // inside strings, but real files occasionally have them. + out[i] = c === '\n' || c === '\r' ? c : ' '; + i++; + } + continue; + } + + out[i] = ch; + i++; + } + return out.join(''); +} + function extractServiceBlocks(content: string): Array<{ name: string; body: string }> { const results: Array<{ name: string; body: string }> = []; - // v1: brace-depth only — braces inside comments or string literals are not filtered (see spec Fix 2) + // Sanitize comments and string literals so braces inside them don't + // throw off the depth counter. The sanitized copy has the same length + // and offsets as the original, so we use it ONLY to scan for service + // headers and braces; the service body we return is sliced from the + // ORIGINAL content to preserve exact source text for downstream use. + const sanitized = stripProtoCommentsAndStrings(content); const headerRe = /service\s+(\w+)\s*\{/g; let headerMatch: RegExpExecArray | null; - while ((headerMatch = headerRe.exec(content)) !== null) { + while ((headerMatch = headerRe.exec(sanitized)) !== null) { const serviceName = headerMatch[1]; const bodyStart = headerMatch.index + headerMatch[0].length; let depth = 1; let pos = bodyStart; - while (pos < content.length && depth > 0) { - const ch = content[pos]; + while (pos < sanitized.length && depth > 0) { + const ch = sanitized[pos]; if (ch === '{') depth++; else if (ch === '}') depth--; pos++; @@ -75,6 +165,163 @@ function makeContract( }; } +export interface ProtoServiceInfo { + package: string; + serviceName: string; + methods: string[]; + protoPath: string; +} + +function normalizeProtoPath(rel: string): string { + return rel.replace(/\\/g, '/'); +} + +function extractProtoImports(content: string): string[] { + const imports: string[] = []; + const re = /^\s*import\s+"([^"]+)"\s*;/gm; + let match: RegExpExecArray | null; + while ((match = re.exec(content)) !== null) { + imports.push(match[1]); + } + return imports; +} + +function longestSharedSegmentRun(aPath: string, bPath: string): number { + const a = aPath.split('/').filter(Boolean); + const b = bPath.split('/').filter(Boolean); + let best = 0; + + for (let i = 0; i < a.length; i++) { + for (let j = 0; j < b.length; j++) { + let run = 0; + while (a[i + run] && b[j + run] && a[i + run] === b[j + run]) { + run++; + } + if (run > best) best = run; + } + } + + return best; +} + +async function buildProtoContext(repoPath: string): Promise<{ + packagesByProto: Map; + servicesByName: Map; +}> { + const servicesByName = new Map(); + const protoFiles = await glob('**/*.proto', { + cwd: repoPath, + absolute: false, + nodir: true, + ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**'], + }); + const contents = new Map(); + + for (const rel of protoFiles) { + const content = readSafe(repoPath, rel); + if (!content) continue; + contents.set(normalizeProtoPath(rel), content); + } + + const packagesByProto = new Map(); + + const resolvePackage = (protoPath: string, seen = new Set()): string => { + if (packagesByProto.has(protoPath)) return packagesByProto.get(protoPath) ?? ''; + if (seen.has(protoPath)) return ''; + + const content = contents.get(protoPath); + if (!content) return ''; + + seen.add(protoPath); + const pkgMatch = content.match(/^\s*package\s+([\w.]+)\s*;/m); + if (pkgMatch?.[1]) { + packagesByProto.set(protoPath, pkgMatch[1]); + return pkgMatch[1]; + } + + for (const importPath of extractProtoImports(content)) { + const normalizedImport = normalizeProtoPath(importPath); + const candidates = [ + normalizeProtoPath( + path.posix.normalize(path.posix.join(path.posix.dirname(protoPath), normalizedImport)), + ), + normalizedImport, + ]; + for (const candidate of candidates) { + if (!contents.has(candidate)) continue; + const inheritedPackage = resolvePackage(candidate, seen); + if (inheritedPackage) { + packagesByProto.set(protoPath, inheritedPackage); + return inheritedPackage; + } + } + } + + packagesByProto.set(protoPath, ''); + return ''; + }; + + for (const rel of protoFiles) { + const normalizedRel = normalizeProtoPath(rel); + const content = contents.get(normalizedRel); + if (!content) continue; + const pkg = resolvePackage(normalizedRel); + + const serviceBlocks = extractServiceBlocks(content); + for (const block of serviceBlocks) { + const rpcRe = /rpc\s+(\w+)\s*\(/g; + const methods: string[] = []; + let m: RegExpExecArray | null; + while ((m = rpcRe.exec(block.body)) !== null) { + methods.push(m[1]); + } + const info: ProtoServiceInfo = { + package: pkg, + serviceName: block.name, + methods, + protoPath: normalizedRel, + }; + const existing = servicesByName.get(block.name) ?? []; + existing.push(info); + servicesByName.set(block.name, existing); + } + } + + return { packagesByProto, servicesByName }; +} + +export async function buildProtoMap(repoPath: string): Promise> { + const { servicesByName } = await buildProtoContext(repoPath); + return servicesByName; +} + +export function resolveProtoConflict( + _serviceName: string, + sourceFilePath: string, + candidates: ProtoServiceInfo[], +): ProtoServiceInfo | null { + if (candidates.length === 0) return null; + if (candidates.length === 1) return candidates[0]; + + const sourceDir = normalizeProtoPath(path.dirname(sourceFilePath)); + let best = candidates[0]; + let bestScore = -1; + for (const c of candidates) { + const protoDir = normalizeProtoPath(path.dirname(c.protoPath)); + const sharedRun = longestSharedSegmentRun(sourceDir, protoDir); + if (sharedRun > bestScore) { + bestScore = sharedRun; + best = c; + } + } + return best; +} + +export function serviceContractId(pkg: string, serviceName: string): string { + const prefix = pkg ? `${pkg}.${serviceName}` : serviceName; + return `grpc::${prefix}/*`; +} + export class GrpcExtractor implements ContractExtractor { type = 'grpc' as const; @@ -88,6 +335,7 @@ export class GrpcExtractor implements ContractExtractor { _repo: RepoHandle, ): Promise { const out: ExtractedContract[] = []; + const protoContext = await buildProtoContext(repoPath); // Proto files — definitive provider source const protoFiles = await glob('**/*.proto', { @@ -97,8 +345,17 @@ export class GrpcExtractor implements ContractExtractor { }); for (const rel of protoFiles) { const content = readSafe(repoPath, rel); - if (content) out.push(...this.parseProtoFile(content, rel)); + if (content) { + out.push( + ...this.parseProtoFile( + content, + rel, + protoContext.packagesByProto.get(normalizeProtoPath(rel)) ?? '', + ), + ); + } } + const protoMap = protoContext.servicesByName; // Source files — server/client detection const sourceFiles = await glob('**/*.{go,java,py,ts,tsx,js,jsx}', { @@ -112,28 +369,26 @@ export class GrpcExtractor implements ContractExtractor { const ext = path.extname(rel).toLowerCase(); if (ext === '.go') { - out.push(...this.scanGoProviders(content, rel)); - out.push(...this.scanGoConsumers(content, rel)); + out.push(...this.scanGoProviders(content, rel, protoMap)); + out.push(...this.scanGoConsumers(content, rel, protoMap)); } else if (ext === '.java') { - out.push(...this.scanJavaProviders(content, rel)); - out.push(...this.scanJavaConsumers(content, rel)); + out.push(...this.scanJavaProviders(content, rel, protoMap)); + out.push(...this.scanJavaConsumers(content, rel, protoMap)); } else if (ext === '.py') { - out.push(...this.scanPythonProviders(content, rel)); - out.push(...this.scanPythonConsumers(content, rel)); + out.push(...this.scanPythonProviders(content, rel, protoMap)); + out.push(...this.scanPythonConsumers(content, rel, protoMap)); } else if (['.ts', '.tsx', '.js', '.jsx'].includes(ext)) { - out.push(...this.scanTsProviders(content, rel)); + out.push(...this.scanTsProviders(content, rel, protoMap)); + out.push(...this.scanTsConsumers(content, rel, protoMap)); } } return this.dedupe(out); } - private parseProtoFile(content: string, filePath: string): ExtractedContract[] { + private parseProtoFile(content: string, filePath: string, pkg: string): ExtractedContract[] { const out: ExtractedContract[] = []; - const pkgMatch = content.match(/^package\s+([\w.]+)\s*;/m); - const pkg = pkgMatch ? pkgMatch[1] : ''; - for (const { name: serviceName, body } of extractServiceBlocks(content)) { const rpcRe = /rpc\s+(\w+)\s*\(/g; let rpcMatch: RegExpExecArray | null; @@ -154,7 +409,11 @@ export class GrpcExtractor implements ContractExtractor { return out; } - private scanGoProviders(content: string, filePath: string): ExtractedContract[] { + private scanGoProviders( + content: string, + filePath: string, + protoMap: Map, + ): ExtractedContract[] { const out: ExtractedContract[] = []; // pb.RegisterXxxServer( @@ -162,15 +421,17 @@ export class GrpcExtractor implements ContractExtractor { let m: RegExpExecArray | null; while ((m = registerRe.exec(content)) !== null) { const serviceName = m[1]; + const candidates = protoMap.get(serviceName); + const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); + const cid = proto + ? serviceContractId(proto.package, proto.serviceName) + : serviceOnlyContractId(serviceName); + const conf = proto ? 0.8 : 0.65; out.push( - makeContract( - serviceOnlyContractId(serviceName), - 'provider', - filePath, - `Register${serviceName}Server`, - 0.8, - { service: serviceName, source: 'go_register' }, - ), + makeContract(cid, 'provider', filePath, `Register${serviceName}Server`, conf, { + service: serviceName, + source: 'go_register', + }), ); } @@ -178,51 +439,74 @@ export class GrpcExtractor implements ContractExtractor { const unimplRe = /\w+\.Unimplemented(\w+)Server\b/g; while ((m = unimplRe.exec(content)) !== null) { const serviceName = m[1]; + const candidates = protoMap.get(serviceName); + const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); + const cid = proto + ? serviceContractId(proto.package, proto.serviceName) + : serviceOnlyContractId(serviceName); + const conf = proto ? 0.8 : 0.65; out.push( - makeContract( - serviceOnlyContractId(serviceName), - 'provider', - filePath, - `Unimplemented${serviceName}Server`, - 0.8, - { service: serviceName, source: 'go_unimplemented' }, - ), + makeContract(cid, 'provider', filePath, `Unimplemented${serviceName}Server`, conf, { + service: serviceName, + source: 'go_unimplemented', + }), ); } return out; } - private scanGoConsumers(content: string, filePath: string): ExtractedContract[] { + private scanGoConsumers( + content: string, + filePath: string, + protoMap: Map, + ): ExtractedContract[] { const out: ExtractedContract[] = []; const re = /\w+\.New(\w+)Client\s*\(/g; let m: RegExpExecArray | null; while ((m = re.exec(content)) !== null) { const serviceName = m[1]; + const candidates = protoMap.get(serviceName); + const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); + const cid = proto + ? serviceContractId(proto.package, proto.serviceName) + : serviceOnlyContractId(serviceName); + const conf = proto ? 0.75 : 0.55; out.push( - makeContract( - serviceOnlyContractId(serviceName), - 'consumer', - filePath, - `New${serviceName}Client`, - 0.7, - { service: serviceName, source: 'go_client' }, - ), + makeContract(cid, 'consumer', filePath, `New${serviceName}Client`, conf, { + service: serviceName, + source: 'go_client', + }), ); } return out; } - private scanJavaProviders(content: string, filePath: string): ExtractedContract[] { + private scanJavaProviders( + content: string, + filePath: string, + protoMap: Map, + ): ExtractedContract[] { const out: ExtractedContract[] = []; + const resolveJava = (svcName: string): { cid: string; conf: number } => { + const candidates = protoMap.get(svcName); + const proto = resolveProtoConflict(svcName, filePath, candidates ?? []); + const cid = proto + ? serviceContractId(proto.package, proto.serviceName) + : serviceOnlyContractId(svcName); + const conf = proto ? 0.8 : 0.65; + return { cid, conf }; + }; + // @GrpcService if (content.includes('@GrpcService')) { const implBaseRe = /extends\s+(\w+)Grpc\.(\w+)ImplBase/; const m = content.match(implBaseRe); if (m) { + const { cid, conf } = resolveJava(m[1]); out.push( - makeContract(serviceOnlyContractId(m[1]), 'provider', filePath, m[2], 0.8, { + makeContract(cid, 'provider', filePath, m[2], conf, { service: m[1], source: 'java_grpc_service', }), @@ -234,8 +518,9 @@ export class GrpcExtractor implements ContractExtractor { const cm = content.match(classRe); if (cm) { const svcName = cm[2].replace(/Grpc$/, ''); + const { cid, conf } = resolveJava(svcName); out.push( - makeContract(serviceOnlyContractId(svcName), 'provider', filePath, cm[1], 0.8, { + makeContract(cid, 'provider', filePath, cm[1], conf, { service: svcName, source: 'java_grpc_service', }), @@ -250,8 +535,9 @@ export class GrpcExtractor implements ContractExtractor { const m = content.match(implRe); if (m) { const svcName = m[2] || m[1].replace(/Grpc$/, ''); + const { cid, conf } = resolveJava(svcName); out.push( - makeContract(serviceOnlyContractId(svcName), 'provider', filePath, svcName, 0.8, { + makeContract(cid, 'provider', filePath, svcName, conf, { service: svcName, source: 'java_impl_base', }), @@ -262,49 +548,65 @@ export class GrpcExtractor implements ContractExtractor { return out; } - private scanJavaConsumers(content: string, filePath: string): ExtractedContract[] { + private scanJavaConsumers( + content: string, + filePath: string, + protoMap: Map, + ): ExtractedContract[] { const out: ExtractedContract[] = []; // XxxGrpc.newBlockingStub( or XxxGrpc.newStub( const re = /(\w+)Grpc\.new(?:Blocking)?Stub\s*\(/g; let m: RegExpExecArray | null; while ((m = re.exec(content)) !== null) { const serviceName = m[1]; + const candidates = protoMap.get(serviceName); + const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); + const cid = proto + ? serviceContractId(proto.package, proto.serviceName) + : serviceOnlyContractId(serviceName); + const conf = proto ? 0.75 : 0.55; out.push( - makeContract( - serviceOnlyContractId(serviceName), - 'consumer', - filePath, - `${serviceName}Stub`, - 0.7, - { service: serviceName, source: 'java_stub' }, - ), + makeContract(cid, 'consumer', filePath, `${serviceName}Stub`, conf, { + service: serviceName, + source: 'java_stub', + }), ); } return out; } - private scanPythonProviders(content: string, filePath: string): ExtractedContract[] { + private scanPythonProviders( + content: string, + filePath: string, + protoMap: Map, + ): ExtractedContract[] { const out: ExtractedContract[] = []; // add_XxxServicer_to_server( const re = /add_(\w+?)Servicer_to_server\s*\(/g; let m: RegExpExecArray | null; while ((m = re.exec(content)) !== null) { const serviceName = m[1]; + const candidates = protoMap.get(serviceName); + const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); + const cid = proto + ? serviceContractId(proto.package, proto.serviceName) + : serviceOnlyContractId(serviceName); + const conf = proto ? 0.8 : 0.65; out.push( - makeContract( - serviceOnlyContractId(serviceName), - 'provider', - filePath, - `add_${serviceName}Servicer_to_server`, - 0.8, - { service: serviceName, source: 'python_servicer' }, - ), + makeContract(cid, 'provider', filePath, `add_${serviceName}Servicer_to_server`, conf, { + service: serviceName, + source: 'python_servicer', + }), ); } return out; } - private scanPythonConsumers(content: string, filePath: string): ExtractedContract[] { + private scanPythonConsumers( + content: string, + filePath: string, + protoMap: Map, + ): ExtractedContract[] { const out: ExtractedContract[] = []; // XxxStub( const re = /(\w+)Stub\s*\(/g; @@ -313,8 +615,14 @@ export class GrpcExtractor implements ContractExtractor { const name = m[1]; // Filter out common false positives if (['Mock', 'Test', 'Fake', 'Stub'].includes(name)) continue; + const candidates = protoMap.get(name); + const proto = resolveProtoConflict(name, filePath, candidates ?? []); + const cid = proto + ? serviceContractId(proto.package, proto.serviceName) + : serviceOnlyContractId(name); + const conf = proto ? 0.75 : 0.55; out.push( - makeContract(serviceOnlyContractId(name), 'consumer', filePath, `${name}Stub`, 0.7, { + makeContract(cid, 'consumer', filePath, `${name}Stub`, conf, { service: name, source: 'python_stub', }), @@ -323,7 +631,11 @@ export class GrpcExtractor implements ContractExtractor { return out; } - private scanTsProviders(content: string, filePath: string): ExtractedContract[] { + private scanTsProviders( + content: string, + filePath: string, + protoMap: Map, + ): ExtractedContract[] { const out: ExtractedContract[] = []; // @GrpcMethod('ServiceName', 'MethodName') const re = /@GrpcMethod\s*\(\s*['"](\w+)['"]\s*,\s*['"](\w+)['"]\s*\)/g; @@ -331,7 +643,10 @@ export class GrpcExtractor implements ContractExtractor { while ((m = re.exec(content)) !== null) { const serviceName = m[1]; const methodName = m[2]; - const cid = contractId('', serviceName, methodName); + const candidates = protoMap.get(serviceName); + const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); + const pkg = proto?.package ?? ''; + const cid = contractId(pkg, serviceName, methodName); out.push( makeContract(cid, 'provider', filePath, `${serviceName}.${methodName}`, 0.8, { service: serviceName, @@ -343,15 +658,74 @@ export class GrpcExtractor implements ContractExtractor { return out; } - private dedupe(items: ExtractedContract[]): ExtractedContract[] { - const seen = new Set(); + private scanTsConsumers( + content: string, + filePath: string, + protoMap: Map, + ): ExtractedContract[] { const out: ExtractedContract[] = []; + const pushConsumer = ( + serviceName: string, + symbolName: string, + source: string, + confidenceWithProto = 0.75, + confidenceWithoutProto = 0.55, + ): void => { + const candidates = protoMap.get(serviceName); + const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); + const cid = proto + ? serviceContractId(proto.package, proto.serviceName) + : serviceOnlyContractId(serviceName); + const conf = proto ? confidenceWithProto : confidenceWithoutProto; + out.push( + makeContract(cid, 'consumer', filePath, symbolName, conf, { + service: serviceName, + source, + }), + ); + }; + + const grpcClientDecoratorRe = + /@GrpcClient\s*\([^)]*\)\s*(?:private|protected|public)?\s*(?:readonly\s+)?\w+[!?]?\s*:\s*(\w+Service)Client\b/g; + let match: RegExpExecArray | null; + while ((match = grpcClientDecoratorRe.exec(content)) !== null) { + pushConsumer(match[1], `${match[1]}Client`, 'ts_grpc_client_decorator'); + } + + const getServiceRe = /\.getService(?:<[^>]+>)?\s*\(\s*['"](\w+)['"]\s*\)/g; + while ((match = getServiceRe.exec(content)) !== null) { + pushConsumer(match[1], `${match[1]}Client`, 'ts_client_grpc_get_service'); + } + + const clientCtorRe = /new\s+(\w+Service)Client\s*\(/g; + while ((match = clientCtorRe.exec(content)) !== null) { + pushConsumer(match[1], `${match[1]}Client`, 'ts_generated_client'); + } + + if (content.includes('loadPackageDefinition')) { + const packageCtorRe = /new\s+[\w$.]*\.([A-Z]\w+)\s*\(/g; + while ((match = packageCtorRe.exec(content)) !== null) { + pushConsumer(match[1], `${match[1]}Client`, 'ts_load_package_definition'); + } + } + + return out; + } + + private dedupe(items: ExtractedContract[]): ExtractedContract[] { + const byKey = new Map(); for (const c of items) { const k = `${c.contractId}|${c.role}|${c.symbolRef.filePath}`; - if (seen.has(k)) continue; - seen.add(k); - out.push(c); + const existing = byKey.get(k); + if ( + !existing || + c.confidence > existing.confidence || + (c.confidence === existing.confidence && + String(c.meta.source) < String(existing.meta.source)) + ) { + byKey.set(k, c); + } } - return out; + return Array.from(byKey.values()); } } diff --git a/gitnexus/src/core/group/extractors/http-route-extractor.ts b/gitnexus/src/core/group/extractors/http-route-extractor.ts index ebb4c668d7..2a9f662c3b 100644 --- a/gitnexus/src/core/group/extractors/http-route-extractor.ts +++ b/gitnexus/src/core/group/extractors/http-route-extractor.ts @@ -226,7 +226,7 @@ export class HttpRouteExtractor implements ContractExtractor { } private async extractProvidersSourceScan(repoPath: string): Promise { - const files = await glob('**/*.{ts,tsx,js,jsx,java,vue,svelte,php,py}', { + const files = await glob('**/*.{ts,tsx,js,jsx,java,vue,svelte,php,py,go}', { cwd: repoPath, ignore: ['**/node_modules/**', '**/.git/**', '**/dist/**', '**/build/**'], nodir: true, @@ -236,7 +236,9 @@ export class HttpRouteExtractor implements ContractExtractor { const content = readSafe(repoPath, rel); if (!content) continue; out.push(...this.scanSpringProviders(content, rel)); + out.push(...this.scanNestProviders(content, rel)); out.push(...this.scanExpressProviders(content, rel)); + out.push(...this.scanGoProviders(content, rel)); out.push(...this.scanLaravelProviders(content, rel)); out.push(...this.scanFastApiProviders(content, rel)); } @@ -257,18 +259,6 @@ export class HttpRouteExtractor implements ContractExtractor { private scanSpringProviders(content: string, filePath: string): ExtractedContract[] { const out: ExtractedContract[] = []; - - // Skip Feign/client interfaces — annotated methods in interfaces are - // consumers (Feign, JAX-RS proxies), not provider endpoints. - // Anchored to line start (with optional access modifier) so we do not - // match "interface" inside comments or string literals. - if ( - /^\s*(?:public\s+)?interface\s+\w+/m.test(content) && - !/@(?:Rest)?Controller\b/.test(content) - ) { - return out; - } - let classPrefix = ''; const classRm = content.match(/@RequestMapping\s*\(\s*"([^"]+)"/); if (classRm) classPrefix = classRm[1].replace(/\/+$/, ''); @@ -324,6 +314,48 @@ export class HttpRouteExtractor implements ContractExtractor { return out; } + private scanNestProviders(content: string, filePath: string): ExtractedContract[] { + const out: ExtractedContract[] = []; + const controllerMatch = content.match(/@Controller\s*\(\s*['"`]([^'"`]+)['"`]\s*\)/); + const controllerPrefix = controllerMatch ? controllerMatch[1].replace(/\/+$/, '') : ''; + const re = /@(Get|Post|Put|Delete|Patch)\s*\(\s*['"`]?([^'"`)]*)['"`]?\s*\)/gi; + let m: RegExpExecArray | null; + while ((m = re.exec(content)) !== null) { + const method = m[1].toUpperCase(); + const routePath = String(m[2] || ''); + const fullPath = controllerPrefix + ? `${controllerPrefix}/${routePath.replace(/^\/+/, '')}` + : routePath; + const pathNorm = normalizeHttpPath(fullPath.startsWith('/') ? fullPath : `/${fullPath}`); + const sub = content.slice(m.index); + const nameMatch = sub.match( + /(?:public|protected|private)?\s*(?:async\s+)?(\w+)\s*\([^)]*\)\s*\{/, + ); + const name = nameMatch ? nameMatch[1] : m[0]; + out.push(this.makeProvider(filePath, method, pathNorm, name, 0.8)); + } + return out; + } + + private scanGoProviders(content: string, filePath: string): ExtractedContract[] { + const out: ExtractedContract[] = []; + const frameworkRe = + /(?:^|\W)\w+\.(GET|POST|PUT|DELETE|PATCH)\s*\(\s*['"]([^'"]+)['"]\s*,\s*(\w+)/gim; + let m: RegExpExecArray | null; + while ((m = frameworkRe.exec(content)) !== null) { + out.push(this.makeProvider(filePath, m[1].toUpperCase(), normalizeHttpPath(m[2]), m[3], 0.8)); + } + + const handleFuncRe = + /(?:http|\w+)\.HandleFunc\s*\(\s*['"]([^'"]+)['"]\s*,\s*(\w+)\s*\)(?:\s*\.\s*Methods\s*\(\s*['"](\w+)['"]\s*\))?/gim; + while ((m = handleFuncRe.exec(content)) !== null) { + const method = (m[3] || 'GET').toUpperCase(); + out.push(this.makeProvider(filePath, method, normalizeHttpPath(m[1]), m[2], 0.8)); + } + + return out; + } + private makeProvider( filePath: string, method: string, @@ -419,7 +451,7 @@ export class HttpRouteExtractor implements ContractExtractor { } private async extractConsumersSourceScan(repoPath: string): Promise { - const files = await glob('**/*.{ts,tsx,js,jsx,vue,svelte}', { + const files = await glob('**/*.{ts,tsx,js,jsx,vue,svelte,py,java,go}', { cwd: repoPath, ignore: ['**/node_modules/**', '**/.git/**'], nodir: true, @@ -430,6 +462,9 @@ export class HttpRouteExtractor implements ContractExtractor { if (!content) continue; out.push(...this.scanFetchConsumers(content, rel)); out.push(...this.scanAxiosConsumers(content, rel)); + out.push(...this.scanPythonRequestsConsumers(content, rel)); + out.push(...this.scanJavaConsumers(content, rel)); + out.push(...this.scanGoConsumers(content, rel)); } return this.dedupeContracts(out); } @@ -451,18 +486,127 @@ export class HttpRouteExtractor implements ContractExtractor { return url.replace(/\$\{[^}]+\}/g, '{param}'); } + private normalizeConsumerPath(url: string): string { + const templated = this.templateToPattern(url.trim()); + let pathOnly = templated; + if (/^https?:\/\//i.test(templated)) { + try { + pathOnly = new URL(templated).pathname; + } catch { + pathOnly = templated.replace(/^https?:\/\/[^/]+/i, ''); + } + } + + const normalized = normalizeHttpPath(pathOnly || '/'); + const segments = normalized + .split('/') + .filter(Boolean) + .map((segment) => { + if (/^\d+$/.test(segment)) return '{param}'; + return segment; + }); + return `/${segments.join('/')}`.replace(/\/+$/, '') || '/'; + } + private scanAxiosConsumers(content: string, filePath: string): ExtractedContract[] { const out: ExtractedContract[] = []; const re = /axios\.(get|post|put|delete|patch)\s*\(\s*[`'"]([^`'"]+)[`'"]/gi; let m: RegExpExecArray | null; while ((m = re.exec(content)) !== null) { const method = m[1].toUpperCase(); - const pathNorm = normalizeHttpPath(this.templateToPattern(m[2])); + const pathNorm = this.normalizeConsumerPath(m[2]); out.push(this.makeConsumer(filePath, method, pathNorm, 0.7)); } return out; } + private scanPythonRequestsConsumers(content: string, filePath: string): ExtractedContract[] { + const out: ExtractedContract[] = []; + const methodRe = /requests\.(get|post|put|delete|patch)\s*\(\s*['"]([^'"]+)['"]/gi; + let m: RegExpExecArray | null; + while ((m = methodRe.exec(content)) !== null) { + out.push( + this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), + ); + } + + const genericRe = /requests\.request\s*\(\s*['"](\w+)['"]\s*,\s*['"]([^'"]+)['"]/gi; + while ((m = genericRe.exec(content)) !== null) { + out.push( + this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), + ); + } + + return out; + } + + private scanJavaConsumers(content: string, filePath: string): ExtractedContract[] { + const out: ExtractedContract[] = []; + const restTemplateMethods: Array<[RegExp, string]> = [ + [/restTemplate\.getFor(?:Object|Entity)\s*\(\s*['"]([^'"]+)['"]/gi, 'GET'], + [/restTemplate\.postFor(?:Object|Entity)\s*\(\s*['"]([^'"]+)['"]/gi, 'POST'], + [/restTemplate\.put\s*\(\s*['"]([^'"]+)['"]/gi, 'PUT'], + [/restTemplate\.delete\s*\(\s*['"]([^'"]+)['"]/gi, 'DELETE'], + [/restTemplate\.patchForObject\s*\(\s*['"]([^'"]+)['"]/gi, 'PATCH'], + ]; + for (const [re, method] of restTemplateMethods) { + let m: RegExpExecArray | null; + while ((m = re.exec(content)) !== null) { + out.push(this.makeConsumer(filePath, method, this.normalizeConsumerPath(m[1]), 0.7)); + } + } + + const webClientMethodRe = + /webClient\.method\s*\(\s*HttpMethod\.(GET|POST|PUT|DELETE|PATCH)\s*,\s*['"]([^'"]+)['"]/gi; + let m: RegExpExecArray | null; + while ((m = webClientMethodRe.exec(content)) !== null) { + out.push( + this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), + ); + } + + const okHttpRe = + /new\s+Request\.Builder\s*\(\)\s*\.url\s*\(\s*['"]([^'"]+)['"]\s*\)(?:\s*\.\s*method\s*\(\s*['"](\w+)['"])?/gim; + while ((m = okHttpRe.exec(content)) !== null) { + out.push( + this.makeConsumer( + filePath, + (m[2] || 'GET').toUpperCase(), + this.normalizeConsumerPath(m[1]), + 0.7, + ), + ); + } + + return out; + } + + private scanGoConsumers(content: string, filePath: string): ExtractedContract[] { + const out: ExtractedContract[] = []; + const httpMethodRe = /\bhttp\.(Get|Post|Head)\s*\(\s*['"]([^'"]+)['"]/gi; + let m: RegExpExecArray | null; + while ((m = httpMethodRe.exec(content)) !== null) { + const method = m[1].toUpperCase() === 'HEAD' ? 'GET' : m[1].toUpperCase(); + out.push(this.makeConsumer(filePath, method, this.normalizeConsumerPath(m[2]), 0.7)); + } + + const newRequestRe = /\bhttp\.NewRequest\s*\(\s*['"](\w+)['"]\s*,\s*['"]([^'"]+)['"]/gi; + while ((m = newRequestRe.exec(content)) !== null) { + out.push( + this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), + ); + } + + const restyRe = /\b\w+\.R\(\)\.(Get|Post|Put|Delete|Patch)\s*\(\s*['"]([^'"]+)['"]/gi; + while ((m = restyRe.exec(content)) !== null) { + out.push( + this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), + ); + } + + return out; + } + private makeConsumer( filePath: string, method: string, diff --git a/gitnexus/src/core/group/extractors/manifest-extractor.ts b/gitnexus/src/core/group/extractors/manifest-extractor.ts new file mode 100644 index 0000000000..09817b356c --- /dev/null +++ b/gitnexus/src/core/group/extractors/manifest-extractor.ts @@ -0,0 +1,228 @@ +import type { ContractType, CrossLink, GroupManifestLink, StoredContract } from '../types.js'; +import type { CypherExecutor } from '../contract-extractor.js'; + +export interface ManifestExtractResult { + contracts: StoredContract[]; + crossLinks: CrossLink[]; +} + +/** + * Canonicalize an HTTP path for matching against Route.name in the graph. + * Mirrors core/ingestion/pipeline.ts ensureSlash semantics: + * - Ensures a leading slash. + * - Strips trailing slashes (except the root "/"). + * - Normalizes consecutive slashes. + * - Does NOT lowercase (route matching is case-sensitive). + */ +function normalizeRoutePath(raw: string): string { + const trimmed = raw.trim(); + if (!trimmed) return '/'; + const withLeading = trimmed.startsWith('/') ? trimmed : `/${trimmed}`; + const collapsed = withLeading.replace(/\/+/g, '/'); + if (collapsed === '/') return '/'; + return collapsed.replace(/\/+$/, ''); +} + +/** + * Stable synthetic symbolUid for a manifest-declared contract whose target + * symbol could not be resolved against the per-repo graph (resolveSymbol + * returned null). Two reasons we don't leave the uid empty: + * + * 1. The bridge stores Contract nodes keyed in part by symbolUid; an empty + * uid means downstream Cypher queries that anchor on `provider.symbolUid` + * can't tell two different unresolved manifest contracts apart. + * 2. The cross-impact bridge query in cross-impact.ts joins local impact + * results to bridge contracts via `WHERE provider.symbolUid IN $localUids`. + * If the local impact engine produces a deterministic identifier for the + * unresolved target, it must agree with the value the bridge stored. A + * synthetic uid keyed off (repo, contractId) is the only thing both sides + * can derive without knowing about each other. + * + * Format: `manifest::::`. Stable across syncs, scoped to a + * single repo within a group, and never collides with real indexer uids + * (which never start with `manifest::`). + */ +export function manifestSymbolUid(repo: string, contractId: string): string { + return `manifest::${repo}::${contractId}`; +} + +export class ManifestExtractor { + async extractFromManifest( + links: GroupManifestLink[], + dbExecutors?: Map, + ): Promise { + const contracts: StoredContract[] = []; + const crossLinks: CrossLink[] = []; + + for (const link of links) { + const contractId = this.buildContractId(link.type, link.contract); + + const providerRepo = link.role === 'provider' ? link.from : link.to; + const consumerRepo = link.role === 'provider' ? link.to : link.from; + + const providerSymbol = await this.resolveSymbol(providerRepo, link, dbExecutors); + const consumerSymbol = await this.resolveSymbol(consumerRepo, link, dbExecutors); + const providerRef = providerSymbol || { filePath: '', name: link.contract }; + const consumerRef = consumerSymbol || { filePath: '', name: link.contract }; + // When the resolver finds a real graph symbol we keep its uid, otherwise + // fall back to the deterministic synthetic uid (see manifestSymbolUid). + const providerUid = providerSymbol?.uid || manifestSymbolUid(providerRepo, contractId); + const consumerUid = consumerSymbol?.uid || manifestSymbolUid(consumerRepo, contractId); + + contracts.push({ + contractId, + type: link.type, + role: 'provider', + symbolUid: providerUid, + symbolRef: providerRef, + symbolName: link.contract, + confidence: 1.0, + meta: { source: 'manifest' }, + repo: providerRepo, + }); + + contracts.push({ + contractId, + type: link.type, + role: 'consumer', + symbolUid: consumerUid, + symbolRef: consumerRef, + symbolName: link.contract, + confidence: 1.0, + meta: { source: 'manifest' }, + repo: consumerRepo, + }); + + crossLinks.push({ + from: { repo: consumerRepo, symbolUid: consumerUid, symbolRef: consumerRef }, + to: { repo: providerRepo, symbolUid: providerUid, symbolRef: providerRef }, + type: link.type, + contractId, + matchType: 'manifest', + confidence: 1.0, + }); + } + + return { contracts, crossLinks }; + } + + private async resolveSymbol( + repoPathKey: string, + link: GroupManifestLink, + dbExecutors?: Map, + ): Promise<{ filePath: string; name: string; uid: string } | null> { + const executor = dbExecutors?.get(repoPathKey); + if (!executor) return null; + + // NOTE: All lookups use EXACT equality on the relevant name field and + // deterministic ORDER BY before LIMIT 1. Previous versions used CONTAINS + // for fuzzy matching (plus an unconditional ".proto" fallback for gRPC) + // which produced silent false positives: e.g. manifest "/orders" would + // match "/suborders", and a gRPC manifest entry in a repo with any + // .proto file would attach to a random proto symbol. + // + // If resolveSymbol returns null, the extractor creates a contract with + // an empty symbolUid/ref — cross-impact still works via name-based + // matching through the `hint` path in runGroupImpact. + try { + let rows: Record[]; + if (link.type === 'http') { + // Route.name is the canonicalized URL path (see + // core/ingestion/pipeline.ts ensureSlash + generateId('Route', ...)). + // Normalize the manifest contract the same way so a user-written + // "/api/orders" matches "api/orders" in the graph. + const normalized = normalizeRoutePath(link.contract); + rows = await executor( + `MATCH (handler)-[r:CodeRelation {type: 'HANDLES_ROUTE'}]->(route:Route) + WHERE route.name = $normalized + RETURN handler.id AS uid, handler.name AS name, handler.filePath AS filePath + ORDER BY handler.filePath ASC + LIMIT 1`, + { normalized }, + ); + } else if (link.type === 'topic') { + rows = await executor( + `MATCH (n) WHERE n.name = $contract + RETURN n.id AS uid, n.name AS name, n.filePath AS filePath + ORDER BY n.filePath ASC + LIMIT 1`, + { contract: link.contract }, + ); + } else if (link.type === 'grpc') { + // Contract is "Service/Method" or just "Service" (or package.Service + // variants). Prefer matching by method name when present, otherwise + // by service name. NO .proto path fallback — that's guaranteed to + // return a wrong symbol in any repo with more than one proto file. + const parts = link.contract.split('/'); + const serviceName = parts[0]?.trim() ?? ''; + const methodName = parts[1]?.trim() ?? ''; + if (methodName) { + rows = await executor( + `MATCH (n) WHERE n.name = $methodName + RETURN n.id AS uid, n.name AS name, n.filePath AS filePath + ORDER BY n.filePath ASC + LIMIT 1`, + { methodName }, + ); + } else if (serviceName) { + rows = await executor( + `MATCH (n) WHERE n.name = $serviceName + RETURN n.id AS uid, n.name AS name, n.filePath AS filePath + ORDER BY n.filePath ASC + LIMIT 1`, + { serviceName }, + ); + } else { + rows = []; + } + } else if (link.type === 'lib') { + // Only exact match on the symbol's name. Previous fallback to + // CONTAINS on n.filePath would promote "react" to "react-native" + // or "@types/react" — silent wrong attribution. + rows = await executor( + `MATCH (n) WHERE n.name = $contract + RETURN n.id AS uid, n.name AS name, n.filePath AS filePath + ORDER BY n.filePath ASC + LIMIT 1`, + { contract: link.contract }, + ); + } else { + return null; + } + if (rows.length > 0) { + return { + filePath: rows[0].filePath as string, + name: rows[0].name as string, + uid: String(rows[0].uid ?? ''), + }; + } + } catch (err) { + // Log but don't throw: a broken graph query in one repo shouldn't + // fail the whole manifest extraction. Unresolved contracts still + // get a synthetic symbolUid below, so cross-impact can proceed. + const message = err instanceof Error ? err.message : String(err); + console.warn( + `[manifest-extractor] resolveSymbol failed for ${link.type}:${link.contract} ` + + `in ${repoPathKey}: ${message}`, + ); + } + return null; + } + + private buildContractId(type: ContractType, contract: string): string { + switch (type) { + case 'http': { + if (/^[A-Za-z]+::/.test(contract)) return `http::${contract}`; + return `http::*::${contract}`; + } + case 'grpc': + return `grpc::${contract}`; + case 'topic': + return `topic::${contract}`; + case 'lib': + return `lib::${contract}`; + case 'custom': + return `custom::${contract}`; + } + } +} diff --git a/gitnexus/src/core/group/extractors/topic-extractor.ts b/gitnexus/src/core/group/extractors/topic-extractor.ts index c27b419bbd..3d5280d862 100644 --- a/gitnexus/src/core/group/extractors/topic-extractor.ts +++ b/gitnexus/src/core/group/extractors/topic-extractor.ts @@ -6,6 +6,9 @@ import type { ExtractedContract, RepoHandle } from '../types.js'; type Broker = 'kafka' | 'rabbitmq' | 'nats'; +const KAFKAJS_CONSUMER_RUN_RE = /consumer\.run\s*\(\s*\{\s*eachMessage:/; +const KAFKAJS_SUBSCRIBE_RE = /consumer\.subscribe\s*\(\s*\{\s*topic:\s*['"]([^'"]+)['"]/g; + function readSafe(repoPath: string, rel: string): string | null { const abs = path.resolve(repoPath, rel); const base = path.resolve(repoPath); @@ -116,6 +119,54 @@ const KAFKA_PATTERNS: PatternDef[] = [ topicGroup: 1, symbolName: 'producer.send', }, + // Go: sarama.ProducerMessage{Topic: "xxx"} struct literal (emitted by + // both NewSyncProducer and NewAsyncProducer client code paths). + // + // Previous pattern was `sarama.NewSyncProducer[\s\S]{0,300}?Topic:...` + // which anchored to the producer constructor and used a 300-char + // lookahead. In a loop like + // producer := sarama.NewSyncProducer(...) + // for _, item := range items { + // msg1 := &sarama.ProducerMessage{Topic: "order.created"} + // msg2 := &sarama.ProducerMessage{Topic: "order.shipped"} + // } + // the regex captured only "order.created" (first Topic after the + // constructor) and silently missed "order.shipped". Matching on the + // struct literal directly fixes both the false negative in loops and + // the spurious cross-message capture when multiple unrelated messages + // sit within 300 chars of the constructor. + { + regex: /sarama\.ProducerMessage\s*\{[\s\S]{0,200}?Topic:\s*"([^"]+)"/g, + role: 'provider', + broker: 'kafka', + confidence: 0.75, + topicGroup: 1, + symbolName: 'sarama.ProducerMessage', + }, + // Go: kafka-go writer construction. kafka-go does NOT wrap messages in + // a struct with a Topic field (the writer owns the topic), so we match + // the Writer itself. A 200-char window bridges the gap between + // `kafka.NewWriter(...)` / `kafka.Writer{` and the Topic field inside + // the config literal — kafka-go writer configs are small and rarely + // contain more than one Topic field, so the risk of cross-message + // capture is low here. + { + regex: /kafka\.(?:NewWriter|Writer)\b[\s\S]{0,200}?Topic:\s*"([^"]+)"/g, + role: 'provider', + broker: 'kafka', + confidence: 0.75, + topicGroup: 1, + symbolName: 'kafka.Writer', + }, + // Go: kafka-go reader construction, mirrors Writer above. + { + regex: /kafka\.(?:NewReader|Reader)\b[\s\S]{0,200}?Topic:\s*"([^"]+)"/g, + role: 'consumer', + broker: 'kafka', + confidence: 0.75, + topicGroup: 1, + symbolName: 'kafka.Reader', + }, ]; // --- RabbitMQ patterns --- @@ -205,6 +256,42 @@ const NATS_PATTERNS: PatternDef[] = [ topicGroup: 1, symbolName: 'nc.Publish', }, + // Go/Node JetStream: js.Subscribe("xxx" + { + regex: /js\.(?:S|s)ubscribe\s*\(\s*"([^"]+)"/g, + role: 'consumer', + broker: 'nats', + confidence: 0.8, + topicGroup: 1, + symbolName: 'js.Subscribe', + }, + // Go/Node JetStream: js.Publish("xxx" + { + regex: /js\.(?:P|p)ublish\s*\(\s*"([^"]+)"/g, + role: 'provider', + broker: 'nats', + confidence: 0.8, + topicGroup: 1, + symbolName: 'js.Publish', + }, + // Python: await nc.subscribe("xxx") + { + regex: /await\s+nc\.subscribe\s*\(\s*['"]([^'"]+)['"]/g, + role: 'consumer', + broker: 'nats', + confidence: 0.75, + topicGroup: 1, + symbolName: 'nc.subscribe', + }, + // Python: await nc.publish("xxx") + { + regex: /await\s+nc\.publish\s*\(\s*['"]([^'"]+)['"]/g, + role: 'provider', + broker: 'nats', + confidence: 0.75, + topicGroup: 1, + symbolName: 'nc.publish', + }, ]; const ALL_PATTERNS: PatternDef[] = [...KAFKA_PATTERNS, ...RABBITMQ_PATTERNS, ...NATS_PATTERNS]; @@ -229,6 +316,7 @@ export class TopicExtractor implements ContractExtractor { const out: ExtractedContract[] = []; for (const rel of files) { + if (rel.endsWith('_test.go')) continue; const content = readSafe(repoPath, rel); if (!content) continue; out.push(...this.scanFile(content, rel)); @@ -260,6 +348,16 @@ export class TopicExtractor implements ContractExtractor { } } + if (KAFKAJS_CONSUMER_RUN_RE.test(content)) { + const subscribeRe = new RegExp(KAFKAJS_SUBSCRIBE_RE.source, KAFKAJS_SUBSCRIBE_RE.flags); + let subscribeMatch: RegExpExecArray | null; + while ((subscribeMatch = subscribeRe.exec(content)) !== null) { + const topicName = subscribeMatch[1]; + if (!topicName) continue; + out.push(makeContract(topicName, 'consumer', filePath, 'consumer.run', 0.75, 'kafka')); + } + } + return out; } diff --git a/gitnexus/test/unit/group/grpc-extractor.test.ts b/gitnexus/test/unit/group/grpc-extractor.test.ts index b4fc63b5c7..82d79cbd69 100644 --- a/gitnexus/test/unit/group/grpc-extractor.test.ts +++ b/gitnexus/test/unit/group/grpc-extractor.test.ts @@ -1,17 +1,23 @@ import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import * as fs from 'node:fs'; +import fsp from 'node:fs/promises'; import * as path from 'node:path'; import * as os from 'node:os'; -import { GrpcExtractor } from '../../../src/core/group/extractors/grpc-extractor.js'; +import { + GrpcExtractor, + buildProtoMap, + resolveProtoConflict, + serviceContractId, +} from '../../../src/core/group/extractors/grpc-extractor.js'; +import type { ProtoServiceInfo } from '../../../src/core/group/extractors/grpc-extractor.js'; import type { RepoHandle } from '../../../src/core/group/types.js'; describe('GrpcExtractor', () => { let tmpDir: string; let extractor: GrpcExtractor; - beforeEach(() => { - tmpDir = path.join(os.tmpdir(), `gitnexus-grpc-${Date.now()}`); - fs.mkdirSync(tmpDir, { recursive: true }); + beforeEach(async () => { + tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), 'gitnexus-grpc-')); extractor = new GrpcExtractor(); }); @@ -205,6 +211,66 @@ service IncompleteService { // The old regex would find partial match; the new parser should skip it expect(providers).toHaveLength(0); }); + + it('test_extract_proto_ignores_braces_inside_string_literals', async () => { + // Regression for a known parser limitation: braces inside string + // literals used to be counted as real service-body braces, which + // would terminate the service early and drop methods after the + // offending string. + writeFile( + 'api/strings.proto', + `syntax = "proto3"; +package strings; + +service TrickyService { + rpc First (Req) returns (Res) { + option (google.api.http).additional_bindings = { + post: "/v1/first"; + }; + } + // Previously the "{" inside this literal would close the service body. + option deprecated_reason = "use NewService { instead"; + rpc Second (Req) returns (Res); + rpc Third (Req) returns (Res); +} +`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const protoProviders = contracts.filter( + (c) => c.role === 'provider' && c.symbolRef.filePath === 'api/strings.proto', + ); + // All three methods must be extracted even though a string literal + // contains an unbalanced "{". + expect(protoProviders.map((c) => c.symbolName).sort()).toEqual([ + 'TrickyService.First', + 'TrickyService.Second', + 'TrickyService.Third', + ]); + }); + + it('test_extract_proto_ignores_braces_inside_comments', async () => { + writeFile( + 'api/commented.proto', + `syntax = "proto3"; +package commented; + +service Svc { + // TODO: move { or } from this comment — parser used to count them + /* A block comment with { unbalanced braces } */ + rpc Alpha (Req) returns (Res); + // }} end of the method block (in comment) + rpc Beta (Req) returns (Res); +} +`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const protoProviders = contracts.filter( + (c) => c.role === 'provider' && c.symbolRef.filePath === 'api/commented.proto', + ); + expect(protoProviders.map((c) => c.symbolName).sort()).toEqual(['Svc.Alpha', 'Svc.Beta']); + }); }); describe('Go server detection', () => { @@ -228,7 +294,7 @@ func main() { expect(providers.length).toBeGreaterThanOrEqual(1); expect(providers[0].contractId).toContain('grpc::'); expect(providers[0].contractId).toContain('AuthService'); - expect(providers[0].confidence).toBe(0.8); + expect(providers[0].confidence).toBe(0.65); }); it('test_extract_go_unimplemented_server_returns_provider', async () => { @@ -267,7 +333,7 @@ func NewAuthClient(conn *grpc.ClientConn) pb.AuthServiceClient { expect(consumers.length).toBeGreaterThanOrEqual(1); expect(consumers[0].contractId).toContain('AuthService'); - expect(consumers[0].confidence).toBe(0.7); + expect(consumers[0].confidence).toBe(0.55); }); }); @@ -287,7 +353,7 @@ public class AuthGrpcService extends AuthServiceGrpc.AuthServiceImplBase { expect(providers.length).toBeGreaterThanOrEqual(1); expect(providers[0].contractId).toContain('AuthService'); - expect(providers[0].confidence).toBe(0.8); + expect(providers[0].confidence).toBe(0.65); }); it('test_extract_java_blocking_stub_returns_consumer', async () => { @@ -306,7 +372,7 @@ public class AuthGrpcService extends AuthServiceGrpc.AuthServiceImplBase { expect(consumers.length).toBeGreaterThanOrEqual(1); expect(consumers[0].contractId).toContain('AuthService'); - expect(consumers[0].confidence).toBe(0.7); + expect(consumers[0].confidence).toBe(0.55); }); }); @@ -328,7 +394,7 @@ def serve(): expect(providers.length).toBeGreaterThanOrEqual(1); expect(providers[0].contractId).toContain('AuthService'); - expect(providers[0].confidence).toBe(0.8); + expect(providers[0].confidence).toBe(0.65); }); it('test_extract_python_stub_returns_consumer', async () => { @@ -346,7 +412,7 @@ stub = auth_pb2_grpc.AuthServiceStub(channel)`, expect(consumers.length).toBeGreaterThanOrEqual(1); expect(consumers[0].contractId).toContain('AuthService'); - expect(consumers[0].confidence).toBe(0.7); + expect(consumers[0].confidence).toBe(0.55); }); }); @@ -372,6 +438,165 @@ export class AuthController { expect(providers[0].contractId).toContain('Login'); expect(providers[0].confidence).toBe(0.8); }); + + it('test_extract_ts_grpc_client_decorator_returns_consumer', async () => { + writeFile( + 'proto/auth.proto', + `syntax = "proto3"; +package auth.v1; +service AuthService { + rpc Login (LoginRequest) returns (LoginResponse); +}`, + ); + writeFile( + 'src/auth.client.ts', + `import { GrpcClient } from '@nestjs/microservices'; +import type { AuthServiceClient } from './generated/auth'; + +export class AuthGateway { + @GrpcClient({ package: 'auth.v1', protoPath: 'proto/auth.proto' }) + private readonly authClient!: AuthServiceClient; +}`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(1); + expect(consumers[0].contractId).toBe('grpc::auth.v1.AuthService/*'); + }); + + it('test_extract_ts_getService_without_decorator_returns_consumer', async () => { + writeFile( + 'proto/auth.proto', + `syntax = "proto3"; +package auth.v1; +service AuthService { + rpc Login (LoginRequest) returns (LoginResponse); +}`, + ); + writeFile( + 'src/auth.client.ts', + `import type { ClientGrpc } from '@nestjs/microservices'; + +export function createAuthClient(client: ClientGrpc) { + return client.getService('AuthService'); +}`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(1); + expect(consumers[0].contractId).toBe('grpc::auth.v1.AuthService/*'); + }); + + it('test_extract_ts_generated_client_constructor_returns_consumer', async () => { + writeFile( + 'proto/auth.proto', + `syntax = "proto3"; +package auth.v1; +service AuthService { + rpc Login (LoginRequest) returns (LoginResponse); +}`, + ); + writeFile( + 'src/auth.client.ts', + `import { credentials } from '@grpc/grpc-js'; +import { AuthServiceClient } from './generated/auth'; + +export const authClient = new AuthServiceClient('localhost:50051', credentials.createInsecure());`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(1); + expect(consumers[0].contractId).toBe('grpc::auth.v1.AuthService/*'); + }); + + it('test_extract_ts_non_service_client_constructor_is_ignored', async () => { + writeFile( + 'proto/auth.proto', + `syntax = "proto3"; +package auth.v1; +service AuthService { + rpc Login (LoginRequest) returns (LoginResponse); +}`, + ); + writeFile( + 'src/auth.client.ts', + `import { AuthClient } from './generated/auth'; + +export const authClient = new AuthClient('localhost:50051');`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(0); + }); + + it('test_extract_ts_loadPackageDefinition_constructor_returns_consumer', async () => { + writeFile( + 'proto/auth.proto', + `syntax = "proto3"; +package auth.v1; +service AuthService { + rpc Login (LoginRequest) returns (LoginResponse); +}`, + ); + writeFile( + 'src/auth.client.ts', + `import * as grpc from '@grpc/grpc-js'; +import * as protoLoader from '@grpc/proto-loader'; + +const definition = protoLoader.loadSync('proto/auth.proto'); +const authProto = grpc.loadPackageDefinition(definition) as any; +export const authClient = new authProto.auth.v1.AuthService( + 'localhost:50051', + grpc.credentials.createInsecure(), +);`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(1); + expect(consumers[0].contractId).toBe('grpc::auth.v1.AuthService/*'); + }); + + it('test_extract_ts_duplicate_consumer_patterns_in_one_file_dedupes_deterministically', async () => { + writeFile( + 'proto/auth.proto', + `syntax = "proto3"; +package auth.v1; +service AuthService { + rpc Login (LoginRequest) returns (LoginResponse); +}`, + ); + writeFile( + 'src/auth.client.ts', + `import * as grpc from '@grpc/grpc-js'; +import type { ClientGrpc } from '@nestjs/microservices'; +import { AuthServiceClient } from './generated/auth'; + +export class AuthGateway { + constructor(private readonly client: ClientGrpc) {} + + connect() { + this.client.getService('AuthService'); + return new AuthServiceClient('localhost:50051', grpc.credentials.createInsecure()); + } +}`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(1); + expect(consumers[0].contractId).toBe('grpc::auth.v1.AuthService/*'); + }); }); describe('edge cases', () => { @@ -389,3 +614,297 @@ export class AuthController { }); }); }); + +describe('buildProtoMap', () => { + let tmpDir: string; + beforeEach(async () => { + tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), 'proto-test-')); + }); + afterEach(async () => { + await fsp.rm(tmpDir, { recursive: true, force: true }); + }); + + it('test_buildProtoMap_single_proto_parses_package_service_methods', async () => { + const protoContent = ` +syntax = "proto3"; +package com.example; + +service UserService { + rpc GetUser (GetUserRequest) returns (GetUserResponse); + rpc ListUsers (ListUsersRequest) returns (ListUsersResponse); +}`; + await fsp.mkdir(path.join(tmpDir, 'proto'), { recursive: true }); + await fsp.writeFile(path.join(tmpDir, 'proto', 'user.proto'), protoContent); + + const map = await buildProtoMap(tmpDir); + expect(map.has('UserService')).toBe(true); + const entries = map.get('UserService')!; + expect(entries).toHaveLength(1); + expect(entries[0].package).toBe('com.example'); + expect(entries[0].serviceName).toBe('UserService'); + expect(entries[0].methods).toEqual(['GetUser', 'ListUsers']); + expect(entries[0].protoPath).toBe('proto/user.proto'); + }); + + it('test_buildProtoMap_no_package_declaration', async () => { + const protoContent = ` +syntax = "proto3"; +service Foo { rpc Bar (Req) returns (Res); }`; + await fsp.writeFile(path.join(tmpDir, 'foo.proto'), protoContent); + + const map = await buildProtoMap(tmpDir); + const entries = map.get('Foo')!; + expect(entries[0].package).toBe(''); + }); + + it('test_buildProtoMap_no_protos_returns_empty', async () => { + const map = await buildProtoMap(tmpDir); + expect(map.size).toBe(0); + }); + + it('test_buildProtoMap_conflicting_names', async () => { + await fsp.mkdir(path.join(tmpDir, 'a'), { recursive: true }); + await fsp.mkdir(path.join(tmpDir, 'b'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'a', 'svc.proto'), + 'package pkg.a;\nservice Svc { rpc Do (R) returns (R); }', + ); + await fsp.writeFile( + path.join(tmpDir, 'b', 'svc.proto'), + 'package pkg.b;\nservice Svc { rpc Do (R) returns (R); }', + ); + + const map = await buildProtoMap(tmpDir); + expect(map.get('Svc')).toHaveLength(2); + }); + + it('test_buildProtoMap_imported_package_is_inherited_for_split_service_definition', async () => { + await fsp.mkdir(path.join(tmpDir, 'proto', 'shared'), { recursive: true }); + await fsp.mkdir(path.join(tmpDir, 'proto', 'services'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'proto', 'shared', 'package.proto'), + 'package auth.v1;\nmessage LoginRequest {}', + ); + await fsp.writeFile( + path.join(tmpDir, 'proto', 'services', 'auth.proto'), + 'import "../shared/package.proto";\nservice AuthService { rpc Login (LoginRequest) returns (LoginRequest); }', + ); + + const map = await buildProtoMap(tmpDir); + const entries = map.get('AuthService')!; + + expect(entries).toHaveLength(1); + expect(entries[0].package).toBe('auth.v1'); + }); +}); + +describe('resolveProtoConflict', () => { + const makeInfo = (pkg: string, protoPath: string): ProtoServiceInfo => ({ + package: pkg, + serviceName: 'Svc', + methods: ['Do'], + protoPath, + }); + + it('test_single_candidate_returns_it', () => { + const result = resolveProtoConflict('Svc', 'src/main.go', [makeInfo('pkg', 'proto/svc.proto')]); + expect(result?.package).toBe('pkg'); + }); + + it('test_multiple_candidates_picks_closest_directory', () => { + const candidates = [ + makeInfo('far', 'other/dir/svc.proto'), + makeInfo('close', 'src/proto/svc.proto'), + ]; + const result = resolveProtoConflict('Svc', 'src/server.go', candidates); + expect(result?.package).toBe('close'); + }); + + it('test_centralized_proto_layout_prefers_shared_path_segments_over_prefix_only', () => { + const candidates = [ + makeInfo('billing', 'proto/services/billing/svc.proto'), + makeInfo('auth', 'proto/services/auth/svc.proto'), + ]; + const result = resolveProtoConflict('Svc', 'services/auth/src/server.ts', candidates); + expect(result?.package).toBe('auth'); + }); + + it('test_no_candidates_returns_null', () => { + expect(resolveProtoConflict('Svc', 'src/main.go', [])).toBeNull(); + }); +}); + +describe('serviceContractId', () => { + it('test_with_package', () => { + expect(serviceContractId('com.example', 'UserService')).toBe('grpc::com.example.UserService/*'); + }); + + it('test_without_package', () => { + expect(serviceContractId('', 'UserService')).toBe('grpc::UserService/*'); + }); +}); + +describe('proto-aware source scanners', () => { + let tmpDir: string; + let extractor: GrpcExtractor; + + beforeEach(async () => { + tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), 'scanner-test-')); + extractor = new GrpcExtractor(); + }); + afterEach(async () => { + await fsp.rm(tmpDir, { recursive: true, force: true }); + }); + + const makeRepo = (repoPath: string): RepoHandle => ({ + id: 'test-repo', + path: '', + repoPath, + storagePath: '', + }); + + it('test_go_provider_with_proto_uses_canonical_service_id', async () => { + await fsp.mkdir(path.join(tmpDir, 'proto'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'proto', 'user.proto'), + 'package com.example;\nservice UserService { rpc GetUser (R) returns (R); }', + ); + await fsp.mkdir(path.join(tmpDir, 'src'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'src', 'server.go'), + 'package main\nfunc init() { pb.RegisterUserServiceServer(srv, &impl{}) }', + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + + const goProvider = contracts.find((c) => c.meta.source === 'go_register'); + expect(goProvider).toBeDefined(); + expect(goProvider!.contractId).toBe('grpc::com.example.UserService/*'); + expect(goProvider!.confidence).toBe(0.8); + }); + + it('test_go_provider_without_proto_reduced_confidence', async () => { + await fsp.mkdir(path.join(tmpDir, 'src'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'src', 'server.go'), + 'package main\nfunc init() { pb.RegisterFooServer(srv, &impl{}) }', + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + + const goProvider = contracts.find((c) => c.meta.source === 'go_register'); + expect(goProvider).toBeDefined(); + expect(goProvider!.contractId).toBe('grpc::Foo/*'); + expect(goProvider!.confidence).toBe(0.65); + }); + + it('test_go_consumer_with_proto_uses_canonical_service_id', async () => { + await fsp.mkdir(path.join(tmpDir, 'proto'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'proto', 'user.proto'), + 'package com.example;\nservice UserService { rpc GetUser (R) returns (R); }', + ); + await fsp.mkdir(path.join(tmpDir, 'src'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'src', 'client.go'), + 'package main\nfunc init() { client := pb.NewUserServiceClient(conn) }', + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + + const goConsumer = contracts.find((c) => c.meta.source === 'go_client'); + expect(goConsumer).toBeDefined(); + expect(goConsumer!.contractId).toBe('grpc::com.example.UserService/*'); + expect(goConsumer!.confidence).toBe(0.75); + }); + + it('test_java_provider_with_proto_uses_canonical_service_id', async () => { + await fsp.mkdir(path.join(tmpDir, 'proto'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'proto', 'user.proto'), + 'package com.example;\nservice UserService { rpc GetUser (R) returns (R); }', + ); + await fsp.mkdir(path.join(tmpDir, 'src', 'main', 'java'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'src', 'main', 'java', 'UserGrpcService.java'), + `@GrpcService +public class UserGrpcService extends UserServiceGrpc.UserServiceImplBase { + @Override + public void getUser(GetUserRequest req, StreamObserver obs) {} +}`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + + const javaProvider = contracts.find((c) => c.meta.source === 'java_grpc_service'); + expect(javaProvider).toBeDefined(); + expect(javaProvider!.contractId).toBe('grpc::com.example.UserService/*'); + expect(javaProvider!.confidence).toBe(0.8); + }); + + it('test_python_consumer_with_proto_uses_canonical_service_id', async () => { + await fsp.mkdir(path.join(tmpDir, 'proto'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'proto', 'user.proto'), + 'package com.example;\nservice UserService { rpc GetUser (R) returns (R); }', + ); + await fsp.writeFile( + path.join(tmpDir, 'client.py'), + `import grpc +channel = grpc.insecure_channel('localhost:50051') +stub = UserServiceStub(channel)`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + + const pyConsumer = contracts.find((c) => c.meta.source === 'python_stub'); + expect(pyConsumer).toBeDefined(); + expect(pyConsumer!.contractId).toBe('grpc::com.example.UserService/*'); + expect(pyConsumer!.confidence).toBe(0.75); + }); + + it('test_ts_provider_with_proto_adds_package', async () => { + await fsp.mkdir(path.join(tmpDir, 'proto'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'proto', 'user.proto'), + 'package com.example;\nservice UserService { rpc GetUser (R) returns (R); }', + ); + await fsp.mkdir(path.join(tmpDir, 'src'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'src', 'controller.ts'), + "@GrpcMethod('UserService', 'GetUser')\nasync getUser() {}", + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + + const tsProvider = contracts.find((c) => c.meta.source === 'ts_grpc_method'); + expect(tsProvider).toBeDefined(); + expect(tsProvider!.contractId).toBe('grpc::com.example.UserService/GetUser'); + expect(tsProvider!.confidence).toBe(0.8); + }); + + it('test_proto_provider_inherits_package_from_imported_definition', async () => { + await fsp.mkdir(path.join(tmpDir, 'proto', 'shared'), { recursive: true }); + await fsp.mkdir(path.join(tmpDir, 'proto', 'services'), { recursive: true }); + await fsp.writeFile( + path.join(tmpDir, 'proto', 'shared', 'package.proto'), + 'package auth.v1;\nmessage LoginRequest {}', + ); + await fsp.writeFile( + path.join(tmpDir, 'proto', 'services', 'auth.proto'), + `syntax = "proto3"; +import "../shared/package.proto"; +service AuthService { + rpc Login (LoginRequest) returns (LoginRequest); +}`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + + const protoProvider = contracts.find( + (c) => c.symbolRef.filePath === 'proto/services/auth.proto', + ); + expect(protoProvider).toBeDefined(); + expect(protoProvider!.contractId).toBe('grpc::auth.v1.AuthService/Login'); + }); +}); diff --git a/gitnexus/test/unit/group/http-route-extractor.test.ts b/gitnexus/test/unit/group/http-route-extractor.test.ts index d4c0db3ebc..653b4952c4 100644 --- a/gitnexus/test/unit/group/http-route-extractor.test.ts +++ b/gitnexus/test/unit/group/http-route-extractor.test.ts @@ -157,6 +157,89 @@ export default router; providers.find((c) => c.contractId === 'http::DELETE::/api/users/{param}'), ).toBeDefined(); }); + + it('extracts Go Gin and Echo route registrations', async () => { + const dir = path.join(tmpDir, 'go-frameworks'); + fs.mkdirSync(path.join(dir, 'cmd'), { recursive: true }); + fs.writeFileSync( + path.join(dir, 'cmd', 'server.go'), + ` +package main + +func createOrder(c *gin.Context) {} +func listOrders(c echo.Context) error { return nil } + +func main() { + r := gin.Default() + r.POST("/api/orders/:id", createOrder) + + e := echo.New() + e.GET("/api/orders", listOrders) +} +`, + ); + + const contracts = await extractor.extract(null, dir, makeRepo(dir)); + const providers = contracts.filter((c) => c.role === 'provider'); + + const ginRoute = providers.find((c) => c.contractId === 'http::POST::/api/orders/{param}'); + expect(ginRoute).toBeDefined(); + expect(ginRoute?.symbolName).toBe('createOrder'); + + const echoRoute = providers.find((c) => c.contractId === 'http::GET::/api/orders'); + expect(echoRoute).toBeDefined(); + expect(echoRoute?.symbolName).toBe('listOrders'); + }); + + it('extracts stdlib HandleFunc providers', async () => { + const dir = path.join(tmpDir, 'go-stdlib-provider'); + fs.mkdirSync(path.join(dir, 'cmd'), { recursive: true }); + fs.writeFileSync( + path.join(dir, 'cmd', 'server.go'), + ` +package main + +func healthHandler(w http.ResponseWriter, r *http.Request) {} + +func main() { + http.HandleFunc("/api/health", healthHandler) +} +`, + ); + + const contracts = await extractor.extract(null, dir, makeRepo(dir)); + const providers = contracts.filter((c) => c.role === 'provider'); + + const healthRoute = providers.find((c) => c.contractId === 'http::GET::/api/health'); + expect(healthRoute).toBeDefined(); + expect(healthRoute?.symbolName).toBe('healthHandler'); + }); + + it('extracts NestJS controller decorators', async () => { + const dir = path.join(tmpDir, 'nestjs'); + fs.mkdirSync(path.join(dir, 'src'), { recursive: true }); + fs.writeFileSync( + path.join(dir, 'src', 'orders.controller.ts'), + ` +import { Controller, Patch } from '@nestjs/common'; + +@Controller('orders') +export class OrdersController { + @Patch(':id') + updateOrder() { + return {}; + } +} +`, + ); + + const contracts = await extractor.extract(null, dir, makeRepo(dir)); + const providers = contracts.filter((c) => c.role === 'provider'); + + const patchRoute = providers.find((c) => c.contractId === 'http::PATCH::/orders/{param}'); + expect(patchRoute).toBeDefined(); + expect(patchRoute?.symbolName).toBe('updateOrder'); + }); }); describe('consumer extraction — fetch patterns', () => { @@ -206,6 +289,91 @@ export const deleteUser = (id: string) => axios.delete(\`/api/users/\${id}\`); consumers.find((c) => c.contractId === 'http::DELETE::/api/users/{param}'), ).toBeDefined(); }); + + it('extracts Python requests calls', async () => { + const dir = path.join(tmpDir, 'python-consumer'); + fs.mkdirSync(path.join(dir, 'src'), { recursive: true }); + fs.writeFileSync( + path.join(dir, 'src', 'client.py'), + ` +import requests + +def create_order(): + return requests.post("https://svc.local/api/orders/42", json={"id": 42}) +`, + ); + + const contracts = await extractor.extract(null, dir, makeRepo(dir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect( + consumers.find((c) => c.contractId === 'http::POST::/api/orders/{param}'), + ).toBeDefined(); + }); + + it('extracts Java RestTemplate, WebClient and OkHttp calls', async () => { + const dir = path.join(tmpDir, 'java-consumer'); + fs.mkdirSync(path.join(dir, 'src'), { recursive: true }); + fs.writeFileSync( + path.join(dir, 'src', 'ApiClient.java'), + ` +import org.springframework.http.HttpMethod; +import org.springframework.web.client.RestTemplate; +import org.springframework.web.reactive.function.client.WebClient; +import okhttp3.Request; + +class ApiClient { + void run(RestTemplate restTemplate, WebClient webClient) { + restTemplate.getForObject("/api/users/{id}", String.class, 42); + webClient.method(HttpMethod.PATCH, "/api/users/42"); + new Request.Builder().url("/api/orders/42").build(); + } +} +`, + ); + + const contracts = await extractor.extract(null, dir, makeRepo(dir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers.find((c) => c.contractId === 'http::GET::/api/users/{param}')).toBeDefined(); + expect( + consumers.find((c) => c.contractId === 'http::PATCH::/api/users/{param}'), + ).toBeDefined(); + expect( + consumers.find((c) => c.contractId === 'http::GET::/api/orders/{param}'), + ).toBeDefined(); + }); + + it('extracts Go stdlib and resty calls', async () => { + const dir = path.join(tmpDir, 'go-consumer'); + fs.mkdirSync(path.join(dir, 'cmd'), { recursive: true }); + fs.writeFileSync( + path.join(dir, 'cmd', 'client.go'), + ` +package main + +import ( + "net/http" + + "github.com/go-resty/resty/v2" +) + +func main() { + http.Get("/api/health") + client := resty.New() + client.R().Delete("/api/orders/42") +} +`, + ); + + const contracts = await extractor.extract(null, dir, makeRepo(dir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers.find((c) => c.contractId === 'http::GET::/api/health')).toBeDefined(); + expect( + consumers.find((c) => c.contractId === 'http::DELETE::/api/orders/{param}'), + ).toBeDefined(); + }); }); describe('provider extraction — Laravel', () => { @@ -326,78 +494,6 @@ async def create_user(user: UserCreate): }); }); - describe('interface regex anchoring', () => { - it('skips Feign client interfaces (no @Controller)', async () => { - const dir = path.join(tmpDir, 'feign-skip'); - fs.mkdirSync(path.join(dir, 'src'), { recursive: true }); - fs.writeFileSync( - path.join(dir, 'src/UserClient.java'), - ` -package com.example; -@FeignClient(name = "user-service") -public interface UserClient { - @GetMapping("/users") - List getUsers(); -} -`, - ); - const contracts = await extractor.extract(null, dir, makeRepo(dir)); - expect(contracts.filter((c) => c.role === 'provider')).toHaveLength(0); - }); - - it('does NOT skip when @RestController is present', async () => { - const dir = path.join(tmpDir, 'ctrl-iface'); - fs.mkdirSync(path.join(dir, 'src'), { recursive: true }); - fs.writeFileSync( - path.join(dir, 'src/UserController.java'), - ` -@RestController -@RequestMapping("/api") -public class UserController { - @GetMapping("/users") - public List list() { return null; } -} -`, - ); - const contracts = await extractor.extract(null, dir, makeRepo(dir)); - expect(contracts.filter((c) => c.role === 'provider').length).toBeGreaterThanOrEqual(1); - }); - - it('does NOT false-positive on interface in comments', async () => { - const dir = path.join(tmpDir, 'iface-comment'); - fs.mkdirSync(path.join(dir, 'src'), { recursive: true }); - fs.writeFileSync( - path.join(dir, 'src/Api.java'), - ` -// implements the interface UserApi -public class Api { - @GetMapping("/health") - public String health() { return "ok"; } -} -`, - ); - const contracts = await extractor.extract(null, dir, makeRepo(dir)); - expect(contracts.filter((c) => c.role === 'provider').length).toBeGreaterThanOrEqual(1); - }); - - it('does NOT false-positive on interface in a string', async () => { - const dir = path.join(tmpDir, 'iface-str'); - fs.mkdirSync(path.join(dir, 'src'), { recursive: true }); - fs.writeFileSync( - path.join(dir, 'src/Svc.java'), - ` -public class Svc { - String desc = "implements interface Foo"; - @GetMapping("/status") - public String status() { return desc; } -} -`, - ); - const contracts = await extractor.extract(null, dir, makeRepo(dir)); - expect(contracts.filter((c) => c.role === 'provider').length).toBeGreaterThanOrEqual(1); - }); - }); - describe('path normalization', () => { it('strips trailing slash', async () => { const dir = path.join(tmpDir, 'trailing'); diff --git a/gitnexus/test/unit/group/manifest-extractor.test.ts b/gitnexus/test/unit/group/manifest-extractor.test.ts new file mode 100644 index 0000000000..c2c67a33ad --- /dev/null +++ b/gitnexus/test/unit/group/manifest-extractor.test.ts @@ -0,0 +1,308 @@ +import { describe, it, expect } from 'vitest'; +import { ManifestExtractor } from '../../../src/core/group/extractors/manifest-extractor.js'; +import type { GroupManifestLink } from '../../../src/core/group/types.js'; + +describe('ManifestExtractor', () => { + const extractor = new ManifestExtractor(); + + it('creates provider + consumer contracts and a cross-link for each manifest link', async () => { + const links: GroupManifestLink[] = [ + { + from: 'hr/payroll/backend', + to: 'hr/hiring/backend', + type: 'topic', + contract: 'employee.hired', + role: 'provider', + }, + ]; + + const result = await extractor.extractFromManifest(links); + + expect(result.contracts).toHaveLength(2); + + const provider = result.contracts.find((c) => c.role === 'provider'); + expect(provider).toBeDefined(); + expect(provider!.contractId).toBe('topic::employee.hired'); + expect(provider!.type).toBe('topic'); + expect(provider!.confidence).toBe(1.0); + + const consumer = result.contracts.find((c) => c.role === 'consumer'); + expect(consumer).toBeDefined(); + expect(consumer!.contractId).toBe('topic::employee.hired'); + + expect(result.crossLinks).toHaveLength(1); + expect(result.crossLinks[0].matchType).toBe('manifest'); + expect(result.crossLinks[0].confidence).toBe(1.0); + expect(result.crossLinks[0].from.repo).toBe('hr/hiring/backend'); + expect(result.crossLinks[0].to.repo).toBe('hr/payroll/backend'); + }); + + it('handles role: consumer (from-repo is consumer)', async () => { + const links: GroupManifestLink[] = [ + { + from: 'sales/admin/bff', + to: 'sales/crm/backend', + type: 'http', + contract: '/api/v2/leads/*', + role: 'consumer', + }, + ]; + + const result = await extractor.extractFromManifest(links); + + const provider = result.contracts.find((c) => c.role === 'provider'); + const consumer = result.contracts.find((c) => c.role === 'consumer'); + + expect(consumer!.contractId).toBe('http::*::/api/v2/leads/*'); + expect(provider!.contractId).toBe('http::*::/api/v2/leads/*'); + + expect(result.crossLinks[0].from.repo).toBe('sales/admin/bff'); + expect(result.crossLinks[0].to.repo).toBe('sales/crm/backend'); + }); + + it('resolves grpc manifest provider by exact method name (no .proto fallback)', async () => { + const links: GroupManifestLink[] = [ + { + from: 'platform/orders', + to: 'platform/auth', + type: 'grpc', + contract: 'auth.AuthService/Login', + role: 'consumer', + }, + ]; + + const dbExecutors = new Map< + string, + (cypher: string, params?: Record) => Promise[]> + >([ + [ + 'platform/auth', + async (_cypher, params) => { + // Exact match on method name. + if (params?.methodName === 'Login') { + return [ + { + uid: 'uid-auth-login', + name: 'Login', + filePath: 'src/auth.proto', + }, + ]; + } + return []; + }, + ], + [ + 'platform/orders', + async (_cypher, params) => { + // No symbol with the exact method name — resolve returns null and + // the consumer contract gets an empty symbolUid, falling back to + // name-based hint at cross-impact time. + if (params?.methodName === 'Login') return []; + return []; + }, + ], + ]); + + const result = await extractor.extractFromManifest(links, dbExecutors); + + const provider = result.contracts.find((c) => c.role === 'provider'); + const consumer = result.contracts.find((c) => c.role === 'consumer'); + + // Provider resolved to the concrete proto symbol. + expect(provider?.symbolUid).toBe('uid-auth-login'); + expect(provider?.symbolRef.filePath).toBe('src/auth.proto'); + + // Consumer falls back to a deterministic synthetic uid + name-based ref. + // The synthetic uid lets the bridge cross-impact query anchor on it + // even when the indexer doesn't expose a matching symbol. + expect(consumer?.symbolUid).toBe('manifest::platform/orders::grpc::auth.AuthService/Login'); + expect(consumer?.symbolRef.name).toBe('auth.AuthService/Login'); + + expect(result.crossLinks[0].to.symbolRef.filePath).toBe('src/auth.proto'); + expect(result.crossLinks[0].from.symbolUid).toBe( + 'manifest::platform/orders::grpc::auth.AuthService/Login', + ); + }); + + it('does NOT resolve grpc manifest to an arbitrary .proto file', async () => { + // Regression test for a previous bug: the extractor had an unconditional + // `OR n.filePath ENDS WITH '.proto'` fallback that returned the first + // proto symbol in the repo, regardless of whether it matched the contract. + const links: GroupManifestLink[] = [ + { + from: 'platform/orders', + to: 'platform/auth', + type: 'grpc', + contract: 'auth.AuthService/Login', + role: 'consumer', + }, + ]; + + const dbExecutors = new Map< + string, + (cypher: string, params?: Record) => Promise[]> + >([ + [ + 'platform/auth', + // Executor returns matches for ANY query (simulates the old buggy + // fallback that returned a random .proto file). The new code must + // only accept a hit when the method/service name matches exactly. + async (_cypher, params) => { + if (params?.methodName === 'Login' || params?.serviceName === 'auth.AuthService') { + return [ + { + uid: 'uid-correct-login', + name: 'Login', + filePath: 'src/auth.proto', + }, + ]; + } + return []; + }, + ], + ['platform/orders', async () => []], + ]); + + const result = await extractor.extractFromManifest(links, dbExecutors); + const provider = result.contracts.find((c) => c.role === 'provider'); + // Must resolve to the correct symbol (not a random proto one). + expect(provider?.symbolUid).toBe('uid-correct-login'); + }); + + it('resolves lib manifest links by exact name only', async () => { + const links: GroupManifestLink[] = [ + { + from: 'platform/web', + to: 'platform/shared-lib', + type: 'lib', + contract: '@platform/contracts', + role: 'consumer', + }, + ]; + + const dbExecutors = new Map< + string, + (cypher: string, params?: Record) => Promise[]> + >([ + [ + 'platform/shared-lib', + async (_cypher, params) => { + if (params?.contract !== '@platform/contracts') return []; + return [ + { + uid: 'uid-lib', + name: '@platform/contracts', + filePath: 'src/index.ts', + }, + ]; + }, + ], + [ + 'platform/web', + async (_cypher, params) => { + if (params?.contract !== '@platform/contracts') return []; + return []; + }, + ], + ]); + + const result = await extractor.extractFromManifest(links, dbExecutors); + + const provider = result.contracts.find((c) => c.role === 'provider'); + const consumer = result.contracts.find((c) => c.role === 'consumer'); + + expect(provider?.symbolUid).toBe('uid-lib'); + // Consumer doesn't have a symbol named exactly '@platform/contracts' — + // exact matching returns null, falling back to the synthetic manifest uid. + expect(consumer?.symbolUid).toBe('manifest::platform/web::lib::@platform/contracts'); + }); + + it('does NOT resolve lib manifest via CONTAINS on name', async () => { + // Regression test: previous CONTAINS fallback would match "react" to + // "react-native" or "@types/react". Exact matching must reject both. + const links: GroupManifestLink[] = [ + { + from: 'web', + to: 'packages/ui', + type: 'lib', + contract: 'react', + role: 'consumer', + }, + ]; + + const dbExecutors = new Map< + string, + (cypher: string, params?: Record) => Promise[]> + >([ + [ + 'packages/ui', + async (_cypher, params) => { + // Executor is called with contract='react'. Only exact matches + // should come back; return only wrong candidates to verify the + // Cypher uses `=` not `CONTAINS`. + if (params?.contract === 'react') { + // Simulated DB returns nothing because it has only "react-native" + // and "@types/react" — neither is an exact match for "react". + return []; + } + return []; + }, + ], + ['web', async () => []], + ]); + + const result = await extractor.extractFromManifest(links, dbExecutors); + const provider = result.contracts.find((c) => c.role === 'provider'); + // No exact match → synthetic manifest uid, not a wrong real one. + expect(provider?.symbolUid).toBe('manifest::packages/ui::lib::react'); + }); + + it('normalizes http contract path for exact Route.name match', async () => { + // Manifest may be written as "/api/orders/" or "api/orders"; both should + // match the canonical "/api/orders" stored in the graph. + const variants = ['/api/orders', '/api/orders/', 'api/orders', '//api//orders']; + for (const raw of variants) { + const links: GroupManifestLink[] = [ + { + from: 'gateway', + to: 'orders-svc', + type: 'http', + contract: raw, + role: 'consumer', + }, + ]; + + let seenParam: string | undefined; + const dbExecutors = new Map< + string, + (cypher: string, params?: Record) => Promise[]> + >([ + [ + 'orders-svc', + async (_cypher, params) => { + seenParam = params?.normalized as string; + return [ + { + uid: 'uid-orders-list', + name: 'listOrders', + filePath: 'src/orders.ts', + }, + ]; + }, + ], + ['gateway', async () => []], + ]); + + const result = await extractor.extractFromManifest(links, dbExecutors); + expect(seenParam).toBe('/api/orders'); + const provider = result.contracts.find((c) => c.role === 'provider'); + expect(provider?.symbolUid).toBe('uid-orders-list'); + } + }); + + it('returns empty for no links', async () => { + const result = await extractor.extractFromManifest([]); + expect(result.contracts).toHaveLength(0); + expect(result.crossLinks).toHaveLength(0); + }); +}); diff --git a/gitnexus/test/unit/group/topic-extractor.test.ts b/gitnexus/test/unit/group/topic-extractor.test.ts index c6a1161a00..bf821de63a 100644 --- a/gitnexus/test/unit/group/topic-extractor.test.ts +++ b/gitnexus/test/unit/group/topic-extractor.test.ts @@ -75,8 +75,7 @@ public void handleUserCreated(ConsumerRecord record) { it('test_extract_kafkajs_subscribe_returns_consumer', async () => { writeFile( 'src/consumer.ts', - `await consumer.subscribe({ topic: 'order.placed', fromBeginning: true }); -await consumer.run({ eachMessage: async ({ message }) => {} });`, + `await consumer.subscribe({ topic: 'order.placed', fromBeginning: true });`, ); const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); @@ -101,6 +100,23 @@ await consumer.run({ eachMessage: async ({ message }) => {} });`, }); }); + describe('KafkaJS consumer run', () => { + it('test_extract_kafkajs_consumer_run_eachmessage_returns_consumer', async () => { + writeFile( + 'src/consumer.ts', + `await consumer.subscribe({ topic: 'user.logged-in' }); +await consumer.run({ eachMessage: async () => {} });`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(1); + expect(consumers[0].contractId).toBe('topic::user.logged-in'); + expect(consumers[0].meta.broker).toBe('kafka'); + }); + }); + describe('RabbitMQ — Java', () => { it('test_extract_rabbit_listener_returns_consumer', async () => { writeFile( @@ -174,6 +190,62 @@ public void processOrder(OrderMessage msg) {}`, }); }); + describe('JetStream', () => { + it('test_extract_jetstream_publish_returns_provider', async () => { + writeFile('src/stream.go', `js.Publish("orders.created", payload)`); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const producers = contracts.filter((c) => c.role === 'provider'); + + expect(producers).toHaveLength(1); + expect(producers[0].contractId).toBe('topic::orders.created'); + expect(producers[0].meta.broker).toBe('nats'); + }); + + it('test_extract_jetstream_subscribe_returns_consumer', async () => { + writeFile('src/stream.go', `js.Subscribe("orders.created", handler)`); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(1); + expect(consumers[0].contractId).toBe('topic::orders.created'); + expect(consumers[0].meta.broker).toBe('nats'); + }); + }); + + describe('Python NATS', () => { + it('test_extract_python_nats_subscribe_returns_consumer', async () => { + writeFile( + 'src/subscriber.py', + `nc = await nats.connect() +await nc.subscribe("orders.created", cb=handler)`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(1); + expect(consumers[0].contractId).toBe('topic::orders.created'); + expect(consumers[0].meta.broker).toBe('nats'); + }); + + it('test_extract_python_nats_publish_returns_provider', async () => { + writeFile( + 'src/publisher.py', + `nc = await nats.connect() +await nc.publish("orders.created", payload)`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const producers = contracts.filter((c) => c.role === 'provider'); + + expect(producers).toHaveLength(1); + expect(producers[0].contractId).toBe('topic::orders.created'); + expect(producers[0].meta.broker).toBe('nats'); + }); + }); + describe('NATS', () => { it('test_extract_nats_subscribe_go_returns_consumer', async () => { writeFile( @@ -248,6 +320,96 @@ partConsumer, _ := consumer.ConsumePartition("inventory.update", 0, sarama.Offse expect(consumers[0].contractId).toBe('topic::inventory.update'); expect(consumers[0].meta.broker).toBe('kafka'); }); + + it('test_extract_sarama_sync_producer_returns_provider', async () => { + writeFile( + 'internal/publisher.go', + `package publisher +producer, _ := sarama.NewSyncProducer(brokers, cfg) +producer.SendMessage(&sarama.ProducerMessage{Topic: "inventory.update"})`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const producers = contracts.filter((c) => c.role === 'provider'); + + expect(producers).toHaveLength(1); + expect(producers[0].contractId).toBe('topic::inventory.update'); + expect(producers[0].meta.broker).toBe('kafka'); + }); + + it('test_extract_sarama_async_producer_returns_provider', async () => { + writeFile( + 'internal/publisher.go', + `package publisher +producer, _ := sarama.NewAsyncProducer(brokers, cfg) +producer.Input() <- &sarama.ProducerMessage{Topic: "inventory.update"}`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const producers = contracts.filter((c) => c.role === 'provider'); + + expect(producers).toHaveLength(1); + expect(producers[0].contractId).toBe('topic::inventory.update'); + expect(producers[0].meta.broker).toBe('kafka'); + }); + + it('test_extract_sarama_producer_in_loop_captures_all_topics', async () => { + // Regression: a for loop that constructs multiple ProducerMessage + // literals inside a single NewSyncProducer scope. The previous + // regex anchored on NewSyncProducer and captured only the first + // Topic within 300 chars, silently dropping the rest. + writeFile( + 'internal/multi-publisher.go', + `package publisher + +func publishAll(producer sarama.SyncProducer, items []Item) error { + _, _ = sarama.NewSyncProducer(brokers, cfg) + for _, item := range items { + msg1 := &sarama.ProducerMessage{Topic: "order.created"} + msg2 := &sarama.ProducerMessage{Topic: "order.shipped"} + _ = msg1 + _ = msg2 + } + return nil +}`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const producers = contracts.filter((c) => c.role === 'provider'); + const topics = producers.map((c) => c.contractId).sort(); + // Both topics must appear (exact set match to catch any duplicates). + expect(topics).toEqual(['topic::order.created', 'topic::order.shipped']); + }); + + it('test_extract_kafka_go_writer_returns_provider', async () => { + writeFile( + 'internal/writer.go', + `package publisher +writer := &kafka.Writer{Topic: "inventory.update"}`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const producers = contracts.filter((c) => c.role === 'provider'); + + expect(producers).toHaveLength(1); + expect(producers[0].contractId).toBe('topic::inventory.update'); + expect(producers[0].meta.broker).toBe('kafka'); + }); + + it('test_extract_kafka_go_reader_returns_consumer', async () => { + writeFile( + 'internal/reader.go', + `package consumer +reader := kafka.NewReader(kafka.ReaderConfig{Topic: "inventory.update"})`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + const consumers = contracts.filter((c) => c.role === 'consumer'); + + expect(consumers).toHaveLength(1); + expect(consumers[0].contractId).toBe('topic::inventory.update'); + expect(consumers[0].meta.broker).toBe('kafka'); + }); }); describe('Kafka — Python', () => { @@ -309,5 +471,16 @@ await consumer.subscribe({ topic: 'order.placed' });`, expect(producers).toHaveLength(2); expect(consumers).toHaveLength(1); }); + + it('test_extract_ignores_go_test_files', async () => { + writeFile( + 'src/orders_test.go', + `consumer.ConsumePartition("fake-topic", 0, sarama.OffsetNewest)`, + ); + + const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir)); + + expect(contracts).toEqual([]); + }); }); }); From 55ed7443bca5fdd72833177980f128ee96a26fa0 Mon Sep 17 00:00:00 2001 From: ivkond Date: Sat, 11 Apr 2026 21:41:17 +0000 Subject: [PATCH 02/10] refactor(group): migrate topic-extractor from regex to tree-sitter queries MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses @magyargergo's feedback on #796 that regex-based lookups should use tree-sitter nodes instead, and that the top-level extractors must NOT carry language dependencies. This is phase 1 of a multi-step migration — topic-extractor first because its patterns are the most uniform (16 "call/annotation with first-arg string literal" variants), which makes it a clean proof of the approach before grpc-extractor and http-route-extractor get the same treatment. ## Architecture: language-agnostic orchestrator + per-language plugins The top-level extractor is a thin orchestrator that never imports a tree-sitter grammar or a query string. Per-language knowledge lives in a new `topic-patterns/` folder with one file per language plus a registry that maps file extensions to compiled plugins: ``` src/core/group/extractors/ ├── tree-sitter-scanner.ts # shared, language-agnostic scanning utilities ├── topic-extractor.ts # thin orchestrator (no grammar imports) └── topic-patterns/ ├── types.ts # TopicMeta, Broker ├── index.ts # registry: extension → compiled provider ├── java.ts # tree-sitter-java + JAVA_TOPIC_PROVIDER ├── go.ts # tree-sitter-go + GO_TOPIC_PROVIDER ├── python.ts # tree-sitter-python + PYTHON_TOPIC_PROVIDER └── node.ts # tree-sitter-javascript + tree-sitter-typescript # → JAVASCRIPT_/TYPESCRIPT_/TSX_TOPIC_PROVIDER ``` **Shared scanner (`tree-sitter-scanner.ts`)** — defines `PatternSpec`, `LanguagePatterns`, `CompiledPatterns` and the `scanFile(parser, plugin, content)` helper. Plugins compile their queries eagerly at module load via `compilePatterns()`, so a broken pattern fails loudly at import time instead of silently at scan time. `unquoteLiteral()` handles single/double/template quotes, Python triple-quoted strings, and Go raw backtick strings. **Per-language plugins** own: - the tree-sitter grammar import (this is the ONLY place in `src/core/group/` where tree-sitter grammars are imported), - the query S-expressions, - the `TopicMeta` payload (role, broker, confidence, symbolName) that the orchestrator receives back on every match. Each plugin uses a `@value` capture name to bind the topic literal node. The JavaScript and TypeScript grammars share AST node names for every construct we query, so `node.ts` defines the pattern sources once and compiles them against `JavaScript`, `TypeScript.typescript`, and `TypeScript.tsx` — exporting three providers because `Parser.Query` objects are NOT portable across grammar instances. **Registry (`topic-patterns/index.ts`)** — maps `.java` → Java provider, `.go` → Go, `.py` → Python, `.js`/`.jsx` → JS, `.ts` → TS, `.tsx` → TSX. Also exports `TOPIC_SCAN_GLOB` so adding a new language is a single file-level edit (drop `topic-patterns/.ts`, import + register it here — zero edits required in `topic-extractor.ts`). **Orchestrator (`topic-extractor.ts`)** — ~110 lines, no grammar or query imports. Per file: `getProviderForFile(rel)` → `scanFile(parser, provider, content)` → `unquoteLiteral(valueText)` → `makeContract(...)`. Reuses one `Parser` instance across files; the scanner calls `setLanguage` per plugin. ## Why this is better than regex 1. **Comments and strings are respected for free.** The old regex would match `// kafkaTemplate.send("fake.topic")` as a real producer; tree-sitter never visits comments or string literals as code nodes, so false positives from commented-out code are eliminated. 2. **Struct/object literal patterns are structural, not textual.** `sarama.ProducerMessage{Topic: "..."}` no longer needs a 300-char lookahead (which was a known cross-match bug partly mitigated by a loop regression test in the self-review). The new query matches a specific `composite_literal` with a specific `qualified_type` and `keyed_element` — exactly one struct literal per match. 3. **No order-of-operations fragility.** Regex for `channel.publish` vs `channel.consume` was independent and file-wide; the AST scopes matches to the specific `call_expression`. 4. **Language-agnostic extension.** Adding Ruby, Rust, or C# topic detection later means dropping one file in `topic-patterns/` — no changes to shared scanner or orchestrator, and no tree-sitter imports leak into top-level code. ## Per-file fault tolerance - Malformed files that tree-sitter can't parse are silently skipped (`parser.parse` is wrapped by `scanFile`). The ingestion pipeline already logs unparseable files at index time. - A syntactically invalid query is caught at `compilePatterns` time, not scan time — broken plugins fail loudly at import. - Per-pattern `matches()` failures are swallowed so one broken query in a plugin doesn't block the rest. ## Tests All 30 existing `topic-extractor.test.ts` tests pass **without any changes to the test file** — they were written as input/output contract tests (given this source file, expect these `ExtractedContract` objects) and that contract is unchanged. Regression coverage includes: - Kafka: Java `@KafkaListener` + `kafkaTemplate.send`; Node `producer.send` + `consumer.subscribe`; Go sarama producer/consumer (sync and async); kafka-go Writer/Reader; Python `KafkaConsumer` + `producer.send/produce` - RabbitMQ: Java `@RabbitListener` + `rabbitTemplate.convertAndSend`; Node `channel.consume/publish/sendToQueue`; Python `basic_consume/ basic_publish` with keyword args - NATS: Go and Node `nc.Subscribe/Publish`; Go and Node JetStream `js.Subscribe/Publish`; Python `await nc.subscribe/publish` Including the regression test for the sarama `ProducerMessage` in-loop case — the AST-based query captures every literal in the file independently, not just the first one after `NewSyncProducer`. ## Neighbor regression check - `topic-extractor.test.ts` — 30/30 pass (rewritten extractor) - `http-route-extractor.test.ts` — 18/18 pass (untouched) - `grpc-extractor.test.ts` — 43/43 pass (untouched) - `manifest-extractor.test.ts` — 8/8 pass (untouched) - Full `npx tsc --noEmit` clean ## Scope discipline (per GUARDRAILS.md) - Only files under `src/core/group/extractors/` are touched; no changes to other extractors, tests, MCP surface, or pipeline.ts. - No CI/release/security config changes, no secrets. - New tree-sitter imports all reference grammars that are already installed as dependencies (`tree-sitter`, `tree-sitter-javascript`, `tree-sitter-typescript`, `tree-sitter-python`, `tree-sitter-java`, `tree-sitter-go` — all in `package.json` for the existing pipeline). ## Phase 2 / phase 3 plan - **Phase 2 (next commit):** rewrite `http-route-extractor.ts` Strategy B (regex fallback) on the same plugin pattern. Graph-assisted Strategy A stays as-is (already uses pipeline-built tree-sitter data via `HANDLES_ROUTE` Cypher queries). - **Phase 3 (commit after):** rewrite `grpc-extractor.ts` for Java / Go / Python / TypeScript detection. `.proto` files are the one outstanding question — there is no `tree-sitter-proto` grammar installed; the in-tree string-sanitizing parser stays as a pragmatic exception with a comment, alternative being to add `tree-sitter-proto` as a dep (open for the maintainer). Co-authored-by: Claude --- .../core/group/extractors/topic-extractor.ts | 351 +++--------------- .../group/extractors/topic-patterns/go.ts | 123 ++++++ .../group/extractors/topic-patterns/index.ts | 49 +++ .../group/extractors/topic-patterns/java.ts | 83 +++++ .../group/extractors/topic-patterns/node.ts | 165 ++++++++ .../group/extractors/topic-patterns/python.ts | 119 ++++++ .../group/extractors/topic-patterns/types.ts | 27 ++ .../group/extractors/tree-sitter-scanner.ts | 177 +++++++++ 8 files changed, 788 insertions(+), 306 deletions(-) create mode 100644 gitnexus/src/core/group/extractors/topic-patterns/go.ts create mode 100644 gitnexus/src/core/group/extractors/topic-patterns/index.ts create mode 100644 gitnexus/src/core/group/extractors/topic-patterns/java.ts create mode 100644 gitnexus/src/core/group/extractors/topic-patterns/node.ts create mode 100644 gitnexus/src/core/group/extractors/topic-patterns/python.ts create mode 100644 gitnexus/src/core/group/extractors/topic-patterns/types.ts create mode 100644 gitnexus/src/core/group/extractors/tree-sitter-scanner.ts diff --git a/gitnexus/src/core/group/extractors/topic-extractor.ts b/gitnexus/src/core/group/extractors/topic-extractor.ts index 3d5280d862..e602273693 100644 --- a/gitnexus/src/core/group/extractors/topic-extractor.ts +++ b/gitnexus/src/core/group/extractors/topic-extractor.ts @@ -1,13 +1,32 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { glob } from 'glob'; +import Parser from 'tree-sitter'; import type { ContractExtractor, CypherExecutor } from '../contract-extractor.js'; import type { ExtractedContract, RepoHandle } from '../types.js'; - -type Broker = 'kafka' | 'rabbitmq' | 'nats'; - -const KAFKAJS_CONSUMER_RUN_RE = /consumer\.run\s*\(\s*\{\s*eachMessage:/; -const KAFKAJS_SUBSCRIBE_RE = /consumer\.subscribe\s*\(\s*\{\s*topic:\s*['"]([^'"]+)['"]/g; +import { scanFile, unquoteLiteral } from './tree-sitter-scanner.js'; +import { + TOPIC_SCAN_GLOB, + getProviderForFile, + type Broker, + type TopicMeta, +} from './topic-patterns/index.js'; + +/** + * Language-agnostic orchestrator for topic (message broker) contract + * extraction. All grammar-specific knowledge lives in `topic-patterns/*` + * — this file must not import any tree-sitter grammar directly. + * + * Flow per file: + * 1. `getProviderForFile(rel)` → compiled plugin (or `undefined` if the + * file's extension isn't registered, in which case we skip it). + * 2. `scanFile(parser, provider, content)` → list of `{meta, valueText}` + * pairs, one per matched literal. + * 3. `unquoteLiteral(valueText)` → the raw topic string. + * 4. `makeContract(topic, meta, relPath)` → `ExtractedContract`. + * + * Adding a new language is a one-file edit in `topic-patterns/index.ts`. + */ function readSafe(repoPath: string, rel: string): string | null { const abs = path.resolve(repoPath, rel); @@ -21,281 +40,23 @@ function readSafe(repoPath: string, rel: string): string | null { } } -function makeContract( - topicName: string, - role: 'provider' | 'consumer', - filePath: string, - symbolName: string, - confidence: number, - broker: Broker, -): ExtractedContract { +function makeContract(topicName: string, meta: TopicMeta, filePath: string): ExtractedContract { return { contractId: `topic::${topicName}`, type: 'topic', - role, + role: meta.role, symbolUid: '', - symbolRef: { filePath: filePath.replace(/\\/g, '/'), name: symbolName }, - symbolName, - confidence, + symbolRef: { filePath: filePath.replace(/\\/g, '/'), name: meta.symbolName }, + symbolName: meta.symbolName, + confidence: meta.confidence, meta: { - broker, + broker: meta.broker satisfies Broker, topicName, - extractionStrategy: 'source_scan', + extractionStrategy: 'tree_sitter', }, }; } -interface PatternDef { - regex: RegExp; - role: 'provider' | 'consumer'; - broker: Broker; - confidence: number; - topicGroup: number; - symbolName: string; -} - -// --- Kafka patterns --- -const KAFKA_PATTERNS: PatternDef[] = [ - // Java: @KafkaListener(topics = "xxx") - { - regex: /@KafkaListener\s*\(\s*topics\s*=\s*"([^"]+)"/g, - role: 'consumer', - broker: 'kafka', - confidence: 0.8, - topicGroup: 1, - symbolName: 'kafkaListener', - }, - // Java: kafkaTemplate.send("xxx" - { - regex: /kafkaTemplate\.send\s*\(\s*"([^"]+)"/gi, - role: 'provider', - broker: 'kafka', - confidence: 0.8, - topicGroup: 1, - symbolName: 'kafkaTemplate.send', - }, - // Node: producer.send({ topic: 'xxx' - { - regex: /producer\.send\s*\(\s*\{\s*topic:\s*['"]([^'"]+)['"]/g, - role: 'provider', - broker: 'kafka', - confidence: 0.8, - topicGroup: 1, - symbolName: 'producer.send', - }, - // Node: consumer.subscribe({ topic: 'xxx' - { - regex: /consumer\.subscribe\s*\(\s*\{\s*topic:\s*['"]([^'"]+)['"]/g, - role: 'consumer', - broker: 'kafka', - confidence: 0.8, - topicGroup: 1, - symbolName: 'consumer.subscribe', - }, - // Go: consumer.ConsumePartition("xxx" - { - regex: /\.ConsumePartition\s*\(\s*"([^"]+)"/g, - role: 'consumer', - broker: 'kafka', - confidence: 0.7, - topicGroup: 1, - symbolName: 'ConsumePartition', - }, - // Python: KafkaConsumer('xxx' - { - regex: /KafkaConsumer\s*\(\s*['"]([^'"]+)['"]/g, - role: 'consumer', - broker: 'kafka', - confidence: 0.7, - topicGroup: 1, - symbolName: 'KafkaConsumer', - }, - // Python: producer.send('xxx' or producer.produce('xxx' - { - regex: /producer\.(?:send|produce)\s*\(\s*['"]([^'"]+)['"]/g, - role: 'provider', - broker: 'kafka', - confidence: 0.7, - topicGroup: 1, - symbolName: 'producer.send', - }, - // Go: sarama.ProducerMessage{Topic: "xxx"} struct literal (emitted by - // both NewSyncProducer and NewAsyncProducer client code paths). - // - // Previous pattern was `sarama.NewSyncProducer[\s\S]{0,300}?Topic:...` - // which anchored to the producer constructor and used a 300-char - // lookahead. In a loop like - // producer := sarama.NewSyncProducer(...) - // for _, item := range items { - // msg1 := &sarama.ProducerMessage{Topic: "order.created"} - // msg2 := &sarama.ProducerMessage{Topic: "order.shipped"} - // } - // the regex captured only "order.created" (first Topic after the - // constructor) and silently missed "order.shipped". Matching on the - // struct literal directly fixes both the false negative in loops and - // the spurious cross-message capture when multiple unrelated messages - // sit within 300 chars of the constructor. - { - regex: /sarama\.ProducerMessage\s*\{[\s\S]{0,200}?Topic:\s*"([^"]+)"/g, - role: 'provider', - broker: 'kafka', - confidence: 0.75, - topicGroup: 1, - symbolName: 'sarama.ProducerMessage', - }, - // Go: kafka-go writer construction. kafka-go does NOT wrap messages in - // a struct with a Topic field (the writer owns the topic), so we match - // the Writer itself. A 200-char window bridges the gap between - // `kafka.NewWriter(...)` / `kafka.Writer{` and the Topic field inside - // the config literal — kafka-go writer configs are small and rarely - // contain more than one Topic field, so the risk of cross-message - // capture is low here. - { - regex: /kafka\.(?:NewWriter|Writer)\b[\s\S]{0,200}?Topic:\s*"([^"]+)"/g, - role: 'provider', - broker: 'kafka', - confidence: 0.75, - topicGroup: 1, - symbolName: 'kafka.Writer', - }, - // Go: kafka-go reader construction, mirrors Writer above. - { - regex: /kafka\.(?:NewReader|Reader)\b[\s\S]{0,200}?Topic:\s*"([^"]+)"/g, - role: 'consumer', - broker: 'kafka', - confidence: 0.75, - topicGroup: 1, - symbolName: 'kafka.Reader', - }, -]; - -// --- RabbitMQ patterns --- -const RABBITMQ_PATTERNS: PatternDef[] = [ - // Java: @RabbitListener(queues = "xxx") - { - regex: /@RabbitListener\s*\(\s*queues\s*=\s*"([^"]+)"/g, - role: 'consumer', - broker: 'rabbitmq', - confidence: 0.8, - topicGroup: 1, - symbolName: 'rabbitListener', - }, - // Java: rabbitTemplate.convertAndSend("xxx" - { - regex: /rabbitTemplate\.convertAndSend\s*\(\s*"([^"]+)"/gi, - role: 'provider', - broker: 'rabbitmq', - confidence: 0.8, - topicGroup: 1, - symbolName: 'rabbitTemplate.convertAndSend', - }, - // Node: channel.consume("xxx" - { - regex: /channel\.consume\s*\(\s*"([^"]+)"/g, - role: 'consumer', - broker: 'rabbitmq', - confidence: 0.8, - topicGroup: 1, - symbolName: 'channel.consume', - }, - // Node: channel.publish("xxx" - { - regex: /channel\.publish\s*\(\s*"([^"]+)"/g, - role: 'provider', - broker: 'rabbitmq', - confidence: 0.8, - topicGroup: 1, - symbolName: 'channel.publish', - }, - // Node: channel.sendToQueue("xxx" - { - regex: /channel\.sendToQueue\s*\(\s*"([^"]+)"/g, - role: 'provider', - broker: 'rabbitmq', - confidence: 0.8, - topicGroup: 1, - symbolName: 'channel.sendToQueue', - }, - // Python: channel.basic_consume(queue='xxx' - { - regex: /channel\.basic_consume\s*\(\s*queue\s*=\s*['"]([^'"]+)['"]/g, - role: 'consumer', - broker: 'rabbitmq', - confidence: 0.7, - topicGroup: 1, - symbolName: 'basic_consume', - }, - // Python: channel.basic_publish(exchange='xxx' - { - regex: /channel\.basic_publish\s*\([^)]*exchange\s*=\s*['"]([^'"]+)['"]/g, - role: 'provider', - broker: 'rabbitmq', - confidence: 0.7, - topicGroup: 1, - symbolName: 'basic_publish', - }, -]; - -// --- NATS patterns --- -const NATS_PATTERNS: PatternDef[] = [ - // Go/Node: nc.Subscribe("xxx" or nc.subscribe("xxx" - { - regex: /nc\.(?:S|s)ubscribe\s*\(\s*"([^"]+)"/g, - role: 'consumer', - broker: 'nats', - confidence: 0.8, - topicGroup: 1, - symbolName: 'nc.Subscribe', - }, - // Go/Node: nc.Publish("xxx" or nc.publish("xxx" - { - regex: /nc\.(?:P|p)ublish\s*\(\s*"([^"]+)"/g, - role: 'provider', - broker: 'nats', - confidence: 0.8, - topicGroup: 1, - symbolName: 'nc.Publish', - }, - // Go/Node JetStream: js.Subscribe("xxx" - { - regex: /js\.(?:S|s)ubscribe\s*\(\s*"([^"]+)"/g, - role: 'consumer', - broker: 'nats', - confidence: 0.8, - topicGroup: 1, - symbolName: 'js.Subscribe', - }, - // Go/Node JetStream: js.Publish("xxx" - { - regex: /js\.(?:P|p)ublish\s*\(\s*"([^"]+)"/g, - role: 'provider', - broker: 'nats', - confidence: 0.8, - topicGroup: 1, - symbolName: 'js.Publish', - }, - // Python: await nc.subscribe("xxx") - { - regex: /await\s+nc\.subscribe\s*\(\s*['"]([^'"]+)['"]/g, - role: 'consumer', - broker: 'nats', - confidence: 0.75, - topicGroup: 1, - symbolName: 'nc.subscribe', - }, - // Python: await nc.publish("xxx") - { - regex: /await\s+nc\.publish\s*\(\s*['"]([^'"]+)['"]/g, - role: 'provider', - broker: 'nats', - confidence: 0.75, - topicGroup: 1, - symbolName: 'nc.publish', - }, -]; - -const ALL_PATTERNS: PatternDef[] = [...KAFKA_PATTERNS, ...RABBITMQ_PATTERNS, ...NATS_PATTERNS]; - export class TopicExtractor implements ContractExtractor { type = 'topic' as const; @@ -308,57 +69,35 @@ export class TopicExtractor implements ContractExtractor { repoPath: string, _repo: RepoHandle, ): Promise { - const files = await glob('**/*.{ts,tsx,js,jsx,java,go,py}', { + const files = await glob(TOPIC_SCAN_GLOB, { cwd: repoPath, ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**', '**/dist/**', '**/build/**'], nodir: true, }); + // One parser reused across files; the scanner calls `setLanguage` per + // file based on which plugin the registry returns. + const parser = new Parser(); const out: ExtractedContract[] = []; + for (const rel of files) { if (rel.endsWith('_test.go')) continue; - const content = readSafe(repoPath, rel); - if (!content) continue; - out.push(...this.scanFile(content, rel)); - } - return this.dedupe(out); - } - - private scanFile(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; + const provider = getProviderForFile(rel); + if (!provider) continue; - for (const pattern of ALL_PATTERNS) { - // Reset regex state for each file - const re = new RegExp(pattern.regex.source, pattern.regex.flags); - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const topicName = m[pattern.topicGroup]; - if (!topicName) continue; - out.push( - makeContract( - topicName, - pattern.role, - filePath, - pattern.symbolName, - pattern.confidence, - pattern.broker, - ), - ); - } - } + const content = readSafe(repoPath, rel); + if (!content) continue; - if (KAFKAJS_CONSUMER_RUN_RE.test(content)) { - const subscribeRe = new RegExp(KAFKAJS_SUBSCRIBE_RE.source, KAFKAJS_SUBSCRIBE_RE.flags); - let subscribeMatch: RegExpExecArray | null; - while ((subscribeMatch = subscribeRe.exec(content)) !== null) { - const topicName = subscribeMatch[1]; + const matches = scanFile(parser, provider, content); + for (const match of matches) { + const topicName = unquoteLiteral(match.valueText); if (!topicName) continue; - out.push(makeContract(topicName, 'consumer', filePath, 'consumer.run', 0.75, 'kafka')); + out.push(makeContract(topicName, match.meta, rel)); } } - return out; + return this.dedupe(out); } private dedupe(items: ExtractedContract[]): ExtractedContract[] { diff --git a/gitnexus/src/core/group/extractors/topic-patterns/go.ts b/gitnexus/src/core/group/extractors/topic-patterns/go.ts new file mode 100644 index 0000000000..df3bab0953 --- /dev/null +++ b/gitnexus/src/core/group/extractors/topic-patterns/go.ts @@ -0,0 +1,123 @@ +import Go from 'tree-sitter-go'; +import { compilePatterns, type LanguagePatterns } from '../tree-sitter-scanner.js'; +import type { TopicMeta } from './types.js'; + +/** + * Go topic extraction patterns. + * + * Detects Sarama, segmentio/kafka-go and nats.go producer/consumer APIs: + * - `X.ConsumePartition("topic", ...)` + * - `sarama.ProducerMessage{Topic: "xxx"}` + * - `kafka.Writer{Topic: "xxx"}` / `kafka.WriterConfig{Topic: ...}` + * - `kafka.Reader{Topic: "xxx"}` / `kafka.ReaderConfig{Topic: ...}` + * - `nc.Subscribe("topic", ...)` / `js.Subscribe("topic", ...)` + * - `nc.Publish("topic", ...)` / `js.Publish("topic", ...)` + * + * Every query MUST bind `@value` to the topic literal node. + */ +const GO_TOPIC_SPEC: LanguagePatterns = { + name: 'go-topic', + language: Go, + patterns: [ + { + meta: { + role: 'consumer', + broker: 'kafka', + confidence: 0.7, + symbolName: 'ConsumePartition', + }, + query: ` + (call_expression + function: (selector_expression + field: (field_identifier) @method (#eq? @method "ConsumePartition")) + arguments: (argument_list . (interpreted_string_literal) @value)) + `, + }, + { + meta: { + role: 'provider', + broker: 'kafka', + confidence: 0.75, + symbolName: 'sarama.ProducerMessage', + }, + query: ` + (composite_literal + type: (qualified_type + package: (package_identifier) @pkg (#eq? @pkg "sarama") + name: (type_identifier) @ty (#eq? @ty "ProducerMessage")) + body: (literal_value + (keyed_element + (literal_element (identifier) @field (#eq? @field "Topic")) + (literal_element (interpreted_string_literal) @value)))) + `, + }, + { + meta: { + role: 'provider', + broker: 'kafka', + confidence: 0.75, + symbolName: 'kafka.Writer', + }, + query: ` + (composite_literal + type: (qualified_type + package: (package_identifier) @pkg (#eq? @pkg "kafka") + name: (type_identifier) @ty (#match? @ty "^(Writer|WriterConfig)$")) + body: (literal_value + (keyed_element + (literal_element (identifier) @field (#eq? @field "Topic")) + (literal_element (interpreted_string_literal) @value)))) + `, + }, + { + meta: { + role: 'consumer', + broker: 'kafka', + confidence: 0.75, + symbolName: 'kafka.Reader', + }, + query: ` + (composite_literal + type: (qualified_type + package: (package_identifier) @pkg (#eq? @pkg "kafka") + name: (type_identifier) @ty (#match? @ty "^(Reader|ReaderConfig)$")) + body: (literal_value + (keyed_element + (literal_element (identifier) @field (#eq? @field "Topic")) + (literal_element (interpreted_string_literal) @value)))) + `, + }, + { + meta: { + role: 'consumer', + broker: 'nats', + confidence: 0.8, + symbolName: 'nc.Subscribe', + }, + query: ` + (call_expression + function: (selector_expression + operand: (identifier) @obj (#match? @obj "^(nc|js)$") + field: (field_identifier) @method (#match? @method "^[Ss]ubscribe$")) + arguments: (argument_list . (interpreted_string_literal) @value)) + `, + }, + { + meta: { + role: 'provider', + broker: 'nats', + confidence: 0.8, + symbolName: 'nc.Publish', + }, + query: ` + (call_expression + function: (selector_expression + operand: (identifier) @obj (#match? @obj "^(nc|js)$") + field: (field_identifier) @method (#match? @method "^[Pp]ublish$")) + arguments: (argument_list . (interpreted_string_literal) @value)) + `, + }, + ], +}; + +export const GO_TOPIC_PROVIDER = compilePatterns(GO_TOPIC_SPEC); diff --git a/gitnexus/src/core/group/extractors/topic-patterns/index.ts b/gitnexus/src/core/group/extractors/topic-patterns/index.ts new file mode 100644 index 0000000000..b6e1b8c1f3 --- /dev/null +++ b/gitnexus/src/core/group/extractors/topic-patterns/index.ts @@ -0,0 +1,49 @@ +import * as path from 'node:path'; +import type { CompiledPatterns } from '../tree-sitter-scanner.js'; +import type { TopicMeta } from './types.js'; +import { JAVA_TOPIC_PROVIDER } from './java.js'; +import { GO_TOPIC_PROVIDER } from './go.js'; +import { PYTHON_TOPIC_PROVIDER } from './python.js'; +import { + JAVASCRIPT_TOPIC_PROVIDER, + TYPESCRIPT_TOPIC_PROVIDER, + TSX_TOPIC_PROVIDER, +} from './node.js'; + +export type { TopicMeta, Broker } from './types.js'; + +/** + * File-extension → compiled-plugin registry for topic extraction. The + * top-level orchestrator (`topic-extractor.ts`) looks up the plugin for + * each file it visits and delegates the scanning to `tree-sitter-scanner`. + * + * Keys are lowercase extensions including the leading dot. To add a new + * language, drop a `topic-patterns/.ts` that exports a compiled + * provider, import it here and register the extension(s). No edits to + * `topic-extractor.ts` are required. + */ +const REGISTRY: Record> = { + '.java': JAVA_TOPIC_PROVIDER, + '.go': GO_TOPIC_PROVIDER, + '.py': PYTHON_TOPIC_PROVIDER, + '.js': JAVASCRIPT_TOPIC_PROVIDER, + '.jsx': JAVASCRIPT_TOPIC_PROVIDER, + '.ts': TYPESCRIPT_TOPIC_PROVIDER, + '.tsx': TSX_TOPIC_PROVIDER, +}; + +/** + * Glob pattern for files worth scanning. Kept here so adding a new + * language to the registry also widens the glob automatically via a + * single edit. + */ +export const TOPIC_SCAN_GLOB = '**/*.{ts,tsx,js,jsx,java,go,py}'; + +/** + * Return the compiled provider registered for the given file's + * extension, or `undefined` if the extension is not registered. + */ +export function getProviderForFile(rel: string): CompiledPatterns | undefined { + const ext = path.extname(rel).toLowerCase(); + return REGISTRY[ext]; +} diff --git a/gitnexus/src/core/group/extractors/topic-patterns/java.ts b/gitnexus/src/core/group/extractors/topic-patterns/java.ts new file mode 100644 index 0000000000..d126f25ce5 --- /dev/null +++ b/gitnexus/src/core/group/extractors/topic-patterns/java.ts @@ -0,0 +1,83 @@ +import Java from 'tree-sitter-java'; +import { compilePatterns, type LanguagePatterns } from '../tree-sitter-scanner.js'; +import type { TopicMeta } from './types.js'; + +/** + * Java topic extraction patterns. + * + * Detects Kafka and RabbitMQ (Spring conventions) producer/consumer APIs: + * - `@KafkaListener(topics = "xxx")` + * - `@RabbitListener(queues = "xxx")` + * - `kafkaTemplate.send("xxx", ...)` + * - `rabbitTemplate.convertAndSend("xxx", ...)` + * + * Every query MUST bind `@value` to the topic literal node. + */ +const JAVA_TOPIC_SPEC: LanguagePatterns = { + name: 'java-topic', + language: Java, + patterns: [ + { + meta: { + role: 'consumer', + broker: 'kafka', + confidence: 0.8, + symbolName: 'kafkaListener', + }, + query: ` + (annotation + name: (identifier) @name (#eq? @name "KafkaListener") + arguments: (annotation_argument_list + (element_value_pair + key: (identifier) @key (#eq? @key "topics") + value: (string_literal) @value))) + `, + }, + { + meta: { + role: 'consumer', + broker: 'rabbitmq', + confidence: 0.8, + symbolName: 'rabbitListener', + }, + query: ` + (annotation + name: (identifier) @name (#eq? @name "RabbitListener") + arguments: (annotation_argument_list + (element_value_pair + key: (identifier) @key (#eq? @key "queues") + value: (string_literal) @value))) + `, + }, + { + meta: { + role: 'provider', + broker: 'kafka', + confidence: 0.8, + symbolName: 'kafkaTemplate.send', + }, + query: ` + (method_invocation + object: (identifier) @obj (#eq? @obj "kafkaTemplate") + name: (identifier) @method (#eq? @method "send") + arguments: (argument_list . (string_literal) @value)) + `, + }, + { + meta: { + role: 'provider', + broker: 'rabbitmq', + confidence: 0.8, + symbolName: 'rabbitTemplate.convertAndSend', + }, + query: ` + (method_invocation + object: (identifier) @obj (#eq? @obj "rabbitTemplate") + name: (identifier) @method (#eq? @method "convertAndSend") + arguments: (argument_list . (string_literal) @value)) + `, + }, + ], +}; + +export const JAVA_TOPIC_PROVIDER = compilePatterns(JAVA_TOPIC_SPEC); diff --git a/gitnexus/src/core/group/extractors/topic-patterns/node.ts b/gitnexus/src/core/group/extractors/topic-patterns/node.ts new file mode 100644 index 0000000000..68f3a4ef8a --- /dev/null +++ b/gitnexus/src/core/group/extractors/topic-patterns/node.ts @@ -0,0 +1,165 @@ +import JavaScript from 'tree-sitter-javascript'; +import TypeScript from 'tree-sitter-typescript'; +import { + compilePatterns, + type LanguagePatterns, + type PatternSpec, +} from '../tree-sitter-scanner.js'; +import type { TopicMeta } from './types.js'; + +/** + * Node.js / TypeScript topic extraction patterns. + * + * Detects kafkajs, amqplib (RabbitMQ), and nats.js producer/consumer APIs: + * - `producer.send({ topic: 'xxx', ... })` (kafkajs) + * - `consumer.subscribe({ topic: 'xxx', ... })` (kafkajs) + * - `channel.consume("queue", ...)` / `channel.publish(...)` / `channel.sendToQueue(...)` + * - `nc.subscribe("topic")` / `js.subscribe("topic")` + * - `nc.publish("topic", ...)` / `js.publish("topic", ...)` + * + * The JavaScript and TypeScript tree-sitter grammars share node type + * names for every construct we query here, so the pattern sources are + * defined once and compiled against each grammar variant. We export three + * providers because Parser.Query objects are NOT portable across grammar + * instances — `.js` files use the JavaScript grammar, `.ts` uses + * TypeScript.typescript, and `.tsx` uses TypeScript.tsx. + * + * Every query MUST bind `@value` to the topic literal node. + */ +const NODE_TOPIC_PATTERNS: PatternSpec[] = [ + { + meta: { + role: 'provider', + broker: 'kafka', + confidence: 0.8, + symbolName: 'producer.send', + }, + query: ` + (call_expression + function: (member_expression + object: (identifier) @obj (#eq? @obj "producer") + property: (property_identifier) @prop (#eq? @prop "send")) + arguments: (arguments + (object + (pair + key: (property_identifier) @key (#eq? @key "topic") + value: [(string) (template_string)] @value)))) + `, + }, + { + meta: { + role: 'consumer', + broker: 'kafka', + confidence: 0.8, + symbolName: 'consumer.subscribe', + }, + query: ` + (call_expression + function: (member_expression + object: (identifier) @obj (#eq? @obj "consumer") + property: (property_identifier) @prop (#eq? @prop "subscribe")) + arguments: (arguments + (object + (pair + key: (property_identifier) @key (#eq? @key "topic") + value: [(string) (template_string)] @value)))) + `, + }, + { + meta: { + role: 'consumer', + broker: 'rabbitmq', + confidence: 0.8, + symbolName: 'channel.consume', + }, + query: ` + (call_expression + function: (member_expression + object: (identifier) @obj (#eq? @obj "channel") + property: (property_identifier) @prop (#eq? @prop "consume")) + arguments: (arguments . [(string) (template_string)] @value)) + `, + }, + { + meta: { + role: 'provider', + broker: 'rabbitmq', + confidence: 0.8, + symbolName: 'channel.publish', + }, + query: ` + (call_expression + function: (member_expression + object: (identifier) @obj (#eq? @obj "channel") + property: (property_identifier) @prop (#eq? @prop "publish")) + arguments: (arguments . [(string) (template_string)] @value)) + `, + }, + { + meta: { + role: 'provider', + broker: 'rabbitmq', + confidence: 0.8, + symbolName: 'channel.sendToQueue', + }, + query: ` + (call_expression + function: (member_expression + object: (identifier) @obj (#eq? @obj "channel") + property: (property_identifier) @prop (#eq? @prop "sendToQueue")) + arguments: (arguments . [(string) (template_string)] @value)) + `, + }, + { + meta: { + role: 'consumer', + broker: 'nats', + confidence: 0.8, + symbolName: 'nc.subscribe', + }, + query: ` + (call_expression + function: (member_expression + object: (identifier) @obj (#match? @obj "^(nc|js)$") + property: (property_identifier) @prop (#match? @prop "^[Ss]ubscribe$")) + arguments: (arguments . [(string) (template_string)] @value)) + `, + }, + { + meta: { + role: 'provider', + broker: 'nats', + confidence: 0.8, + symbolName: 'nc.publish', + }, + query: ` + (call_expression + function: (member_expression + object: (identifier) @obj (#match? @obj "^(nc|js)$") + property: (property_identifier) @prop (#match? @prop "^[Pp]ublish$")) + arguments: (arguments . [(string) (template_string)] @value)) + `, + }, +]; + +const JAVASCRIPT_TOPIC_SPEC: LanguagePatterns = { + name: 'javascript-topic', + language: JavaScript, + patterns: NODE_TOPIC_PATTERNS, +}; + +const TYPESCRIPT_TOPIC_SPEC: LanguagePatterns = { + name: 'typescript-topic', + language: TypeScript.typescript, + patterns: NODE_TOPIC_PATTERNS, +}; + +const TSX_TOPIC_SPEC: LanguagePatterns = { + name: 'tsx-topic', + language: TypeScript.tsx, + patterns: NODE_TOPIC_PATTERNS, +}; + +export const JAVASCRIPT_TOPIC_PROVIDER = compilePatterns(JAVASCRIPT_TOPIC_SPEC); +export const TYPESCRIPT_TOPIC_PROVIDER = compilePatterns(TYPESCRIPT_TOPIC_SPEC); +export const TSX_TOPIC_PROVIDER = compilePatterns(TSX_TOPIC_SPEC); diff --git a/gitnexus/src/core/group/extractors/topic-patterns/python.ts b/gitnexus/src/core/group/extractors/topic-patterns/python.ts new file mode 100644 index 0000000000..d84cae9990 --- /dev/null +++ b/gitnexus/src/core/group/extractors/topic-patterns/python.ts @@ -0,0 +1,119 @@ +import Python from 'tree-sitter-python'; +import { compilePatterns, type LanguagePatterns } from '../tree-sitter-scanner.js'; +import type { TopicMeta } from './types.js'; + +/** + * Python topic extraction patterns. + * + * Detects kafka-python, pika (RabbitMQ), and nats-py producer/consumer APIs: + * - `KafkaConsumer('topic', ...)` + * - `producer.send('topic', ...)` / `producer.produce('topic', ...)` + * - `channel.basic_consume(queue='xxx', ...)` + * - `channel.basic_publish(exchange='xxx', ...)` + * - `await nc.subscribe('topic')` + * - `await nc.publish('topic', ...)` + * + * Every query MUST bind `@value` to the topic literal node. + */ +const PYTHON_TOPIC_SPEC: LanguagePatterns = { + name: 'python-topic', + language: Python, + patterns: [ + { + meta: { + role: 'consumer', + broker: 'kafka', + confidence: 0.7, + symbolName: 'KafkaConsumer', + }, + query: ` + (call + function: (identifier) @func (#eq? @func "KafkaConsumer") + arguments: (argument_list . (string) @value)) + `, + }, + { + meta: { + role: 'provider', + broker: 'kafka', + confidence: 0.7, + symbolName: 'producer.send', + }, + query: ` + (call + function: (attribute + object: (identifier) @obj (#eq? @obj "producer") + attribute: (identifier) @method (#match? @method "^(send|produce)$")) + arguments: (argument_list . (string) @value)) + `, + }, + { + meta: { + role: 'consumer', + broker: 'rabbitmq', + confidence: 0.7, + symbolName: 'basic_consume', + }, + query: ` + (call + function: (attribute + object: (identifier) @obj (#eq? @obj "channel") + attribute: (identifier) @method (#eq? @method "basic_consume")) + arguments: (argument_list + (keyword_argument + name: (identifier) @kw (#eq? @kw "queue") + value: (string) @value))) + `, + }, + { + meta: { + role: 'provider', + broker: 'rabbitmq', + confidence: 0.7, + symbolName: 'basic_publish', + }, + query: ` + (call + function: (attribute + object: (identifier) @obj (#eq? @obj "channel") + attribute: (identifier) @method (#eq? @method "basic_publish")) + arguments: (argument_list + (keyword_argument + name: (identifier) @kw (#eq? @kw "exchange") + value: (string) @value))) + `, + }, + { + meta: { + role: 'consumer', + broker: 'nats', + confidence: 0.75, + symbolName: 'nc.subscribe', + }, + query: ` + (call + function: (attribute + object: (identifier) @obj (#eq? @obj "nc") + attribute: (identifier) @method (#eq? @method "subscribe")) + arguments: (argument_list . (string) @value)) + `, + }, + { + meta: { + role: 'provider', + broker: 'nats', + confidence: 0.75, + symbolName: 'nc.publish', + }, + query: ` + (call + function: (attribute + object: (identifier) @obj (#eq? @obj "nc") + attribute: (identifier) @method (#eq? @method "publish")) + arguments: (argument_list . (string) @value)) + `, + }, + ], +}; + +export const PYTHON_TOPIC_PROVIDER = compilePatterns(PYTHON_TOPIC_SPEC); diff --git a/gitnexus/src/core/group/extractors/topic-patterns/types.ts b/gitnexus/src/core/group/extractors/topic-patterns/types.ts new file mode 100644 index 0000000000..3a27f21d3d --- /dev/null +++ b/gitnexus/src/core/group/extractors/topic-patterns/types.ts @@ -0,0 +1,27 @@ +/** + * Shared types for the topic-extractor language plugins. + * + * Each plugin lives in its own file (java.ts, go.ts, ...) and owns the + * tree-sitter grammar import + query sources. The top-level + * `topic-extractor.ts` orchestrator only knows about this type module and + * the plugin registry (`./index.ts`). It MUST NOT import any grammar or + * query text directly — that's the whole point of the split. + */ + +export type Broker = 'kafka' | 'rabbitmq' | 'nats'; + +/** + * Per-pattern payload every topic plugin attaches to its query. Whatever + * the pattern matches, the orchestrator receives this object verbatim + * and uses it to build an `ExtractedContract`. + * + * Plugins produce one `TopicMeta` per pattern (not per match) because a + * single query uniquely identifies its broker/role/confidence triple. + */ +export interface TopicMeta { + role: 'provider' | 'consumer'; + broker: Broker; + confidence: number; + /** Short human-readable label of the API being detected. */ + symbolName: string; +} diff --git a/gitnexus/src/core/group/extractors/tree-sitter-scanner.ts b/gitnexus/src/core/group/extractors/tree-sitter-scanner.ts new file mode 100644 index 0000000000..d792002caf --- /dev/null +++ b/gitnexus/src/core/group/extractors/tree-sitter-scanner.ts @@ -0,0 +1,177 @@ +import Parser from 'tree-sitter'; + +/** + * Shared, language-agnostic tree-sitter scanning utilities used by group + * extractors (topic, http, grpc, ...). + * + * Design goals: + * - The top-level extractors must not import any tree-sitter grammar. + * - Per-language plugins own their grammar import, their query sources, + * and the mapping from capture → meta. + * - This module provides the plumbing: compile queries once per plugin, + * parse a file with a given grammar, run all patterns, and return the + * captured `string_literal`-style nodes together with the plugin's meta. + */ + +/** + * One pattern owned by a language plugin. Each pattern owns a tree-sitter + * S-expression query. The query MUST contain a capture named `@value` + * whose node text is the literal we want to extract (string/template/etc). + * + * `TMeta` is the plugin-specific payload the orchestrator receives back + * when this pattern matches — e.g. for topic extraction it carries the + * broker name, role, confidence, symbol name. + */ +export interface PatternSpec { + /** Tree-sitter S-expression. MUST contain a `@value` capture. */ + query: string; + /** Plugin-specific payload returned on every match. */ + meta: TMeta; +} + +/** + * A set of patterns owned by one language plugin, bound to a specific + * tree-sitter grammar. + * + * `language` is typed as `unknown` because tree-sitter's TypeScript + * declarations use `any` for the grammar object, and the grammar modules + * export different shapes (plain grammar vs. namespace with `typescript` + * / `tsx` members). Callers pass the concrete grammar object; this + * module forwards it to `parser.setLanguage` / `new Parser.Query`. + */ +export interface LanguagePatterns { + /** Human-readable plugin name for diagnostics. */ + name: string; + /** tree-sitter grammar object. */ + language: unknown; + /** Patterns authored against `language`. */ + patterns: PatternSpec[]; +} + +/** + * Compiled form of a `LanguagePatterns` bundle. Queries are compiled + * eagerly at module load time so a broken grammar/query pair fails + * loudly the first time the plugin is imported, instead of silently + * at scan time when no contract is produced. + */ +export interface CompiledPatterns { + name: string; + language: unknown; + patterns: CompiledPattern[]; +} + +export interface CompiledPattern { + query: Parser.Query; + meta: TMeta; +} + +/** + * One match returned by `scanFile`. The orchestrator receives the raw + * literal text (still including any surrounding quotes) together with + * the plugin meta, and is responsible for calling `unquoteLiteral` / + * emitting a domain object (ExtractedContract, Route, ...). + */ +export interface ScanMatch { + meta: TMeta; + /** The node captured as `@value` (the literal). */ + valueNode: Parser.SyntaxNode; + /** Raw text of the captured value node — caller must unquote. */ + valueText: string; +} + +/** + * Compile a LanguagePatterns bundle. Call this once per plugin, at + * module load time, and export the result. Throws if any pattern + * fails to compile against the grammar — that's a bug in the plugin + * author's query, not a runtime condition. + */ +export function compilePatterns(bundle: LanguagePatterns): CompiledPatterns { + const compiled: CompiledPattern[] = []; + for (const spec of bundle.patterns) { + try { + const query = new Parser.Query(bundle.language, spec.query); + compiled.push({ query, meta: spec.meta }); + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + throw new Error( + `[tree-sitter-scanner] Failed to compile pattern in ${bundle.name}: ${message}\n` + + `Query source:\n${spec.query}`, + ); + } + } + return { name: bundle.name, language: bundle.language, patterns: compiled }; +} + +/** + * Parse `content` as source code of the plugin's language and run every + * compiled pattern against the resulting AST. Returns one `ScanMatch` per + * matched `@value` capture, carrying the plugin's meta payload. + * + * Errors are swallowed at the file level (malformed file must not abort + * the whole extract). Individual pattern failures are swallowed too so + * a single unusable query doesn't block the rest of the plugin. + */ +export function scanFile( + parser: Parser, + plugin: CompiledPatterns, + content: string, +): ScanMatch[] { + const out: ScanMatch[] = []; + let tree: Parser.Tree; + try { + parser.setLanguage(plugin.language); + tree = parser.parse(content); + } catch { + return out; + } + + for (const compiled of plugin.patterns) { + let matches: Parser.QueryMatch[]; + try { + matches = compiled.query.matches(tree.rootNode); + } catch { + continue; + } + for (const match of matches) { + const valueCapture = match.captures.find((c) => c.name === 'value'); + if (!valueCapture) continue; + out.push({ + meta: compiled.meta, + valueNode: valueCapture.node, + valueText: valueCapture.node.text, + }); + } + } + + return out; +} + +/** + * Strip enclosing quotes from a tree-sitter string literal node's text. + * Handles single / double / template quotes, Python triple-quoted strings, + * and Go raw string literals (backticks). + * + * Returns null for empty/nullish input so callers can uniformly skip + * captures whose value is missing. + */ +export function unquoteLiteral(raw: string): string | null { + if (!raw) return null; + + // Python triple-quoted + if ( + (raw.startsWith('"""') && raw.endsWith('"""')) || + (raw.startsWith("'''") && raw.endsWith("'''")) + ) { + return raw.slice(3, -3); + } + + const first = raw[0]; + const last = raw[raw.length - 1]; + if ((first === '"' || first === "'" || first === '`') && last === first && raw.length >= 2) { + return raw.slice(1, -1); + } + + // Some grammars expose the string content without quotes already (e.g. + // Python `string_content` child). Return as-is. + return raw; +} From 6a76ac0753f9229441897c1bec8c6d7c602018e0 Mon Sep 17 00:00:00 2001 From: ivkond Date: Sun, 12 Apr 2026 00:22:54 +0000 Subject: [PATCH 03/10] refactor(group): migrate http-route-extractor Strategy B to tree-sitter plugins MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2 of the extractor refactor requested by @magyargergo on #796. Same architecture as the phase 1 topic-extractor rewrite: a thin, language-agnostic orchestrator plus per-language plugins that own tree-sitter grammars and query sources. The top-level extractor file no longer imports any tree-sitter grammar or query string. ## Architecture ``` src/core/group/extractors/ ├── tree-sitter-scanner.ts # shared, language-agnostic primitives ├── http-route-extractor.ts # thin orchestrator (no grammar imports) └── http-patterns/ ├── types.ts # HttpDetection, HttpLanguagePlugin, HttpRole ├── index.ts # registry: ext → plugin + HTTP_SCAN_GLOB ├── java.ts # tree-sitter-java: Spring + RestTemplate/WebClient/OkHttp ├── go.ts # tree-sitter-go: gin/echo/HandleFunc + http/resty consumers ├── python.ts # tree-sitter-python: FastAPI + requests ├── php.ts # tree-sitter-php: Laravel Route::get/... └── node.ts # tree-sitter-javascript + tree-sitter-typescript: # NestJS controllers, Express, fetch, axios ``` **Shared scanner (`tree-sitter-scanner.ts`)** — generalised from phase 1: - `ScanMatch.captures` is now a full `CaptureMap` (every named capture the query binds, not just a single `@value`). Topic extractor updated to read `match.captures.value` accordingly. - New `runCompiledPatterns(plugin, tree)` helper lets plugins run multiple query bundles against the same pre-parsed tree. This is needed for HTTP plugins that combine a class-prefix query with a method-route query (Spring, NestJS). - `scanFile` becomes a thin wrapper over `parser.parse + runCompiledPatterns`. **HTTP plugin shape** — unlike topic plugins, HTTP plugins expose a `scan(tree)` function rather than a flat pattern list. This reflects HTTP's more complex extraction: each detection needs method + path + handler name, and framework patterns like Spring `@RequestMapping` / NestJS `@Controller` require cross-referencing a class-level prefix with method-level annotations. Plugins internally use `compilePatterns` + `runCompiledPatterns` and walk the AST to resolve the class/method relationships. **Per-framework coverage:** - **Java (`java.ts`)** - Spring: `@RequestMapping("/api/v2")` class prefix + `@(Get|Post|Put| Delete|Patch)Mapping("/sub")` method routes, joined via the enclosing `class_declaration` node id. - `RestTemplate.getForObject/postForEntity/put/delete/patchForObject` → method derived from API name. - `WebClient.method(HttpMethod.X, "/path")` → method from `HttpMethod.X` capture. - `new Request.Builder().url("/path")` → OkHttp consumer. - **Go (`go.ts`)** - gin / echo / chi frameworks: `\w+.GET("/path", handler)` captures upper-case verb + handler identifier. - `net/http.HandleFunc("/path", handler)` → provider (default GET). - `http.Get/Post/Head` consumer, `http.NewRequest("METHOD", ...)`, resty `client.R().Get/Post/...`. - **Python (`python.ts`)** - `@app.get("/path")` FastAPI decorators. - `requests.get/post/...` and `requests.request("METHOD", "url")`. - **PHP (`php.ts`)** - Laravel `Route::get/post/.../patch('/path', ...)` via `scoped_call_expression`. Uses `PHP.php_only` to match the existing ingestion pipeline's grammar selection. - **Node (`node.ts`) — JS + TS + TSX** - Pattern sources defined once, compiled against three grammar variants (`JavaScript`, `TypeScript.typescript`, `TypeScript.tsx`) because `Parser.Query` objects are not portable across grammars. Exports three plugins sharing the same `scan` logic. - NestJS: `@Controller('prefix')` decorators are siblings of the class in `export_statement` / `program`; `@Get(':id')` decorators are siblings of the method in `class_body`. The plugin walks decorator → next named sibling to find the decorated class / method, then combines the class prefix with the method path. Only emits NestJS detections when the enclosing class has a real `@Controller` decorator — prevents false positives from generic classes that happen to use `@Get` from another library. - Express: `(router|app).('/path', ...)`. - `fetch(url)` (default GET) + `fetch(url, { method: 'X' })` (uses two queries + a SyntaxNode-id dedupe set so URL literals aren't double-emitted by the options variant). - `axios.get/post/...`. ## Orchestrator changes `http-route-extractor.ts` drops every `scanXxxProviders` / `scanXxxConsumers` regex method and replaces them with a single source-scan loop that delegates to `getPluginForFile(rel).scan(tree)`. The orchestrator still owns: - **Path normalization** (`normalizeHttpPath`, `normalizeConsumerPath`) — language-agnostic string processing shared by both strategies. - **Graph-assisted Strategy A** (`HANDLES_ROUTE` / `FETCHES` / `CONTAINS` Cypher queries) — unchanged in spirit. The only regex helpers it used (`inferMethodFromFileScan`, `pickJavaHandlerName`) are now replaced by a lookup against the plugin's detections for the same file: for each route row, find the detection whose normalized path matches, and pull the HTTP method + handler name from it. - **Per-file parse cache** — the orchestrator parses each relevant file at most once per `extract()` call. Both the graph-assisted enrichment loop and the source-scan fallback share the same `cachedDetections` map, so we never run the plugin twice for the same file. ## Why this is better than the regex version 1. **Comments and strings for free.** The old regex would match `// router.get('/fake')` as a real Express route; tree-sitter never visits string/comment nodes. 2. **Structural controller-prefix.** Spring and NestJS class-prefix joining is now scoped to the enclosing class via `class_declaration` node ids, eliminating file-wide state that broke when a file had multiple controllers. 3. **Precise NestJS disambiguation.** The plugin only emits a NestJS detection when the enclosing class has a real `@Controller` decorator — the old regex would fire on any `@Get(...)` in the file regardless of surrounding context. 4. **Language-agnostic extension.** Adding Ruby / Rust / Kotlin HTTP detection later means dropping one file in `http-patterns/` — no changes to the shared scanner, the orchestrator, or the Strategy A Cypher queries. ## Tests - `http-route-extractor.test.ts` — **18/18 pass** (tests unchanged; they're contract-style input/output tests and the contract shape is unchanged). Covers Spring class prefix, Express, gin/echo, stdlib HandleFunc, NestJS, Laravel, FastAPI for providers and fetch/axios/python-requests/rest-template/webClient/okhttp/go-stdlib/ resty for consumers, plus graph-first Strategy A for both. - `topic-extractor.test.ts` — **30/30 pass** after the `captures.value` API migration. - `grpc-extractor.test.ts` — 43/43 pass (untouched; phase 3). - `manifest-extractor.test.ts` — 8/8 pass (untouched). - `service.test.ts`, `sync.test.ts`, `storage.test.ts` — 41/41 pass. - `npx tsc -p tsconfig.json --noEmit` clean. ## Scope discipline (per GUARDRAILS.md) - Only files under `src/core/group/extractors/` are touched. - No changes to pipeline.ts, MCP surface, ingestion, or tests. - No CI / release / security / secrets changes. - Tree-sitter grammars imported by plugins (`tree-sitter-java`, `tree-sitter-go`, `tree-sitter-python`, `tree-sitter-php`, `tree-sitter-javascript`, `tree-sitter-typescript`) are all already in `package.json` for the existing ingestion pipeline. ## Phase 3 plan - **grpc-extractor** gets the same treatment: plugin-per-language under `grpc-patterns/` for Java / Go / Python / TS detection. `.proto` files remain an open question — no `tree-sitter-proto` grammar is installed, so the in-tree string-sanitizing parser from PR #796's self-review stays as a pragmatic exception unless the maintainer wants us to add `tree-sitter-proto` as a new dep. Co-authored-by: Claude --- .../core/group/extractors/http-patterns/go.ts | 224 +++++++ .../group/extractors/http-patterns/index.ts | 50 ++ .../group/extractors/http-patterns/java.ts | 267 ++++++++ .../group/extractors/http-patterns/node.ts | 373 +++++++++++ .../group/extractors/http-patterns/php.ts | 79 +++ .../group/extractors/http-patterns/python.ts | 142 ++++ .../group/extractors/http-patterns/types.ts | 65 ++ .../group/extractors/http-route-extractor.ts | 606 ++++++------------ .../core/group/extractors/topic-extractor.ts | 4 +- .../group/extractors/tree-sitter-scanner.ts | 88 +-- 10 files changed, 1461 insertions(+), 437 deletions(-) create mode 100644 gitnexus/src/core/group/extractors/http-patterns/go.ts create mode 100644 gitnexus/src/core/group/extractors/http-patterns/index.ts create mode 100644 gitnexus/src/core/group/extractors/http-patterns/java.ts create mode 100644 gitnexus/src/core/group/extractors/http-patterns/node.ts create mode 100644 gitnexus/src/core/group/extractors/http-patterns/php.ts create mode 100644 gitnexus/src/core/group/extractors/http-patterns/python.ts create mode 100644 gitnexus/src/core/group/extractors/http-patterns/types.ts diff --git a/gitnexus/src/core/group/extractors/http-patterns/go.ts b/gitnexus/src/core/group/extractors/http-patterns/go.ts new file mode 100644 index 0000000000..afbfaad562 --- /dev/null +++ b/gitnexus/src/core/group/extractors/http-patterns/go.ts @@ -0,0 +1,224 @@ +import Go from 'tree-sitter-go'; +import { + compilePatterns, + runCompiledPatterns, + unquoteLiteral, + type LanguagePatterns, +} from '../tree-sitter-scanner.js'; +import type { HttpDetection, HttpLanguagePlugin } from './types.js'; + +/** + * Go HTTP plugin. Handles: + * - gin / echo / chi framework routing — `r.GET("/path", handler)` + * - net/http stdlib — `http.HandleFunc("/path", handler)` + * - net/http consumer — `http.Get(...)`, `http.NewRequest("METHOD", ...)` + * - resty consumer — `client.R().Delete("/path")` + */ + +// ─── Provider: framework routing ────────────────────────────────────── +// Matches `\w+\.GET(...)` etc. (gin, echo, chi all share this shape). +// Captures the HTTP method (field name), path literal, and handler +// identifier passed as the second argument. +const FRAMEWORK_ROUTE_PATTERNS = compilePatterns({ + name: 'go-framework-route', + language: Go, + patterns: [ + { + meta: {}, + query: ` + (call_expression + function: (selector_expression + field: (field_identifier) @http_method (#match? @http_method "^(GET|POST|PUT|DELETE|PATCH)$")) + arguments: (argument_list + (interpreted_string_literal) @path + (identifier) @handler)) + `, + }, + ], +} satisfies LanguagePatterns>); + +// ─── Provider: net/http `http.HandleFunc("/p", handler)` ───────────── +const HANDLE_FUNC_PATTERNS = compilePatterns({ + name: 'go-handle-func', + language: Go, + patterns: [ + { + meta: {}, + query: ` + (call_expression + function: (selector_expression + operand: (identifier) @pkg (#eq? @pkg "http") + field: (field_identifier) @fn (#eq? @fn "HandleFunc")) + arguments: (argument_list + (interpreted_string_literal) @path + (identifier) @handler)) + `, + }, + ], +} satisfies LanguagePatterns>); + +// ─── Consumer: net/http stdlib Get / Post / Head ───────────────────── +const HTTP_CLIENT_METHOD_TO_HTTP: Record = { + Get: 'GET', + Post: 'POST', + Head: 'GET', // HEAD has no body semantics we care about — treat as GET for contract matching +}; + +const HTTP_CLIENT_PATTERNS = compilePatterns({ + name: 'go-http-client', + language: Go, + patterns: [ + { + meta: {}, + query: ` + (call_expression + function: (selector_expression + operand: (identifier) @pkg (#eq? @pkg "http") + field: (field_identifier) @fn (#match? @fn "^(Get|Post|Head)$")) + arguments: (argument_list . (interpreted_string_literal) @path)) + `, + }, + ], +} satisfies LanguagePatterns>); + +// ─── Consumer: net/http `http.NewRequest("METHOD", "/path", ...)` ──── +const NEW_REQUEST_PATTERNS = compilePatterns({ + name: 'go-new-request', + language: Go, + patterns: [ + { + meta: {}, + query: ` + (call_expression + function: (selector_expression + operand: (identifier) @pkg (#eq? @pkg "http") + field: (field_identifier) @fn (#eq? @fn "NewRequest")) + arguments: (argument_list + . + (interpreted_string_literal) @http_method + (interpreted_string_literal) @path)) + `, + }, + ], +} satisfies LanguagePatterns>); + +// ─── Consumer: resty `client.R().Delete("/path")` ───────────────────── +// Matches any chained call whose receiver is `something.R()` and whose +// method name is an HTTP verb. This is how go-resty's fluent API looks. +const RESTY_PATTERNS = compilePatterns({ + name: 'go-resty', + language: Go, + patterns: [ + { + meta: {}, + query: ` + (call_expression + function: (selector_expression + operand: (call_expression + function: (selector_expression + field: (field_identifier) @r (#eq? @r "R"))) + field: (field_identifier) @http_method (#match? @http_method "^(Get|Post|Put|Delete|Patch)$")) + arguments: (argument_list . (interpreted_string_literal) @path)) + `, + }, + ], +} satisfies LanguagePatterns>); + +export const GO_HTTP_PLUGIN: HttpLanguagePlugin = { + name: 'go-http', + language: Go, + scan(tree) { + const out: HttpDetection[] = []; + + // Framework providers: r.GET/POST/... with handler identifier + for (const match of runCompiledPatterns(FRAMEWORK_ROUTE_PATTERNS, tree)) { + const methodNode = match.captures.http_method; + const pathNode = match.captures.path; + const handlerNode = match.captures.handler; + if (!methodNode || !pathNode) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'provider', + framework: 'go-framework', + method: methodNode.text.toUpperCase(), + path, + name: handlerNode?.text ?? null, + confidence: 0.8, + }); + } + + // net/http HandleFunc: default method GET + for (const match of runCompiledPatterns(HANDLE_FUNC_PATTERNS, tree)) { + const pathNode = match.captures.path; + const handlerNode = match.captures.handler; + if (!pathNode) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'provider', + framework: 'go-stdlib', + method: 'GET', + path, + name: handlerNode?.text ?? null, + confidence: 0.8, + }); + } + + // net/http client: http.Get/Post/Head + for (const match of runCompiledPatterns(HTTP_CLIENT_PATTERNS, tree)) { + const fnNode = match.captures.fn; + const pathNode = match.captures.path; + if (!fnNode || !pathNode) continue; + const httpMethod = HTTP_CLIENT_METHOD_TO_HTTP[fnNode.text]; + if (!httpMethod) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'consumer', + framework: 'go-stdlib', + method: httpMethod, + path, + name: null, + confidence: 0.7, + }); + } + + // net/http NewRequest + for (const match of runCompiledPatterns(NEW_REQUEST_PATTERNS, tree)) { + const methodNode = match.captures.http_method; + const pathNode = match.captures.path; + if (!methodNode || !pathNode) continue; + const method = unquoteLiteral(methodNode.text); + const path = unquoteLiteral(pathNode.text); + if (method === null || path === null) continue; + out.push({ + role: 'consumer', + framework: 'go-stdlib', + method: method.toUpperCase(), + path, + name: null, + confidence: 0.7, + }); + } + + // resty + for (const match of runCompiledPatterns(RESTY_PATTERNS, tree)) { + const methodNode = match.captures.http_method; + const pathNode = match.captures.path; + if (!methodNode || !pathNode) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'consumer', + framework: 'go-resty', + method: methodNode.text.toUpperCase(), + path, + name: null, + confidence: 0.7, + }); + } + + return out; + }, +}; diff --git a/gitnexus/src/core/group/extractors/http-patterns/index.ts b/gitnexus/src/core/group/extractors/http-patterns/index.ts new file mode 100644 index 0000000000..e33d32a79e --- /dev/null +++ b/gitnexus/src/core/group/extractors/http-patterns/index.ts @@ -0,0 +1,50 @@ +import * as path from 'node:path'; +import type { HttpLanguagePlugin } from './types.js'; +import { JAVA_HTTP_PLUGIN } from './java.js'; +import { GO_HTTP_PLUGIN } from './go.js'; +import { PYTHON_HTTP_PLUGIN } from './python.js'; +import { PHP_HTTP_PLUGIN } from './php.js'; +import { JAVASCRIPT_HTTP_PLUGIN, TYPESCRIPT_HTTP_PLUGIN, TSX_HTTP_PLUGIN } from './node.js'; + +export type { HttpDetection, HttpLanguagePlugin, HttpRole } from './types.js'; + +/** + * File-extension → HTTP language plugin registry. The top-level + * orchestrator (`http-route-extractor.ts`) looks up the plugin for each + * file it visits and delegates the tree-sitter scanning to the plugin. + * + * Keys are lowercase extensions including the leading dot. To add a + * new language, drop a `http-patterns/.ts` that exports a + * `HttpLanguagePlugin`, import it here and register the extension(s). + * No edits to `http-route-extractor.ts` are required. + */ +const REGISTRY: Record = { + '.java': JAVA_HTTP_PLUGIN, + '.go': GO_HTTP_PLUGIN, + '.py': PYTHON_HTTP_PLUGIN, + '.php': PHP_HTTP_PLUGIN, + '.js': JAVASCRIPT_HTTP_PLUGIN, + '.jsx': JAVASCRIPT_HTTP_PLUGIN, + '.ts': TYPESCRIPT_HTTP_PLUGIN, + '.tsx': TSX_HTTP_PLUGIN, +}; + +/** + * Glob for files worth scanning for HTTP routes. Kept alongside the + * registry so adding a new language widens the glob in one edit. + * + * `.vue` / `.svelte` files are intentionally omitted for the source-scan + * path — they need their own grammar-aware extraction and the existing + * regex fallback for them was never very accurate. The graph-assisted + * Strategy A still handles them via the ingestion pipeline. + */ +export const HTTP_SCAN_GLOB = '**/*.{ts,tsx,js,jsx,java,go,py,php}'; + +/** + * Return the HTTP plugin registered for the given file's extension, + * or `undefined` if the extension is not registered. + */ +export function getPluginForFile(rel: string): HttpLanguagePlugin | undefined { + const ext = path.extname(rel).toLowerCase(); + return REGISTRY[ext]; +} diff --git a/gitnexus/src/core/group/extractors/http-patterns/java.ts b/gitnexus/src/core/group/extractors/http-patterns/java.ts new file mode 100644 index 0000000000..484f74fb2e --- /dev/null +++ b/gitnexus/src/core/group/extractors/http-patterns/java.ts @@ -0,0 +1,267 @@ +import Parser from 'tree-sitter'; +import Java from 'tree-sitter-java'; +import { + compilePatterns, + runCompiledPatterns, + unquoteLiteral, + type LanguagePatterns, +} from '../tree-sitter-scanner.js'; +import type { HttpDetection, HttpLanguagePlugin } from './types.js'; + +/** + * Java HTTP plugin. Handles: + * - Spring `@RequestMapping` class prefixes + `@(Get|Post|...)Mapping` method annotations + * - Spring `RestTemplate.getForObject/...`, `WebClient.method(HttpMethod.X, ...)` + * - OkHttp `new Request.Builder().url("...")` + * + * The plugin runs two pattern bundles: one to collect class-level + * `@RequestMapping` prefixes keyed by the enclosing class node, and a + * second to match method-level annotations. The `scan` function walks + * up from each matched annotation to find its enclosing class and + * combines the prefix with the method path. + */ + +const METHOD_ANNOTATION_TO_HTTP: Record = { + GetMapping: 'GET', + PostMapping: 'POST', + PutMapping: 'PUT', + DeleteMapping: 'DELETE', + PatchMapping: 'PATCH', +}; + +// ─── Provider: Spring class-level @RequestMapping prefix ────────────── +const SPRING_CLASS_PREFIX_PATTERNS = compilePatterns({ + name: 'java-spring-class-prefix', + language: Java, + patterns: [ + { + meta: {}, + query: ` + (class_declaration + (modifiers + (annotation + name: (identifier) @ann (#eq? @ann "RequestMapping") + arguments: (annotation_argument_list (string_literal) @prefix)))) @class + `, + }, + ], +} satisfies LanguagePatterns>); + +// ─── Provider: Spring @(Get|Post|...)Mapping method annotations ─────── +const SPRING_METHOD_ROUTE_PATTERNS = compilePatterns({ + name: 'java-spring-method-route', + language: Java, + patterns: [ + { + meta: {}, + query: ` + (method_declaration + (modifiers + (annotation + name: (identifier) @ann (#match? @ann "^(Get|Post|Put|Delete|Patch)Mapping$") + arguments: (annotation_argument_list (string_literal) @path))) + name: (identifier) @method_name) @method + `, + }, + ], +} satisfies LanguagePatterns>); + +// ─── Consumer: Spring RestTemplate (object-named + method-named) ────── +// RestTemplate.getForObject / getForEntity → GET +// RestTemplate.postForObject / postForEntity → POST +// RestTemplate.put → PUT +// RestTemplate.delete → DELETE +// RestTemplate.patchForObject → PATCH +const REST_TEMPLATE_TO_HTTP: Record = { + getForObject: 'GET', + getForEntity: 'GET', + postForObject: 'POST', + postForEntity: 'POST', + put: 'PUT', + delete: 'DELETE', + patchForObject: 'PATCH', +}; + +interface RestTemplateMeta { + framework: 'spring-rest-template'; +} + +const REST_TEMPLATE_PATTERNS = compilePatterns({ + name: 'java-rest-template', + language: Java, + patterns: [ + { + meta: { framework: 'spring-rest-template' }, + query: ` + (method_invocation + object: (identifier) @obj (#eq? @obj "restTemplate") + name: (identifier) @method + arguments: (argument_list . (string_literal) @path)) + `, + }, + ], +} satisfies LanguagePatterns); + +// ─── Consumer: Spring WebClient — webClient.method(HttpMethod.X, "path") ─ +const WEB_CLIENT_PATTERNS = compilePatterns({ + name: 'java-web-client', + language: Java, + patterns: [ + { + meta: {}, + query: ` + (method_invocation + object: (identifier) @obj (#eq? @obj "webClient") + name: (identifier) @method (#eq? @method "method") + arguments: (argument_list + (field_access + object: (identifier) @httpMethodCls (#eq? @httpMethodCls "HttpMethod") + field: (identifier) @http_method) + (string_literal) @path)) + `, + }, + ], +} satisfies LanguagePatterns>); + +// ─── Consumer: OkHttp `new Request.Builder().url("path")` ───────────── +// Note: `Request.Builder` is a `scoped_type_identifier` whose text includes +// the dot, so `#eq?` against the literal string matches cleanly (no need +// to escape a regex dot). +const OK_HTTP_PATTERNS = compilePatterns({ + name: 'java-okhttp', + language: Java, + patterns: [ + { + meta: {}, + query: ` + (method_invocation + object: (object_creation_expression + type: (scoped_type_identifier) @type (#eq? @type "Request.Builder")) + name: (identifier) @method (#eq? @method "url") + arguments: (argument_list . (string_literal) @path)) + `, + }, + ], +} satisfies LanguagePatterns>); + +/** + * Find the nearest enclosing class_declaration ancestor for a node, or + * null if the node is top-level. Tree-sitter's SyntaxNode.parent walks + * one level at a time. + */ +function findEnclosingClass(node: Parser.SyntaxNode): Parser.SyntaxNode | null { + let cur: Parser.SyntaxNode | null = node.parent; + while (cur) { + if (cur.type === 'class_declaration') return cur; + cur = cur.parent; + } + return null; +} + +/** + * Join a class-level prefix and a method-level path into a single URL + * path. Mirrors the semantics of the original regex implementation: + * strip trailing slashes on the prefix, then ensure a single slash + * between prefix and method path. + */ +function joinPath(prefix: string, methodPath: string): string { + const cleanPrefix = prefix.replace(/^\/+/, '').replace(/\/+$/, ''); + const cleanSub = methodPath.replace(/^\/+/, ''); + if (!cleanPrefix) return `/${cleanSub}`; + return `/${cleanPrefix}/${cleanSub}`; +} + +export const JAVA_HTTP_PLUGIN: HttpLanguagePlugin = { + name: 'java-http', + language: Java, + scan(tree) { + const out: HttpDetection[] = []; + + // ─── Providers: Spring class prefix + method annotations ──────── + const prefixByClassId = new Map(); + for (const match of runCompiledPatterns(SPRING_CLASS_PREFIX_PATTERNS, tree)) { + const prefixNode = match.captures.prefix; + const classNode = match.captures.class; + if (!prefixNode || !classNode) continue; + const prefix = unquoteLiteral(prefixNode.text); + if (prefix !== null) prefixByClassId.set(classNode.id, prefix); + } + + for (const match of runCompiledPatterns(SPRING_METHOD_ROUTE_PATTERNS, tree)) { + const annNode = match.captures.ann; + const pathNode = match.captures.path; + const nameNode = match.captures.method_name; + const methodNode = match.captures.method; + if (!annNode || !pathNode || !methodNode) continue; + const httpMethod = METHOD_ANNOTATION_TO_HTTP[annNode.text]; + if (!httpMethod) continue; + const rawPath = unquoteLiteral(pathNode.text); + if (rawPath === null) continue; + const enclosingClass = findEnclosingClass(methodNode); + const prefix = enclosingClass ? (prefixByClassId.get(enclosingClass.id) ?? '') : ''; + const fullPath = joinPath(prefix, rawPath); + out.push({ + role: 'provider', + framework: 'spring', + method: httpMethod, + path: fullPath, + name: nameNode?.text ?? null, + confidence: 0.8, + }); + } + + // ─── Consumers: RestTemplate ──────────────────────────────────── + for (const match of runCompiledPatterns(REST_TEMPLATE_PATTERNS, tree)) { + const methodNode = match.captures.method; + const pathNode = match.captures.path; + if (!methodNode || !pathNode) continue; + const httpMethod = REST_TEMPLATE_TO_HTTP[methodNode.text]; + if (!httpMethod) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'consumer', + framework: 'spring-rest-template', + method: httpMethod, + path, + name: null, + confidence: 0.7, + }); + } + + // ─── Consumers: WebClient.method(HttpMethod.X, "path") ────────── + for (const match of runCompiledPatterns(WEB_CLIENT_PATTERNS, tree)) { + const httpMethodNode = match.captures.http_method; + const pathNode = match.captures.path; + if (!httpMethodNode || !pathNode) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'consumer', + framework: 'spring-web-client', + method: httpMethodNode.text.toUpperCase(), + path, + name: null, + confidence: 0.7, + }); + } + + // ─── Consumers: OkHttp Request.Builder().url("path") ──────────── + for (const match of runCompiledPatterns(OK_HTTP_PATTERNS, tree)) { + const pathNode = match.captures.path; + if (!pathNode) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'consumer', + framework: 'okhttp', + method: 'GET', + path, + name: null, + confidence: 0.7, + }); + } + + return out; + }, +}; diff --git a/gitnexus/src/core/group/extractors/http-patterns/node.ts b/gitnexus/src/core/group/extractors/http-patterns/node.ts new file mode 100644 index 0000000000..587f48e8c7 --- /dev/null +++ b/gitnexus/src/core/group/extractors/http-patterns/node.ts @@ -0,0 +1,373 @@ +import Parser from 'tree-sitter'; +import JavaScript from 'tree-sitter-javascript'; +import TypeScript from 'tree-sitter-typescript'; +import { + compilePatterns, + runCompiledPatterns, + unquoteLiteral, + type CompiledPatterns, + type LanguagePatterns, + type PatternSpec, +} from '../tree-sitter-scanner.js'; +import type { HttpDetection, HttpLanguagePlugin } from './types.js'; + +/** + * Node.js / TypeScript HTTP plugin family. Handles: + * - NestJS `@Controller('prefix')` classes with `@Get(':id')` methods + * - Express `router.get(...)` / `app.post(...)` providers + * - `fetch(url)` / `fetch(url, { method: 'POST' })` consumers + * - `axios.get(url)` / `axios.delete(url)` consumers + * + * Because the JavaScript and TypeScript tree-sitter grammars share + * node type names for every construct we query, pattern sources are + * defined once and compiled against each grammar variant. The plugin + * exports three `HttpLanguagePlugin`s (JS, TS, TSX) that share the + * same `scan` function but bind to different grammars. + */ + +// ─── Provider: NestJS — class-level @Controller('prefix') ──────────── +// In tree-sitter-typescript decorators are NOT children of +// class_declaration / method_definition — they're siblings in the +// surrounding class_body / program node. We therefore match the +// decorator standalone and walk to its related class/method in JS. +const NEST_CONTROLLER_SPEC: PatternSpec> = { + meta: {}, + query: ` + (decorator + (call_expression + function: (identifier) @dec (#eq? @dec "Controller") + arguments: (arguments . [(string) (template_string)] @prefix))) @ctrl_decorator + `, +}; + +// ─── Provider: NestJS — method-level @Get/@Post/... decorators ─────── +// Matches either `@Get('path')` or `@Get()`. The `@path` capture is +// optional — when the first argument isn't a string, the plugin falls +// back to '/' for the method-level path. +const NEST_METHOD_SPEC: PatternSpec> = { + meta: {}, + query: ` + (decorator + (call_expression + function: (identifier) @dec (#match? @dec "^(Get|Post|Put|Delete|Patch)$") + arguments: (arguments) @args)) @method_decorator + `, +}; + +// ─── Provider: Express — router.get/app.post/... ───────────────────── +const EXPRESS_SPEC: PatternSpec> = { + meta: {}, + query: ` + (call_expression + function: (member_expression + object: (identifier) @obj (#match? @obj "^(router|app)$") + property: (property_identifier) @http_method (#match? @http_method "^(get|post|put|delete|patch)$")) + arguments: (arguments . [(string) (template_string)] @path)) + `, +}; + +// ─── Consumer: fetch(url) with NO options ───────────────────────────── +const FETCH_NO_OPTIONS_SPEC: PatternSpec> = { + meta: {}, + query: ` + (call_expression + function: (identifier) @fn (#eq? @fn "fetch") + arguments: (arguments . [(string) (template_string)] @path .)) + `, +}; + +// ─── Consumer: fetch(url, { method: 'X', ... }) ────────────────────── +const FETCH_WITH_OPTIONS_SPEC: PatternSpec> = { + meta: {}, + query: ` + (call_expression + function: (identifier) @fn (#eq? @fn "fetch") + arguments: (arguments + . [(string) (template_string)] @path + (object + (pair + key: (property_identifier) @key (#eq? @key "method") + value: (string) @http_method)))) + `, +}; + +// ─── Consumer: axios.get/post/... ──────────────────────────────────── +const AXIOS_SPEC: PatternSpec> = { + meta: {}, + query: ` + (call_expression + function: (member_expression + object: (identifier) @obj (#eq? @obj "axios") + property: (property_identifier) @http_method (#match? @http_method "^(get|post|put|delete|patch)$")) + arguments: (arguments . [(string) (template_string)] @path)) + `, +}; + +interface NodePatternBundle { + controller: CompiledPatterns>; + methodDecorator: CompiledPatterns>; + express: CompiledPatterns>; + fetchNoOptions: CompiledPatterns>; + fetchWithOptions: CompiledPatterns>; + axios: CompiledPatterns>; +} + +function compileBundle(language: unknown, name: string): NodePatternBundle { + const mk = (spec: PatternSpec>, suffix: string) => + compilePatterns({ + name: `${name}-${suffix}`, + language, + patterns: [spec], + } satisfies LanguagePatterns>); + return { + controller: mk(NEST_CONTROLLER_SPEC, 'nest-controller'), + methodDecorator: mk(NEST_METHOD_SPEC, 'nest-method-decorator'), + express: mk(EXPRESS_SPEC, 'express'), + fetchNoOptions: mk(FETCH_NO_OPTIONS_SPEC, 'fetch-no-options'), + fetchWithOptions: mk(FETCH_WITH_OPTIONS_SPEC, 'fetch-with-options'), + axios: mk(AXIOS_SPEC, 'axios'), + }; +} + +const JAVASCRIPT_BUNDLE = compileBundle(JavaScript, 'javascript-http'); +const TYPESCRIPT_BUNDLE = compileBundle(TypeScript.typescript, 'typescript-http'); +const TSX_BUNDLE = compileBundle(TypeScript.tsx, 'tsx-http'); + +const NEST_DECORATOR_TO_HTTP: Record = { + Get: 'GET', + Post: 'POST', + Put: 'PUT', + Delete: 'DELETE', + Patch: 'PATCH', +}; + +/** + * Find the nearest enclosing class_declaration for a node, or null. + */ +function findEnclosingClass(node: Parser.SyntaxNode): Parser.SyntaxNode | null { + let cur: Parser.SyntaxNode | null = node.parent; + while (cur) { + if (cur.type === 'class_declaration') return cur; + cur = cur.parent; + } + return null; +} + +function joinPath(prefix: string, sub: string): string { + const cleanPrefix = prefix.replace(/^\/+/, '').replace(/\/+$/, ''); + const cleanSub = sub.replace(/^\/+/, ''); + if (!cleanPrefix) return `/${cleanSub}`; + return `/${cleanPrefix}/${cleanSub}`; +} + +/** + * For a standalone `decorator` node (child of class_body / program), + * find the related `class_declaration` node that it decorates. In + * tree-sitter-typescript the decorator is placed before the class + * declaration as a sibling (when decorating a class) or inside the + * class_body before a method_definition (when decorating a method); + * we walk the parent chain until we find the enclosing class. + */ +function findDecoratedClass(decoratorNode: Parser.SyntaxNode): Parser.SyntaxNode | null { + const parent = decoratorNode.parent; + if (!parent) return null; + // Case 1: decorator is a sibling of the class_declaration at program / + // export_statement level. Walk forward through siblings until we find + // the class_declaration this decorator belongs to. + for (let i = 0; i < parent.namedChildCount; i++) { + const child = parent.namedChild(i); + if (child && child.id === decoratorNode.id) { + for (let j = i + 1; j < parent.namedChildCount; j++) { + const next = parent.namedChild(j); + if (!next) continue; + if (next.type === 'decorator') continue; // adjacent decorators stack + if (next.type === 'class_declaration') return next; + if (next.type === 'export_statement') { + // `export class Foo { ... }` wraps the declaration. + for (let k = 0; k < next.namedChildCount; k++) { + const inner = next.namedChild(k); + if (inner?.type === 'class_declaration') return inner; + } + } + break; + } + break; + } + } + // Case 2: decorator is inside a class_body (decorating a method) — + // walk up to the enclosing class_declaration. + return findEnclosingClass(decoratorNode); +} + +/** + * For a method-level decorator node (child of class_body before a + * method_definition), find the method_definition it decorates. + */ +function findDecoratedMethod(decoratorNode: Parser.SyntaxNode): Parser.SyntaxNode | null { + const parent = decoratorNode.parent; + if (!parent || parent.type !== 'class_body') return null; + for (let i = 0; i < parent.namedChildCount; i++) { + const child = parent.namedChild(i); + if (child && child.id === decoratorNode.id) { + for (let j = i + 1; j < parent.namedChildCount; j++) { + const next = parent.namedChild(j); + if (!next) continue; + if (next.type === 'decorator') continue; + if (next.type === 'method_definition') return next; + return null; + } + return null; + } + } + return null; +} + +function scanBundle(bundle: NodePatternBundle, tree: Parser.Tree): HttpDetection[] { + const out: HttpDetection[] = []; + + // NestJS: collect `@Controller('prefix')` class decorators, keyed by + // the `class_declaration` they decorate. + const prefixByClassId = new Map(); + for (const match of runCompiledPatterns(bundle.controller, tree)) { + const prefixNode = match.captures.prefix; + const decoratorNode = match.captures.ctrl_decorator; + if (!prefixNode || !decoratorNode) continue; + const prefix = unquoteLiteral(prefixNode.text); + if (prefix === null) continue; + const classNode = findDecoratedClass(decoratorNode); + if (!classNode) continue; + prefixByClassId.set(classNode.id, prefix); + } + + // NestJS: method-level @Get/@Post/... decorators. The decorator's + // arguments list may be empty (`@Get()`), a string (`@Get('path')`), + // or something else (which we skip). + for (const match of runCompiledPatterns(bundle.methodDecorator, tree)) { + const decNode = match.captures.dec; + const argsNode = match.captures.args; + const decoratorNode = match.captures.method_decorator; + if (!decNode || !argsNode || !decoratorNode) continue; + const httpMethod = NEST_DECORATOR_TO_HTTP[decNode.text]; + if (!httpMethod) continue; + const methodNode = findDecoratedMethod(decoratorNode); + if (!methodNode) continue; + const enclosingClass = findEnclosingClass(methodNode); + // Only emit NestJS detections when the class actually has a + // @Controller decorator — without it, the match is almost certainly + // something else (e.g. an unrelated library using similar names). + if (!enclosingClass || !prefixByClassId.has(enclosingClass.id)) continue; + const prefix = prefixByClassId.get(enclosingClass.id) ?? ''; + + let rawPath = '/'; + const firstArg = argsNode.namedChild(0); + if (firstArg && (firstArg.type === 'string' || firstArg.type === 'template_string')) { + const unquoted = unquoteLiteral(firstArg.text); + if (unquoted !== null) rawPath = unquoted; + } + + // Get the method name from the decorated method_definition. + const methodNameNode = methodNode.childForFieldName('name'); + const name = methodNameNode?.text ?? null; + + out.push({ + role: 'provider', + framework: 'nest', + method: httpMethod, + path: joinPath(prefix, rawPath), + name, + confidence: 0.8, + }); + } + + // Express: router/app.(...) + for (const match of runCompiledPatterns(bundle.express, tree)) { + const methodNode = match.captures.http_method; + const pathNode = match.captures.path; + if (!methodNode || !pathNode) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'provider', + framework: 'express', + method: methodNode.text.toUpperCase(), + path, + name: 'handler', + confidence: 0.8, + }); + } + + // Consumer: fetch with options { method: 'X' } + const fetchSeen = new Set(); + for (const match of runCompiledPatterns(bundle.fetchWithOptions, tree)) { + const pathNode = match.captures.path; + const methodNode = match.captures.http_method; + if (!pathNode || !methodNode) continue; + const path = unquoteLiteral(pathNode.text); + const method = unquoteLiteral(methodNode.text); + if (path === null || method === null) continue; + fetchSeen.add(pathNode.id); + out.push({ + role: 'consumer', + framework: 'fetch', + method: method.toUpperCase(), + path, + name: null, + confidence: 0.7, + }); + } + + // Consumer: plain fetch(path) — default GET. Skip path nodes we already + // matched with the options variant so we don't double-emit. + for (const match of runCompiledPatterns(bundle.fetchNoOptions, tree)) { + const pathNode = match.captures.path; + if (!pathNode) continue; + if (fetchSeen.has(pathNode.id)) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'consumer', + framework: 'fetch', + method: 'GET', + path, + name: null, + confidence: 0.7, + }); + } + + // Consumer: axios.(url) + for (const match of runCompiledPatterns(bundle.axios, tree)) { + const methodNode = match.captures.http_method; + const pathNode = match.captures.path; + if (!methodNode || !pathNode) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'consumer', + framework: 'axios', + method: methodNode.text.toUpperCase(), + path, + name: null, + confidence: 0.7, + }); + } + + return out; +} + +export const JAVASCRIPT_HTTP_PLUGIN: HttpLanguagePlugin = { + name: 'javascript-http', + language: JavaScript, + scan: (tree) => scanBundle(JAVASCRIPT_BUNDLE, tree), +}; + +export const TYPESCRIPT_HTTP_PLUGIN: HttpLanguagePlugin = { + name: 'typescript-http', + language: TypeScript.typescript, + scan: (tree) => scanBundle(TYPESCRIPT_BUNDLE, tree), +}; + +export const TSX_HTTP_PLUGIN: HttpLanguagePlugin = { + name: 'tsx-http', + language: TypeScript.tsx, + scan: (tree) => scanBundle(TSX_BUNDLE, tree), +}; diff --git a/gitnexus/src/core/group/extractors/http-patterns/php.ts b/gitnexus/src/core/group/extractors/http-patterns/php.ts new file mode 100644 index 0000000000..ae91c141ba --- /dev/null +++ b/gitnexus/src/core/group/extractors/http-patterns/php.ts @@ -0,0 +1,79 @@ +import PHP from 'tree-sitter-php'; +import { + compilePatterns, + runCompiledPatterns, + unquoteLiteral, + type LanguagePatterns, +} from '../tree-sitter-scanner.js'; +import type { HttpDetection, HttpLanguagePlugin } from './types.js'; + +/** + * PHP HTTP plugin — Laravel `Route::get/post/...` declarations. + * + * The pipeline already uses `PHP.php_only` for ingesting plain `.php` + * files (see `core/tree-sitter/parser-loader.ts`), and we do the same + * here so Laravel route files are parsed with the right grammar dialect. + */ + +const LARAVEL_PATTERNS = compilePatterns({ + name: 'php-laravel', + language: PHP.php_only, + patterns: [ + { + meta: {}, + query: ` + (scoped_call_expression + scope: (name) @scope (#eq? @scope "Route") + name: (name) @method (#match? @method "^(get|post|put|delete|patch)$") + arguments: (arguments . (argument (string) @path))) + `, + }, + ], +} satisfies LanguagePatterns>); + +/** + * Extract the inner text of a PHP `string` node. The tree-sitter-php + * grammar wraps single / double-quoted literals differently depending + * on content; we try both the raw `text` (with quotes) through + * `unquoteLiteral`, and a fallback via the `string_value` / `string_content` + * child nodes. + */ +function phpStringText(node: import('tree-sitter').SyntaxNode): string | null { + // Most single-quoted strings expose their inner content through the + // full node text (including quotes), which unquoteLiteral strips. + const direct = unquoteLiteral(node.text); + if (direct !== null && direct !== node.text) return direct; + // Fall back to child string_content / string_value node if present. + for (const child of node.children) { + if (child.type === 'string_content' || child.type === 'string_value') { + return child.text; + } + } + return direct; +} + +export const PHP_HTTP_PLUGIN: HttpLanguagePlugin = { + name: 'php-http', + language: PHP.php_only, + scan(tree) { + const out: HttpDetection[] = []; + + for (const match of runCompiledPatterns(LARAVEL_PATTERNS, tree)) { + const methodNode = match.captures.method; + const pathNode = match.captures.path; + if (!methodNode || !pathNode) continue; + const path = phpStringText(pathNode); + if (path === null) continue; + out.push({ + role: 'provider', + framework: 'laravel', + method: methodNode.text.toUpperCase(), + path, + name: 'route', + confidence: 0.8, + }); + } + + return out; + }, +}; diff --git a/gitnexus/src/core/group/extractors/http-patterns/python.ts b/gitnexus/src/core/group/extractors/http-patterns/python.ts new file mode 100644 index 0000000000..27ddf6633c --- /dev/null +++ b/gitnexus/src/core/group/extractors/http-patterns/python.ts @@ -0,0 +1,142 @@ +import Python from 'tree-sitter-python'; +import { + compilePatterns, + runCompiledPatterns, + unquoteLiteral, + type LanguagePatterns, +} from '../tree-sitter-scanner.js'; +import type { HttpDetection, HttpLanguagePlugin } from './types.js'; + +/** + * Python HTTP plugin. Handles: + * - FastAPI `@app.get("/path")` provider decorators + * - `requests.get/post/...("url")` consumer calls + * - Generic `requests.request("METHOD", "url")` consumer calls + */ + +const FASTAPI_VERBS: Record = { + get: 'GET', + post: 'POST', + put: 'PUT', + delete: 'DELETE', + patch: 'PATCH', +}; + +// ─── Provider: FastAPI @app.get/... ────────────────────────────────── +const FASTAPI_PATTERNS = compilePatterns({ + name: 'python-fastapi', + language: Python, + patterns: [ + { + meta: {}, + query: ` + (decorator + (call + function: (attribute + object: (identifier) @obj (#eq? @obj "app") + attribute: (identifier) @method (#match? @method "^(get|post|put|delete|patch)$")) + arguments: (argument_list . (string) @path))) + `, + }, + ], +} satisfies LanguagePatterns>); + +// ─── Consumer: requests.get/post/... ────────────────────────────────── +const REQUESTS_VERB_PATTERNS = compilePatterns({ + name: 'python-requests-verb', + language: Python, + patterns: [ + { + meta: {}, + query: ` + (call + function: (attribute + object: (identifier) @obj (#eq? @obj "requests") + attribute: (identifier) @method (#match? @method "^(get|post|put|delete|patch)$")) + arguments: (argument_list . (string) @path)) + `, + }, + ], +} satisfies LanguagePatterns>); + +// ─── Consumer: requests.request("METHOD", "url") ───────────────────── +const REQUESTS_GENERIC_PATTERNS = compilePatterns({ + name: 'python-requests-generic', + language: Python, + patterns: [ + { + meta: {}, + query: ` + (call + function: (attribute + object: (identifier) @obj (#eq? @obj "requests") + attribute: (identifier) @method (#eq? @method "request")) + arguments: (argument_list . (string) @http_method (string) @path)) + `, + }, + ], +} satisfies LanguagePatterns>); + +export const PYTHON_HTTP_PLUGIN: HttpLanguagePlugin = { + name: 'python-http', + language: Python, + scan(tree) { + const out: HttpDetection[] = []; + + // Providers: FastAPI + for (const match of runCompiledPatterns(FASTAPI_PATTERNS, tree)) { + const methodNode = match.captures.method; + const pathNode = match.captures.path; + if (!methodNode || !pathNode) continue; + const httpMethod = FASTAPI_VERBS[methodNode.text]; + if (!httpMethod) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'provider', + framework: 'fastapi', + method: httpMethod, + path, + name: null, + confidence: 0.8, + }); + } + + // Consumers: requests. + for (const match of runCompiledPatterns(REQUESTS_VERB_PATTERNS, tree)) { + const methodNode = match.captures.method; + const pathNode = match.captures.path; + if (!methodNode || !pathNode) continue; + const path = unquoteLiteral(pathNode.text); + if (path === null) continue; + out.push({ + role: 'consumer', + framework: 'python-requests', + method: methodNode.text.toUpperCase(), + path, + name: null, + confidence: 0.7, + }); + } + + // Consumers: requests.request("METHOD", "url") + for (const match of runCompiledPatterns(REQUESTS_GENERIC_PATTERNS, tree)) { + const methodNode = match.captures.http_method; + const pathNode = match.captures.path; + if (!methodNode || !pathNode) continue; + const methodRaw = unquoteLiteral(methodNode.text); + const path = unquoteLiteral(pathNode.text); + if (methodRaw === null || path === null) continue; + out.push({ + role: 'consumer', + framework: 'python-requests', + method: methodRaw.toUpperCase(), + path, + name: null, + confidence: 0.7, + }); + } + + return out; + }, +}; diff --git a/gitnexus/src/core/group/extractors/http-patterns/types.ts b/gitnexus/src/core/group/extractors/http-patterns/types.ts new file mode 100644 index 0000000000..6df0ede286 --- /dev/null +++ b/gitnexus/src/core/group/extractors/http-patterns/types.ts @@ -0,0 +1,65 @@ +import type Parser from 'tree-sitter'; + +/** + * Shared types for the http-route-extractor language plugins. + * + * Each plugin lives in its own file (java.ts, node.ts, ...) and owns + * the tree-sitter grammar import + queries. The top-level + * `http-route-extractor.ts` orchestrator only knows about this type + * module and the plugin registry (`./index.ts`). It MUST NOT import + * any grammar or query text directly — language-specific knowledge + * belongs in the plugins. + */ + +export type HttpRole = 'provider' | 'consumer'; + +/** + * One raw HTTP detection produced by a plugin's `scan()` function. The + * orchestrator converts this into a full `ExtractedContract` by running + * path normalization and building the contract id. + * + * `path` is the raw literal string as it appeared in source (with + * `${...}` template placeholders still in place); the orchestrator + * runs the appropriate normalizer for provider vs. consumer paths. + */ +export interface HttpDetection { + role: HttpRole; + /** Short framework label, e.g. `'spring'`, `'nest'`, `'express'`. */ + framework: string; + /** HTTP method in upper case (`'GET'`, `'POST'`, ...). */ + method: string; + /** Raw path literal as seen in source (template placeholders intact). */ + path: string; + /** + * Symbol name of the handler (for providers) or calling function + * (for consumers) when the plugin can determine it structurally. + * Null when no good candidate is available. + */ + name: string | null; + /** Confidence in (0, 1]. Source-scan plugins typically use 0.7–0.8. */ + confidence: number; +} + +/** + * One language-scoped HTTP plugin. The plugin owns the tree-sitter + * grammar and the `scan` function that translates a parsed tree into + * zero or more `HttpDetection`s. Plugins are free to run multiple + * compiled pattern bundles internally (see the shared scanner's + * `runCompiledPatterns` helper). + * + * `language` is typed as `unknown` for the same reason as + * `LanguagePatterns.language` in `tree-sitter-scanner.ts` — the + * grammar modules export different shapes. + */ +export interface HttpLanguagePlugin { + /** Human-readable plugin name for diagnostics. */ + name: string; + /** tree-sitter grammar object (passed to the shared parser). */ + language: unknown; + /** + * Scan a parsed tree and return zero or more HTTP detections. Plugins + * must not throw — they should swallow per-match errors so a single + * malformed construct does not abort the whole file. + */ + scan(tree: Parser.Tree): HttpDetection[]; +} diff --git a/gitnexus/src/core/group/extractors/http-route-extractor.ts b/gitnexus/src/core/group/extractors/http-route-extractor.ts index 2a9f662c3b..96a7ffe200 100644 --- a/gitnexus/src/core/group/extractors/http-route-extractor.ts +++ b/gitnexus/src/core/group/extractors/http-route-extractor.ts @@ -1,8 +1,34 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { glob } from 'glob'; +import Parser from 'tree-sitter'; import type { ContractExtractor, CypherExecutor } from '../contract-extractor.js'; import type { ExtractedContract, RepoHandle } from '../types.js'; +import { getPluginForFile, HTTP_SCAN_GLOB, type HttpDetection } from './http-patterns/index.js'; + +/** + * Language-agnostic orchestrator for HTTP route (provider + consumer) + * contract extraction. Two strategies, in order of preference per role: + * + * 1. **Graph-assisted (Strategy A)** — if a per-repo LadybugDB executor + * is available, read `HANDLES_ROUTE` / `FETCHES` Cypher edges that + * the ingestion pipeline already produced via tree-sitter. This is + * the preferred path because the graph has richer symbol metadata + * (real uids, class/method structure, etc.). + * + * 2. **Source-scan fallback (Strategy B)** — parse files directly with + * the per-language plugin registry in `./http-patterns/`. Used when + * the graph has no routes/fetches for this repo (e.g. a repo that + * hasn't been indexed yet, or whose indexer doesn't know the + * framework). Each plugin owns its tree-sitter grammar and query + * sources — this orchestrator imports NO grammars or query strings. + * + * Adding a new language for Strategy B is a one-file edit in + * `http-patterns/index.ts`: register a new `HttpLanguagePlugin` and + * widen `HTTP_SCAN_GLOB` if needed. + */ + +// ─── Graph-assisted queries ────────────────────────────────────────── const HANDLES_ROUTE_QUERY = ` MATCH (handlerFile:File)-[r:CodeRelation {type: 'HANDLES_ROUTE'}]->(route:Route) @@ -23,6 +49,15 @@ WHERE sym.startLine IS NOT NULL RETURN sym.id AS uid, sym.name AS name, sym.filePath AS filePath, labels(sym) AS labels ORDER BY sym.startLine`; +// ─── Path normalization (shared between provider / consumer paths) ── + +/** + * Canonicalize a provider-side HTTP path for contract-id generation: + * - strip query string + * - lower-case + * - drop trailing slash + * - collapse `:id`, `{id}`, `[id]` path params into a single `{param}` + */ export function normalizeHttpPath(p: string): string { let s = p.trim().split('?')[0].toLowerCase().replace(/\/+$/, ''); s = s.replace(/:\w+/g, '{param}'); @@ -31,20 +66,36 @@ export function normalizeHttpPath(p: string): string { return s; } -function methodFromRouteReason(reason: string): string | null { - const r = reason || ''; - if (/GetMapping|decorator-Get/i.test(r)) return 'GET'; - if (/PostMapping|decorator-Post/i.test(r)) return 'POST'; - if (/PutMapping|decorator-Put/i.test(r)) return 'PUT'; - if (/DeleteMapping|decorator-Delete/i.test(r)) return 'DELETE'; - if (/PatchMapping|decorator-Patch/i.test(r)) return 'PATCH'; - return null; +/** + * Consumer-side normalization is more aggressive: + * - template literals (`${x}`) → `{param}` + * - strip protocol + host if the URL is absolute + * - numeric segments → `{param}` (so `/api/orders/42` → `/api/orders/{param}`) + */ +function normalizeConsumerPath(url: string): string { + const templated = url.replace(/\$\{[^}]+\}/g, '{param}').trim(); + let pathOnly = templated; + if (/^https?:\/\//i.test(templated)) { + try { + pathOnly = new URL(templated).pathname; + } catch { + pathOnly = templated.replace(/^https?:\/\/[^/]+/i, ''); + } + } + const normalized = normalizeHttpPath(pathOnly || '/'); + const segments = normalized + .split('/') + .filter(Boolean) + .map((segment) => (/^\d+$/.test(segment) ? '{param}' : segment)); + return `/${segments.join('/')}`.replace(/\/+$/, '') || '/'; } function contractIdFor(method: string, pathNorm: string): string { return `http::${method.toUpperCase()}::${pathNorm}`; } +// ─── File read helper (path-traversal safe) ────────────────────────── + function readSafe(repoPath: string, rel: string): string | null { const abs = path.resolve(repoPath, rel); const base = path.resolve(repoPath); @@ -57,31 +108,15 @@ function readSafe(repoPath: string, rel: string): string | null { } } -function pickJavaHandlerName( - content: string, - routePath: string, - httpMethod: string, -): string | null { - const tail = routePath.split('/').filter(Boolean).pop() || ''; - const mapNames: Record = { - GET: 'GetMapping', - POST: 'PostMapping', - PUT: 'PutMapping', - DELETE: 'DeleteMapping', - PATCH: 'PatchMapping', - }; - const ann = mapNames[httpMethod] || 'GetMapping'; - const lines = content.split(/\r?\n/); - for (let i = 0; i < lines.length; i++) { - const line = lines[i]; - if (!line.includes(`@${ann}`)) continue; - if (!line.includes(`"${tail}"`) && !line.includes(`'${tail}'`) && tail && !line.includes(tail)) - continue; - for (let j = i + 1; j < Math.min(i + 8, lines.length); j++) { - const m = lines[j].match(/(?:public|protected|private)\s+[\w<>,\s\[\]]+\s+(\w+)\s*\(/); - if (m) return m[1]; - } - } +// ─── Graph row helpers ─────────────────────────────────────────────── + +function methodFromRouteReason(reason: string): string | null { + const r = reason || ''; + if (/GetMapping|decorator-Get/i.test(r)) return 'GET'; + if (/PostMapping|decorator-Post/i.test(r)) return 'POST'; + if (/PutMapping|decorator-Put/i.test(r)) return 'PUT'; + if (/DeleteMapping|decorator-Delete/i.test(r)) return 'DELETE'; + if (/PatchMapping|decorator-Patch/i.test(r)) return 'PATCH'; return null; } @@ -114,6 +149,8 @@ function pickSymbolUid( }; } +// ─── Orchestrator ──────────────────────────────────────────────────── + export class HttpRouteExtractor implements ContractExtractor { type = 'http' as const; @@ -124,20 +161,67 @@ export class HttpRouteExtractor implements ContractExtractor { async extract( dbExecutor: CypherExecutor | null, repoPath: string, - repo: RepoHandle, + _repo: RepoHandle, ): Promise { - const graphP = dbExecutor != null ? await this.extractProvidersGraph(dbExecutor, repoPath) : []; - const providers = graphP.length > 0 ? graphP : await this.extractProvidersSourceScan(repoPath); + // Parse each file at most once and reuse the plugin results across + // both graph-assisted enrichment and source-scan emission. + const parser = new Parser(); + const cachedDetections = new Map(); + const getDetections = (rel: string): HttpDetection[] => { + const cached = cachedDetections.get(rel); + if (cached) return cached; + const plugin = getPluginForFile(rel); + if (!plugin) { + cachedDetections.set(rel, []); + return []; + } + const content = readSafe(repoPath, rel); + if (!content) { + cachedDetections.set(rel, []); + return []; + } + try { + parser.setLanguage(plugin.language); + const tree = parser.parse(content); + const detections = plugin.scan(tree); + cachedDetections.set(rel, detections); + return detections; + } catch { + cachedDetections.set(rel, []); + return []; + } + }; - const graphC = dbExecutor != null ? await this.extractConsumersGraph(dbExecutor, repoPath) : []; - const consumers = graphC.length > 0 ? graphC : await this.extractConsumersSourceScan(repoPath); + const graphProviders = + dbExecutor != null ? await this.extractProvidersGraph(dbExecutor, getDetections) : []; + const providers = + graphProviders.length > 0 + ? graphProviders + : this.extractProvidersSourceScan(await this.scanFiles(repoPath), getDetections); + + const graphConsumers = + dbExecutor != null ? await this.extractConsumersGraph(dbExecutor, getDetections) : []; + const consumers = + graphConsumers.length > 0 + ? graphConsumers + : this.extractConsumersSourceScan(await this.scanFiles(repoPath), getDetections); return [...providers, ...consumers]; } + private async scanFiles(repoPath: string): Promise { + return glob(HTTP_SCAN_GLOB, { + cwd: repoPath, + ignore: ['**/node_modules/**', '**/.git/**', '**/dist/**', '**/build/**'], + nodir: true, + }); + } + + // ─── Graph-assisted providers ────────────────────────────────────── + private async extractProvidersGraph( db: CypherExecutor, - repoPath: string, + getDetections: (rel: string) => HttpDetection[], ): Promise { const out: ExtractedContract[] = []; let rows: Record[]; @@ -152,16 +236,26 @@ export class HttpRouteExtractor implements ContractExtractor { const routePath = String(row.routePath ?? ''); const routeSource = String(row.routeSource ?? row.routeReason ?? ''); let method = methodFromRouteReason(routeSource); - const content = readSafe(repoPath, filePath); - if (!method && content) { - method = this.inferMethodFromFileScan(content, routePath, 'provider'); + + // Fallback: look up method + handler name from the plugin's scan + // of the handler file. This replaces the old regex-based + // `inferMethodFromFileScan` and `pickJavaHandlerName` helpers — + // tree-sitter gives both pieces of information structurally. + const detections = filePath ? getDetections(filePath) : []; + const providerDetections = detections.filter((d) => d.role === 'provider'); + let handlerName: string | null = null; + if (!method || !handlerName) { + const normalizedRoute = normalizeHttpPath(routePath); + const match = providerDetections.find((d) => normalizeHttpPath(d.path) === normalizedRoute); + if (match) { + if (!method) method = match.method; + handlerName = match.name; + } } if (!method) method = 'GET'; const pathNorm = normalizeHttpPath(routePath); const cid = contractIdFor(method, pathNorm); - const handlerName = - content && routePath ? pickJavaHandlerName(content, routePath, method) : null; let symbolUid = ''; let symbolName = path.basename(filePath) || 'handler'; @@ -201,189 +295,44 @@ export class HttpRouteExtractor implements ContractExtractor { return out; } - private inferMethodFromFileScan( - content: string, - routePath: string, - _role: string, - ): string | null { - const tail = routePath.split('/').filter(Boolean).pop() || ''; - for (const m of ['GET', 'POST', 'PUT', 'DELETE', 'PATCH'] as const) { - const mapNames: Record = { - GET: 'GetMapping', - POST: 'PostMapping', - PUT: 'PutMapping', - DELETE: 'DeleteMapping', - PATCH: 'PatchMapping', - }; - if ( - content.includes(`@${mapNames[m]}`) && - (content.includes(tail) || routePath.includes(tail)) - ) { - return m; - } - } - return null; - } + // ─── Source-scan providers ───────────────────────────────────────── - private async extractProvidersSourceScan(repoPath: string): Promise { - const files = await glob('**/*.{ts,tsx,js,jsx,java,vue,svelte,php,py,go}', { - cwd: repoPath, - ignore: ['**/node_modules/**', '**/.git/**', '**/dist/**', '**/build/**'], - nodir: true, - }); + private extractProvidersSourceScan( + files: string[], + getDetections: (rel: string) => HttpDetection[], + ): ExtractedContract[] { const out: ExtractedContract[] = []; for (const rel of files) { - const content = readSafe(repoPath, rel); - if (!content) continue; - out.push(...this.scanSpringProviders(content, rel)); - out.push(...this.scanNestProviders(content, rel)); - out.push(...this.scanExpressProviders(content, rel)); - out.push(...this.scanGoProviders(content, rel)); - out.push(...this.scanLaravelProviders(content, rel)); - out.push(...this.scanFastApiProviders(content, rel)); + const detections = getDetections(rel); + for (const d of detections) { + if (d.role !== 'provider') continue; + const pathNorm = normalizeHttpPath(d.path); + out.push({ + contractId: contractIdFor(d.method, pathNorm), + type: 'http', + role: 'provider', + symbolUid: '', + symbolRef: { filePath: rel, name: d.name ?? 'handler' }, + symbolName: d.name ?? 'handler', + confidence: d.confidence, + meta: { + method: d.method, + path: pathNorm, + pathSegments: pathNorm.split('/').filter(Boolean), + extractionStrategy: 'source_scan', + framework: d.framework, + }, + }); + } } return this.dedupeContracts(out); } - private dedupeContracts(items: ExtractedContract[]): ExtractedContract[] { - const seen = new Set(); - const out: ExtractedContract[] = []; - for (const c of items) { - const k = `${c.contractId}|${c.symbolRef.filePath}|${c.symbolRef.name}`; - if (seen.has(k)) continue; - seen.add(k); - out.push(c); - } - return out; - } - - private scanSpringProviders(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - let classPrefix = ''; - const classRm = content.match(/@RequestMapping\s*\(\s*"([^"]+)"/); - if (classRm) classPrefix = classRm[1].replace(/\/+$/, ''); - - const re = /@(Get|Post|Put|Delete|Patch)Mapping\s*\(\s*"([^"]+)"/gi; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const method = m[1].toUpperCase(); - let p = m[2]; - if (classPrefix) p = `${classPrefix}/${p.replace(/^\//, '')}`; - const pathNorm = normalizeHttpPath(p); - const sub = content.slice(m.index); - const nameM = sub.match(/(?:public|protected|private)\s+[\w<>,\s\[\]]+\s+(\w+)\s*\(/); - const name = nameM ? nameM[1] : m[0]; - out.push(this.makeProvider(filePath, method, pathNorm, name, 0.8)); - } - return out; - } - - private scanExpressProviders(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - const re = /(?:router|app)\.(get|post|put|delete|patch)\s*\(\s*['"]([^'"]+)['"]/gi; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const method = m[1].toUpperCase(); - const pathNorm = normalizeHttpPath(m[2]); - out.push(this.makeProvider(filePath, method, pathNorm, 'handler', 0.8)); - } - return out; - } - - private scanLaravelProviders(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - const re = /Route::(get|post|put|delete|patch)\s*\(\s*['"]([^'"]+)['"]/gi; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const method = m[1].toUpperCase(); - const pathNorm = normalizeHttpPath(m[2]); - out.push(this.makeProvider(filePath, method, pathNorm, 'route', 0.8)); - } - return out; - } - - private scanFastApiProviders(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - const re = /@app\.(get|post|put|delete|patch)\s*\(\s*['"]([^'"]+)['"]/gi; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const method = m[1].toUpperCase(); - const pathNorm = normalizeHttpPath(m[2]); - out.push(this.makeProvider(filePath, method, pathNorm, 'handler', 0.8)); - } - return out; - } - - private scanNestProviders(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - const controllerMatch = content.match(/@Controller\s*\(\s*['"`]([^'"`]+)['"`]\s*\)/); - const controllerPrefix = controllerMatch ? controllerMatch[1].replace(/\/+$/, '') : ''; - const re = /@(Get|Post|Put|Delete|Patch)\s*\(\s*['"`]?([^'"`)]*)['"`]?\s*\)/gi; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const method = m[1].toUpperCase(); - const routePath = String(m[2] || ''); - const fullPath = controllerPrefix - ? `${controllerPrefix}/${routePath.replace(/^\/+/, '')}` - : routePath; - const pathNorm = normalizeHttpPath(fullPath.startsWith('/') ? fullPath : `/${fullPath}`); - const sub = content.slice(m.index); - const nameMatch = sub.match( - /(?:public|protected|private)?\s*(?:async\s+)?(\w+)\s*\([^)]*\)\s*\{/, - ); - const name = nameMatch ? nameMatch[1] : m[0]; - out.push(this.makeProvider(filePath, method, pathNorm, name, 0.8)); - } - return out; - } - - private scanGoProviders(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - const frameworkRe = - /(?:^|\W)\w+\.(GET|POST|PUT|DELETE|PATCH)\s*\(\s*['"]([^'"]+)['"]\s*,\s*(\w+)/gim; - let m: RegExpExecArray | null; - while ((m = frameworkRe.exec(content)) !== null) { - out.push(this.makeProvider(filePath, m[1].toUpperCase(), normalizeHttpPath(m[2]), m[3], 0.8)); - } - - const handleFuncRe = - /(?:http|\w+)\.HandleFunc\s*\(\s*['"]([^'"]+)['"]\s*,\s*(\w+)\s*\)(?:\s*\.\s*Methods\s*\(\s*['"](\w+)['"]\s*\))?/gim; - while ((m = handleFuncRe.exec(content)) !== null) { - const method = (m[3] || 'GET').toUpperCase(); - out.push(this.makeProvider(filePath, method, normalizeHttpPath(m[1]), m[2], 0.8)); - } - - return out; - } - - private makeProvider( - filePath: string, - method: string, - pathNorm: string, - name: string, - confidence: number, - ): ExtractedContract { - const cid = contractIdFor(method, pathNorm); - return { - contractId: cid, - type: 'http', - role: 'provider', - symbolUid: '', - symbolRef: { filePath, name }, - symbolName: name, - confidence, - meta: { - method, - path: pathNorm, - pathSegments: pathNorm.split('/').filter(Boolean), - extractionStrategy: 'source_scan', - }, - }; - } + // ─── Graph-assisted consumers ────────────────────────────────────── private async extractConsumersGraph( db: CypherExecutor, - repoPath: string, + getDetections: (rel: string) => HttpDetection[], ): Promise { const out: ExtractedContract[] = []; let rows: Record[]; @@ -397,11 +346,14 @@ export class HttpRouteExtractor implements ContractExtractor { const routePath = String(row.routePath ?? ''); const pathNorm = normalizeHttpPath(routePath); let method = 'GET'; - const content = readSafe(repoPath, filePath); - if (content) { - const inferred = this.inferFetchMethod(content, pathNorm); - if (inferred) method = inferred; - } + // Prefer the plugin's detected method if we can find a matching + // fetch/axios call in the same file. + const detections = filePath ? getDetections(filePath) : []; + const inferred = detections.find( + (d) => d.role === 'consumer' && normalizeConsumerPath(d.path) === pathNorm, + ); + if (inferred) method = inferred.method; + const cid = contractIdFor(method, pathNorm); let symbolUid = ''; let symbolName = 'fetch'; @@ -439,193 +391,47 @@ export class HttpRouteExtractor implements ContractExtractor { return out; } - private inferFetchMethod(content: string, pathNorm: string): string | null { - const esc = pathNorm.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); - const fetchRe = new RegExp( - `fetch\\s*\\(\\s*['"\`]([^'"\`]*${esc}[^'"\`]*)['"\`]\\s*,\\s*\\{[^}]*method:\\s*['"](\\w+)['"]`, - 'i', - ); - const m = content.match(fetchRe); - if (m) return m[2].toUpperCase(); - return null; - } + // ─── Source-scan consumers ───────────────────────────────────────── - private async extractConsumersSourceScan(repoPath: string): Promise { - const files = await glob('**/*.{ts,tsx,js,jsx,vue,svelte,py,java,go}', { - cwd: repoPath, - ignore: ['**/node_modules/**', '**/.git/**'], - nodir: true, - }); + private extractConsumersSourceScan( + files: string[], + getDetections: (rel: string) => HttpDetection[], + ): ExtractedContract[] { const out: ExtractedContract[] = []; for (const rel of files) { - const content = readSafe(repoPath, rel); - if (!content) continue; - out.push(...this.scanFetchConsumers(content, rel)); - out.push(...this.scanAxiosConsumers(content, rel)); - out.push(...this.scanPythonRequestsConsumers(content, rel)); - out.push(...this.scanJavaConsumers(content, rel)); - out.push(...this.scanGoConsumers(content, rel)); - } - return this.dedupeContracts(out); - } - - private scanFetchConsumers(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - const re = - /fetch\s*\(\s*['"`]([^'"`]+)['"`](?:\s*,\s*\{[^}]*method:\s*['"](\w+)['"][^}]*\})?\s*\)/gi; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const pathNorm = normalizeHttpPath(this.templateToPattern(m[1])); - const method = (m[2] || 'GET').toUpperCase(); - out.push(this.makeConsumer(filePath, method, pathNorm, 0.7)); - } - return out; - } - - private templateToPattern(url: string): string { - return url.replace(/\$\{[^}]+\}/g, '{param}'); - } - - private normalizeConsumerPath(url: string): string { - const templated = this.templateToPattern(url.trim()); - let pathOnly = templated; - if (/^https?:\/\//i.test(templated)) { - try { - pathOnly = new URL(templated).pathname; - } catch { - pathOnly = templated.replace(/^https?:\/\/[^/]+/i, ''); + const detections = getDetections(rel); + for (const d of detections) { + if (d.role !== 'consumer') continue; + const pathNorm = normalizeConsumerPath(d.path); + out.push({ + contractId: contractIdFor(d.method, pathNorm), + type: 'http', + role: 'consumer', + symbolUid: '', + symbolRef: { filePath: rel, name: 'fetch' }, + symbolName: 'fetch', + confidence: d.confidence, + meta: { + method: d.method, + path: pathNorm, + extractionStrategy: 'source_scan', + framework: d.framework, + }, + }); } } - - const normalized = normalizeHttpPath(pathOnly || '/'); - const segments = normalized - .split('/') - .filter(Boolean) - .map((segment) => { - if (/^\d+$/.test(segment)) return '{param}'; - return segment; - }); - return `/${segments.join('/')}`.replace(/\/+$/, '') || '/'; - } - - private scanAxiosConsumers(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - const re = /axios\.(get|post|put|delete|patch)\s*\(\s*[`'"]([^`'"]+)[`'"]/gi; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const method = m[1].toUpperCase(); - const pathNorm = this.normalizeConsumerPath(m[2]); - out.push(this.makeConsumer(filePath, method, pathNorm, 0.7)); - } - return out; - } - - private scanPythonRequestsConsumers(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - const methodRe = /requests\.(get|post|put|delete|patch)\s*\(\s*['"]([^'"]+)['"]/gi; - let m: RegExpExecArray | null; - while ((m = methodRe.exec(content)) !== null) { - out.push( - this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), - ); - } - - const genericRe = /requests\.request\s*\(\s*['"](\w+)['"]\s*,\s*['"]([^'"]+)['"]/gi; - while ((m = genericRe.exec(content)) !== null) { - out.push( - this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), - ); - } - - return out; - } - - private scanJavaConsumers(content: string, filePath: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - const restTemplateMethods: Array<[RegExp, string]> = [ - [/restTemplate\.getFor(?:Object|Entity)\s*\(\s*['"]([^'"]+)['"]/gi, 'GET'], - [/restTemplate\.postFor(?:Object|Entity)\s*\(\s*['"]([^'"]+)['"]/gi, 'POST'], - [/restTemplate\.put\s*\(\s*['"]([^'"]+)['"]/gi, 'PUT'], - [/restTemplate\.delete\s*\(\s*['"]([^'"]+)['"]/gi, 'DELETE'], - [/restTemplate\.patchForObject\s*\(\s*['"]([^'"]+)['"]/gi, 'PATCH'], - ]; - for (const [re, method] of restTemplateMethods) { - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - out.push(this.makeConsumer(filePath, method, this.normalizeConsumerPath(m[1]), 0.7)); - } - } - - const webClientMethodRe = - /webClient\.method\s*\(\s*HttpMethod\.(GET|POST|PUT|DELETE|PATCH)\s*,\s*['"]([^'"]+)['"]/gi; - let m: RegExpExecArray | null; - while ((m = webClientMethodRe.exec(content)) !== null) { - out.push( - this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), - ); - } - - const okHttpRe = - /new\s+Request\.Builder\s*\(\)\s*\.url\s*\(\s*['"]([^'"]+)['"]\s*\)(?:\s*\.\s*method\s*\(\s*['"](\w+)['"])?/gim; - while ((m = okHttpRe.exec(content)) !== null) { - out.push( - this.makeConsumer( - filePath, - (m[2] || 'GET').toUpperCase(), - this.normalizeConsumerPath(m[1]), - 0.7, - ), - ); - } - - return out; + return this.dedupeContracts(out); } - private scanGoConsumers(content: string, filePath: string): ExtractedContract[] { + private dedupeContracts(items: ExtractedContract[]): ExtractedContract[] { + const seen = new Set(); const out: ExtractedContract[] = []; - const httpMethodRe = /\bhttp\.(Get|Post|Head)\s*\(\s*['"]([^'"]+)['"]/gi; - let m: RegExpExecArray | null; - while ((m = httpMethodRe.exec(content)) !== null) { - const method = m[1].toUpperCase() === 'HEAD' ? 'GET' : m[1].toUpperCase(); - out.push(this.makeConsumer(filePath, method, this.normalizeConsumerPath(m[2]), 0.7)); - } - - const newRequestRe = /\bhttp\.NewRequest\s*\(\s*['"](\w+)['"]\s*,\s*['"]([^'"]+)['"]/gi; - while ((m = newRequestRe.exec(content)) !== null) { - out.push( - this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), - ); - } - - const restyRe = /\b\w+\.R\(\)\.(Get|Post|Put|Delete|Patch)\s*\(\s*['"]([^'"]+)['"]/gi; - while ((m = restyRe.exec(content)) !== null) { - out.push( - this.makeConsumer(filePath, m[1].toUpperCase(), this.normalizeConsumerPath(m[2]), 0.7), - ); + for (const c of items) { + const k = `${c.contractId}|${c.symbolRef.filePath}|${c.symbolRef.name}`; + if (seen.has(k)) continue; + seen.add(k); + out.push(c); } - return out; } - - private makeConsumer( - filePath: string, - method: string, - pathNorm: string, - confidence: number, - ): ExtractedContract { - return { - contractId: contractIdFor(method, pathNorm), - type: 'http', - role: 'consumer', - symbolUid: '', - symbolRef: { filePath, name: 'fetch' }, - symbolName: 'fetch', - confidence, - meta: { - method, - path: pathNorm, - extractionStrategy: 'source_scan', - }, - }; - } } diff --git a/gitnexus/src/core/group/extractors/topic-extractor.ts b/gitnexus/src/core/group/extractors/topic-extractor.ts index e602273693..faab19cf30 100644 --- a/gitnexus/src/core/group/extractors/topic-extractor.ts +++ b/gitnexus/src/core/group/extractors/topic-extractor.ts @@ -91,7 +91,9 @@ export class TopicExtractor implements ContractExtractor { const matches = scanFile(parser, provider, content); for (const match of matches) { - const topicName = unquoteLiteral(match.valueText); + const valueNode = match.captures.value; + if (!valueNode) continue; + const topicName = unquoteLiteral(valueNode.text); if (!topicName) continue; out.push(makeContract(topicName, match.meta, rel)); } diff --git a/gitnexus/src/core/group/extractors/tree-sitter-scanner.ts b/gitnexus/src/core/group/extractors/tree-sitter-scanner.ts index d792002caf..cd50456aaf 100644 --- a/gitnexus/src/core/group/extractors/tree-sitter-scanner.ts +++ b/gitnexus/src/core/group/extractors/tree-sitter-scanner.ts @@ -15,15 +15,16 @@ import Parser from 'tree-sitter'; /** * One pattern owned by a language plugin. Each pattern owns a tree-sitter - * S-expression query. The query MUST contain a capture named `@value` - * whose node text is the literal we want to extract (string/template/etc). + * S-expression query. Plugins can freely choose which capture names to + * use — the scanner exposes every capture in the returned `captures` + * map and does not privilege any particular name. * * `TMeta` is the plugin-specific payload the orchestrator receives back * when this pattern matches — e.g. for topic extraction it carries the * broker name, role, confidence, symbol name. */ export interface PatternSpec { - /** Tree-sitter S-expression. MUST contain a `@value` capture. */ + /** Tree-sitter S-expression. */ query: string; /** Plugin-specific payload returned on every match. */ meta: TMeta; @@ -66,17 +67,23 @@ export interface CompiledPattern { } /** - * One match returned by `scanFile`. The orchestrator receives the raw - * literal text (still including any surrounding quotes) together with - * the plugin meta, and is responsible for calling `unquoteLiteral` / - * emitting a domain object (ExtractedContract, Route, ...). + * Map from capture name → syntax node. Every named capture the query + * binds is exposed as an entry. If a query captures the same name more + * than once (unusual), the first occurrence wins — plugins that need + * all occurrences should use distinct capture names or fall back to + * `match.captures` array directly by iterating `query.matches()` + * themselves. + */ +export type CaptureMap = Record; + +/** + * One match returned by `scanFile` / `runCompiledPatterns`. The caller + * receives the full capture map plus the plugin meta, and is + * responsible for turning it into a domain object. */ export interface ScanMatch { meta: TMeta; - /** The node captured as `@value` (the literal). */ - valueNode: Parser.SyntaxNode; - /** Raw text of the captured value node — caller must unquote. */ - valueText: string; + captures: CaptureMap; } /** @@ -103,9 +110,38 @@ export function compilePatterns(bundle: LanguagePatterns): Compile } /** - * Parse `content` as source code of the plugin's language and run every - * compiled pattern against the resulting AST. Returns one `ScanMatch` per - * matched `@value` capture, carrying the plugin's meta payload. + * Run every compiled pattern in `plugin` against an already-parsed + * tree. Use this when a plugin needs multiple query bundles against + * the same file (e.g. one query for class-level prefixes and another + * for method-level annotations) and wants to avoid re-parsing. + */ +export function runCompiledPatterns( + plugin: CompiledPatterns, + tree: Parser.Tree, +): ScanMatch[] { + const out: ScanMatch[] = []; + for (const compiled of plugin.patterns) { + let matches: Parser.QueryMatch[]; + try { + matches = compiled.query.matches(tree.rootNode); + } catch { + continue; + } + for (const match of matches) { + const captures: CaptureMap = {}; + for (const cap of match.captures) { + if (!(cap.name in captures)) captures[cap.name] = cap.node; + } + out.push({ meta: compiled.meta, captures }); + } + } + return out; +} + +/** + * Parse `content` with the plugin's grammar and run every compiled + * pattern against the AST. Returns one `ScanMatch` per matched query + * occurrence, carrying the plugin's meta payload. * * Errors are swallowed at the file level (malformed file must not abort * the whole extract). Individual pattern failures are swallowed too so @@ -116,34 +152,14 @@ export function scanFile( plugin: CompiledPatterns, content: string, ): ScanMatch[] { - const out: ScanMatch[] = []; let tree: Parser.Tree; try { parser.setLanguage(plugin.language); tree = parser.parse(content); } catch { - return out; + return []; } - - for (const compiled of plugin.patterns) { - let matches: Parser.QueryMatch[]; - try { - matches = compiled.query.matches(tree.rootNode); - } catch { - continue; - } - for (const match of matches) { - const valueCapture = match.captures.find((c) => c.name === 'value'); - if (!valueCapture) continue; - out.push({ - meta: compiled.meta, - valueNode: valueCapture.node, - valueText: valueCapture.node.text, - }); - } - } - - return out; + return runCompiledPatterns(plugin, tree); } /** From 2d9a830aa0d0bb973260194044d2e943c6a3c776 Mon Sep 17 00:00:00 2001 From: ivkond Date: Sun, 12 Apr 2026 00:34:06 +0000 Subject: [PATCH 04/10] refactor(group): migrate grpc-extractor source scans to tree-sitter plugins MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 3 (final) of the extractor refactor requested by @magyargergo on #796. Same architecture as phase 1 (topic) and phase 2 (http): thin language-agnostic orchestrator + per-language plugins that own tree-sitter grammars and query sources. With this commit the top-level extractors under `src/core/group/extractors/` import ZERO tree-sitter grammars and ZERO query strings — every grammar import lives in a `*-patterns/.ts` plugin file, and the orchestrators go through the registry indirection. ## Architecture ``` src/core/group/extractors/ ├── tree-sitter-scanner.ts # shared primitives (unchanged) ├── grpc-extractor.ts # orchestrator (only `.proto` parser left) └── grpc-patterns/ ├── types.ts # GrpcDetection, GrpcLanguagePlugin, GrpcRole ├── index.ts # registry: ext → plugin + GRPC_SCAN_GLOB ├── go.ts # tree-sitter-go: RegisterXxxServer, Unimplemented, NewXxxClient ├── java.ts # tree-sitter-java: @GrpcService + XxxImplBase + newBlockingStub ├── python.ts # tree-sitter-python: add_XxxServicer_to_server + XxxStub └── node.ts # tree-sitter-javascript + tree-sitter-typescript: # @GrpcMethod, @GrpcClient field type, # .getService('Svc'), new XxxServiceClient, # loadPackageDefinition dynamic constructors ``` ## Per-language coverage **Go (`go.ts`)** - Provider: `\w+.RegisterXxxServer(...)` via `call_expression → selector_expression → field_identifier` + JS regex filter `^Register(\w+)Server$`. - Provider: `pb.UnimplementedXxxServer` embedded in a struct via `struct_type → field_declaration_list → field_declaration → qualified_type → type_identifier` + JS filter. - Consumer: `\w+.NewXxxClient(...)` via the same call_expression query + JS filter `^New(\w+)Client$`. **Java (`java.ts`)** - Provider: `class X extends YyyGrpc.YyyImplBase` — two queries handle the scoped and plain forms. `scoped_type_identifier`'s children are positional (no `scope:`/`name:` fields), so the query matches the two `type_identifier` children by position. - `#match? @inner "ImplBase$"` restricts matches at query time. - Whether the class has `@GrpcService` or not controls only the `source` metadata label — the plugin walks the class_declaration's `modifiers` child in JS to detect the marker_annotation. - Consumer: `YyyGrpc.newStub(ch)` / `newBlockingStub(ch)` via a `method_invocation` query with `#match? @method "^new(Blocking)?Stub$"`, service name extracted via `^(\w+)Grpc$` on the object identifier. **Python (`python.ts`)** - Single call-expression query covers both bare identifier and `obj.method` attribute forms: `(call function: [(identifier) @fn (attribute attribute: (identifier) @fn)])`. - Plugin filters `@fn.text` against two JS regexes: `^add_(\w+)Servicer_to_server$` (provider) and `^(\w+)Stub$` (consumer), with a reserved-names ignore list for the Stub case (Mock / Test / Fake / Stub). **Node — JavaScript + TypeScript + TSX (`node.ts`)** - Pattern sources defined once, compiled three times (one per grammar) because `Parser.Query` objects are not portable across grammars. Exports three `GrpcLanguagePlugin`s sharing the same `scan`. - `@GrpcMethod('Service', 'Method')`: decorator query captures the two string literals. Confidence is hard-coded 0.8 regardless of proto map resolution (matches the original regex version's behaviour). - `@GrpcClient(...) field: XxxServiceClient`: decorator query captures the decorator node, plugin walks up to find the enclosing `public_field_definition` (decorators on fields are CHILDREN of the field definition in tree-sitter-typescript, not siblings) and reads its first `type_annotation → type_identifier`, then runs the `^(\w+Service)Client$` JS filter. - `client.getService('AuthService')`: call-expression query on `member_expression.property = "getService"` + string literal arg. - `new XxxServiceClient(...)`: `new_expression` with a bare identifier constructor, filtered by `^(\w+Service)Client$` so generic `new AuthClient(...)` (missing the `Service` infix) does NOT falsely register as a consumer. Preserves the regression test `test_extract_ts_non_service_client_constructor_is_ignored`. - `loadPackageDefinition` dynamic loader: gated on `tree.rootNode.text.includes('loadPackageDefinition')`. When set, `new foo.bar.Xxx(...)` qualified constructors with a capitalised property name register as consumers. ## Orchestrator changes `grpc-extractor.ts` loses every `scanGoProviders` / `scanJavaProviders` / ... helper and replaces them with a single source-scan loop that: 1. Parses each file with the plugin's grammar (one shared `Parser` instance across all files, `setLanguage` called per plugin). 2. Calls `plugin.scan(tree)` to get `GrpcDetection[]`. 3. Converts each detection to an `ExtractedContract` via the private `detectionToContract` helper, which: - Looks the short service name up in the proto map (filled by the `.proto` parser). - Picks confidence = `confidenceWithProto` if resolved, else `confidenceWithoutProto`. - Builds a method-level contract id (`grpc::pkg.Svc/Method`) when the detection carries a `methodName` (TS `@GrpcMethod` only), otherwise a service-level id (`grpc::pkg.Svc/*`). Everything else — the `.proto` parser, `buildProtoContext`, `buildProtoMap`, `resolveProtoConflict`, `serviceContractId`, `stripProtoCommentsAndStrings`, `extractServiceBlocks`, the dedupe function — stays exactly as before. The `.proto` parser is kept as a pragmatic exception to the "no regex in extractors" rule because no `tree-sitter-proto` grammar is installed in the repo; a comment at the top of the file explains this and flags the maintainer option of adding `tree-sitter-proto` as a dependency. ## Why this is better than the regex version 1. **Comments and strings are respected for free.** Matched node types are only code constructs, never text inside comments or string literals. 2. **No false positives on partial names.** The old `(\w+?)Grpc`-style regexes would cross-match unrelated identifiers; structural queries restrict matches to the exact AST shape (`scoped_type_identifier → type_identifier` pairs, `method_invocation → identifier` etc.). 3. **NestJS `@GrpcClient` is structural, not regex-based.** The old regex required a specific textual layout (`@GrpcClient(...) private readonly foo!: XxxServiceClient`); the plugin now walks the AST, so modifier order / optional modifiers / multi-line formatting don't break it. 4. **Language-agnostic extension.** Adding Kotlin / Rust / C# gRPC detection later is a one-file edit in `grpc-patterns/index.ts` — no touches to the shared scanner, the orchestrator, or the proto parser. ## Tests - `grpc-extractor.test.ts` — **43/43 pass** (tests unchanged; the contract shape is identical). Covers .proto parsing (including the brace-inside-string regression), Go provider/consumer, Java @GrpcService / plain ImplBase provider + newBlockingStub consumer, Python servicer + stub, TS @GrpcMethod + @GrpcClient + .getService + new XxxServiceClient + loadPackageDefinition + the `AuthClient` vs `AuthServiceClient` discrimination, dedupe across multiple patterns in one file, proto-aware confidence, and the inherited-package resolution for split proto definitions. - `topic-extractor.test.ts` — 30/30 pass. - `http-route-extractor.test.ts` — 18/18 pass. - `manifest-extractor.test.ts` — 8/8 pass. - `service.test.ts`, `sync.test.ts`, `storage.test.ts` — 41/41 pass. - `npx tsc -p tsconfig.json --noEmit` clean. ## Scope discipline (per GUARDRAILS.md) - Only files under `src/core/group/extractors/` are touched. - No pipeline.ts, MCP surface, ingestion, CI / release / security, or test changes. - New tree-sitter grammar imports (`tree-sitter-go`, `tree-sitter-java`, `tree-sitter-python`, `tree-sitter-javascript`, `tree-sitter-typescript`) are all already installed for the ingestion pipeline. ## End of phase series This commit completes the three-phase extractor refactor: - **Phase 1** (`ea06d11`): topic-extractor → `topic-patterns/` - **Phase 2** (`b6015f6`): http-route-extractor → `http-patterns/` - **Phase 3** (this commit): grpc-extractor → `grpc-patterns/` Every remaining regex-based extractor helper under the `src/core/group/ extractors/` directory is either (a) language-agnostic string processing (path normalization, dedupe keys) or (b) the `.proto` parser, which is documented as an explicit exception. Co-authored-by: Claude --- .../core/group/extractors/grpc-extractor.ts | 382 +++--------------- .../core/group/extractors/grpc-patterns/go.ts | 109 +++++ .../group/extractors/grpc-patterns/index.ts | 43 ++ .../group/extractors/grpc-patterns/java.ts | 179 ++++++++ .../group/extractors/grpc-patterns/node.ts | 295 ++++++++++++++ .../group/extractors/grpc-patterns/python.ts | 77 ++++ .../group/extractors/grpc-patterns/types.ts | 54 +++ 7 files changed, 824 insertions(+), 315 deletions(-) create mode 100644 gitnexus/src/core/group/extractors/grpc-patterns/go.ts create mode 100644 gitnexus/src/core/group/extractors/grpc-patterns/index.ts create mode 100644 gitnexus/src/core/group/extractors/grpc-patterns/java.ts create mode 100644 gitnexus/src/core/group/extractors/grpc-patterns/node.ts create mode 100644 gitnexus/src/core/group/extractors/grpc-patterns/python.ts create mode 100644 gitnexus/src/core/group/extractors/grpc-patterns/types.ts diff --git a/gitnexus/src/core/group/extractors/grpc-extractor.ts b/gitnexus/src/core/group/extractors/grpc-extractor.ts index 6e52c0b746..5910043a5f 100644 --- a/gitnexus/src/core/group/extractors/grpc-extractor.ts +++ b/gitnexus/src/core/group/extractors/grpc-extractor.ts @@ -1,8 +1,32 @@ import * as fs from 'node:fs'; import * as path from 'node:path'; import { glob } from 'glob'; +import Parser from 'tree-sitter'; import type { ContractExtractor, CypherExecutor } from '../contract-extractor.js'; import type { ExtractedContract, RepoHandle } from '../types.js'; +import { GRPC_SCAN_GLOB, getPluginForFile, type GrpcDetection } from './grpc-patterns/index.js'; + +/** + * Language-agnostic orchestrator for gRPC (provider + consumer) contract + * extraction. + * + * Two parts: + * + * 1. **`.proto` parsing** — done in-process by a small string-sanitizing + * parser (see `stripProtoCommentsAndStrings` + `extractServiceBlocks` + * below). NOT tree-sitter-based because no `tree-sitter-proto` + * grammar is installed in the repo. The parser preserves offsets so + * downstream regex scans run against a sanitized copy without + * affecting the line numbers of the original. Kept as a pragmatic + * exception to the "no regex in extractors" rule until / unless + * maintainers want to add a proto grammar. + * + * 2. **Source-scan providers / consumers** — delegated to per-language + * plugins in `./grpc-patterns/`. The orchestrator imports NO + * tree-sitter grammars or query strings — each plugin owns its own. + */ + +// ─── .proto parsing (not tree-sitter) ──────────────────────────────── function readSafe(repoPath: string, rel: string): string | null { const abs = path.resolve(repoPath, rel); @@ -322,6 +346,8 @@ export function serviceContractId(pkg: string, serviceName: string): string { return `grpc::${prefix}/*`; } +// ─── Orchestrator ──────────────────────────────────────────────────── + export class GrpcExtractor implements ContractExtractor { type = 'grpc' as const; @@ -337,7 +363,7 @@ export class GrpcExtractor implements ContractExtractor { const out: ExtractedContract[] = []; const protoContext = await buildProtoContext(repoPath); - // Proto files — definitive provider source + // ─── Proto files — definitive provider source ───────────────── const protoFiles = await glob('**/*.proto', { cwd: repoPath, ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**'], @@ -357,29 +383,29 @@ export class GrpcExtractor implements ContractExtractor { } const protoMap = protoContext.servicesByName; - // Source files — server/client detection - const sourceFiles = await glob('**/*.{go,java,py,ts,tsx,js,jsx}', { + // ─── Source files — delegate to per-language plugins ────────── + const sourceFiles = await glob(GRPC_SCAN_GLOB, { cwd: repoPath, ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**', '**/dist/**', '**/build/**'], nodir: true, }); + + const parser = new Parser(); for (const rel of sourceFiles) { + const plugin = getPluginForFile(rel); + if (!plugin) continue; const content = readSafe(repoPath, rel); if (!content) continue; - const ext = path.extname(rel).toLowerCase(); - - if (ext === '.go') { - out.push(...this.scanGoProviders(content, rel, protoMap)); - out.push(...this.scanGoConsumers(content, rel, protoMap)); - } else if (ext === '.java') { - out.push(...this.scanJavaProviders(content, rel, protoMap)); - out.push(...this.scanJavaConsumers(content, rel, protoMap)); - } else if (ext === '.py') { - out.push(...this.scanPythonProviders(content, rel, protoMap)); - out.push(...this.scanPythonConsumers(content, rel, protoMap)); - } else if (['.ts', '.tsx', '.js', '.jsx'].includes(ext)) { - out.push(...this.scanTsProviders(content, rel, protoMap)); - out.push(...this.scanTsConsumers(content, rel, protoMap)); + let detections: GrpcDetection[] = []; + try { + parser.setLanguage(plugin.language); + const tree = parser.parse(content); + detections = plugin.scan(tree); + } catch { + continue; + } + for (const d of detections) { + out.push(this.detectionToContract(d, rel, protoMap)); } } @@ -409,307 +435,33 @@ export class GrpcExtractor implements ContractExtractor { return out; } - private scanGoProviders( - content: string, - filePath: string, - protoMap: Map, - ): ExtractedContract[] { - const out: ExtractedContract[] = []; - - // pb.RegisterXxxServer( - const registerRe = /\w+\.Register(\w+)Server\s*\(/g; - let m: RegExpExecArray | null; - while ((m = registerRe.exec(content)) !== null) { - const serviceName = m[1]; - const candidates = protoMap.get(serviceName); - const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); - const cid = proto - ? serviceContractId(proto.package, proto.serviceName) - : serviceOnlyContractId(serviceName); - const conf = proto ? 0.8 : 0.65; - out.push( - makeContract(cid, 'provider', filePath, `Register${serviceName}Server`, conf, { - service: serviceName, - source: 'go_register', - }), - ); - } - - // pb.UnimplementedXxxServer - const unimplRe = /\w+\.Unimplemented(\w+)Server\b/g; - while ((m = unimplRe.exec(content)) !== null) { - const serviceName = m[1]; - const candidates = protoMap.get(serviceName); - const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); - const cid = proto - ? serviceContractId(proto.package, proto.serviceName) - : serviceOnlyContractId(serviceName); - const conf = proto ? 0.8 : 0.65; - out.push( - makeContract(cid, 'provider', filePath, `Unimplemented${serviceName}Server`, conf, { - service: serviceName, - source: 'go_unimplemented', - }), - ); - } - - return out; - } - - private scanGoConsumers( - content: string, - filePath: string, - protoMap: Map, - ): ExtractedContract[] { - const out: ExtractedContract[] = []; - const re = /\w+\.New(\w+)Client\s*\(/g; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const serviceName = m[1]; - const candidates = protoMap.get(serviceName); - const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); - const cid = proto - ? serviceContractId(proto.package, proto.serviceName) - : serviceOnlyContractId(serviceName); - const conf = proto ? 0.75 : 0.55; - out.push( - makeContract(cid, 'consumer', filePath, `New${serviceName}Client`, conf, { - service: serviceName, - source: 'go_client', - }), - ); - } - return out; - } - - private scanJavaProviders( - content: string, - filePath: string, - protoMap: Map, - ): ExtractedContract[] { - const out: ExtractedContract[] = []; - - const resolveJava = (svcName: string): { cid: string; conf: number } => { - const candidates = protoMap.get(svcName); - const proto = resolveProtoConflict(svcName, filePath, candidates ?? []); - const cid = proto - ? serviceContractId(proto.package, proto.serviceName) - : serviceOnlyContractId(svcName); - const conf = proto ? 0.8 : 0.65; - return { cid, conf }; - }; - - // @GrpcService - if (content.includes('@GrpcService')) { - const implBaseRe = /extends\s+(\w+)Grpc\.(\w+)ImplBase/; - const m = content.match(implBaseRe); - if (m) { - const { cid, conf } = resolveJava(m[1]); - out.push( - makeContract(cid, 'provider', filePath, m[2], conf, { - service: m[1], - source: 'java_grpc_service', - }), - ); - } else { - // Try extracting service name from class name - const classRe = - /class\s+(\w*?)(?:Grpc)?(?:Service)?\s+extends\s+(\w+)(?:Grpc\.(\w+))?ImplBase/; - const cm = content.match(classRe); - if (cm) { - const svcName = cm[2].replace(/Grpc$/, ''); - const { cid, conf } = resolveJava(svcName); - out.push( - makeContract(cid, 'provider', filePath, cm[1], conf, { - service: svcName, - source: 'java_grpc_service', - }), - ); - } - } - } - - // extends XxxImplBase (without @GrpcService) - if (!content.includes('@GrpcService')) { - const implRe = /extends\s+(\w+?)(?:Grpc\.(\w+))?ImplBase/; - const m = content.match(implRe); - if (m) { - const svcName = m[2] || m[1].replace(/Grpc$/, ''); - const { cid, conf } = resolveJava(svcName); - out.push( - makeContract(cid, 'provider', filePath, svcName, conf, { - service: svcName, - source: 'java_impl_base', - }), - ); - } - } - - return out; - } - - private scanJavaConsumers( - content: string, - filePath: string, - protoMap: Map, - ): ExtractedContract[] { - const out: ExtractedContract[] = []; - // XxxGrpc.newBlockingStub( or XxxGrpc.newStub( - const re = /(\w+)Grpc\.new(?:Blocking)?Stub\s*\(/g; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const serviceName = m[1]; - const candidates = protoMap.get(serviceName); - const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); - const cid = proto - ? serviceContractId(proto.package, proto.serviceName) - : serviceOnlyContractId(serviceName); - const conf = proto ? 0.75 : 0.55; - out.push( - makeContract(cid, 'consumer', filePath, `${serviceName}Stub`, conf, { - service: serviceName, - source: 'java_stub', - }), - ); - } - return out; - } - - private scanPythonProviders( - content: string, + /** + * Convert a plugin `GrpcDetection` into a concrete `ExtractedContract` + * by resolving the short service name against the proto map, building + * either a service-level (`grpc::pkg.Svc/*`) or method-level + * (`grpc::pkg.Svc/Method`) contract id, and selecting confidence + * based on whether the proto map had an entry. + */ + private detectionToContract( + d: GrpcDetection, filePath: string, protoMap: Map, - ): ExtractedContract[] { - const out: ExtractedContract[] = []; - // add_XxxServicer_to_server( - const re = /add_(\w+?)Servicer_to_server\s*\(/g; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const serviceName = m[1]; - const candidates = protoMap.get(serviceName); - const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); - const cid = proto - ? serviceContractId(proto.package, proto.serviceName) - : serviceOnlyContractId(serviceName); - const conf = proto ? 0.8 : 0.65; - out.push( - makeContract(cid, 'provider', filePath, `add_${serviceName}Servicer_to_server`, conf, { - service: serviceName, - source: 'python_servicer', - }), - ); - } - return out; - } - - private scanPythonConsumers( - content: string, - filePath: string, - protoMap: Map, - ): ExtractedContract[] { - const out: ExtractedContract[] = []; - // XxxStub( - const re = /(\w+)Stub\s*\(/g; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const name = m[1]; - // Filter out common false positives - if (['Mock', 'Test', 'Fake', 'Stub'].includes(name)) continue; - const candidates = protoMap.get(name); - const proto = resolveProtoConflict(name, filePath, candidates ?? []); - const cid = proto - ? serviceContractId(proto.package, proto.serviceName) - : serviceOnlyContractId(name); - const conf = proto ? 0.75 : 0.55; - out.push( - makeContract(cid, 'consumer', filePath, `${name}Stub`, conf, { - service: name, - source: 'python_stub', - }), - ); - } - return out; - } - - private scanTsProviders( - content: string, - filePath: string, - protoMap: Map, - ): ExtractedContract[] { - const out: ExtractedContract[] = []; - // @GrpcMethod('ServiceName', 'MethodName') - const re = /@GrpcMethod\s*\(\s*['"](\w+)['"]\s*,\s*['"](\w+)['"]\s*\)/g; - let m: RegExpExecArray | null; - while ((m = re.exec(content)) !== null) { - const serviceName = m[1]; - const methodName = m[2]; - const candidates = protoMap.get(serviceName); - const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); - const pkg = proto?.package ?? ''; - const cid = contractId(pkg, serviceName, methodName); - out.push( - makeContract(cid, 'provider', filePath, `${serviceName}.${methodName}`, 0.8, { - service: serviceName, - method: methodName, - source: 'ts_grpc_method', - }), - ); - } - return out; - } - - private scanTsConsumers( - content: string, - filePath: string, - protoMap: Map, - ): ExtractedContract[] { - const out: ExtractedContract[] = []; - const pushConsumer = ( - serviceName: string, - symbolName: string, - source: string, - confidenceWithProto = 0.75, - confidenceWithoutProto = 0.55, - ): void => { - const candidates = protoMap.get(serviceName); - const proto = resolveProtoConflict(serviceName, filePath, candidates ?? []); - const cid = proto - ? serviceContractId(proto.package, proto.serviceName) - : serviceOnlyContractId(serviceName); - const conf = proto ? confidenceWithProto : confidenceWithoutProto; - out.push( - makeContract(cid, 'consumer', filePath, symbolName, conf, { - service: serviceName, - source, - }), - ); + ): ExtractedContract { + const candidates = protoMap.get(d.serviceName); + const proto = resolveProtoConflict(d.serviceName, filePath, candidates ?? []); + const pkg = proto?.package ?? ''; + const cid = d.methodName + ? contractId(pkg, d.serviceName, d.methodName) + : proto + ? serviceContractId(pkg, d.serviceName) + : serviceOnlyContractId(d.serviceName); + const confidence = proto ? d.confidenceWithProto : d.confidenceWithoutProto; + const meta: Record = { + service: d.serviceName, + source: d.source, }; - - const grpcClientDecoratorRe = - /@GrpcClient\s*\([^)]*\)\s*(?:private|protected|public)?\s*(?:readonly\s+)?\w+[!?]?\s*:\s*(\w+Service)Client\b/g; - let match: RegExpExecArray | null; - while ((match = grpcClientDecoratorRe.exec(content)) !== null) { - pushConsumer(match[1], `${match[1]}Client`, 'ts_grpc_client_decorator'); - } - - const getServiceRe = /\.getService(?:<[^>]+>)?\s*\(\s*['"](\w+)['"]\s*\)/g; - while ((match = getServiceRe.exec(content)) !== null) { - pushConsumer(match[1], `${match[1]}Client`, 'ts_client_grpc_get_service'); - } - - const clientCtorRe = /new\s+(\w+Service)Client\s*\(/g; - while ((match = clientCtorRe.exec(content)) !== null) { - pushConsumer(match[1], `${match[1]}Client`, 'ts_generated_client'); - } - - if (content.includes('loadPackageDefinition')) { - const packageCtorRe = /new\s+[\w$.]*\.([A-Z]\w+)\s*\(/g; - while ((match = packageCtorRe.exec(content)) !== null) { - pushConsumer(match[1], `${match[1]}Client`, 'ts_load_package_definition'); - } - } - - return out; + if (d.methodName) meta.method = d.methodName; + return makeContract(cid, d.role, filePath, d.symbolName, confidence, meta); } private dedupe(items: ExtractedContract[]): ExtractedContract[] { diff --git a/gitnexus/src/core/group/extractors/grpc-patterns/go.ts b/gitnexus/src/core/group/extractors/grpc-patterns/go.ts new file mode 100644 index 0000000000..b1abbaeb76 --- /dev/null +++ b/gitnexus/src/core/group/extractors/grpc-patterns/go.ts @@ -0,0 +1,109 @@ +import Go from 'tree-sitter-go'; +import { + compilePatterns, + runCompiledPatterns, + type LanguagePatterns, +} from '../tree-sitter-scanner.js'; +import type { GrpcDetection, GrpcLanguagePlugin } from './types.js'; + +/** + * Go gRPC plugin. Detects: + * - Provider: `pb.RegisterXxxServer(...)` calls + * - Provider: `pb.UnimplementedXxxServer` embedded in a struct + * - Consumer: `pb.NewXxxClient(conn)` calls + */ + +const REGISTER_RE = /^Register(\w+)Server$/; +const UNIMPLEMENTED_RE = /^Unimplemented(\w+)Server$/; +const NEW_CLIENT_RE = /^New(\w+)Client$/; + +// Any `xxx.(...)` call — plugin filters the field identifier text. +const SELECTOR_CALL_PATTERNS = compilePatterns({ + name: 'go-grpc-selector-call', + language: Go, + patterns: [ + { + meta: {}, + query: ` + (call_expression + function: (selector_expression + field: (field_identifier) @fn)) + `, + }, + ], +} satisfies LanguagePatterns>); + +// Any `qualified_type` used as a struct field — for `pb.UnimplementedXxxServer`. +const STRUCT_EMBEDDING_PATTERNS = compilePatterns({ + name: 'go-grpc-struct-embedding', + language: Go, + patterns: [ + { + meta: {}, + query: ` + (struct_type + (field_declaration_list + (field_declaration + type: (qualified_type + name: (type_identifier) @field_type)))) + `, + }, + ], +} satisfies LanguagePatterns>); + +export const GO_GRPC_PLUGIN: GrpcLanguagePlugin = { + name: 'go-grpc', + language: Go, + scan(tree) { + const out: GrpcDetection[] = []; + + for (const match of runCompiledPatterns(SELECTOR_CALL_PATTERNS, tree)) { + const fnNode = match.captures.fn; + if (!fnNode) continue; + const fnText = fnNode.text; + + const registerMatch = REGISTER_RE.exec(fnText); + if (registerMatch) { + out.push({ + role: 'provider', + serviceName: registerMatch[1], + symbolName: fnText, + source: 'go_register', + confidenceWithProto: 0.8, + confidenceWithoutProto: 0.65, + }); + continue; + } + + const newClientMatch = NEW_CLIENT_RE.exec(fnText); + if (newClientMatch) { + out.push({ + role: 'consumer', + serviceName: newClientMatch[1], + symbolName: fnText, + source: 'go_client', + confidenceWithProto: 0.75, + confidenceWithoutProto: 0.55, + }); + continue; + } + } + + for (const match of runCompiledPatterns(STRUCT_EMBEDDING_PATTERNS, tree)) { + const fieldNode = match.captures.field_type; + if (!fieldNode) continue; + const unimpl = UNIMPLEMENTED_RE.exec(fieldNode.text); + if (!unimpl) continue; + out.push({ + role: 'provider', + serviceName: unimpl[1], + symbolName: fieldNode.text, + source: 'go_unimplemented', + confidenceWithProto: 0.8, + confidenceWithoutProto: 0.65, + }); + } + + return out; + }, +}; diff --git a/gitnexus/src/core/group/extractors/grpc-patterns/index.ts b/gitnexus/src/core/group/extractors/grpc-patterns/index.ts new file mode 100644 index 0000000000..10d33d4266 --- /dev/null +++ b/gitnexus/src/core/group/extractors/grpc-patterns/index.ts @@ -0,0 +1,43 @@ +import * as path from 'node:path'; +import type { GrpcLanguagePlugin } from './types.js'; +import { GO_GRPC_PLUGIN } from './go.js'; +import { JAVA_GRPC_PLUGIN } from './java.js'; +import { PYTHON_GRPC_PLUGIN } from './python.js'; +import { JAVASCRIPT_GRPC_PLUGIN, TYPESCRIPT_GRPC_PLUGIN, TSX_GRPC_PLUGIN } from './node.js'; + +export type { GrpcDetection, GrpcLanguagePlugin, GrpcRole } from './types.js'; + +/** + * File-extension → gRPC language plugin registry. Mirrors the shape + * of `http-patterns/index.ts` and `topic-patterns/index.ts`. + * + * To add a new language drop a `grpc-patterns/.ts` exporting a + * `GrpcLanguagePlugin`, import + register it here, and widen + * `GRPC_SCAN_GLOB` if needed. No edits to `grpc-extractor.ts` required. + */ +const REGISTRY: Record = { + '.go': GO_GRPC_PLUGIN, + '.java': JAVA_GRPC_PLUGIN, + '.py': PYTHON_GRPC_PLUGIN, + '.js': JAVASCRIPT_GRPC_PLUGIN, + '.jsx': JAVASCRIPT_GRPC_PLUGIN, + '.ts': TYPESCRIPT_GRPC_PLUGIN, + '.tsx': TSX_GRPC_PLUGIN, +}; + +/** + * Glob for source files worth scanning for gRPC server/client patterns. + * `.proto` files are handled directly by the orchestrator's in-tree + * string-sanitizing parser (no `tree-sitter-proto` grammar is + * installed). + */ +export const GRPC_SCAN_GLOB = '**/*.{go,java,py,ts,tsx,js,jsx}'; + +/** + * Return the gRPC plugin registered for the given file's extension, + * or `undefined` if the extension is not registered. + */ +export function getPluginForFile(rel: string): GrpcLanguagePlugin | undefined { + const ext = path.extname(rel).toLowerCase(); + return REGISTRY[ext]; +} diff --git a/gitnexus/src/core/group/extractors/grpc-patterns/java.ts b/gitnexus/src/core/group/extractors/grpc-patterns/java.ts new file mode 100644 index 0000000000..bf1cf48162 --- /dev/null +++ b/gitnexus/src/core/group/extractors/grpc-patterns/java.ts @@ -0,0 +1,179 @@ +import Parser from 'tree-sitter'; +import Java from 'tree-sitter-java'; +import { + compilePatterns, + runCompiledPatterns, + type LanguagePatterns, +} from '../tree-sitter-scanner.js'; +import type { GrpcDetection, GrpcLanguagePlugin } from './types.js'; + +/** + * Java gRPC plugin. Detects: + * - Provider: classes extending `XxxServiceGrpc.XxxServiceImplBase` + * (with or without a `@GrpcService` annotation; the annotation + * only affects confidence labelling in the original regex version + * — here we emit a single detection per class and pick the source + * label based on whether the annotation is present). + * - Consumer: `XxxServiceGrpc.newBlockingStub(ch)` / + * `XxxServiceGrpc.newStub(ch)` calls. + */ + +const IMPL_BASE_RE = /^(\w+)ImplBase$/; +const GRPC_SUFFIX_RE = /^(\w+)Grpc$/; + +// Classes extending `ScopedType.ScopedType` where the inner name ends +// in ImplBase. Covers `XxxServiceGrpc.XxxServiceImplBase`. +// Note: tree-sitter-java's `scoped_type_identifier` exposes its two +// segments as positional `type_identifier` children, NOT as named +// `scope:`/`name:` fields. We match positionally here and rely on the +// grammar's left-to-right ordering: first child = outer, second = inner. +const SCOPED_IMPL_BASE_PATTERNS = compilePatterns({ + name: 'java-grpc-scoped-impl-base', + language: Java, + patterns: [ + { + meta: {}, + query: ` + (class_declaration + name: (identifier) @class_name + superclass: (superclass + (scoped_type_identifier + (type_identifier) @outer + (type_identifier) @inner (#match? @inner "ImplBase$")))) @class + `, + }, + ], +} satisfies LanguagePatterns>); + +// Classes extending a simple `XxxImplBase` identifier (no scope). +const PLAIN_IMPL_BASE_PATTERNS = compilePatterns({ + name: 'java-grpc-plain-impl-base', + language: Java, + patterns: [ + { + meta: {}, + query: ` + (class_declaration + name: (identifier) @class_name + superclass: (superclass + (type_identifier) @plain_type (#match? @plain_type "ImplBase$"))) @class + `, + }, + ], +} satisfies LanguagePatterns>); + +// gRPC stub factories: `XxxGrpc.newStub(ch)` / `XxxGrpc.newBlockingStub(ch)`. +const STUB_PATTERNS = compilePatterns({ + name: 'java-grpc-stub', + language: Java, + patterns: [ + { + meta: {}, + query: ` + (method_invocation + object: (identifier) @grpc_cls + name: (identifier) @method (#match? @method "^new(Blocking)?Stub$")) + `, + }, + ], +} satisfies LanguagePatterns>); + +/** + * Check whether a `class_declaration` node has a `@GrpcService` + * annotation in its modifiers list. In tree-sitter-java, class-level + * annotations live under `(class_declaration (modifiers (marker_annotation|annotation)))`. + */ +function hasGrpcServiceAnnotation(classNode: Parser.SyntaxNode): boolean { + for (let i = 0; i < classNode.namedChildCount; i++) { + const child = classNode.namedChild(i); + if (!child || child.type !== 'modifiers') continue; + for (let j = 0; j < child.namedChildCount; j++) { + const mod = child.namedChild(j); + if (!mod) continue; + if (mod.type !== 'marker_annotation' && mod.type !== 'annotation') continue; + const nameNode = mod.childForFieldName('name'); + if (nameNode?.text === 'GrpcService') return true; + } + } + return false; +} + +/** + * Given the inner type_identifier text like `AuthServiceImplBase`, + * return the service name (`AuthService`), or null if the text + * doesn't end in `ImplBase`. + */ +function extractServiceFromImplBase(text: string): string | null { + const m = IMPL_BASE_RE.exec(text); + if (!m) return null; + // Strip a trailing `Grpc` on the service name too — the original + // regex replaces `Grpc$` on the extracted prefix. + return m[1].replace(/Grpc$/, ''); +} + +export const JAVA_GRPC_PLUGIN: GrpcLanguagePlugin = { + name: 'java-grpc', + language: Java, + scan(tree) { + const out: GrpcDetection[] = []; + const emittedClassIds = new Set(); + + // ─── Providers: scoped form (`...Grpc.XxxImplBase`) ───────────── + for (const match of runCompiledPatterns(SCOPED_IMPL_BASE_PATTERNS, tree)) { + const classNode = match.captures.class; + const innerNode = match.captures.inner; + if (!classNode || !innerNode) continue; + const serviceName = extractServiceFromImplBase(innerNode.text); + if (!serviceName) continue; + emittedClassIds.add(classNode.id); + const annotated = hasGrpcServiceAnnotation(classNode); + out.push({ + role: 'provider', + serviceName, + symbolName: serviceName, + source: annotated ? 'java_grpc_service' : 'java_impl_base', + confidenceWithProto: 0.8, + confidenceWithoutProto: 0.65, + }); + } + + // ─── Providers: plain form (`XxxImplBase`) ────────────────────── + for (const match of runCompiledPatterns(PLAIN_IMPL_BASE_PATTERNS, tree)) { + const classNode = match.captures.class; + const plainNode = match.captures.plain_type; + if (!classNode || !plainNode) continue; + if (emittedClassIds.has(classNode.id)) continue; + const serviceName = extractServiceFromImplBase(plainNode.text); + if (!serviceName) continue; + emittedClassIds.add(classNode.id); + const annotated = hasGrpcServiceAnnotation(classNode); + out.push({ + role: 'provider', + serviceName, + symbolName: serviceName, + source: annotated ? 'java_grpc_service' : 'java_impl_base', + confidenceWithProto: 0.8, + confidenceWithoutProto: 0.65, + }); + } + + // ─── Consumers: `XxxGrpc.newBlockingStub(...)` / `newStub(...)` ─ + for (const match of runCompiledPatterns(STUB_PATTERNS, tree)) { + const grpcClsNode = match.captures.grpc_cls; + if (!grpcClsNode) continue; + const grpcMatch = GRPC_SUFFIX_RE.exec(grpcClsNode.text); + if (!grpcMatch) continue; + const serviceName = grpcMatch[1]; + out.push({ + role: 'consumer', + serviceName, + symbolName: `${serviceName}Stub`, + source: 'java_stub', + confidenceWithProto: 0.75, + confidenceWithoutProto: 0.55, + }); + } + + return out; + }, +}; diff --git a/gitnexus/src/core/group/extractors/grpc-patterns/node.ts b/gitnexus/src/core/group/extractors/grpc-patterns/node.ts new file mode 100644 index 0000000000..2abe33ea32 --- /dev/null +++ b/gitnexus/src/core/group/extractors/grpc-patterns/node.ts @@ -0,0 +1,295 @@ +import Parser from 'tree-sitter'; +import JavaScript from 'tree-sitter-javascript'; +import TypeScript from 'tree-sitter-typescript'; +import { + compilePatterns, + runCompiledPatterns, + unquoteLiteral, + type CompiledPatterns, + type LanguagePatterns, + type PatternSpec, +} from '../tree-sitter-scanner.js'; +import type { GrpcDetection, GrpcLanguagePlugin } from './types.js'; + +/** + * Node.js / TypeScript gRPC plugin family. Detects: + * - Provider: NestJS `@GrpcMethod('Service', 'Method')` decorators + * - Consumer: NestJS `@GrpcClient(...) readonly x!: XxxServiceClient` + * - Consumer: `client.getService('AuthService')` + * - Consumer: `new XxxServiceClient(...)` (generated client constructor) + * - Consumer: `new foo.bar.Xxx(...)` when the file uses + * `loadPackageDefinition` (gRPC dynamic proto loader) + * + * As with the HTTP `node.ts`, pattern sources are defined once and + * compiled against three grammar variants (JS / TS / TSX) because + * `Parser.Query` is not portable across grammar objects. + */ + +const SERVICE_CLIENT_RE = /^(\w+Service)Client$/; +const CAPITALIZED_SERVICE_RE = /^[A-Z]\w+$/; + +// @GrpcMethod('Service', 'Method') +const GRPC_METHOD_SPEC: PatternSpec> = { + meta: {}, + query: ` + (decorator + (call_expression + function: (identifier) @dec (#eq? @dec "GrpcMethod") + arguments: (arguments + . [(string) (template_string)] @service + . [(string) (template_string)] @method))) + `, +}; + +// @GrpcClient(...) standalone decorator — the plugin walks to the next +// sibling (a field definition) to read its type annotation. +const GRPC_CLIENT_SPEC: PatternSpec> = { + meta: {}, + query: ` + (decorator + (call_expression + function: (identifier) @dec (#eq? @dec "GrpcClient"))) @grpc_client_decorator + `, +}; + +// `.getService('AuthService')` / `.getService('AuthService')` +const GET_SERVICE_SPEC: PatternSpec> = { + meta: {}, + query: ` + (call_expression + function: (member_expression + property: (property_identifier) @method (#eq? @method "getService")) + arguments: (arguments . [(string) (template_string)] @service)) + `, +}; + +// `new XxxServiceClient(...)` — bare identifier constructor. +const NEW_SIMPLE_CTOR_SPEC: PatternSpec> = { + meta: {}, + query: ` + (new_expression + constructor: (identifier) @ctor) + `, +}; + +// `new foo.bar.XxxService(...)` — qualified constructor. +const NEW_QUALIFIED_CTOR_SPEC: PatternSpec> = { + meta: {}, + query: ` + (new_expression + constructor: (member_expression + property: (property_identifier) @ctor)) + `, +}; + +interface NodeGrpcPatternBundle { + grpcMethod: CompiledPatterns>; + grpcClient: CompiledPatterns>; + getService: CompiledPatterns>; + newSimpleCtor: CompiledPatterns>; + newQualifiedCtor: CompiledPatterns>; +} + +function compileBundle(language: unknown, name: string): NodeGrpcPatternBundle { + const mk = (spec: PatternSpec>, suffix: string) => + compilePatterns({ + name: `${name}-${suffix}`, + language, + patterns: [spec], + } satisfies LanguagePatterns>); + return { + grpcMethod: mk(GRPC_METHOD_SPEC, 'grpc-method'), + grpcClient: mk(GRPC_CLIENT_SPEC, 'grpc-client'), + getService: mk(GET_SERVICE_SPEC, 'get-service'), + newSimpleCtor: mk(NEW_SIMPLE_CTOR_SPEC, 'new-simple-ctor'), + newQualifiedCtor: mk(NEW_QUALIFIED_CTOR_SPEC, 'new-qualified-ctor'), + }; +} + +const JAVASCRIPT_BUNDLE = compileBundle(JavaScript, 'javascript-grpc'); +const TYPESCRIPT_BUNDLE = compileBundle(TypeScript.typescript, 'typescript-grpc'); +const TSX_BUNDLE = compileBundle(TypeScript.tsx, 'tsx-grpc'); + +/** + * Given a `@GrpcClient(...)` decorator node, find the type annotation + * text of the field it decorates (e.g. `AuthServiceClient`). + * + * In tree-sitter-typescript, decorators on class fields can appear in + * two configurations: + * - As a CHILD of `public_field_definition` alongside the field's + * type annotation (the common case for NestJS `@GrpcClient`). + * - As a SIBLING of the field in `class_body` (for method + * decorators, but kept for resilience against grammar variants). + * We walk the parent container and search for a type annotation. + */ +function resolveGrpcClientFieldType(decoratorNode: Parser.SyntaxNode): string | null { + const parent = decoratorNode.parent; + if (!parent) return null; + + // Case 1: decorator is a child of the field definition — search + // the parent itself (which is the field definition) for a + // type_annotation child. + if (parent.type === 'public_field_definition' || parent.type.endsWith('field_definition')) { + return findFirstTypeAnnotationText(parent); + } + + // Case 2: decorator is a sibling of the field in a class_body — walk + // forward through subsequent siblings until we find a node containing + // a type annotation. + for (let i = 0; i < parent.namedChildCount; i++) { + const child = parent.namedChild(i); + if (child && child.id === decoratorNode.id) { + for (let j = i + 1; j < parent.namedChildCount; j++) { + const next = parent.namedChild(j); + if (!next) continue; + if (next.type === 'decorator') continue; + const typeText = findFirstTypeAnnotationText(next); + if (typeText) return typeText; + return null; + } + return null; + } + } + return null; +} + +/** + * Recursively search `node` for the first `type_annotation` child and + * return the text of its inner `type_identifier`, or null. Handles + * both `public_field_definition` and its variants. + */ +function findFirstTypeAnnotationText(node: Parser.SyntaxNode): string | null { + if (node.type === 'type_annotation') { + for (let i = 0; i < node.namedChildCount; i++) { + const child = node.namedChild(i); + if (!child) continue; + if (child.type === 'type_identifier') return child.text; + } + return null; + } + for (let i = 0; i < node.namedChildCount; i++) { + const child = node.namedChild(i); + if (!child) continue; + const found = findFirstTypeAnnotationText(child); + if (found) return found; + } + return null; +} + +function scanBundle(bundle: NodeGrpcPatternBundle, tree: Parser.Tree): GrpcDetection[] { + const out: GrpcDetection[] = []; + + // ─── Provider: @GrpcMethod('Service', 'Method') ────────────────── + for (const match of runCompiledPatterns(bundle.grpcMethod, tree)) { + const svcNode = match.captures.service; + const methodNode = match.captures.method; + if (!svcNode || !methodNode) continue; + const svc = unquoteLiteral(svcNode.text); + const mth = unquoteLiteral(methodNode.text); + if (!svc || !mth) continue; + out.push({ + role: 'provider', + serviceName: svc, + symbolName: `${svc}.${mth}`, + source: 'ts_grpc_method', + methodName: mth, + // @GrpcMethod hard-coded confidence 0.8 in the original code + // regardless of whether the proto map has a match. + confidenceWithProto: 0.8, + confidenceWithoutProto: 0.8, + }); + } + + // ─── Consumer: @GrpcClient() field with XxxServiceClient type ──── + for (const match of runCompiledPatterns(bundle.grpcClient, tree)) { + const decoratorNode = match.captures.grpc_client_decorator; + if (!decoratorNode) continue; + const typeText = resolveGrpcClientFieldType(decoratorNode); + if (!typeText) continue; + const svcMatch = SERVICE_CLIENT_RE.exec(typeText); + if (!svcMatch) continue; + const serviceName = svcMatch[1]; + out.push({ + role: 'consumer', + serviceName, + symbolName: `${serviceName}Client`, + source: 'ts_grpc_client_decorator', + confidenceWithProto: 0.75, + confidenceWithoutProto: 0.55, + }); + } + + // ─── Consumer: client.getService('Service') ─────────────────── + for (const match of runCompiledPatterns(bundle.getService, tree)) { + const svcNode = match.captures.service; + if (!svcNode) continue; + const svc = unquoteLiteral(svcNode.text); + if (!svc) continue; + out.push({ + role: 'consumer', + serviceName: svc, + symbolName: `${svc}Client`, + source: 'ts_client_grpc_get_service', + confidenceWithProto: 0.75, + confidenceWithoutProto: 0.55, + }); + } + + // ─── Consumer: new XxxServiceClient(...) ───────────────────────── + for (const match of runCompiledPatterns(bundle.newSimpleCtor, tree)) { + const ctorNode = match.captures.ctor; + if (!ctorNode) continue; + const svcMatch = SERVICE_CLIENT_RE.exec(ctorNode.text); + if (!svcMatch) continue; + const serviceName = svcMatch[1]; + out.push({ + role: 'consumer', + serviceName, + symbolName: `${serviceName}Client`, + source: 'ts_generated_client', + confidenceWithProto: 0.75, + confidenceWithoutProto: 0.55, + }); + } + + // ─── Consumer: loadPackageDefinition dynamic proto loader ──────── + // Only emit when the file uses loadPackageDefinition, otherwise a + // generic `new foo.bar.Something()` in unrelated code would falsely + // register as a gRPC consumer. + const usesLoadPackage = tree.rootNode.text.includes('loadPackageDefinition'); + if (usesLoadPackage) { + for (const match of runCompiledPatterns(bundle.newQualifiedCtor, tree)) { + const ctorNode = match.captures.ctor; + if (!ctorNode) continue; + if (!CAPITALIZED_SERVICE_RE.test(ctorNode.text)) continue; + out.push({ + role: 'consumer', + serviceName: ctorNode.text, + symbolName: `${ctorNode.text}Client`, + source: 'ts_load_package_definition', + confidenceWithProto: 0.75, + confidenceWithoutProto: 0.55, + }); + } + } + + return out; +} + +export const JAVASCRIPT_GRPC_PLUGIN: GrpcLanguagePlugin = { + name: 'javascript-grpc', + language: JavaScript, + scan: (tree) => scanBundle(JAVASCRIPT_BUNDLE, tree), +}; + +export const TYPESCRIPT_GRPC_PLUGIN: GrpcLanguagePlugin = { + name: 'typescript-grpc', + language: TypeScript.typescript, + scan: (tree) => scanBundle(TYPESCRIPT_BUNDLE, tree), +}; + +export const TSX_GRPC_PLUGIN: GrpcLanguagePlugin = { + name: 'tsx-grpc', + language: TypeScript.tsx, + scan: (tree) => scanBundle(TSX_BUNDLE, tree), +}; diff --git a/gitnexus/src/core/group/extractors/grpc-patterns/python.ts b/gitnexus/src/core/group/extractors/grpc-patterns/python.ts new file mode 100644 index 0000000000..a19896c1fa --- /dev/null +++ b/gitnexus/src/core/group/extractors/grpc-patterns/python.ts @@ -0,0 +1,77 @@ +import Python from 'tree-sitter-python'; +import { + compilePatterns, + runCompiledPatterns, + type LanguagePatterns, +} from '../tree-sitter-scanner.js'; +import type { GrpcDetection, GrpcLanguagePlugin } from './types.js'; + +/** + * Python gRPC plugin. Detects: + * - Provider: `add_XxxServicer_to_server(...)` calls (bare identifier + * or qualified attribute form `auth_pb2_grpc.add_XxxServicer_to_server`) + * - Consumer: `XxxStub(channel)` calls (bare or `auth_pb2_grpc.XxxStub`) + */ + +const ADD_SERVICER_RE = /^add_(\w+)Servicer_to_server$/; +const STUB_RE = /^(\w+)Stub$/; +/** Reserved names that would produce garbage service names. */ +const STUB_IGNORE = new Set(['Mock', 'Test', 'Fake', 'Stub']); + +// Any call whose target is either a bare identifier or an attribute +// access (`obj.method`). The plugin filters the function name in JS. +const CALL_PATTERNS = compilePatterns({ + name: 'python-grpc-call', + language: Python, + patterns: [ + { + meta: {}, + query: ` + (call + function: [ + (identifier) @fn + (attribute attribute: (identifier) @fn) + ]) + `, + }, + ], +} satisfies LanguagePatterns>); + +export const PYTHON_GRPC_PLUGIN: GrpcLanguagePlugin = { + name: 'python-grpc', + language: Python, + scan(tree) { + const out: GrpcDetection[] = []; + for (const match of runCompiledPatterns(CALL_PATTERNS, tree)) { + const fnNode = match.captures.fn; + if (!fnNode) continue; + const fnText = fnNode.text; + + const addServicer = ADD_SERVICER_RE.exec(fnText); + if (addServicer) { + out.push({ + role: 'provider', + serviceName: addServicer[1], + symbolName: fnText, + source: 'python_servicer', + confidenceWithProto: 0.8, + confidenceWithoutProto: 0.65, + }); + continue; + } + + const stubMatch = STUB_RE.exec(fnText); + if (stubMatch && !STUB_IGNORE.has(stubMatch[1])) { + out.push({ + role: 'consumer', + serviceName: stubMatch[1], + symbolName: fnText, + source: 'python_stub', + confidenceWithProto: 0.75, + confidenceWithoutProto: 0.55, + }); + } + } + return out; + }, +}; diff --git a/gitnexus/src/core/group/extractors/grpc-patterns/types.ts b/gitnexus/src/core/group/extractors/grpc-patterns/types.ts new file mode 100644 index 0000000000..606d9629bc --- /dev/null +++ b/gitnexus/src/core/group/extractors/grpc-patterns/types.ts @@ -0,0 +1,54 @@ +import type Parser from 'tree-sitter'; + +/** + * Shared types for the grpc-extractor language plugins. + * + * Each plugin lives in its own file (java.ts, go.ts, ...) and owns the + * tree-sitter grammar import + query sources. The top-level + * `grpc-extractor.ts` orchestrator only knows about this type module + * and the plugin registry (`./index.ts`). It MUST NOT import any + * grammar or query text directly. + */ + +export type GrpcRole = 'provider' | 'consumer'; + +/** + * One raw gRPC detection produced by a plugin's `scan()` function. The + * orchestrator uses the proto map to resolve the full package-qualified + * contract id and choose a confidence based on whether the proto was + * found. + * + * Most patterns produce service-level detections; `TS @GrpcMethod` is + * the only pattern that captures an explicit `methodName`, producing + * a method-level contract (`grpc::pkg.Service/Method`). + */ +export interface GrpcDetection { + role: GrpcRole; + /** Short service name, e.g. `"AuthService"`. */ + serviceName: string; + /** Symbol name emitted into the contract's symbolRef. */ + symbolName: string; + /** Metadata source label (goes into `meta.source`). */ + source: string; + /** Explicit method name; set only by TS `@GrpcMethod`. */ + methodName?: string; + /** Confidence when the proto map resolves the service. */ + confidenceWithProto: number; + /** Confidence when the proto map has no entry. */ + confidenceWithoutProto: number; +} + +/** + * One language-scoped gRPC plugin. Plugins own the tree-sitter grammar + * and a `scan(tree)` function that returns zero or more + * `GrpcDetection`s. The plugin is free to run multiple compiled query + * bundles and walk the AST to cross-reference captures. + * + * `language` is typed `unknown` for the same reason as in + * `tree-sitter-scanner.ts`. + */ +export interface GrpcLanguagePlugin { + name: string; + language: unknown; + scan(tree: Parser.Tree): GrpcDetection[]; +} From 62d2dc1c75cd7842eeaaa9b45e43b571bf8af124 Mon Sep 17 00:00:00 2001 From: ivkond Date: Sun, 12 Apr 2026 12:39:50 +0000 Subject: [PATCH 05/10] feat(group): add tree-sitter-proto for .proto file parsing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses @magyargergo's suggestion on #796 to replace the manual string-sanitizing .proto parser with a tree-sitter grammar. - **Vendored `tree-sitter-proto`** in `vendor/tree-sitter-proto/`. Grammar source from [coder3101/tree-sitter-proto](https://github.com/coder3101/tree-sitter-proto) (latest `grammar.js`), parser.c regenerated with `tree-sitter-cli 0.24` to produce ABI version 14 — compatible with the project's `tree-sitter 0.25` runtime (which supports ABI ≤ 14). Added as `optionalDependency` with `file:./vendor/tree-sitter-proto`. - **New `grpc-patterns/proto.ts` plugin** — uses the same `compilePatterns` + `runCompiledPatterns` infrastructure as every other plugin. Two queries: - `(package (full_ident) @pkg)` — package declaration - `(service (service_name) @service_name (rpc (rpc_name) @rpc_name))` — one match per (service, rpc) pair - **Graceful fallback** — `tree-sitter-proto` is an optional dependency. If it fails to install (platform incompatibility) or fails the runtime smoke-test (`setLanguage` + `parse` on a trivial proto), `PROTO_GRPC_PLUGIN` stays `null` and the orchestrator uses the existing manual parser. The smoke-test catches the `SyntaxNode` TDZ error that occurs in vitest's fork-based test runner. - **Orchestrator updated** — when `hasProtoPlugin` is true, `.proto` files are handled by the plugin loop (they're included in `GRPC_SCAN_GLOB`), and the manual `parseProtoFile` loop is skipped. `buildProtoContext` still runs to build the proto map for cross-referencing source-file detections. 1. **No manual comment/string stripping.** The old parser needed `stripProtoCommentsAndStrings` (110 lines) to avoid counting braces inside comments and string literals. tree-sitter handles this natively. 2. **No brace-depth tracking.** `extractServiceBlocks` used a manual depth counter to find service boundaries. tree-sitter's AST gives us `service` → `service_name` + `rpc` → `rpc_name` directly. 3. **Performance.** tree-sitter's C-based parser is faster than character-by-character JS scanning + regex on large proto files. - `grpc-extractor.test.ts` — **43/43 pass** (unchanged) - All other extractor tests — 99/99 pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude --- gitnexus/package-lock.json | 28 + gitnexus/package.json | 1 + .../core/group/extractors/grpc-extractor.ts | 46 +- .../group/extractors/grpc-patterns/index.ts | 24 +- .../group/extractors/grpc-patterns/proto.ts | 147 + gitnexus/vendor/tree-sitter-proto/binding.gyp | 30 + .../bindings/node/binding.cc | 20 + .../bindings/node/index.d.ts | 28 + .../tree-sitter-proto/bindings/node/index.js | 7 + .../vendor/tree-sitter-proto/package.json | 18 + .../tree-sitter-proto/src/node-types.json | 1145 ++ .../vendor/tree-sitter-proto/src/parser.c | 10149 ++++++++++++++++ .../tree-sitter-proto/src/tree_sitter/alloc.h | 54 + .../tree-sitter-proto/src/tree_sitter/array.h | 291 + .../src/tree_sitter/parser.h | 266 + 15 files changed, 12229 insertions(+), 25 deletions(-) create mode 100644 gitnexus/src/core/group/extractors/grpc-patterns/proto.ts create mode 100644 gitnexus/vendor/tree-sitter-proto/binding.gyp create mode 100644 gitnexus/vendor/tree-sitter-proto/bindings/node/binding.cc create mode 100644 gitnexus/vendor/tree-sitter-proto/bindings/node/index.d.ts create mode 100644 gitnexus/vendor/tree-sitter-proto/bindings/node/index.js create mode 100644 gitnexus/vendor/tree-sitter-proto/package.json create mode 100644 gitnexus/vendor/tree-sitter-proto/src/node-types.json create mode 100644 gitnexus/vendor/tree-sitter-proto/src/parser.c create mode 100644 gitnexus/vendor/tree-sitter-proto/src/tree_sitter/alloc.h create mode 100644 gitnexus/vendor/tree-sitter-proto/src/tree_sitter/array.h create mode 100644 gitnexus/vendor/tree-sitter-proto/src/tree_sitter/parser.h diff --git a/gitnexus/package-lock.json b/gitnexus/package-lock.json index ee80faecf4..a746292e8b 100644 --- a/gitnexus/package-lock.json +++ b/gitnexus/package-lock.json @@ -65,6 +65,7 @@ "optionalDependencies": { "tree-sitter-dart": "github:UserNobody14/tree-sitter-dart#80e23c07b64494f7e21090bb3450223ef0b192f4", "tree-sitter-kotlin": "^0.3.8", + "tree-sitter-proto": "file:./vendor/tree-sitter-proto", "tree-sitter-swift": "^0.6.0" } }, @@ -5296,6 +5297,10 @@ "node": "^18 || ^20 || >= 21" } }, + "node_modules/tree-sitter-proto": { + "resolved": "vendor/tree-sitter-proto", + "link": true + }, "node_modules/tree-sitter-python": { "version": "0.23.4", "resolved": "https://registry.npmjs.org/tree-sitter-python/-/tree-sitter-python-0.23.4.tgz", @@ -5877,6 +5882,29 @@ "peerDependencies": { "zod": "^3.25.28 || ^4" } + }, + "vendor/tree-sitter-proto": { + "version": "0.4.1", + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "dependencies": { + "node-addon-api": "^8.0.0", + "node-gyp-build": "^4.8.0" + }, + "peerDependencies": { + "tree-sitter": ">=0.21.0" + } + }, + "vendor/tree-sitter-proto/node_modules/node-addon-api": { + "version": "8.7.0", + "resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-8.7.0.tgz", + "integrity": "sha512-9MdFxmkKaOYVTV+XVRG8ArDwwQ77XIgIPyKASB1k3JPq3M8fGQQQE3YpMOrKm6g//Ktx8ivZr8xo1Qmtqub+GA==", + "license": "MIT", + "optional": true, + "engines": { + "node": "^18 || ^20 || >= 21" + } } } } diff --git a/gitnexus/package.json b/gitnexus/package.json index 864f281015..435f9b325d 100644 --- a/gitnexus/package.json +++ b/gitnexus/package.json @@ -87,6 +87,7 @@ "optionalDependencies": { "tree-sitter-dart": "github:UserNobody14/tree-sitter-dart#80e23c07b64494f7e21090bb3450223ef0b192f4", "tree-sitter-kotlin": "^0.3.8", + "tree-sitter-proto": "file:./vendor/tree-sitter-proto", "tree-sitter-swift": "^0.6.0" }, "devDependencies": { diff --git a/gitnexus/src/core/group/extractors/grpc-extractor.ts b/gitnexus/src/core/group/extractors/grpc-extractor.ts index 5910043a5f..584b1e49ef 100644 --- a/gitnexus/src/core/group/extractors/grpc-extractor.ts +++ b/gitnexus/src/core/group/extractors/grpc-extractor.ts @@ -4,7 +4,12 @@ import { glob } from 'glob'; import Parser from 'tree-sitter'; import type { ContractExtractor, CypherExecutor } from '../contract-extractor.js'; import type { ExtractedContract, RepoHandle } from '../types.js'; -import { GRPC_SCAN_GLOB, getPluginForFile, type GrpcDetection } from './grpc-patterns/index.js'; +import { + GRPC_SCAN_GLOB, + getPluginForFile, + hasProtoPlugin, + type GrpcDetection, +} from './grpc-patterns/index.js'; /** * Language-agnostic orchestrator for gRPC (provider + consumer) contract @@ -362,28 +367,33 @@ export class GrpcExtractor implements ContractExtractor { ): Promise { const out: ExtractedContract[] = []; const protoContext = await buildProtoContext(repoPath); + const protoMap = protoContext.servicesByName; // ─── Proto files — definitive provider source ───────────────── - const protoFiles = await glob('**/*.proto', { - cwd: repoPath, - ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**'], - nodir: true, - }); - for (const rel of protoFiles) { - const content = readSafe(repoPath, rel); - if (content) { - out.push( - ...this.parseProtoFile( - content, - rel, - protoContext.packagesByProto.get(normalizeProtoPath(rel)) ?? '', - ), - ); + // When tree-sitter-proto is available, .proto files are handled by + // the plugin loop below (they're in GRPC_SCAN_GLOB). Otherwise + // fall back to the manual string-sanitizing parser. + if (!hasProtoPlugin) { + const protoFiles = await glob('**/*.proto', { + cwd: repoPath, + ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**'], + nodir: true, + }); + for (const rel of protoFiles) { + const content = readSafe(repoPath, rel); + if (content) { + out.push( + ...this.parseProtoFile( + content, + rel, + protoContext.packagesByProto.get(normalizeProtoPath(rel)) ?? '', + ), + ); + } } } - const protoMap = protoContext.servicesByName; - // ─── Source files — delegate to per-language plugins ────────── + // ─── Source files (+ .proto when plugin available) ──────────── const sourceFiles = await glob(GRPC_SCAN_GLOB, { cwd: repoPath, ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**', '**/dist/**', '**/build/**'], diff --git a/gitnexus/src/core/group/extractors/grpc-patterns/index.ts b/gitnexus/src/core/group/extractors/grpc-patterns/index.ts index 10d33d4266..617c14beb2 100644 --- a/gitnexus/src/core/group/extractors/grpc-patterns/index.ts +++ b/gitnexus/src/core/group/extractors/grpc-patterns/index.ts @@ -4,16 +4,18 @@ import { GO_GRPC_PLUGIN } from './go.js'; import { JAVA_GRPC_PLUGIN } from './java.js'; import { PYTHON_GRPC_PLUGIN } from './python.js'; import { JAVASCRIPT_GRPC_PLUGIN, TYPESCRIPT_GRPC_PLUGIN, TSX_GRPC_PLUGIN } from './node.js'; +import { PROTO_GRPC_PLUGIN } from './proto.js'; export type { GrpcDetection, GrpcLanguagePlugin, GrpcRole } from './types.js'; +export { PROTO_GRPC_PLUGIN, extractPackageFromTree } from './proto.js'; /** * File-extension → gRPC language plugin registry. Mirrors the shape * of `http-patterns/index.ts` and `topic-patterns/index.ts`. * - * To add a new language drop a `grpc-patterns/.ts` exporting a - * `GrpcLanguagePlugin`, import + register it here, and widen - * `GRPC_SCAN_GLOB` if needed. No edits to `grpc-extractor.ts` required. + * `.proto` files are registered only when `tree-sitter-proto` is + * available (it's an optionalDependency). When absent, the orchestrator + * falls back to the built-in manual proto parser. */ const REGISTRY: Record = { '.go': GO_GRPC_PLUGIN, @@ -23,15 +25,23 @@ const REGISTRY: Record = { '.jsx': JAVASCRIPT_GRPC_PLUGIN, '.ts': TYPESCRIPT_GRPC_PLUGIN, '.tsx': TSX_GRPC_PLUGIN, + ...(PROTO_GRPC_PLUGIN ? { '.proto': PROTO_GRPC_PLUGIN } : {}), }; /** * Glob for source files worth scanning for gRPC server/client patterns. - * `.proto` files are handled directly by the orchestrator's in-tree - * string-sanitizing parser (no `tree-sitter-proto` grammar is - * installed). + * Includes `.proto` when the grammar is available. */ -export const GRPC_SCAN_GLOB = '**/*.{go,java,py,ts,tsx,js,jsx}'; +export const GRPC_SCAN_GLOB = PROTO_GRPC_PLUGIN + ? '**/*.{go,java,py,ts,tsx,js,jsx,proto}' + : '**/*.{go,java,py,ts,tsx,js,jsx}'; + +/** + * Whether the tree-sitter proto plugin is available. The orchestrator + * uses this to decide between the tree-sitter path and the fallback + * manual parser for `.proto` files. + */ +export const hasProtoPlugin = PROTO_GRPC_PLUGIN !== null; /** * Return the gRPC plugin registered for the given file's extension, diff --git a/gitnexus/src/core/group/extractors/grpc-patterns/proto.ts b/gitnexus/src/core/group/extractors/grpc-patterns/proto.ts new file mode 100644 index 0000000000..69b446e554 --- /dev/null +++ b/gitnexus/src/core/group/extractors/grpc-patterns/proto.ts @@ -0,0 +1,147 @@ +import { createRequire } from 'node:module'; +import { + compilePatterns, + runCompiledPatterns, + type CompiledPatterns, + type LanguagePatterns, +} from '../tree-sitter-scanner.js'; +import type { GrpcDetection, GrpcLanguagePlugin } from './types.js'; + +/** + * Protobuf (.proto) tree-sitter plugin for gRPC contract extraction. + * + * Uses `tree-sitter-proto` (coder3101/tree-sitter-proto) as an + * optionalDependency — if the grammar is not installed (e.g. native + * compilation failed on an unusual platform), the plugin exports + * `null` and the orchestrator falls back to the existing manual + * string-sanitizing parser. + * + * The grammar is vendored in `vendor/tree-sitter-proto/` with + * parser.c regenerated against tree-sitter-cli 0.24 (ABI version 14) + * so it is compatible with the project's tree-sitter 0.25 runtime. + */ + +const _require = createRequire(import.meta.url); +let ProtoGrammar: unknown = null; +try { + ProtoGrammar = _require('tree-sitter-proto'); +} catch { + // Grammar not installed — PROTO_GRPC_PLUGIN will be null. +} + +let PACKAGE_PATTERNS: CompiledPatterns> | null = null; +let SERVICE_PATTERNS: CompiledPatterns> | null = null; + +if (ProtoGrammar) { + try { + // Validate that the grammar actually loads end-to-end: compile queries + // AND parse + walk a trivial proto file. tree-sitter's internal + // `initializeLanguageNodeClasses` can fail with a TDZ error in some + // test runners (vitest forks) when SyntaxNode isn't fully initialized + // yet. Catching that here ensures `PROTO_GRPC_PLUGIN` stays null and + // the orchestrator falls back to the manual parser. + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const _Parser = _require('tree-sitter') as any; + // Smoke-test: parse + setLanguage to verify the grammar is + // end-to-end compatible with this tree-sitter runtime. + const _testParser = new _Parser(); + _testParser.setLanguage(ProtoGrammar); + _testParser.parse('service X { rpc Y (R) returns (R); }'); + + PACKAGE_PATTERNS = compilePatterns({ + name: 'proto-package', + language: ProtoGrammar, + patterns: [ + { + meta: {}, + query: `(package (full_ident) @pkg)`, + }, + ], + } satisfies LanguagePatterns>); + + SERVICE_PATTERNS = compilePatterns({ + name: 'proto-service', + language: ProtoGrammar, + patterns: [ + { + meta: {}, + query: ` + (service + (service_name) @service_name + (rpc + (rpc_name) @rpc_name)) + `, + }, + ], + } satisfies LanguagePatterns>); + } catch { + // Compilation failed (grammar ABI mismatch?) — fall back to null. + PACKAGE_PATTERNS = null; + SERVICE_PATTERNS = null; + ProtoGrammar = null; + } +} + +function buildPlugin(): GrpcLanguagePlugin | null { + if (!ProtoGrammar || !PACKAGE_PATTERNS || !SERVICE_PATTERNS) return null; + const pkgPatterns = PACKAGE_PATTERNS; + const svcPatterns = SERVICE_PATTERNS; + + return { + name: 'proto-grpc', + language: ProtoGrammar, + scan(tree) { + const out: GrpcDetection[] = []; + + // Extract `package` declaration (first match wins). + let pkg = ''; + for (const match of runCompiledPatterns(pkgPatterns, tree)) { + const pkgNode = match.captures.pkg; + if (pkgNode) { + pkg = pkgNode.text; + break; + } + } + + // Extract `service → rpc` pairs. The query returns one match per + // (service, rpc) combination thanks to the nested structure. + for (const match of runCompiledPatterns(svcPatterns, tree)) { + const serviceNode = match.captures.service_name; + const rpcNode = match.captures.rpc_name; + if (!serviceNode || !rpcNode) continue; + const serviceName = serviceNode.text; + const methodName = rpcNode.text; + out.push({ + role: 'provider', + serviceName, + symbolName: `${serviceName}.${methodName}`, + source: 'proto', + methodName, + // Proto definitions are the canonical source of truth — always + // high confidence regardless of cross-referencing. + confidenceWithProto: 0.85, + confidenceWithoutProto: 0.85, + }); + } + + return out; + }, + }; +} + +/** + * The proto plugin, or `null` if tree-sitter-proto is not available. + * The orchestrator checks this at import time and decides whether to + * use the tree-sitter path or the fallback manual parser. + */ +export const PROTO_GRPC_PLUGIN: GrpcLanguagePlugin | null = buildPlugin(); + +/** The package declaration text from a proto file's tree. */ +export function extractPackageFromTree(tree: import('tree-sitter').Tree): string { + if (!PACKAGE_PATTERNS) return ''; + for (const match of runCompiledPatterns(PACKAGE_PATTERNS, tree)) { + const pkgNode = match.captures.pkg; + if (pkgNode) return pkgNode.text; + } + return ''; +} diff --git a/gitnexus/vendor/tree-sitter-proto/binding.gyp b/gitnexus/vendor/tree-sitter-proto/binding.gyp new file mode 100644 index 0000000000..53ec4feb84 --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/binding.gyp @@ -0,0 +1,30 @@ +{ + "targets": [ + { + "target_name": "tree_sitter_proto_binding", + "dependencies": [ + " + +typedef struct TSLanguage TSLanguage; + +extern "C" TSLanguage *tree_sitter_proto(); + +// "tree-sitter", "language" hashed with BLAKE2 +const napi_type_tag LANGUAGE_TYPE_TAG = { + 0x8AF2E5212AD58ABF, 0xD5006CAD83ABBA16 +}; + +Napi::Object Init(Napi::Env env, Napi::Object exports) { + exports["name"] = Napi::String::New(env, "proto"); + auto language = Napi::External::New(env, tree_sitter_proto()); + language.TypeTag(&LANGUAGE_TYPE_TAG); + exports["language"] = language; + return exports; +} + +NODE_API_MODULE(tree_sitter_proto_binding, Init) diff --git a/gitnexus/vendor/tree-sitter-proto/bindings/node/index.d.ts b/gitnexus/vendor/tree-sitter-proto/bindings/node/index.d.ts new file mode 100644 index 0000000000..efe259eed0 --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/bindings/node/index.d.ts @@ -0,0 +1,28 @@ +type BaseNode = { + type: string; + named: boolean; +}; + +type ChildNode = { + multiple: boolean; + required: boolean; + types: BaseNode[]; +}; + +type NodeInfo = + | (BaseNode & { + subtypes: BaseNode[]; + }) + | (BaseNode & { + fields: { [name: string]: ChildNode }; + children: ChildNode[]; + }); + +type Language = { + name: string; + language: unknown; + nodeTypeInfo: NodeInfo[]; +}; + +declare const language: Language; +export = language; diff --git a/gitnexus/vendor/tree-sitter-proto/bindings/node/index.js b/gitnexus/vendor/tree-sitter-proto/bindings/node/index.js new file mode 100644 index 0000000000..6657bcf42d --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/bindings/node/index.js @@ -0,0 +1,7 @@ +const root = require("path").join(__dirname, "..", ".."); + +module.exports = require("node-gyp-build")(root); + +try { + module.exports.nodeTypeInfo = require("../../src/node-types.json"); +} catch (_) {} diff --git a/gitnexus/vendor/tree-sitter-proto/package.json b/gitnexus/vendor/tree-sitter-proto/package.json new file mode 100644 index 0000000000..387f3d9bb1 --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/package.json @@ -0,0 +1,18 @@ +{ + "name": "tree-sitter-proto", + "version": "0.4.1", + "description": "tree-sitter grammar for protobuf — ABI 14 build from coder3101/tree-sitter-proto latest grammar.js, compatible with tree-sitter 0.25", + "repository": "https://github.com/coder3101/tree-sitter-proto", + "license": "MIT", + "main": "bindings/node", + "scripts": { + "install": "node-gyp-build" + }, + "peerDependencies": { + "tree-sitter": ">=0.21.0" + }, + "dependencies": { + "node-addon-api": "^8.0.0", + "node-gyp-build": "^4.8.0" + } +} diff --git a/gitnexus/vendor/tree-sitter-proto/src/node-types.json b/gitnexus/vendor/tree-sitter-proto/src/node-types.json new file mode 100644 index 0000000000..63f8942a65 --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/src/node-types.json @@ -0,0 +1,1145 @@ +[ + { + "type": "block_lit", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": false, + "types": [ + { + "type": "constant", + "named": true + }, + { + "type": "full_ident", + "named": true + }, + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "bool", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "false", + "named": true + }, + { + "type": "true", + "named": true + } + ] + } + }, + { + "type": "constant", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "block_lit", + "named": true + }, + { + "type": "bool", + "named": true + }, + { + "type": "float_lit", + "named": true + }, + { + "type": "full_ident", + "named": true + }, + { + "type": "int_lit", + "named": true + }, + { + "type": "string", + "named": true + } + ] + } + }, + { + "type": "edition", + "named": true, + "fields": { + "year": { + "multiple": false, + "required": true, + "types": [ + { + "type": "string", + "named": true + } + ] + } + } + }, + { + "type": "empty_statement", + "named": true, + "fields": {} + }, + { + "type": "enum", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "enum_body", + "named": true + }, + { + "type": "enum_name", + "named": true + } + ] + } + }, + { + "type": "enum_body", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": false, + "types": [ + { + "type": "empty_statement", + "named": true + }, + { + "type": "enum_field", + "named": true + }, + { + "type": "option", + "named": true + }, + { + "type": "reserved", + "named": true + } + ] + } + }, + { + "type": "enum_field", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "enum_value_option", + "named": true + }, + { + "type": "identifier", + "named": true + }, + { + "type": "int_lit", + "named": true + } + ] + } + }, + { + "type": "enum_name", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "enum_value_option", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "constant", + "named": true + }, + { + "type": "full_ident", + "named": true + }, + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "extend", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "full_ident", + "named": true + }, + { + "type": "message_body", + "named": true + } + ] + } + }, + { + "type": "extensions", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "ranges", + "named": true + } + ] + } + }, + { + "type": "field", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "field_number", + "named": true + }, + { + "type": "field_options", + "named": true + }, + { + "type": "identifier", + "named": true + }, + { + "type": "type", + "named": true + } + ] + } + }, + { + "type": "field_number", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "int_lit", + "named": true + } + ] + } + }, + { + "type": "field_option", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "constant", + "named": true + }, + { + "type": "full_ident", + "named": true + }, + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "field_options", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "field_option", + "named": true + } + ] + } + }, + { + "type": "full_ident", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "group", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "field_number", + "named": true + }, + { + "type": "message_body", + "named": true + }, + { + "type": "message_name", + "named": true + } + ] + } + }, + { + "type": "import", + "named": true, + "fields": { + "path": { + "multiple": false, + "required": true, + "types": [ + { + "type": "string", + "named": true + } + ] + } + } + }, + { + "type": "int_lit", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "decimal_lit", + "named": true + }, + { + "type": "hex_lit", + "named": true + }, + { + "type": "octal_lit", + "named": true + } + ] + } + }, + { + "type": "key_type", + "named": true, + "fields": {} + }, + { + "type": "map_field", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "field_number", + "named": true + }, + { + "type": "field_options", + "named": true + }, + { + "type": "identifier", + "named": true + }, + { + "type": "key_type", + "named": true + }, + { + "type": "type", + "named": true + } + ] + } + }, + { + "type": "message", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "message_body", + "named": true + }, + { + "type": "message_name", + "named": true + } + ] + } + }, + { + "type": "message_body", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": false, + "types": [ + { + "type": "empty_statement", + "named": true + }, + { + "type": "enum", + "named": true + }, + { + "type": "extend", + "named": true + }, + { + "type": "extensions", + "named": true + }, + { + "type": "field", + "named": true + }, + { + "type": "group", + "named": true + }, + { + "type": "map_field", + "named": true + }, + { + "type": "message", + "named": true + }, + { + "type": "oneof", + "named": true + }, + { + "type": "option", + "named": true + }, + { + "type": "reserved", + "named": true + } + ] + } + }, + { + "type": "message_name", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "message_or_enum_type", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "oneof", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "empty_statement", + "named": true + }, + { + "type": "identifier", + "named": true + }, + { + "type": "oneof_field", + "named": true + }, + { + "type": "option", + "named": true + } + ] + } + }, + { + "type": "oneof_field", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "field_number", + "named": true + }, + { + "type": "field_options", + "named": true + }, + { + "type": "identifier", + "named": true + }, + { + "type": "type", + "named": true + } + ] + } + }, + { + "type": "option", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "constant", + "named": true + }, + { + "type": "full_ident", + "named": true + }, + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "package", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "full_ident", + "named": true + } + ] + } + }, + { + "type": "range", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "int_lit", + "named": true + } + ] + } + }, + { + "type": "ranges", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "range", + "named": true + } + ] + } + }, + { + "type": "reserved", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "ranges", + "named": true + }, + { + "type": "reserved_field_names", + "named": true + } + ] + } + }, + { + "type": "reserved_field_names", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "reserved_identifier", + "named": true + } + ] + } + }, + { + "type": "rpc", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "empty_statement", + "named": true + }, + { + "type": "message_or_enum_type", + "named": true + }, + { + "type": "option", + "named": true + }, + { + "type": "rpc_name", + "named": true + } + ] + } + }, + { + "type": "rpc_name", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "service", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": true, + "types": [ + { + "type": "empty_statement", + "named": true + }, + { + "type": "option", + "named": true + }, + { + "type": "rpc", + "named": true + }, + { + "type": "service_name", + "named": true + } + ] + } + }, + { + "type": "service_name", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": true, + "types": [ + { + "type": "identifier", + "named": true + } + ] + } + }, + { + "type": "source_file", + "named": true, + "root": true, + "fields": {}, + "children": { + "multiple": true, + "required": false, + "types": [ + { + "type": "edition", + "named": true + }, + { + "type": "empty_statement", + "named": true + }, + { + "type": "enum", + "named": true + }, + { + "type": "extend", + "named": true + }, + { + "type": "import", + "named": true + }, + { + "type": "message", + "named": true + }, + { + "type": "option", + "named": true + }, + { + "type": "package", + "named": true + }, + { + "type": "service", + "named": true + }, + { + "type": "syntax", + "named": true + } + ] + } + }, + { + "type": "string", + "named": true, + "fields": {}, + "children": { + "multiple": true, + "required": false, + "types": [ + { + "type": "escape_sequence", + "named": true + } + ] + } + }, + { + "type": "syntax", + "named": true, + "fields": {} + }, + { + "type": "type", + "named": true, + "fields": {}, + "children": { + "multiple": false, + "required": false, + "types": [ + { + "type": "message_or_enum_type", + "named": true + } + ] + } + }, + { + "type": "\"", + "named": false + }, + { + "type": "\"proto2\"", + "named": false + }, + { + "type": "\"proto3\"", + "named": false + }, + { + "type": "'", + "named": false + }, + { + "type": "(", + "named": false + }, + { + "type": ")", + "named": false + }, + { + "type": "+", + "named": false + }, + { + "type": ",", + "named": false + }, + { + "type": "-", + "named": false + }, + { + "type": ".", + "named": false + }, + { + "type": ":", + "named": false + }, + { + "type": ";", + "named": false + }, + { + "type": "<", + "named": false + }, + { + "type": "=", + "named": false + }, + { + "type": ">", + "named": false + }, + { + "type": "[", + "named": false + }, + { + "type": "]", + "named": false + }, + { + "type": "bool", + "named": false + }, + { + "type": "bytes", + "named": false + }, + { + "type": "comment", + "named": true + }, + { + "type": "decimal_lit", + "named": true + }, + { + "type": "double", + "named": false + }, + { + "type": "edition", + "named": false + }, + { + "type": "enum", + "named": false + }, + { + "type": "escape_sequence", + "named": true + }, + { + "type": "export", + "named": false + }, + { + "type": "extend", + "named": false + }, + { + "type": "extensions", + "named": false + }, + { + "type": "false", + "named": true + }, + { + "type": "fixed32", + "named": false + }, + { + "type": "fixed64", + "named": false + }, + { + "type": "float", + "named": false + }, + { + "type": "float_lit", + "named": true + }, + { + "type": "group", + "named": false + }, + { + "type": "hex_lit", + "named": true + }, + { + "type": "identifier", + "named": true + }, + { + "type": "import", + "named": false + }, + { + "type": "int32", + "named": false + }, + { + "type": "int64", + "named": false + }, + { + "type": "local", + "named": false + }, + { + "type": "map", + "named": false + }, + { + "type": "max", + "named": false + }, + { + "type": "message", + "named": false + }, + { + "type": "octal_lit", + "named": true + }, + { + "type": "oneof", + "named": false + }, + { + "type": "option", + "named": false + }, + { + "type": "optional", + "named": false + }, + { + "type": "package", + "named": false + }, + { + "type": "public", + "named": false + }, + { + "type": "repeated", + "named": false + }, + { + "type": "required", + "named": false + }, + { + "type": "reserved", + "named": false + }, + { + "type": "reserved_identifier", + "named": true + }, + { + "type": "returns", + "named": false + }, + { + "type": "rpc", + "named": false + }, + { + "type": "service", + "named": false + }, + { + "type": "sfixed32", + "named": false + }, + { + "type": "sfixed64", + "named": false + }, + { + "type": "sint32", + "named": false + }, + { + "type": "sint64", + "named": false + }, + { + "type": "stream", + "named": false + }, + { + "type": "string", + "named": false + }, + { + "type": "syntax", + "named": false + }, + { + "type": "to", + "named": false + }, + { + "type": "true", + "named": true + }, + { + "type": "uint32", + "named": false + }, + { + "type": "uint64", + "named": false + }, + { + "type": "weak", + "named": false + }, + { + "type": "{", + "named": false + }, + { + "type": "}", + "named": false + } +] \ No newline at end of file diff --git a/gitnexus/vendor/tree-sitter-proto/src/parser.c b/gitnexus/vendor/tree-sitter-proto/src/parser.c new file mode 100644 index 0000000000..96b661b8eb --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/src/parser.c @@ -0,0 +1,10149 @@ +#include "tree_sitter/parser.h" + +#if defined(__GNUC__) || defined(__clang__) +#pragma GCC diagnostic ignored "-Wmissing-field-initializers" +#endif + +#ifdef _MSC_VER +#pragma optimize("", off) +#elif defined(__clang__) +#pragma clang optimize off +#elif defined(__GNUC__) +#pragma GCC optimize ("O0") +#endif + +#define LANGUAGE_VERSION 14 +#define STATE_COUNT 345 +#define LARGE_STATE_COUNT 2 +#define SYMBOL_COUNT 133 +#define ALIAS_COUNT 0 +#define TOKEN_COUNT 73 +#define EXTERNAL_TOKEN_COUNT 0 +#define FIELD_COUNT 2 +#define MAX_ALIAS_SEQUENCE_LENGTH 14 +#define PRODUCTION_ID_COUNT 4 + +enum ts_symbol_identifiers { + anon_sym_SEMI = 1, + anon_sym_edition = 2, + anon_sym_EQ = 3, + anon_sym_syntax = 4, + anon_sym_DQUOTEproto3_DQUOTE = 5, + anon_sym_DQUOTEproto2_DQUOTE = 6, + anon_sym_import = 7, + anon_sym_weak = 8, + anon_sym_public = 9, + anon_sym_option = 10, + anon_sym_package = 11, + anon_sym_LPAREN = 12, + anon_sym_RPAREN = 13, + anon_sym_DOT = 14, + anon_sym_export = 15, + anon_sym_local = 16, + anon_sym_enum = 17, + anon_sym_LBRACE = 18, + anon_sym_RBRACE = 19, + anon_sym_DASH = 20, + anon_sym_LBRACK = 21, + anon_sym_COMMA = 22, + anon_sym_RBRACK = 23, + anon_sym_message = 24, + anon_sym_extend = 25, + anon_sym_optional = 26, + anon_sym_required = 27, + anon_sym_repeated = 28, + anon_sym_group = 29, + anon_sym_oneof = 30, + anon_sym_map = 31, + anon_sym_LT = 32, + anon_sym_GT = 33, + anon_sym_int32 = 34, + anon_sym_int64 = 35, + anon_sym_uint32 = 36, + anon_sym_uint64 = 37, + anon_sym_sint32 = 38, + anon_sym_sint64 = 39, + anon_sym_fixed32 = 40, + anon_sym_fixed64 = 41, + anon_sym_sfixed32 = 42, + anon_sym_sfixed64 = 43, + anon_sym_bool = 44, + anon_sym_string = 45, + anon_sym_double = 46, + anon_sym_float = 47, + anon_sym_bytes = 48, + anon_sym_reserved = 49, + anon_sym_extensions = 50, + anon_sym_to = 51, + anon_sym_max = 52, + anon_sym_service = 53, + anon_sym_rpc = 54, + anon_sym_stream = 55, + anon_sym_returns = 56, + anon_sym_PLUS = 57, + anon_sym_COLON = 58, + sym_identifier = 59, + sym_reserved_identifier = 60, + sym_true = 61, + sym_false = 62, + sym_decimal_lit = 63, + sym_octal_lit = 64, + sym_hex_lit = 65, + sym_float_lit = 66, + anon_sym_DQUOTE = 67, + aux_sym_string_token1 = 68, + anon_sym_SQUOTE = 69, + aux_sym_string_token2 = 70, + sym_escape_sequence = 71, + sym_comment = 72, + sym_source_file = 73, + sym_empty_statement = 74, + sym_edition = 75, + sym_syntax = 76, + sym_import = 77, + sym_package = 78, + sym_option = 79, + sym__option_name = 80, + sym_enum = 81, + sym_enum_name = 82, + sym_enum_body = 83, + sym_enum_field = 84, + sym_enum_value_option = 85, + sym_message = 86, + sym_message_body = 87, + sym_message_name = 88, + sym_extend = 89, + sym_group = 90, + sym_field = 91, + sym_field_options = 92, + sym_field_option = 93, + sym_oneof = 94, + sym_oneof_field = 95, + sym_map_field = 96, + sym_key_type = 97, + sym_type = 98, + sym_reserved = 99, + sym_extensions = 100, + sym_ranges = 101, + sym_range = 102, + sym_reserved_field_names = 103, + sym_message_or_enum_type = 104, + sym_field_number = 105, + sym_service = 106, + sym_service_name = 107, + sym_rpc = 108, + sym_rpc_name = 109, + sym_constant = 110, + sym_block_lit = 111, + sym_full_ident = 112, + sym_bool = 113, + sym_int_lit = 114, + sym_string = 115, + aux_sym_source_file_repeat1 = 116, + aux_sym__option_name_repeat1 = 117, + aux_sym_enum_body_repeat1 = 118, + aux_sym_enum_field_repeat1 = 119, + aux_sym_message_body_repeat1 = 120, + aux_sym_field_options_repeat1 = 121, + aux_sym_oneof_repeat1 = 122, + aux_sym_ranges_repeat1 = 123, + aux_sym_reserved_field_names_repeat1 = 124, + aux_sym_message_or_enum_type_repeat1 = 125, + aux_sym_service_repeat1 = 126, + aux_sym_rpc_repeat1 = 127, + aux_sym_block_lit_repeat1 = 128, + aux_sym_block_lit_repeat2 = 129, + aux_sym_string_repeat1 = 130, + aux_sym_string_repeat2 = 131, + aux_sym_string_repeat3 = 132, +}; + +static const char * const ts_symbol_names[] = { + [ts_builtin_sym_end] = "end", + [anon_sym_SEMI] = ";", + [anon_sym_edition] = "edition", + [anon_sym_EQ] = "=", + [anon_sym_syntax] = "syntax", + [anon_sym_DQUOTEproto3_DQUOTE] = "\"proto3\"", + [anon_sym_DQUOTEproto2_DQUOTE] = "\"proto2\"", + [anon_sym_import] = "import", + [anon_sym_weak] = "weak", + [anon_sym_public] = "public", + [anon_sym_option] = "option", + [anon_sym_package] = "package", + [anon_sym_LPAREN] = "(", + [anon_sym_RPAREN] = ")", + [anon_sym_DOT] = ".", + [anon_sym_export] = "export", + [anon_sym_local] = "local", + [anon_sym_enum] = "enum", + [anon_sym_LBRACE] = "{", + [anon_sym_RBRACE] = "}", + [anon_sym_DASH] = "-", + [anon_sym_LBRACK] = "[", + [anon_sym_COMMA] = ",", + [anon_sym_RBRACK] = "]", + [anon_sym_message] = "message", + [anon_sym_extend] = "extend", + [anon_sym_optional] = "optional", + [anon_sym_required] = "required", + [anon_sym_repeated] = "repeated", + [anon_sym_group] = "group", + [anon_sym_oneof] = "oneof", + [anon_sym_map] = "map", + [anon_sym_LT] = "<", + [anon_sym_GT] = ">", + [anon_sym_int32] = "int32", + [anon_sym_int64] = "int64", + [anon_sym_uint32] = "uint32", + [anon_sym_uint64] = "uint64", + [anon_sym_sint32] = "sint32", + [anon_sym_sint64] = "sint64", + [anon_sym_fixed32] = "fixed32", + [anon_sym_fixed64] = "fixed64", + [anon_sym_sfixed32] = "sfixed32", + [anon_sym_sfixed64] = "sfixed64", + [anon_sym_bool] = "bool", + [anon_sym_string] = "string", + [anon_sym_double] = "double", + [anon_sym_float] = "float", + [anon_sym_bytes] = "bytes", + [anon_sym_reserved] = "reserved", + [anon_sym_extensions] = "extensions", + [anon_sym_to] = "to", + [anon_sym_max] = "max", + [anon_sym_service] = "service", + [anon_sym_rpc] = "rpc", + [anon_sym_stream] = "stream", + [anon_sym_returns] = "returns", + [anon_sym_PLUS] = "+", + [anon_sym_COLON] = ":", + [sym_identifier] = "identifier", + [sym_reserved_identifier] = "reserved_identifier", + [sym_true] = "true", + [sym_false] = "false", + [sym_decimal_lit] = "decimal_lit", + [sym_octal_lit] = "octal_lit", + [sym_hex_lit] = "hex_lit", + [sym_float_lit] = "float_lit", + [anon_sym_DQUOTE] = "\"", + [aux_sym_string_token1] = "string_token1", + [anon_sym_SQUOTE] = "'", + [aux_sym_string_token2] = "string_token2", + [sym_escape_sequence] = "escape_sequence", + [sym_comment] = "comment", + [sym_source_file] = "source_file", + [sym_empty_statement] = "empty_statement", + [sym_edition] = "edition", + [sym_syntax] = "syntax", + [sym_import] = "import", + [sym_package] = "package", + [sym_option] = "option", + [sym__option_name] = "_option_name", + [sym_enum] = "enum", + [sym_enum_name] = "enum_name", + [sym_enum_body] = "enum_body", + [sym_enum_field] = "enum_field", + [sym_enum_value_option] = "enum_value_option", + [sym_message] = "message", + [sym_message_body] = "message_body", + [sym_message_name] = "message_name", + [sym_extend] = "extend", + [sym_group] = "group", + [sym_field] = "field", + [sym_field_options] = "field_options", + [sym_field_option] = "field_option", + [sym_oneof] = "oneof", + [sym_oneof_field] = "oneof_field", + [sym_map_field] = "map_field", + [sym_key_type] = "key_type", + [sym_type] = "type", + [sym_reserved] = "reserved", + [sym_extensions] = "extensions", + [sym_ranges] = "ranges", + [sym_range] = "range", + [sym_reserved_field_names] = "reserved_field_names", + [sym_message_or_enum_type] = "message_or_enum_type", + [sym_field_number] = "field_number", + [sym_service] = "service", + [sym_service_name] = "service_name", + [sym_rpc] = "rpc", + [sym_rpc_name] = "rpc_name", + [sym_constant] = "constant", + [sym_block_lit] = "block_lit", + [sym_full_ident] = "full_ident", + [sym_bool] = "bool", + [sym_int_lit] = "int_lit", + [sym_string] = "string", + [aux_sym_source_file_repeat1] = "source_file_repeat1", + [aux_sym__option_name_repeat1] = "_option_name_repeat1", + [aux_sym_enum_body_repeat1] = "enum_body_repeat1", + [aux_sym_enum_field_repeat1] = "enum_field_repeat1", + [aux_sym_message_body_repeat1] = "message_body_repeat1", + [aux_sym_field_options_repeat1] = "field_options_repeat1", + [aux_sym_oneof_repeat1] = "oneof_repeat1", + [aux_sym_ranges_repeat1] = "ranges_repeat1", + [aux_sym_reserved_field_names_repeat1] = "reserved_field_names_repeat1", + [aux_sym_message_or_enum_type_repeat1] = "message_or_enum_type_repeat1", + [aux_sym_service_repeat1] = "service_repeat1", + [aux_sym_rpc_repeat1] = "rpc_repeat1", + [aux_sym_block_lit_repeat1] = "block_lit_repeat1", + [aux_sym_block_lit_repeat2] = "block_lit_repeat2", + [aux_sym_string_repeat1] = "string_repeat1", + [aux_sym_string_repeat2] = "string_repeat2", + [aux_sym_string_repeat3] = "string_repeat3", +}; + +static const TSSymbol ts_symbol_map[] = { + [ts_builtin_sym_end] = ts_builtin_sym_end, + [anon_sym_SEMI] = anon_sym_SEMI, + [anon_sym_edition] = anon_sym_edition, + [anon_sym_EQ] = anon_sym_EQ, + [anon_sym_syntax] = anon_sym_syntax, + [anon_sym_DQUOTEproto3_DQUOTE] = anon_sym_DQUOTEproto3_DQUOTE, + [anon_sym_DQUOTEproto2_DQUOTE] = anon_sym_DQUOTEproto2_DQUOTE, + [anon_sym_import] = anon_sym_import, + [anon_sym_weak] = anon_sym_weak, + [anon_sym_public] = anon_sym_public, + [anon_sym_option] = anon_sym_option, + [anon_sym_package] = anon_sym_package, + [anon_sym_LPAREN] = anon_sym_LPAREN, + [anon_sym_RPAREN] = anon_sym_RPAREN, + [anon_sym_DOT] = anon_sym_DOT, + [anon_sym_export] = anon_sym_export, + [anon_sym_local] = anon_sym_local, + [anon_sym_enum] = anon_sym_enum, + [anon_sym_LBRACE] = anon_sym_LBRACE, + [anon_sym_RBRACE] = anon_sym_RBRACE, + [anon_sym_DASH] = anon_sym_DASH, + [anon_sym_LBRACK] = anon_sym_LBRACK, + [anon_sym_COMMA] = anon_sym_COMMA, + [anon_sym_RBRACK] = anon_sym_RBRACK, + [anon_sym_message] = anon_sym_message, + [anon_sym_extend] = anon_sym_extend, + [anon_sym_optional] = anon_sym_optional, + [anon_sym_required] = anon_sym_required, + [anon_sym_repeated] = anon_sym_repeated, + [anon_sym_group] = anon_sym_group, + [anon_sym_oneof] = anon_sym_oneof, + [anon_sym_map] = anon_sym_map, + [anon_sym_LT] = anon_sym_LT, + [anon_sym_GT] = anon_sym_GT, + [anon_sym_int32] = anon_sym_int32, + [anon_sym_int64] = anon_sym_int64, + [anon_sym_uint32] = anon_sym_uint32, + [anon_sym_uint64] = anon_sym_uint64, + [anon_sym_sint32] = anon_sym_sint32, + [anon_sym_sint64] = anon_sym_sint64, + [anon_sym_fixed32] = anon_sym_fixed32, + [anon_sym_fixed64] = anon_sym_fixed64, + [anon_sym_sfixed32] = anon_sym_sfixed32, + [anon_sym_sfixed64] = anon_sym_sfixed64, + [anon_sym_bool] = anon_sym_bool, + [anon_sym_string] = anon_sym_string, + [anon_sym_double] = anon_sym_double, + [anon_sym_float] = anon_sym_float, + [anon_sym_bytes] = anon_sym_bytes, + [anon_sym_reserved] = anon_sym_reserved, + [anon_sym_extensions] = anon_sym_extensions, + [anon_sym_to] = anon_sym_to, + [anon_sym_max] = anon_sym_max, + [anon_sym_service] = anon_sym_service, + [anon_sym_rpc] = anon_sym_rpc, + [anon_sym_stream] = anon_sym_stream, + [anon_sym_returns] = anon_sym_returns, + [anon_sym_PLUS] = anon_sym_PLUS, + [anon_sym_COLON] = anon_sym_COLON, + [sym_identifier] = sym_identifier, + [sym_reserved_identifier] = sym_reserved_identifier, + [sym_true] = sym_true, + [sym_false] = sym_false, + [sym_decimal_lit] = sym_decimal_lit, + [sym_octal_lit] = sym_octal_lit, + [sym_hex_lit] = sym_hex_lit, + [sym_float_lit] = sym_float_lit, + [anon_sym_DQUOTE] = anon_sym_DQUOTE, + [aux_sym_string_token1] = aux_sym_string_token1, + [anon_sym_SQUOTE] = anon_sym_SQUOTE, + [aux_sym_string_token2] = aux_sym_string_token2, + [sym_escape_sequence] = sym_escape_sequence, + [sym_comment] = sym_comment, + [sym_source_file] = sym_source_file, + [sym_empty_statement] = sym_empty_statement, + [sym_edition] = sym_edition, + [sym_syntax] = sym_syntax, + [sym_import] = sym_import, + [sym_package] = sym_package, + [sym_option] = sym_option, + [sym__option_name] = sym__option_name, + [sym_enum] = sym_enum, + [sym_enum_name] = sym_enum_name, + [sym_enum_body] = sym_enum_body, + [sym_enum_field] = sym_enum_field, + [sym_enum_value_option] = sym_enum_value_option, + [sym_message] = sym_message, + [sym_message_body] = sym_message_body, + [sym_message_name] = sym_message_name, + [sym_extend] = sym_extend, + [sym_group] = sym_group, + [sym_field] = sym_field, + [sym_field_options] = sym_field_options, + [sym_field_option] = sym_field_option, + [sym_oneof] = sym_oneof, + [sym_oneof_field] = sym_oneof_field, + [sym_map_field] = sym_map_field, + [sym_key_type] = sym_key_type, + [sym_type] = sym_type, + [sym_reserved] = sym_reserved, + [sym_extensions] = sym_extensions, + [sym_ranges] = sym_ranges, + [sym_range] = sym_range, + [sym_reserved_field_names] = sym_reserved_field_names, + [sym_message_or_enum_type] = sym_message_or_enum_type, + [sym_field_number] = sym_field_number, + [sym_service] = sym_service, + [sym_service_name] = sym_service_name, + [sym_rpc] = sym_rpc, + [sym_rpc_name] = sym_rpc_name, + [sym_constant] = sym_constant, + [sym_block_lit] = sym_block_lit, + [sym_full_ident] = sym_full_ident, + [sym_bool] = sym_bool, + [sym_int_lit] = sym_int_lit, + [sym_string] = sym_string, + [aux_sym_source_file_repeat1] = aux_sym_source_file_repeat1, + [aux_sym__option_name_repeat1] = aux_sym__option_name_repeat1, + [aux_sym_enum_body_repeat1] = aux_sym_enum_body_repeat1, + [aux_sym_enum_field_repeat1] = aux_sym_enum_field_repeat1, + [aux_sym_message_body_repeat1] = aux_sym_message_body_repeat1, + [aux_sym_field_options_repeat1] = aux_sym_field_options_repeat1, + [aux_sym_oneof_repeat1] = aux_sym_oneof_repeat1, + [aux_sym_ranges_repeat1] = aux_sym_ranges_repeat1, + [aux_sym_reserved_field_names_repeat1] = aux_sym_reserved_field_names_repeat1, + [aux_sym_message_or_enum_type_repeat1] = aux_sym_message_or_enum_type_repeat1, + [aux_sym_service_repeat1] = aux_sym_service_repeat1, + [aux_sym_rpc_repeat1] = aux_sym_rpc_repeat1, + [aux_sym_block_lit_repeat1] = aux_sym_block_lit_repeat1, + [aux_sym_block_lit_repeat2] = aux_sym_block_lit_repeat2, + [aux_sym_string_repeat1] = aux_sym_string_repeat1, + [aux_sym_string_repeat2] = aux_sym_string_repeat2, + [aux_sym_string_repeat3] = aux_sym_string_repeat3, +}; + +static const TSSymbolMetadata ts_symbol_metadata[] = { + [ts_builtin_sym_end] = { + .visible = false, + .named = true, + }, + [anon_sym_SEMI] = { + .visible = true, + .named = false, + }, + [anon_sym_edition] = { + .visible = true, + .named = false, + }, + [anon_sym_EQ] = { + .visible = true, + .named = false, + }, + [anon_sym_syntax] = { + .visible = true, + .named = false, + }, + [anon_sym_DQUOTEproto3_DQUOTE] = { + .visible = true, + .named = false, + }, + [anon_sym_DQUOTEproto2_DQUOTE] = { + .visible = true, + .named = false, + }, + [anon_sym_import] = { + .visible = true, + .named = false, + }, + [anon_sym_weak] = { + .visible = true, + .named = false, + }, + [anon_sym_public] = { + .visible = true, + .named = false, + }, + [anon_sym_option] = { + .visible = true, + .named = false, + }, + [anon_sym_package] = { + .visible = true, + .named = false, + }, + [anon_sym_LPAREN] = { + .visible = true, + .named = false, + }, + [anon_sym_RPAREN] = { + .visible = true, + .named = false, + }, + [anon_sym_DOT] = { + .visible = true, + .named = false, + }, + [anon_sym_export] = { + .visible = true, + .named = false, + }, + [anon_sym_local] = { + .visible = true, + .named = false, + }, + [anon_sym_enum] = { + .visible = true, + .named = false, + }, + [anon_sym_LBRACE] = { + .visible = true, + .named = false, + }, + [anon_sym_RBRACE] = { + .visible = true, + .named = false, + }, + [anon_sym_DASH] = { + .visible = true, + .named = false, + }, + [anon_sym_LBRACK] = { + .visible = true, + .named = false, + }, + [anon_sym_COMMA] = { + .visible = true, + .named = false, + }, + [anon_sym_RBRACK] = { + .visible = true, + .named = false, + }, + [anon_sym_message] = { + .visible = true, + .named = false, + }, + [anon_sym_extend] = { + .visible = true, + .named = false, + }, + [anon_sym_optional] = { + .visible = true, + .named = false, + }, + [anon_sym_required] = { + .visible = true, + .named = false, + }, + [anon_sym_repeated] = { + .visible = true, + .named = false, + }, + [anon_sym_group] = { + .visible = true, + .named = false, + }, + [anon_sym_oneof] = { + .visible = true, + .named = false, + }, + [anon_sym_map] = { + .visible = true, + .named = false, + }, + [anon_sym_LT] = { + .visible = true, + .named = false, + }, + [anon_sym_GT] = { + .visible = true, + .named = false, + }, + [anon_sym_int32] = { + .visible = true, + .named = false, + }, + [anon_sym_int64] = { + .visible = true, + .named = false, + }, + [anon_sym_uint32] = { + .visible = true, + .named = false, + }, + [anon_sym_uint64] = { + .visible = true, + .named = false, + }, + [anon_sym_sint32] = { + .visible = true, + .named = false, + }, + [anon_sym_sint64] = { + .visible = true, + .named = false, + }, + [anon_sym_fixed32] = { + .visible = true, + .named = false, + }, + [anon_sym_fixed64] = { + .visible = true, + .named = false, + }, + [anon_sym_sfixed32] = { + .visible = true, + .named = false, + }, + [anon_sym_sfixed64] = { + .visible = true, + .named = false, + }, + [anon_sym_bool] = { + .visible = true, + .named = false, + }, + [anon_sym_string] = { + .visible = true, + .named = false, + }, + [anon_sym_double] = { + .visible = true, + .named = false, + }, + [anon_sym_float] = { + .visible = true, + .named = false, + }, + [anon_sym_bytes] = { + .visible = true, + .named = false, + }, + [anon_sym_reserved] = { + .visible = true, + .named = false, + }, + [anon_sym_extensions] = { + .visible = true, + .named = false, + }, + [anon_sym_to] = { + .visible = true, + .named = false, + }, + [anon_sym_max] = { + .visible = true, + .named = false, + }, + [anon_sym_service] = { + .visible = true, + .named = false, + }, + [anon_sym_rpc] = { + .visible = true, + .named = false, + }, + [anon_sym_stream] = { + .visible = true, + .named = false, + }, + [anon_sym_returns] = { + .visible = true, + .named = false, + }, + [anon_sym_PLUS] = { + .visible = true, + .named = false, + }, + [anon_sym_COLON] = { + .visible = true, + .named = false, + }, + [sym_identifier] = { + .visible = true, + .named = true, + }, + [sym_reserved_identifier] = { + .visible = true, + .named = true, + }, + [sym_true] = { + .visible = true, + .named = true, + }, + [sym_false] = { + .visible = true, + .named = true, + }, + [sym_decimal_lit] = { + .visible = true, + .named = true, + }, + [sym_octal_lit] = { + .visible = true, + .named = true, + }, + [sym_hex_lit] = { + .visible = true, + .named = true, + }, + [sym_float_lit] = { + .visible = true, + .named = true, + }, + [anon_sym_DQUOTE] = { + .visible = true, + .named = false, + }, + [aux_sym_string_token1] = { + .visible = false, + .named = false, + }, + [anon_sym_SQUOTE] = { + .visible = true, + .named = false, + }, + [aux_sym_string_token2] = { + .visible = false, + .named = false, + }, + [sym_escape_sequence] = { + .visible = true, + .named = true, + }, + [sym_comment] = { + .visible = true, + .named = true, + }, + [sym_source_file] = { + .visible = true, + .named = true, + }, + [sym_empty_statement] = { + .visible = true, + .named = true, + }, + [sym_edition] = { + .visible = true, + .named = true, + }, + [sym_syntax] = { + .visible = true, + .named = true, + }, + [sym_import] = { + .visible = true, + .named = true, + }, + [sym_package] = { + .visible = true, + .named = true, + }, + [sym_option] = { + .visible = true, + .named = true, + }, + [sym__option_name] = { + .visible = false, + .named = true, + }, + [sym_enum] = { + .visible = true, + .named = true, + }, + [sym_enum_name] = { + .visible = true, + .named = true, + }, + [sym_enum_body] = { + .visible = true, + .named = true, + }, + [sym_enum_field] = { + .visible = true, + .named = true, + }, + [sym_enum_value_option] = { + .visible = true, + .named = true, + }, + [sym_message] = { + .visible = true, + .named = true, + }, + [sym_message_body] = { + .visible = true, + .named = true, + }, + [sym_message_name] = { + .visible = true, + .named = true, + }, + [sym_extend] = { + .visible = true, + .named = true, + }, + [sym_group] = { + .visible = true, + .named = true, + }, + [sym_field] = { + .visible = true, + .named = true, + }, + [sym_field_options] = { + .visible = true, + .named = true, + }, + [sym_field_option] = { + .visible = true, + .named = true, + }, + [sym_oneof] = { + .visible = true, + .named = true, + }, + [sym_oneof_field] = { + .visible = true, + .named = true, + }, + [sym_map_field] = { + .visible = true, + .named = true, + }, + [sym_key_type] = { + .visible = true, + .named = true, + }, + [sym_type] = { + .visible = true, + .named = true, + }, + [sym_reserved] = { + .visible = true, + .named = true, + }, + [sym_extensions] = { + .visible = true, + .named = true, + }, + [sym_ranges] = { + .visible = true, + .named = true, + }, + [sym_range] = { + .visible = true, + .named = true, + }, + [sym_reserved_field_names] = { + .visible = true, + .named = true, + }, + [sym_message_or_enum_type] = { + .visible = true, + .named = true, + }, + [sym_field_number] = { + .visible = true, + .named = true, + }, + [sym_service] = { + .visible = true, + .named = true, + }, + [sym_service_name] = { + .visible = true, + .named = true, + }, + [sym_rpc] = { + .visible = true, + .named = true, + }, + [sym_rpc_name] = { + .visible = true, + .named = true, + }, + [sym_constant] = { + .visible = true, + .named = true, + }, + [sym_block_lit] = { + .visible = true, + .named = true, + }, + [sym_full_ident] = { + .visible = true, + .named = true, + }, + [sym_bool] = { + .visible = true, + .named = true, + }, + [sym_int_lit] = { + .visible = true, + .named = true, + }, + [sym_string] = { + .visible = true, + .named = true, + }, + [aux_sym_source_file_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym__option_name_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_enum_body_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_enum_field_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_message_body_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_field_options_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_oneof_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_ranges_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_reserved_field_names_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_message_or_enum_type_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_service_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_rpc_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_block_lit_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_block_lit_repeat2] = { + .visible = false, + .named = false, + }, + [aux_sym_string_repeat1] = { + .visible = false, + .named = false, + }, + [aux_sym_string_repeat2] = { + .visible = false, + .named = false, + }, + [aux_sym_string_repeat3] = { + .visible = false, + .named = false, + }, +}; + +enum ts_field_identifiers { + field_path = 1, + field_year = 2, +}; + +static const char * const ts_field_names[] = { + [0] = NULL, + [field_path] = "path", + [field_year] = "year", +}; + +static const TSFieldMapSlice ts_field_map_slices[PRODUCTION_ID_COUNT] = { + [1] = {.index = 0, .length = 1}, + [2] = {.index = 1, .length = 1}, + [3] = {.index = 2, .length = 1}, +}; + +static const TSFieldMapEntry ts_field_map_entries[] = { + [0] = + {field_path, 1}, + [1] = + {field_year, 2}, + [2] = + {field_path, 2}, +}; + +static const TSSymbol ts_alias_sequences[PRODUCTION_ID_COUNT][MAX_ALIAS_SEQUENCE_LENGTH] = { + [0] = {0}, +}; + +static const uint16_t ts_non_terminal_alias_map[] = { + 0, +}; + +static const TSStateId ts_primary_state_ids[STATE_COUNT] = { + [0] = 0, + [1] = 1, + [2] = 2, + [3] = 3, + [4] = 4, + [5] = 3, + [6] = 2, + [7] = 7, + [8] = 8, + [9] = 9, + [10] = 10, + [11] = 11, + [12] = 12, + [13] = 13, + [14] = 14, + [15] = 15, + [16] = 16, + [17] = 17, + [18] = 18, + [19] = 19, + [20] = 20, + [21] = 21, + [22] = 22, + [23] = 23, + [24] = 24, + [25] = 25, + [26] = 26, + [27] = 27, + [28] = 28, + [29] = 29, + [30] = 30, + [31] = 31, + [32] = 32, + [33] = 33, + [34] = 34, + [35] = 35, + [36] = 36, + [37] = 37, + [38] = 38, + [39] = 39, + [40] = 40, + [41] = 41, + [42] = 42, + [43] = 43, + [44] = 44, + [45] = 45, + [46] = 46, + [47] = 47, + [48] = 48, + [49] = 49, + [50] = 50, + [51] = 51, + [52] = 52, + [53] = 53, + [54] = 54, + [55] = 30, + [56] = 8, + [57] = 57, + [58] = 57, + [59] = 59, + [60] = 60, + [61] = 61, + [62] = 57, + [63] = 57, + [64] = 8, + [65] = 65, + [66] = 30, + [67] = 67, + [68] = 68, + [69] = 28, + [70] = 22, + [71] = 71, + [72] = 7, + [73] = 73, + [74] = 21, + [75] = 23, + [76] = 76, + [77] = 77, + [78] = 78, + [79] = 24, + [80] = 25, + [81] = 26, + [82] = 27, + [83] = 83, + [84] = 84, + [85] = 85, + [86] = 86, + [87] = 87, + [88] = 86, + [89] = 89, + [90] = 90, + [91] = 87, + [92] = 92, + [93] = 93, + [94] = 94, + [95] = 95, + [96] = 96, + [97] = 97, + [98] = 95, + [99] = 99, + [100] = 100, + [101] = 101, + [102] = 102, + [103] = 103, + [104] = 104, + [105] = 105, + [106] = 106, + [107] = 107, + [108] = 108, + [109] = 39, + [110] = 110, + [111] = 111, + [112] = 112, + [113] = 113, + [114] = 114, + [115] = 115, + [116] = 116, + [117] = 117, + [118] = 118, + [119] = 119, + [120] = 120, + [121] = 121, + [122] = 122, + [123] = 123, + [124] = 30, + [125] = 125, + [126] = 126, + [127] = 127, + [128] = 39, + [129] = 129, + [130] = 130, + [131] = 131, + [132] = 132, + [133] = 8, + [134] = 134, + [135] = 135, + [136] = 136, + [137] = 137, + [138] = 138, + [139] = 139, + [140] = 140, + [141] = 141, + [142] = 142, + [143] = 143, + [144] = 116, + [145] = 145, + [146] = 146, + [147] = 147, + [148] = 29, + [149] = 149, + [150] = 150, + [151] = 151, + [152] = 152, + [153] = 153, + [154] = 154, + [155] = 155, + [156] = 156, + [157] = 157, + [158] = 158, + [159] = 159, + [160] = 160, + [161] = 161, + [162] = 162, + [163] = 163, + [164] = 164, + [165] = 165, + [166] = 166, + [167] = 167, + [168] = 168, + [169] = 169, + [170] = 170, + [171] = 171, + [172] = 172, + [173] = 173, + [174] = 174, + [175] = 175, + [176] = 176, + [177] = 177, + [178] = 178, + [179] = 179, + [180] = 180, + [181] = 181, + [182] = 182, + [183] = 183, + [184] = 184, + [185] = 185, + [186] = 186, + [187] = 187, + [188] = 188, + [189] = 189, + [190] = 190, + [191] = 191, + [192] = 192, + [193] = 193, + [194] = 194, + [195] = 195, + [196] = 196, + [197] = 197, + [198] = 198, + [199] = 199, + [200] = 200, + [201] = 201, + [202] = 202, + [203] = 203, + [204] = 204, + [205] = 188, + [206] = 206, + [207] = 207, + [208] = 208, + [209] = 209, + [210] = 210, + [211] = 211, + [212] = 212, + [213] = 213, + [214] = 214, + [215] = 188, + [216] = 188, + [217] = 217, + [218] = 218, + [219] = 219, + [220] = 220, + [221] = 221, + [222] = 222, + [223] = 223, + [224] = 224, + [225] = 225, + [226] = 226, + [227] = 227, + [228] = 228, + [229] = 229, + [230] = 230, + [231] = 231, + [232] = 232, + [233] = 233, + [234] = 234, + [235] = 235, + [236] = 236, + [237] = 237, + [238] = 238, + [239] = 239, + [240] = 240, + [241] = 241, + [242] = 242, + [243] = 243, + [244] = 244, + [245] = 245, + [246] = 246, + [247] = 247, + [248] = 248, + [249] = 249, + [250] = 250, + [251] = 251, + [252] = 252, + [253] = 253, + [254] = 254, + [255] = 255, + [256] = 256, + [257] = 247, + [258] = 258, + [259] = 259, + [260] = 243, + [261] = 244, + [262] = 258, + [263] = 221, + [264] = 228, + [265] = 255, + [266] = 227, + [267] = 248, + [268] = 259, + [269] = 269, + [270] = 254, + [271] = 271, + [272] = 272, + [273] = 273, + [274] = 274, + [275] = 275, + [276] = 276, + [277] = 277, + [278] = 278, + [279] = 279, + [280] = 280, + [281] = 281, + [282] = 282, + [283] = 283, + [284] = 284, + [285] = 285, + [286] = 286, + [287] = 287, + [288] = 288, + [289] = 289, + [290] = 290, + [291] = 291, + [292] = 292, + [293] = 293, + [294] = 294, + [295] = 295, + [296] = 296, + [297] = 297, + [298] = 298, + [299] = 299, + [300] = 300, + [301] = 301, + [302] = 302, + [303] = 303, + [304] = 304, + [305] = 305, + [306] = 306, + [307] = 307, + [308] = 308, + [309] = 309, + [310] = 310, + [311] = 311, + [312] = 312, + [313] = 313, + [314] = 314, + [315] = 315, + [316] = 316, + [317] = 317, + [318] = 318, + [319] = 319, + [320] = 320, + [321] = 321, + [322] = 308, + [323] = 323, + [324] = 324, + [325] = 303, + [326] = 308, + [327] = 308, + [328] = 328, + [329] = 329, + [330] = 330, + [331] = 331, + [332] = 332, + [333] = 333, + [334] = 334, + [335] = 335, + [336] = 336, + [337] = 337, + [338] = 338, + [339] = 338, + [340] = 338, + [341] = 341, + [342] = 342, + [343] = 343, + [344] = 338, +}; + +static bool ts_lex(TSLexer *lexer, TSStateId state) { + START_LEXER(); + eof = lexer->eof(lexer); + switch (state) { + case 0: + if (eof) ADVANCE(212); + ADVANCE_MAP( + '"', 450, + '\'', 457, + '(', 227, + ')', 228, + '+', 302, + ',', 241, + '-', 239, + '.', 230, + '/', 11, + '0', 442, + ':', 303, + ';', 213, + '<', 259, + '=', 215, + '>', 260, + '[', 240, + '\\', 38, + ']', 242, + 'b', 138, + 'd', 132, + 'e', 64, + 'f', 39, + 'g', 165, + 'i', 115, + 'l', 133, + 'm', 40, + 'n', 41, + 'o', 123, + 'p', 44, + 'r', 68, + 's', 76, + 't', 134, + 'u', 105, + 'w', 78, + '{', 237, + '}', 238, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(210); + if (('1' <= lookahead && lookahead <= '9')) ADVANCE(440); + END_STATE(); + case 1: + ADVANCE_MAP( + '"', 450, + '\'', 457, + '(', 227, + ')', 228, + ',', 241, + '.', 229, + '/', 11, + ';', 213, + '=', 215, + '>', 260, + '[', 240, + ']', 242, + '{', 237, + '}', 238, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(1); + if (('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 2: + ADVANCE_MAP( + '"', 450, + '\'', 457, + '+', 302, + '-', 239, + '.', 197, + '/', 11, + '0', 442, + ':', 303, + '[', 240, + ']', 242, + 'f', 324, + 'i', 380, + 'n', 325, + 't', 405, + '{', 237, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(2); + if (('1' <= lookahead && lookahead <= '9')) ADVANCE(440); + if (('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 3: + if (lookahead == '"') ADVANCE(450); + if (lookahead == '/') ADVANCE(452); + if (lookahead == '\\') ADVANCE(38); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') ADVANCE(455); + if (lookahead != 0) ADVANCE(456); + END_STATE(); + case 4: + if (lookahead == '"') ADVANCE(218); + END_STATE(); + case 5: + if (lookahead == '"') ADVANCE(217); + END_STATE(); + case 6: + if (lookahead == '"') ADVANCE(208); + if (lookahead == '\'') ADVANCE(209); + if (lookahead == '/') ADVANCE(11); + if (lookahead == '0') ADVANCE(444); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(6); + if (('1' <= lookahead && lookahead <= '9')) ADVANCE(441); + if (('A' <= lookahead && lookahead <= 'Z') || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(435); + END_STATE(); + case 7: + if (lookahead == '"') ADVANCE(434); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(7); + END_STATE(); + case 8: + ADVANCE_MAP( + '"', 153, + '.', 229, + '/', 11, + ';', 213, + 'b', 390, + 'd', 385, + 'e', 379, + 'f', 358, + 'g', 408, + 'i', 378, + 'l', 386, + 'm', 319, + 'o', 377, + 'r', 335, + 's', 355, + 'u', 364, + '}', 238, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(8); + if (('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 9: + if (lookahead == '\'') ADVANCE(457); + if (lookahead == '/') ADVANCE(459); + if (lookahead == '\\') ADVANCE(38); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') ADVANCE(462); + if (lookahead != 0) ADVANCE(463); + END_STATE(); + case 10: + if (lookahead == '\'') ADVANCE(434); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(10); + END_STATE(); + case 11: + if (lookahead == '*') ADVANCE(13); + if (lookahead == '/') ADVANCE(468); + END_STATE(); + case 12: + if (lookahead == '*') ADVANCE(12); + if (lookahead == '/') ADVANCE(467); + if (lookahead != 0) ADVANCE(13); + END_STATE(); + case 13: + if (lookahead == '*') ADVANCE(12); + if (lookahead != 0) ADVANCE(13); + END_STATE(); + case 14: + if (lookahead == '.') ADVANCE(448); + if (lookahead == 'E' || + lookahead == 'e') ADVANCE(196); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(14); + END_STATE(); + case 15: + ADVANCE_MAP( + '.', 229, + '/', 11, + ';', 213, + '[', 240, + 'b', 390, + 'd', 385, + 'f', 358, + 'i', 378, + 'o', 401, + 's', 355, + 'u', 364, + '{', 237, + '}', 238, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(15); + if (('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 16: + ADVANCE_MAP( + '.', 229, + '/', 11, + 'b', 390, + 'd', 385, + 'f', 358, + 'g', 408, + 'i', 378, + 'r', 346, + 's', 355, + 'u', 364, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(16); + if (('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 17: + ADVANCE_MAP( + '.', 229, + '/', 11, + 'b', 390, + 'd', 385, + 'f', 358, + 'g', 408, + 'i', 378, + 's', 355, + 'u', 364, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(17); + if (('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 18: + ADVANCE_MAP( + '.', 229, + '/', 11, + 'b', 390, + 'd', 385, + 'f', 358, + 'i', 378, + 's', 355, + 'u', 364, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(18); + if (('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 19: + if (lookahead == '.') ADVANCE(229); + if (lookahead == '/') ADVANCE(11); + if (lookahead == 's') ADVANCE(422); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(19); + if (('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 20: + if (lookahead == '.') ADVANCE(197); + if (lookahead == '/') ADVANCE(11); + if (lookahead == '0') ADVANCE(442); + if (lookahead == 'i') ADVANCE(124); + if (lookahead == 'n') ADVANCE(41); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(20); + if (('1' <= lookahead && lookahead <= '9')) ADVANCE(440); + END_STATE(); + case 21: + if (lookahead == '/') ADVANCE(11); + if (lookahead == ';') ADVANCE(213); + if (lookahead == 'o') ADVANCE(401); + if (lookahead == 'r') ADVANCE(351); + if (lookahead == '}') ADVANCE(238); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(21); + if (('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 22: + if (lookahead == '2') ADVANCE(261); + END_STATE(); + case 23: + if (lookahead == '2') ADVANCE(269); + END_STATE(); + case 24: + if (lookahead == '2') ADVANCE(265); + END_STATE(); + case 25: + if (lookahead == '2') ADVANCE(273); + END_STATE(); + case 26: + if (lookahead == '2') ADVANCE(277); + END_STATE(); + case 27: + if (lookahead == '2') ADVANCE(4); + if (lookahead == '3') ADVANCE(5); + END_STATE(); + case 28: + if (lookahead == '3') ADVANCE(22); + if (lookahead == '6') ADVANCE(33); + END_STATE(); + case 29: + if (lookahead == '3') ADVANCE(23); + if (lookahead == '6') ADVANCE(34); + END_STATE(); + case 30: + if (lookahead == '3') ADVANCE(24); + if (lookahead == '6') ADVANCE(35); + END_STATE(); + case 31: + if (lookahead == '3') ADVANCE(25); + if (lookahead == '6') ADVANCE(36); + END_STATE(); + case 32: + if (lookahead == '3') ADVANCE(26); + if (lookahead == '6') ADVANCE(37); + END_STATE(); + case 33: + if (lookahead == '4') ADVANCE(263); + END_STATE(); + case 34: + if (lookahead == '4') ADVANCE(271); + END_STATE(); + case 35: + if (lookahead == '4') ADVANCE(267); + END_STATE(); + case 36: + if (lookahead == '4') ADVANCE(275); + END_STATE(); + case 37: + if (lookahead == '4') ADVANCE(279); + END_STATE(); + case 38: + if (lookahead == 'U') ADVANCE(207); + if (lookahead == 'u') ADVANCE(203); + if (lookahead == 'x') ADVANCE(201); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(466); + if (lookahead != 0) ADVANCE(464); + END_STATE(); + case 39: + if (lookahead == 'a') ADVANCE(111); + if (lookahead == 'i') ADVANCE(194); + if (lookahead == 'l') ADVANCE(139); + END_STATE(); + case 40: + if (lookahead == 'a') ADVANCE(148); + if (lookahead == 'e') ADVANCE(169); + END_STATE(); + case 41: + if (lookahead == 'a') ADVANCE(118); + END_STATE(); + case 42: + if (lookahead == 'a') ADVANCE(94); + END_STATE(); + case 43: + if (lookahead == 'a') ADVANCE(193); + END_STATE(); + case 44: + if (lookahead == 'a') ADVANCE(54); + if (lookahead == 'u') ADVANCE(52); + END_STATE(); + case 45: + if (lookahead == 'a') ADVANCE(117); + END_STATE(); + case 46: + if (lookahead == 'a') ADVANCE(106); + END_STATE(); + case 47: + if (lookahead == 'a') ADVANCE(192); + if (lookahead == 'e') ADVANCE(169); + END_STATE(); + case 48: + if (lookahead == 'a') ADVANCE(172); + END_STATE(); + case 49: + if (lookahead == 'a') ADVANCE(109); + END_STATE(); + case 50: + if (lookahead == 'a') ADVANCE(179); + END_STATE(); + case 51: + if (lookahead == 'a') ADVANCE(95); + END_STATE(); + case 52: + if (lookahead == 'b') ADVANCE(112); + END_STATE(); + case 53: + if (lookahead == 'b') ADVANCE(113); + END_STATE(); + case 54: + if (lookahead == 'c') ADVANCE(107); + END_STATE(); + case 55: + if (lookahead == 'c') ADVANCE(298); + END_STATE(); + case 56: + if (lookahead == 'c') ADVANCE(221); + END_STATE(); + case 57: + if (lookahead == 'c') ADVANCE(49); + END_STATE(); + case 58: + if (lookahead == 'c') ADVANCE(75); + END_STATE(); + case 59: + if (lookahead == 'd') ADVANCE(245); + END_STATE(); + case 60: + if (lookahead == 'd') ADVANCE(245); + if (lookahead == 's') ADVANCE(101); + END_STATE(); + case 61: + if (lookahead == 'd') ADVANCE(251); + END_STATE(); + case 62: + if (lookahead == 'd') ADVANCE(249); + END_STATE(); + case 63: + if (lookahead == 'd') ADVANCE(291); + END_STATE(); + case 64: + if (lookahead == 'd') ADVANCE(103); + if (lookahead == 'n') ADVANCE(184); + if (lookahead == 'x') ADVANCE(151); + END_STATE(); + case 65: + if (lookahead == 'd') ADVANCE(103); + if (lookahead == 'n') ADVANCE(184); + if (lookahead == 'x') ADVANCE(152); + END_STATE(); + case 66: + if (lookahead == 'd') ADVANCE(31); + END_STATE(); + case 67: + if (lookahead == 'd') ADVANCE(32); + END_STATE(); + case 68: + if (lookahead == 'e') ADVANCE(154); + if (lookahead == 'p') ADVANCE(55); + END_STATE(); + case 69: + if (lookahead == 'e') ADVANCE(66); + END_STATE(); + case 70: + if (lookahead == 'e') ADVANCE(436); + END_STATE(); + case 71: + if (lookahead == 'e') ADVANCE(438); + END_STATE(); + case 72: + if (lookahead == 'e') ADVANCE(285); + END_STATE(); + case 73: + if (lookahead == 'e') ADVANCE(243); + END_STATE(); + case 74: + if (lookahead == 'e') ADVANCE(226); + END_STATE(); + case 75: + if (lookahead == 'e') ADVANCE(297); + END_STATE(); + case 76: + if (lookahead == 'e') ADVANCE(157); + if (lookahead == 'f') ADVANCE(104); + if (lookahead == 'i') ADVANCE(126); + if (lookahead == 't') ADVANCE(158); + if (lookahead == 'y') ADVANCE(127); + END_STATE(); + case 77: + if (lookahead == 'e') ADVANCE(157); + if (lookahead == 'y') ADVANCE(127); + END_STATE(); + case 78: + if (lookahead == 'e') ADVANCE(46); + END_STATE(); + case 79: + if (lookahead == 'e') ADVANCE(61); + END_STATE(); + case 80: + if (lookahead == 'e') ADVANCE(119); + END_STATE(); + case 81: + if (lookahead == 'e') ADVANCE(62); + END_STATE(); + case 82: + if (lookahead == 'e') ADVANCE(166); + END_STATE(); + case 83: + if (lookahead == 'e') ADVANCE(63); + END_STATE(); + case 84: + if (lookahead == 'e') ADVANCE(159); + END_STATE(); + case 85: + if (lookahead == 'e') ADVANCE(135); + END_STATE(); + case 86: + if (lookahead == 'e') ADVANCE(50); + END_STATE(); + case 87: + if (lookahead == 'e') ADVANCE(45); + if (lookahead == 'i') ADVANCE(125); + END_STATE(); + case 88: + if (lookahead == 'e') ADVANCE(129); + END_STATE(); + case 89: + if (lookahead == 'e') ADVANCE(67); + END_STATE(); + case 90: + if (lookahead == 'f') ADVANCE(447); + END_STATE(); + case 91: + if (lookahead == 'f') ADVANCE(447); + if (lookahead == 't') ADVANCE(28); + END_STATE(); + case 92: + if (lookahead == 'f') ADVANCE(255); + END_STATE(); + case 93: + if (lookahead == 'g') ADVANCE(283); + END_STATE(); + case 94: + if (lookahead == 'g') ADVANCE(73); + END_STATE(); + case 95: + if (lookahead == 'g') ADVANCE(74); + END_STATE(); + case 96: + if (lookahead == 'i') ADVANCE(56); + END_STATE(); + case 97: + if (lookahead == 'i') ADVANCE(58); + END_STATE(); + case 98: + if (lookahead == 'i') ADVANCE(142); + END_STATE(); + case 99: + if (lookahead == 'i') ADVANCE(164); + END_STATE(); + case 100: + if (lookahead == 'i') ADVANCE(143); + END_STATE(); + case 101: + if (lookahead == 'i') ADVANCE(145); + END_STATE(); + case 102: + if (lookahead == 'i') ADVANCE(146); + END_STATE(); + case 103: + if (lookahead == 'i') ADVANCE(181); + END_STATE(); + case 104: + if (lookahead == 'i') ADVANCE(195); + END_STATE(); + case 105: + if (lookahead == 'i') ADVANCE(131); + END_STATE(); + case 106: + if (lookahead == 'k') ADVANCE(220); + END_STATE(); + case 107: + if (lookahead == 'k') ADVANCE(51); + END_STATE(); + case 108: + if (lookahead == 'l') ADVANCE(281); + END_STATE(); + case 109: + if (lookahead == 'l') ADVANCE(233); + END_STATE(); + case 110: + if (lookahead == 'l') ADVANCE(247); + END_STATE(); + case 111: + if (lookahead == 'l') ADVANCE(171); + END_STATE(); + case 112: + if (lookahead == 'l') ADVANCE(96); + END_STATE(); + case 113: + if (lookahead == 'l') ADVANCE(72); + END_STATE(); + case 114: + if (lookahead == 'm') ADVANCE(155); + END_STATE(); + case 115: + if (lookahead == 'm') ADVANCE(155); + if (lookahead == 'n') ADVANCE(91); + END_STATE(); + case 116: + if (lookahead == 'm') ADVANCE(235); + END_STATE(); + case 117: + if (lookahead == 'm') ADVANCE(299); + END_STATE(); + case 118: + if (lookahead == 'n') ADVANCE(447); + END_STATE(); + case 119: + if (lookahead == 'n') ADVANCE(60); + END_STATE(); + case 120: + if (lookahead == 'n') ADVANCE(224); + END_STATE(); + case 121: + if (lookahead == 'n') ADVANCE(214); + END_STATE(); + case 122: + if (lookahead == 'n') ADVANCE(222); + END_STATE(); + case 123: + if (lookahead == 'n') ADVANCE(85); + if (lookahead == 'p') ADVANCE(176); + END_STATE(); + case 124: + if (lookahead == 'n') ADVANCE(90); + END_STATE(); + case 125: + if (lookahead == 'n') ADVANCE(93); + END_STATE(); + case 126: + if (lookahead == 'n') ADVANCE(180); + END_STATE(); + case 127: + if (lookahead == 'n') ADVANCE(177); + END_STATE(); + case 128: + if (lookahead == 'n') ADVANCE(167); + END_STATE(); + case 129: + if (lookahead == 'n') ADVANCE(59); + END_STATE(); + case 130: + if (lookahead == 'n') ADVANCE(168); + END_STATE(); + case 131: + if (lookahead == 'n') ADVANCE(182); + END_STATE(); + case 132: + if (lookahead == 'o') ADVANCE(189); + END_STATE(); + case 133: + if (lookahead == 'o') ADVANCE(57); + END_STATE(); + case 134: + if (lookahead == 'o') ADVANCE(295); + if (lookahead == 'r') ADVANCE(188); + END_STATE(); + case 135: + if (lookahead == 'o') ADVANCE(92); + END_STATE(); + case 136: + if (lookahead == 'o') ADVANCE(27); + END_STATE(); + case 137: + if (lookahead == 'o') ADVANCE(108); + END_STATE(); + case 138: + if (lookahead == 'o') ADVANCE(137); + if (lookahead == 'y') ADVANCE(175); + END_STATE(); + case 139: + if (lookahead == 'o') ADVANCE(48); + END_STATE(); + case 140: + if (lookahead == 'o') ADVANCE(160); + END_STATE(); + case 141: + if (lookahead == 'o') ADVANCE(185); + END_STATE(); + case 142: + if (lookahead == 'o') ADVANCE(120); + END_STATE(); + case 143: + if (lookahead == 'o') ADVANCE(121); + END_STATE(); + case 144: + if (lookahead == 'o') ADVANCE(178); + END_STATE(); + case 145: + if (lookahead == 'o') ADVANCE(130); + END_STATE(); + case 146: + if (lookahead == 'o') ADVANCE(122); + END_STATE(); + case 147: + if (lookahead == 'o') ADVANCE(162); + END_STATE(); + case 148: + if (lookahead == 'p') ADVANCE(257); + if (lookahead == 'x') ADVANCE(296); + END_STATE(); + case 149: + if (lookahead == 'p') ADVANCE(253); + END_STATE(); + case 150: + if (lookahead == 'p') ADVANCE(55); + END_STATE(); + case 151: + if (lookahead == 'p') ADVANCE(140); + if (lookahead == 't') ADVANCE(80); + END_STATE(); + case 152: + if (lookahead == 'p') ADVANCE(140); + if (lookahead == 't') ADVANCE(88); + END_STATE(); + case 153: + if (lookahead == 'p') ADVANCE(163); + END_STATE(); + case 154: + if (lookahead == 'p') ADVANCE(86); + if (lookahead == 'q') ADVANCE(187); + if (lookahead == 's') ADVANCE(84); + if (lookahead == 't') ADVANCE(186); + END_STATE(); + case 155: + if (lookahead == 'p') ADVANCE(147); + END_STATE(); + case 156: + if (lookahead == 'p') ADVANCE(183); + END_STATE(); + case 157: + if (lookahead == 'r') ADVANCE(190); + END_STATE(); + case 158: + if (lookahead == 'r') ADVANCE(87); + END_STATE(); + case 159: + if (lookahead == 'r') ADVANCE(191); + END_STATE(); + case 160: + if (lookahead == 'r') ADVANCE(173); + END_STATE(); + case 161: + if (lookahead == 'r') ADVANCE(128); + END_STATE(); + case 162: + if (lookahead == 'r') ADVANCE(174); + END_STATE(); + case 163: + if (lookahead == 'r') ADVANCE(144); + END_STATE(); + case 164: + if (lookahead == 'r') ADVANCE(81); + END_STATE(); + case 165: + if (lookahead == 'r') ADVANCE(141); + END_STATE(); + case 166: + if (lookahead == 's') ADVANCE(289); + END_STATE(); + case 167: + if (lookahead == 's') ADVANCE(301); + END_STATE(); + case 168: + if (lookahead == 's') ADVANCE(293); + END_STATE(); + case 169: + if (lookahead == 's') ADVANCE(170); + END_STATE(); + case 170: + if (lookahead == 's') ADVANCE(42); + END_STATE(); + case 171: + if (lookahead == 's') ADVANCE(71); + END_STATE(); + case 172: + if (lookahead == 't') ADVANCE(287); + END_STATE(); + case 173: + if (lookahead == 't') ADVANCE(231); + END_STATE(); + case 174: + if (lookahead == 't') ADVANCE(219); + END_STATE(); + case 175: + if (lookahead == 't') ADVANCE(82); + END_STATE(); + case 176: + if (lookahead == 't') ADVANCE(98); + END_STATE(); + case 177: + if (lookahead == 't') ADVANCE(43); + END_STATE(); + case 178: + if (lookahead == 't') ADVANCE(136); + END_STATE(); + case 179: + if (lookahead == 't') ADVANCE(79); + END_STATE(); + case 180: + if (lookahead == 't') ADVANCE(29); + END_STATE(); + case 181: + if (lookahead == 't') ADVANCE(100); + END_STATE(); + case 182: + if (lookahead == 't') ADVANCE(30); + END_STATE(); + case 183: + if (lookahead == 't') ADVANCE(102); + END_STATE(); + case 184: + if (lookahead == 'u') ADVANCE(116); + END_STATE(); + case 185: + if (lookahead == 'u') ADVANCE(149); + END_STATE(); + case 186: + if (lookahead == 'u') ADVANCE(161); + END_STATE(); + case 187: + if (lookahead == 'u') ADVANCE(99); + END_STATE(); + case 188: + if (lookahead == 'u') ADVANCE(70); + END_STATE(); + case 189: + if (lookahead == 'u') ADVANCE(53); + END_STATE(); + case 190: + if (lookahead == 'v') ADVANCE(97); + END_STATE(); + case 191: + if (lookahead == 'v') ADVANCE(83); + END_STATE(); + case 192: + if (lookahead == 'x') ADVANCE(296); + END_STATE(); + case 193: + if (lookahead == 'x') ADVANCE(216); + END_STATE(); + case 194: + if (lookahead == 'x') ADVANCE(69); + END_STATE(); + case 195: + if (lookahead == 'x') ADVANCE(89); + END_STATE(); + case 196: + if (lookahead == '+' || + lookahead == '-') ADVANCE(198); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(449); + END_STATE(); + case 197: + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(448); + END_STATE(); + case 198: + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(449); + END_STATE(); + case 199: + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(464); + END_STATE(); + case 200: + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(446); + END_STATE(); + case 201: + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(199); + END_STATE(); + case 202: + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(201); + END_STATE(); + case 203: + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(202); + END_STATE(); + case 204: + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(203); + END_STATE(); + case 205: + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(204); + END_STATE(); + case 206: + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(205); + END_STATE(); + case 207: + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(206); + END_STATE(); + case 208: + if (('A' <= lookahead && lookahead <= 'Z') || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(7); + END_STATE(); + case 209: + if (('A' <= lookahead && lookahead <= 'Z') || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(10); + END_STATE(); + case 210: + if (eof) ADVANCE(212); + ADVANCE_MAP( + '"', 450, + '\'', 457, + '(', 227, + ')', 228, + '+', 302, + ',', 241, + '-', 239, + '.', 230, + '/', 11, + '0', 442, + ':', 303, + ';', 213, + '<', 259, + '=', 215, + '>', 260, + '[', 240, + ']', 242, + 'b', 138, + 'd', 132, + 'e', 64, + 'f', 39, + 'g', 165, + 'i', 115, + 'l', 133, + 'm', 40, + 'n', 41, + 'o', 123, + 'p', 44, + 'r', 68, + 's', 76, + 't', 134, + 'u', 105, + 'w', 78, + '{', 237, + '}', 238, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(210); + if (('1' <= lookahead && lookahead <= '9')) ADVANCE(440); + END_STATE(); + case 211: + if (eof) ADVANCE(212); + ADVANCE_MAP( + '"', 450, + '\'', 457, + '-', 239, + '.', 229, + '/', 11, + '0', 444, + ';', 213, + '=', 215, + 'e', 65, + 'i', 114, + 'l', 133, + 'm', 47, + 'o', 156, + 'p', 44, + 'r', 150, + 's', 77, + 'w', 78, + '}', 238, + ); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') SKIP(211); + if (('1' <= lookahead && lookahead <= '9')) ADVANCE(441); + END_STATE(); + case 212: + ACCEPT_TOKEN(ts_builtin_sym_end); + END_STATE(); + case 213: + ACCEPT_TOKEN(anon_sym_SEMI); + END_STATE(); + case 214: + ACCEPT_TOKEN(anon_sym_edition); + END_STATE(); + case 215: + ACCEPT_TOKEN(anon_sym_EQ); + END_STATE(); + case 216: + ACCEPT_TOKEN(anon_sym_syntax); + END_STATE(); + case 217: + ACCEPT_TOKEN(anon_sym_DQUOTEproto3_DQUOTE); + END_STATE(); + case 218: + ACCEPT_TOKEN(anon_sym_DQUOTEproto2_DQUOTE); + END_STATE(); + case 219: + ACCEPT_TOKEN(anon_sym_import); + END_STATE(); + case 220: + ACCEPT_TOKEN(anon_sym_weak); + END_STATE(); + case 221: + ACCEPT_TOKEN(anon_sym_public); + END_STATE(); + case 222: + ACCEPT_TOKEN(anon_sym_option); + END_STATE(); + case 223: + ACCEPT_TOKEN(anon_sym_option); + if (lookahead == 'a') ADVANCE(368); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('b' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 224: + ACCEPT_TOKEN(anon_sym_option); + if (lookahead == 'a') ADVANCE(110); + END_STATE(); + case 225: + ACCEPT_TOKEN(anon_sym_option); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 226: + ACCEPT_TOKEN(anon_sym_package); + END_STATE(); + case 227: + ACCEPT_TOKEN(anon_sym_LPAREN); + END_STATE(); + case 228: + ACCEPT_TOKEN(anon_sym_RPAREN); + END_STATE(); + case 229: + ACCEPT_TOKEN(anon_sym_DOT); + END_STATE(); + case 230: + ACCEPT_TOKEN(anon_sym_DOT); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(448); + END_STATE(); + case 231: + ACCEPT_TOKEN(anon_sym_export); + END_STATE(); + case 232: + ACCEPT_TOKEN(anon_sym_export); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 233: + ACCEPT_TOKEN(anon_sym_local); + END_STATE(); + case 234: + ACCEPT_TOKEN(anon_sym_local); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 235: + ACCEPT_TOKEN(anon_sym_enum); + END_STATE(); + case 236: + ACCEPT_TOKEN(anon_sym_enum); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 237: + ACCEPT_TOKEN(anon_sym_LBRACE); + END_STATE(); + case 238: + ACCEPT_TOKEN(anon_sym_RBRACE); + END_STATE(); + case 239: + ACCEPT_TOKEN(anon_sym_DASH); + END_STATE(); + case 240: + ACCEPT_TOKEN(anon_sym_LBRACK); + END_STATE(); + case 241: + ACCEPT_TOKEN(anon_sym_COMMA); + END_STATE(); + case 242: + ACCEPT_TOKEN(anon_sym_RBRACK); + END_STATE(); + case 243: + ACCEPT_TOKEN(anon_sym_message); + END_STATE(); + case 244: + ACCEPT_TOKEN(anon_sym_message); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 245: + ACCEPT_TOKEN(anon_sym_extend); + END_STATE(); + case 246: + ACCEPT_TOKEN(anon_sym_extend); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 247: + ACCEPT_TOKEN(anon_sym_optional); + END_STATE(); + case 248: + ACCEPT_TOKEN(anon_sym_optional); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 249: + ACCEPT_TOKEN(anon_sym_required); + END_STATE(); + case 250: + ACCEPT_TOKEN(anon_sym_required); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 251: + ACCEPT_TOKEN(anon_sym_repeated); + END_STATE(); + case 252: + ACCEPT_TOKEN(anon_sym_repeated); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 253: + ACCEPT_TOKEN(anon_sym_group); + END_STATE(); + case 254: + ACCEPT_TOKEN(anon_sym_group); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 255: + ACCEPT_TOKEN(anon_sym_oneof); + END_STATE(); + case 256: + ACCEPT_TOKEN(anon_sym_oneof); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 257: + ACCEPT_TOKEN(anon_sym_map); + END_STATE(); + case 258: + ACCEPT_TOKEN(anon_sym_map); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 259: + ACCEPT_TOKEN(anon_sym_LT); + END_STATE(); + case 260: + ACCEPT_TOKEN(anon_sym_GT); + END_STATE(); + case 261: + ACCEPT_TOKEN(anon_sym_int32); + END_STATE(); + case 262: + ACCEPT_TOKEN(anon_sym_int32); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 263: + ACCEPT_TOKEN(anon_sym_int64); + END_STATE(); + case 264: + ACCEPT_TOKEN(anon_sym_int64); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 265: + ACCEPT_TOKEN(anon_sym_uint32); + END_STATE(); + case 266: + ACCEPT_TOKEN(anon_sym_uint32); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 267: + ACCEPT_TOKEN(anon_sym_uint64); + END_STATE(); + case 268: + ACCEPT_TOKEN(anon_sym_uint64); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 269: + ACCEPT_TOKEN(anon_sym_sint32); + END_STATE(); + case 270: + ACCEPT_TOKEN(anon_sym_sint32); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 271: + ACCEPT_TOKEN(anon_sym_sint64); + END_STATE(); + case 272: + ACCEPT_TOKEN(anon_sym_sint64); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 273: + ACCEPT_TOKEN(anon_sym_fixed32); + END_STATE(); + case 274: + ACCEPT_TOKEN(anon_sym_fixed32); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 275: + ACCEPT_TOKEN(anon_sym_fixed64); + END_STATE(); + case 276: + ACCEPT_TOKEN(anon_sym_fixed64); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 277: + ACCEPT_TOKEN(anon_sym_sfixed32); + END_STATE(); + case 278: + ACCEPT_TOKEN(anon_sym_sfixed32); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 279: + ACCEPT_TOKEN(anon_sym_sfixed64); + END_STATE(); + case 280: + ACCEPT_TOKEN(anon_sym_sfixed64); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 281: + ACCEPT_TOKEN(anon_sym_bool); + END_STATE(); + case 282: + ACCEPT_TOKEN(anon_sym_bool); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 283: + ACCEPT_TOKEN(anon_sym_string); + END_STATE(); + case 284: + ACCEPT_TOKEN(anon_sym_string); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 285: + ACCEPT_TOKEN(anon_sym_double); + END_STATE(); + case 286: + ACCEPT_TOKEN(anon_sym_double); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 287: + ACCEPT_TOKEN(anon_sym_float); + END_STATE(); + case 288: + ACCEPT_TOKEN(anon_sym_float); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 289: + ACCEPT_TOKEN(anon_sym_bytes); + END_STATE(); + case 290: + ACCEPT_TOKEN(anon_sym_bytes); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 291: + ACCEPT_TOKEN(anon_sym_reserved); + END_STATE(); + case 292: + ACCEPT_TOKEN(anon_sym_reserved); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 293: + ACCEPT_TOKEN(anon_sym_extensions); + END_STATE(); + case 294: + ACCEPT_TOKEN(anon_sym_extensions); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 295: + ACCEPT_TOKEN(anon_sym_to); + END_STATE(); + case 296: + ACCEPT_TOKEN(anon_sym_max); + END_STATE(); + case 297: + ACCEPT_TOKEN(anon_sym_service); + END_STATE(); + case 298: + ACCEPT_TOKEN(anon_sym_rpc); + END_STATE(); + case 299: + ACCEPT_TOKEN(anon_sym_stream); + END_STATE(); + case 300: + ACCEPT_TOKEN(anon_sym_stream); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 301: + ACCEPT_TOKEN(anon_sym_returns); + END_STATE(); + case 302: + ACCEPT_TOKEN(anon_sym_PLUS); + END_STATE(); + case 303: + ACCEPT_TOKEN(anon_sym_COLON); + END_STATE(); + case 304: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '2') ADVANCE(262); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 305: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '2') ADVANCE(270); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 306: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '2') ADVANCE(266); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 307: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '2') ADVANCE(274); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 308: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '2') ADVANCE(278); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 309: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '3') ADVANCE(304); + if (lookahead == '6') ADVANCE(314); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 310: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '3') ADVANCE(305); + if (lookahead == '6') ADVANCE(315); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 311: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '3') ADVANCE(306); + if (lookahead == '6') ADVANCE(316); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 312: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '3') ADVANCE(307); + if (lookahead == '6') ADVANCE(317); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 313: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '3') ADVANCE(308); + if (lookahead == '6') ADVANCE(318); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 314: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '4') ADVANCE(264); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 315: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '4') ADVANCE(272); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 316: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '4') ADVANCE(268); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 317: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '4') ADVANCE(276); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 318: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == '4') ADVANCE(280); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 319: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'a') ADVANCE(396); + if (lookahead == 'e') ADVANCE(411); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('b' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 320: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'a') ADVANCE(357); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('b' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 321: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'a') ADVANCE(372); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('b' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 322: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'a') ADVANCE(367); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('b' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 323: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'a') ADVANCE(416); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('b' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 324: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'a') ADVANCE(369); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('b' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 325: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'a') ADVANCE(373); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('b' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 326: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'a') ADVANCE(420); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('b' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 327: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'b') ADVANCE(370); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 328: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'c') ADVANCE(322); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 329: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'd') ADVANCE(246); + if (lookahead == 's') ADVANCE(362); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 330: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'd') ADVANCE(252); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 331: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'd') ADVANCE(250); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 332: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'd') ADVANCE(292); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 333: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'd') ADVANCE(312); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 334: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'd') ADVANCE(313); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 335: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(399); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 336: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(333); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 337: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(286); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 338: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(244); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 339: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(437); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 340: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(439); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 341: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(374); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 342: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(330); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 343: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(409); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 344: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(331); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 345: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(402); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 346: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(400); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 347: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(326); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 348: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(332); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 349: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(389); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 350: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(321); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 351: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(413); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 352: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'e') ADVANCE(334); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 353: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'f') ADVANCE(433); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 354: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'f') ADVANCE(256); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 355: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'f') ADVANCE(365); + if (lookahead == 'i') ADVANCE(383); + if (lookahead == 't') ADVANCE(403); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 356: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'g') ADVANCE(284); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 357: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'g') ADVANCE(338); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 358: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'i') ADVANCE(431); + if (lookahead == 'l') ADVANCE(388); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 359: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'i') ADVANCE(381); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 360: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'i') ADVANCE(406); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 361: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'i') ADVANCE(393); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 362: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'i') ADVANCE(394); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 363: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'i') ADVANCE(395); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 364: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'i') ADVANCE(384); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 365: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'i') ADVANCE(432); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 366: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'l') ADVANCE(282); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 367: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'l') ADVANCE(234); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 368: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'l') ADVANCE(248); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 369: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'l') ADVANCE(414); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 370: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'l') ADVANCE(337); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 371: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'm') ADVANCE(236); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 372: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'm') ADVANCE(300); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 373: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(433); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 374: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(329); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 375: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(223); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 376: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(225); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 377: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(349); + if (lookahead == 'p') ADVANCE(419); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 378: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(415); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 379: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(426); + if (lookahead == 'x') ADVANCE(398); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 380: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(353); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 381: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(356); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 382: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(410); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 383: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(421); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 384: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'n') ADVANCE(423); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 385: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(425); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 386: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(328); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 387: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(366); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 388: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(323); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 389: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(354); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 390: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(387); + if (lookahead == 'y') ADVANCE(418); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 391: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(404); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 392: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(427); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 393: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(375); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 394: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(382); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 395: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'o') ADVANCE(376); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 396: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'p') ADVANCE(258); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 397: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'p') ADVANCE(254); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 398: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'p') ADVANCE(391); + if (lookahead == 't') ADVANCE(341); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 399: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'p') ADVANCE(347); + if (lookahead == 'q') ADVANCE(428); + if (lookahead == 's') ADVANCE(345); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 400: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'p') ADVANCE(347); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 401: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'p') ADVANCE(424); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 402: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'r') ADVANCE(430); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 403: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'r') ADVANCE(359); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 404: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'r') ADVANCE(417); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 405: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'r') ADVANCE(429); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 406: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'r') ADVANCE(344); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 407: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'r') ADVANCE(350); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 408: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'r') ADVANCE(392); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 409: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 's') ADVANCE(290); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 410: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 's') ADVANCE(294); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 411: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 's') ADVANCE(412); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 412: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 's') ADVANCE(320); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 413: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 's') ADVANCE(345); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 414: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 's') ADVANCE(340); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 415: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(309); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 416: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(288); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 417: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(232); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 418: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(343); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 419: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(361); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 420: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(342); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 421: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(310); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 422: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(407); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 423: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(311); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 424: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 't') ADVANCE(363); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 425: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'u') ADVANCE(327); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 426: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'u') ADVANCE(371); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 427: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'u') ADVANCE(397); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 428: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'u') ADVANCE(360); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 429: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'u') ADVANCE(339); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 430: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'v') ADVANCE(348); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 431: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'x') ADVANCE(336); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 432: + ACCEPT_TOKEN(sym_identifier); + if (lookahead == 'x') ADVANCE(352); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 433: + ACCEPT_TOKEN(sym_identifier); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 434: + ACCEPT_TOKEN(sym_reserved_identifier); + END_STATE(); + case 435: + ACCEPT_TOKEN(sym_reserved_identifier); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(435); + END_STATE(); + case 436: + ACCEPT_TOKEN(sym_true); + END_STATE(); + case 437: + ACCEPT_TOKEN(sym_true); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 438: + ACCEPT_TOKEN(sym_false); + END_STATE(); + case 439: + ACCEPT_TOKEN(sym_false); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'Z') || + lookahead == '_' || + ('a' <= lookahead && lookahead <= 'z')) ADVANCE(433); + END_STATE(); + case 440: + ACCEPT_TOKEN(sym_decimal_lit); + if (lookahead == '.') ADVANCE(448); + if (lookahead == 'E' || + lookahead == 'e') ADVANCE(196); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(440); + END_STATE(); + case 441: + ACCEPT_TOKEN(sym_decimal_lit); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(441); + END_STATE(); + case 442: + ACCEPT_TOKEN(sym_octal_lit); + if (lookahead == '.') ADVANCE(448); + if (lookahead == 'E' || + lookahead == 'e') ADVANCE(196); + if (lookahead == 'X' || + lookahead == 'x') ADVANCE(200); + if (lookahead == '8' || + lookahead == '9') ADVANCE(14); + if (('0' <= lookahead && lookahead <= '7')) ADVANCE(443); + END_STATE(); + case 443: + ACCEPT_TOKEN(sym_octal_lit); + if (lookahead == '.') ADVANCE(448); + if (lookahead == 'E' || + lookahead == 'e') ADVANCE(196); + if (lookahead == '8' || + lookahead == '9') ADVANCE(14); + if (('0' <= lookahead && lookahead <= '7')) ADVANCE(443); + END_STATE(); + case 444: + ACCEPT_TOKEN(sym_octal_lit); + if (lookahead == 'X' || + lookahead == 'x') ADVANCE(200); + if (('0' <= lookahead && lookahead <= '7')) ADVANCE(445); + END_STATE(); + case 445: + ACCEPT_TOKEN(sym_octal_lit); + if (('0' <= lookahead && lookahead <= '7')) ADVANCE(445); + END_STATE(); + case 446: + ACCEPT_TOKEN(sym_hex_lit); + if (('0' <= lookahead && lookahead <= '9') || + ('A' <= lookahead && lookahead <= 'F') || + ('a' <= lookahead && lookahead <= 'f')) ADVANCE(446); + END_STATE(); + case 447: + ACCEPT_TOKEN(sym_float_lit); + END_STATE(); + case 448: + ACCEPT_TOKEN(sym_float_lit); + if (lookahead == 'E' || + lookahead == 'e') ADVANCE(196); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(448); + END_STATE(); + case 449: + ACCEPT_TOKEN(sym_float_lit); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(449); + END_STATE(); + case 450: + ACCEPT_TOKEN(anon_sym_DQUOTE); + END_STATE(); + case 451: + ACCEPT_TOKEN(aux_sym_string_token1); + if (lookahead == '\n') ADVANCE(456); + if (lookahead != 0 && + lookahead != '"' && + lookahead != '\\') ADVANCE(451); + END_STATE(); + case 452: + ACCEPT_TOKEN(aux_sym_string_token1); + if (lookahead == '*') ADVANCE(454); + if (lookahead == '/') ADVANCE(451); + if (lookahead != 0 && + lookahead != '"' && + lookahead != '\\') ADVANCE(456); + END_STATE(); + case 453: + ACCEPT_TOKEN(aux_sym_string_token1); + if (lookahead == '*') ADVANCE(453); + if (lookahead == '/') ADVANCE(456); + if (lookahead != 0 && + lookahead != '"' && + lookahead != '\\') ADVANCE(454); + END_STATE(); + case 454: + ACCEPT_TOKEN(aux_sym_string_token1); + if (lookahead == '*') ADVANCE(453); + if (lookahead != 0 && + lookahead != '"' && + lookahead != '\\') ADVANCE(454); + END_STATE(); + case 455: + ACCEPT_TOKEN(aux_sym_string_token1); + if (lookahead == '/') ADVANCE(452); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') ADVANCE(455); + if (lookahead != 0 && + lookahead != '"' && + lookahead != '\\') ADVANCE(456); + END_STATE(); + case 456: + ACCEPT_TOKEN(aux_sym_string_token1); + if (lookahead != 0 && + lookahead != '"' && + lookahead != '\\') ADVANCE(456); + END_STATE(); + case 457: + ACCEPT_TOKEN(anon_sym_SQUOTE); + END_STATE(); + case 458: + ACCEPT_TOKEN(aux_sym_string_token2); + if (lookahead == '\n') ADVANCE(463); + if (lookahead != 0 && + lookahead != '\'' && + lookahead != '\\') ADVANCE(458); + END_STATE(); + case 459: + ACCEPT_TOKEN(aux_sym_string_token2); + if (lookahead == '*') ADVANCE(461); + if (lookahead == '/') ADVANCE(458); + if (lookahead != 0 && + lookahead != '\'' && + lookahead != '\\') ADVANCE(463); + END_STATE(); + case 460: + ACCEPT_TOKEN(aux_sym_string_token2); + if (lookahead == '*') ADVANCE(460); + if (lookahead == '/') ADVANCE(463); + if (lookahead != 0 && + lookahead != '\'' && + lookahead != '\\') ADVANCE(461); + END_STATE(); + case 461: + ACCEPT_TOKEN(aux_sym_string_token2); + if (lookahead == '*') ADVANCE(460); + if (lookahead != 0 && + lookahead != '\'' && + lookahead != '\\') ADVANCE(461); + END_STATE(); + case 462: + ACCEPT_TOKEN(aux_sym_string_token2); + if (lookahead == '/') ADVANCE(459); + if (('\t' <= lookahead && lookahead <= '\r') || + lookahead == ' ') ADVANCE(462); + if (lookahead != 0 && + lookahead != '\'' && + lookahead != '\\') ADVANCE(463); + END_STATE(); + case 463: + ACCEPT_TOKEN(aux_sym_string_token2); + if (lookahead != 0 && + lookahead != '\'' && + lookahead != '\\') ADVANCE(463); + END_STATE(); + case 464: + ACCEPT_TOKEN(sym_escape_sequence); + END_STATE(); + case 465: + ACCEPT_TOKEN(sym_escape_sequence); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(464); + END_STATE(); + case 466: + ACCEPT_TOKEN(sym_escape_sequence); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(465); + END_STATE(); + case 467: + ACCEPT_TOKEN(sym_comment); + END_STATE(); + case 468: + ACCEPT_TOKEN(sym_comment); + if (lookahead != 0 && + lookahead != '\n') ADVANCE(468); + END_STATE(); + default: + return false; + } +} + +static const TSLexMode ts_lex_modes[STATE_COUNT] = { + [0] = {.lex_state = 0}, + [1] = {.lex_state = 211}, + [2] = {.lex_state = 8}, + [3] = {.lex_state = 8}, + [4] = {.lex_state = 8}, + [5] = {.lex_state = 8}, + [6] = {.lex_state = 8}, + [7] = {.lex_state = 8}, + [8] = {.lex_state = 8}, + [9] = {.lex_state = 8}, + [10] = {.lex_state = 8}, + [11] = {.lex_state = 8}, + [12] = {.lex_state = 8}, + [13] = {.lex_state = 8}, + [14] = {.lex_state = 8}, + [15] = {.lex_state = 8}, + [16] = {.lex_state = 8}, + [17] = {.lex_state = 8}, + [18] = {.lex_state = 8}, + [19] = {.lex_state = 8}, + [20] = {.lex_state = 8}, + [21] = {.lex_state = 8}, + [22] = {.lex_state = 8}, + [23] = {.lex_state = 8}, + [24] = {.lex_state = 8}, + [25] = {.lex_state = 8}, + [26] = {.lex_state = 8}, + [27] = {.lex_state = 8}, + [28] = {.lex_state = 8}, + [29] = {.lex_state = 8}, + [30] = {.lex_state = 8}, + [31] = {.lex_state = 8}, + [32] = {.lex_state = 15}, + [33] = {.lex_state = 15}, + [34] = {.lex_state = 15}, + [35] = {.lex_state = 16}, + [36] = {.lex_state = 15}, + [37] = {.lex_state = 15}, + [38] = {.lex_state = 17}, + [39] = {.lex_state = 15}, + [40] = {.lex_state = 2}, + [41] = {.lex_state = 2}, + [42] = {.lex_state = 211}, + [43] = {.lex_state = 18}, + [44] = {.lex_state = 211}, + [45] = {.lex_state = 211}, + [46] = {.lex_state = 2}, + [47] = {.lex_state = 2}, + [48] = {.lex_state = 2}, + [49] = {.lex_state = 18}, + [50] = {.lex_state = 2}, + [51] = {.lex_state = 2}, + [52] = {.lex_state = 15}, + [53] = {.lex_state = 211}, + [54] = {.lex_state = 2}, + [55] = {.lex_state = 15}, + [56] = {.lex_state = 15}, + [57] = {.lex_state = 2}, + [58] = {.lex_state = 2}, + [59] = {.lex_state = 2}, + [60] = {.lex_state = 2}, + [61] = {.lex_state = 2}, + [62] = {.lex_state = 2}, + [63] = {.lex_state = 2}, + [64] = {.lex_state = 211}, + [65] = {.lex_state = 0}, + [66] = {.lex_state = 211}, + [67] = {.lex_state = 211}, + [68] = {.lex_state = 211}, + [69] = {.lex_state = 211}, + [70] = {.lex_state = 211}, + [71] = {.lex_state = 211}, + [72] = {.lex_state = 211}, + [73] = {.lex_state = 211}, + [74] = {.lex_state = 211}, + [75] = {.lex_state = 211}, + [76] = {.lex_state = 211}, + [77] = {.lex_state = 211}, + [78] = {.lex_state = 1}, + [79] = {.lex_state = 211}, + [80] = {.lex_state = 211}, + [81] = {.lex_state = 211}, + [82] = {.lex_state = 211}, + [83] = {.lex_state = 211}, + [84] = {.lex_state = 1}, + [85] = {.lex_state = 21}, + [86] = {.lex_state = 21}, + [87] = {.lex_state = 21}, + [88] = {.lex_state = 21}, + [89] = {.lex_state = 1}, + [90] = {.lex_state = 1}, + [91] = {.lex_state = 21}, + [92] = {.lex_state = 1}, + [93] = {.lex_state = 1}, + [94] = {.lex_state = 1}, + [95] = {.lex_state = 6}, + [96] = {.lex_state = 211}, + [97] = {.lex_state = 1}, + [98] = {.lex_state = 6}, + [99] = {.lex_state = 211}, + [100] = {.lex_state = 211}, + [101] = {.lex_state = 211}, + [102] = {.lex_state = 211}, + [103] = {.lex_state = 211}, + [104] = {.lex_state = 211}, + [105] = {.lex_state = 211}, + [106] = {.lex_state = 211}, + [107] = {.lex_state = 211}, + [108] = {.lex_state = 211}, + [109] = {.lex_state = 0}, + [110] = {.lex_state = 1}, + [111] = {.lex_state = 1}, + [112] = {.lex_state = 1}, + [113] = {.lex_state = 211}, + [114] = {.lex_state = 1}, + [115] = {.lex_state = 1}, + [116] = {.lex_state = 20}, + [117] = {.lex_state = 1}, + [118] = {.lex_state = 1}, + [119] = {.lex_state = 211}, + [120] = {.lex_state = 19}, + [121] = {.lex_state = 1}, + [122] = {.lex_state = 1}, + [123] = {.lex_state = 21}, + [124] = {.lex_state = 21}, + [125] = {.lex_state = 1}, + [126] = {.lex_state = 211}, + [127] = {.lex_state = 1}, + [128] = {.lex_state = 1}, + [129] = {.lex_state = 211}, + [130] = {.lex_state = 211}, + [131] = {.lex_state = 21}, + [132] = {.lex_state = 211}, + [133] = {.lex_state = 21}, + [134] = {.lex_state = 1}, + [135] = {.lex_state = 211}, + [136] = {.lex_state = 21}, + [137] = {.lex_state = 1}, + [138] = {.lex_state = 21}, + [139] = {.lex_state = 1}, + [140] = {.lex_state = 211}, + [141] = {.lex_state = 1}, + [142] = {.lex_state = 1}, + [143] = {.lex_state = 211}, + [144] = {.lex_state = 20}, + [145] = {.lex_state = 211}, + [146] = {.lex_state = 21}, + [147] = {.lex_state = 1}, + [148] = {.lex_state = 21}, + [149] = {.lex_state = 211}, + [150] = {.lex_state = 19}, + [151] = {.lex_state = 19}, + [152] = {.lex_state = 9}, + [153] = {.lex_state = 1}, + [154] = {.lex_state = 1}, + [155] = {.lex_state = 1}, + [156] = {.lex_state = 1}, + [157] = {.lex_state = 1}, + [158] = {.lex_state = 1}, + [159] = {.lex_state = 3}, + [160] = {.lex_state = 9}, + [161] = {.lex_state = 1}, + [162] = {.lex_state = 1}, + [163] = {.lex_state = 1}, + [164] = {.lex_state = 1}, + [165] = {.lex_state = 1}, + [166] = {.lex_state = 0}, + [167] = {.lex_state = 211}, + [168] = {.lex_state = 211}, + [169] = {.lex_state = 211}, + [170] = {.lex_state = 211}, + [171] = {.lex_state = 211}, + [172] = {.lex_state = 0}, + [173] = {.lex_state = 3}, + [174] = {.lex_state = 1}, + [175] = {.lex_state = 211}, + [176] = {.lex_state = 1}, + [177] = {.lex_state = 3}, + [178] = {.lex_state = 9}, + [179] = {.lex_state = 0}, + [180] = {.lex_state = 0}, + [181] = {.lex_state = 0}, + [182] = {.lex_state = 0}, + [183] = {.lex_state = 0}, + [184] = {.lex_state = 0}, + [185] = {.lex_state = 1}, + [186] = {.lex_state = 0}, + [187] = {.lex_state = 0}, + [188] = {.lex_state = 1}, + [189] = {.lex_state = 0}, + [190] = {.lex_state = 211}, + [191] = {.lex_state = 0}, + [192] = {.lex_state = 0}, + [193] = {.lex_state = 1}, + [194] = {.lex_state = 0}, + [195] = {.lex_state = 0}, + [196] = {.lex_state = 1}, + [197] = {.lex_state = 0}, + [198] = {.lex_state = 1}, + [199] = {.lex_state = 0}, + [200] = {.lex_state = 0}, + [201] = {.lex_state = 0}, + [202] = {.lex_state = 0}, + [203] = {.lex_state = 0}, + [204] = {.lex_state = 1}, + [205] = {.lex_state = 1}, + [206] = {.lex_state = 1}, + [207] = {.lex_state = 0}, + [208] = {.lex_state = 211}, + [209] = {.lex_state = 0}, + [210] = {.lex_state = 0}, + [211] = {.lex_state = 1}, + [212] = {.lex_state = 211}, + [213] = {.lex_state = 0}, + [214] = {.lex_state = 211}, + [215] = {.lex_state = 1}, + [216] = {.lex_state = 1}, + [217] = {.lex_state = 0}, + [218] = {.lex_state = 0}, + [219] = {.lex_state = 0}, + [220] = {.lex_state = 0}, + [221] = {.lex_state = 1}, + [222] = {.lex_state = 0}, + [223] = {.lex_state = 0}, + [224] = {.lex_state = 1}, + [225] = {.lex_state = 1}, + [226] = {.lex_state = 1}, + [227] = {.lex_state = 1}, + [228] = {.lex_state = 1}, + [229] = {.lex_state = 0}, + [230] = {.lex_state = 0}, + [231] = {.lex_state = 0}, + [232] = {.lex_state = 0}, + [233] = {.lex_state = 0}, + [234] = {.lex_state = 0}, + [235] = {.lex_state = 0}, + [236] = {.lex_state = 1}, + [237] = {.lex_state = 1}, + [238] = {.lex_state = 0}, + [239] = {.lex_state = 1}, + [240] = {.lex_state = 0}, + [241] = {.lex_state = 1}, + [242] = {.lex_state = 1}, + [243] = {.lex_state = 0}, + [244] = {.lex_state = 0}, + [245] = {.lex_state = 8}, + [246] = {.lex_state = 1}, + [247] = {.lex_state = 0}, + [248] = {.lex_state = 1}, + [249] = {.lex_state = 1}, + [250] = {.lex_state = 1}, + [251] = {.lex_state = 0}, + [252] = {.lex_state = 0}, + [253] = {.lex_state = 0}, + [254] = {.lex_state = 0}, + [255] = {.lex_state = 1}, + [256] = {.lex_state = 0}, + [257] = {.lex_state = 0}, + [258] = {.lex_state = 0}, + [259] = {.lex_state = 0}, + [260] = {.lex_state = 0}, + [261] = {.lex_state = 0}, + [262] = {.lex_state = 0}, + [263] = {.lex_state = 1}, + [264] = {.lex_state = 1}, + [265] = {.lex_state = 1}, + [266] = {.lex_state = 1}, + [267] = {.lex_state = 1}, + [268] = {.lex_state = 0}, + [269] = {.lex_state = 0}, + [270] = {.lex_state = 0}, + [271] = {.lex_state = 1}, + [272] = {.lex_state = 0}, + [273] = {.lex_state = 0}, + [274] = {.lex_state = 0}, + [275] = {.lex_state = 0}, + [276] = {.lex_state = 1}, + [277] = {.lex_state = 0}, + [278] = {.lex_state = 0}, + [279] = {.lex_state = 0}, + [280] = {.lex_state = 0}, + [281] = {.lex_state = 0}, + [282] = {.lex_state = 211}, + [283] = {.lex_state = 0}, + [284] = {.lex_state = 0}, + [285] = {.lex_state = 0}, + [286] = {.lex_state = 0}, + [287] = {.lex_state = 0}, + [288] = {.lex_state = 0}, + [289] = {.lex_state = 1}, + [290] = {.lex_state = 0}, + [291] = {.lex_state = 0}, + [292] = {.lex_state = 1}, + [293] = {.lex_state = 0}, + [294] = {.lex_state = 0}, + [295] = {.lex_state = 0}, + [296] = {.lex_state = 0}, + [297] = {.lex_state = 0}, + [298] = {.lex_state = 0}, + [299] = {.lex_state = 1}, + [300] = {.lex_state = 0}, + [301] = {.lex_state = 0}, + [302] = {.lex_state = 0}, + [303] = {.lex_state = 0}, + [304] = {.lex_state = 0}, + [305] = {.lex_state = 0}, + [306] = {.lex_state = 6}, + [307] = {.lex_state = 0}, + [308] = {.lex_state = 0}, + [309] = {.lex_state = 0}, + [310] = {.lex_state = 0}, + [311] = {.lex_state = 0}, + [312] = {.lex_state = 0}, + [313] = {.lex_state = 0}, + [314] = {.lex_state = 0}, + [315] = {.lex_state = 1}, + [316] = {.lex_state = 0}, + [317] = {.lex_state = 0}, + [318] = {.lex_state = 0}, + [319] = {.lex_state = 0}, + [320] = {.lex_state = 1}, + [321] = {.lex_state = 0}, + [322] = {.lex_state = 0}, + [323] = {.lex_state = 0}, + [324] = {.lex_state = 0}, + [325] = {.lex_state = 0}, + [326] = {.lex_state = 0}, + [327] = {.lex_state = 0}, + [328] = {.lex_state = 0}, + [329] = {.lex_state = 0}, + [330] = {.lex_state = 0}, + [331] = {.lex_state = 0}, + [332] = {.lex_state = 0}, + [333] = {.lex_state = 0}, + [334] = {.lex_state = 0}, + [335] = {.lex_state = 0}, + [336] = {.lex_state = 1}, + [337] = {.lex_state = 0}, + [338] = {.lex_state = 0}, + [339] = {.lex_state = 0}, + [340] = {.lex_state = 0}, + [341] = {.lex_state = 0}, + [342] = {.lex_state = 1}, + [343] = {.lex_state = 0}, + [344] = {.lex_state = 0}, +}; + +static const uint16_t ts_parse_table[LARGE_STATE_COUNT][SYMBOL_COUNT] = { + [0] = { + [ts_builtin_sym_end] = ACTIONS(1), + [anon_sym_SEMI] = ACTIONS(1), + [anon_sym_edition] = ACTIONS(1), + [anon_sym_EQ] = ACTIONS(1), + [anon_sym_syntax] = ACTIONS(1), + [anon_sym_import] = ACTIONS(1), + [anon_sym_weak] = ACTIONS(1), + [anon_sym_public] = ACTIONS(1), + [anon_sym_option] = ACTIONS(1), + [anon_sym_package] = ACTIONS(1), + [anon_sym_LPAREN] = ACTIONS(1), + [anon_sym_RPAREN] = ACTIONS(1), + [anon_sym_DOT] = ACTIONS(1), + [anon_sym_export] = ACTIONS(1), + [anon_sym_local] = ACTIONS(1), + [anon_sym_enum] = ACTIONS(1), + [anon_sym_LBRACE] = ACTIONS(1), + [anon_sym_RBRACE] = ACTIONS(1), + [anon_sym_DASH] = ACTIONS(1), + [anon_sym_LBRACK] = ACTIONS(1), + [anon_sym_COMMA] = ACTIONS(1), + [anon_sym_RBRACK] = ACTIONS(1), + [anon_sym_message] = ACTIONS(1), + [anon_sym_extend] = ACTIONS(1), + [anon_sym_optional] = ACTIONS(1), + [anon_sym_required] = ACTIONS(1), + [anon_sym_repeated] = ACTIONS(1), + [anon_sym_group] = ACTIONS(1), + [anon_sym_oneof] = ACTIONS(1), + [anon_sym_map] = ACTIONS(1), + [anon_sym_LT] = ACTIONS(1), + [anon_sym_GT] = ACTIONS(1), + [anon_sym_int32] = ACTIONS(1), + [anon_sym_int64] = ACTIONS(1), + [anon_sym_uint32] = ACTIONS(1), + [anon_sym_uint64] = ACTIONS(1), + [anon_sym_sint32] = ACTIONS(1), + [anon_sym_sint64] = ACTIONS(1), + [anon_sym_fixed32] = ACTIONS(1), + [anon_sym_fixed64] = ACTIONS(1), + [anon_sym_sfixed32] = ACTIONS(1), + [anon_sym_sfixed64] = ACTIONS(1), + [anon_sym_bool] = ACTIONS(1), + [anon_sym_string] = ACTIONS(1), + [anon_sym_double] = ACTIONS(1), + [anon_sym_float] = ACTIONS(1), + [anon_sym_bytes] = ACTIONS(1), + [anon_sym_reserved] = ACTIONS(1), + [anon_sym_extensions] = ACTIONS(1), + [anon_sym_to] = ACTIONS(1), + [anon_sym_max] = ACTIONS(1), + [anon_sym_service] = ACTIONS(1), + [anon_sym_rpc] = ACTIONS(1), + [anon_sym_stream] = ACTIONS(1), + [anon_sym_returns] = ACTIONS(1), + [anon_sym_PLUS] = ACTIONS(1), + [anon_sym_COLON] = ACTIONS(1), + [sym_true] = ACTIONS(1), + [sym_false] = ACTIONS(1), + [sym_decimal_lit] = ACTIONS(1), + [sym_octal_lit] = ACTIONS(1), + [sym_hex_lit] = ACTIONS(1), + [sym_float_lit] = ACTIONS(1), + [anon_sym_DQUOTE] = ACTIONS(1), + [anon_sym_SQUOTE] = ACTIONS(1), + [sym_escape_sequence] = ACTIONS(1), + [sym_comment] = ACTIONS(3), + }, + [1] = { + [sym_source_file] = STATE(319), + [sym_empty_statement] = STATE(53), + [sym_edition] = STATE(44), + [sym_syntax] = STATE(44), + [sym_import] = STATE(53), + [sym_package] = STATE(53), + [sym_option] = STATE(53), + [sym_enum] = STATE(53), + [sym_message] = STATE(53), + [sym_extend] = STATE(53), + [sym_service] = STATE(53), + [aux_sym_source_file_repeat1] = STATE(53), + [ts_builtin_sym_end] = ACTIONS(5), + [anon_sym_SEMI] = ACTIONS(7), + [anon_sym_edition] = ACTIONS(9), + [anon_sym_syntax] = ACTIONS(11), + [anon_sym_import] = ACTIONS(13), + [anon_sym_option] = ACTIONS(15), + [anon_sym_package] = ACTIONS(17), + [anon_sym_export] = ACTIONS(19), + [anon_sym_local] = ACTIONS(19), + [anon_sym_enum] = ACTIONS(21), + [anon_sym_message] = ACTIONS(23), + [anon_sym_extend] = ACTIONS(25), + [anon_sym_service] = ACTIONS(27), + [sym_comment] = ACTIONS(3), + }, +}; + +static const uint16_t ts_small_parse_table[] = { + [0] = 22, + ACTIONS(3), 1, + sym_comment, + ACTIONS(29), 1, + anon_sym_SEMI, + ACTIONS(31), 1, + anon_sym_option, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(37), 1, + anon_sym_enum, + ACTIONS(39), 1, + anon_sym_RBRACE, + ACTIONS(41), 1, + anon_sym_message, + ACTIONS(43), 1, + anon_sym_extend, + ACTIONS(47), 1, + anon_sym_repeated, + ACTIONS(49), 1, + anon_sym_group, + ACTIONS(51), 1, + anon_sym_oneof, + ACTIONS(53), 1, + anon_sym_map, + ACTIONS(57), 1, + anon_sym_reserved, + ACTIONS(59), 1, + anon_sym_extensions, + ACTIONS(61), 1, + sym_identifier, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(299), 1, + sym_type, + ACTIONS(35), 2, + anon_sym_export, + anon_sym_local, + ACTIONS(45), 2, + anon_sym_optional, + anon_sym_required, + STATE(3), 12, + sym_empty_statement, + sym_option, + sym_enum, + sym_message, + sym_extend, + sym_group, + sym_field, + sym_oneof, + sym_map_field, + sym_reserved, + sym_extensions, + aux_sym_message_body_repeat1, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [94] = 22, + ACTIONS(3), 1, + sym_comment, + ACTIONS(29), 1, + anon_sym_SEMI, + ACTIONS(31), 1, + anon_sym_option, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(37), 1, + anon_sym_enum, + ACTIONS(41), 1, + anon_sym_message, + ACTIONS(43), 1, + anon_sym_extend, + ACTIONS(47), 1, + anon_sym_repeated, + ACTIONS(49), 1, + anon_sym_group, + ACTIONS(51), 1, + anon_sym_oneof, + ACTIONS(53), 1, + anon_sym_map, + ACTIONS(57), 1, + anon_sym_reserved, + ACTIONS(59), 1, + anon_sym_extensions, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(63), 1, + anon_sym_RBRACE, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(299), 1, + sym_type, + ACTIONS(35), 2, + anon_sym_export, + anon_sym_local, + ACTIONS(45), 2, + anon_sym_optional, + anon_sym_required, + STATE(4), 12, + sym_empty_statement, + sym_option, + sym_enum, + sym_message, + sym_extend, + sym_group, + sym_field, + sym_oneof, + sym_map_field, + sym_reserved, + sym_extensions, + aux_sym_message_body_repeat1, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [188] = 22, + ACTIONS(3), 1, + sym_comment, + ACTIONS(65), 1, + anon_sym_SEMI, + ACTIONS(68), 1, + anon_sym_option, + ACTIONS(71), 1, + anon_sym_DOT, + ACTIONS(77), 1, + anon_sym_enum, + ACTIONS(80), 1, + anon_sym_RBRACE, + ACTIONS(82), 1, + anon_sym_message, + ACTIONS(85), 1, + anon_sym_extend, + ACTIONS(91), 1, + anon_sym_repeated, + ACTIONS(94), 1, + anon_sym_group, + ACTIONS(97), 1, + anon_sym_oneof, + ACTIONS(100), 1, + anon_sym_map, + ACTIONS(106), 1, + anon_sym_reserved, + ACTIONS(109), 1, + anon_sym_extensions, + ACTIONS(112), 1, + sym_identifier, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(299), 1, + sym_type, + ACTIONS(74), 2, + anon_sym_export, + anon_sym_local, + ACTIONS(88), 2, + anon_sym_optional, + anon_sym_required, + STATE(4), 12, + sym_empty_statement, + sym_option, + sym_enum, + sym_message, + sym_extend, + sym_group, + sym_field, + sym_oneof, + sym_map_field, + sym_reserved, + sym_extensions, + aux_sym_message_body_repeat1, + ACTIONS(103), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [282] = 22, + ACTIONS(3), 1, + sym_comment, + ACTIONS(29), 1, + anon_sym_SEMI, + ACTIONS(31), 1, + anon_sym_option, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(37), 1, + anon_sym_enum, + ACTIONS(41), 1, + anon_sym_message, + ACTIONS(43), 1, + anon_sym_extend, + ACTIONS(47), 1, + anon_sym_repeated, + ACTIONS(49), 1, + anon_sym_group, + ACTIONS(51), 1, + anon_sym_oneof, + ACTIONS(53), 1, + anon_sym_map, + ACTIONS(57), 1, + anon_sym_reserved, + ACTIONS(59), 1, + anon_sym_extensions, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(115), 1, + anon_sym_RBRACE, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(299), 1, + sym_type, + ACTIONS(35), 2, + anon_sym_export, + anon_sym_local, + ACTIONS(45), 2, + anon_sym_optional, + anon_sym_required, + STATE(4), 12, + sym_empty_statement, + sym_option, + sym_enum, + sym_message, + sym_extend, + sym_group, + sym_field, + sym_oneof, + sym_map_field, + sym_reserved, + sym_extensions, + aux_sym_message_body_repeat1, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [376] = 22, + ACTIONS(3), 1, + sym_comment, + ACTIONS(29), 1, + anon_sym_SEMI, + ACTIONS(31), 1, + anon_sym_option, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(37), 1, + anon_sym_enum, + ACTIONS(41), 1, + anon_sym_message, + ACTIONS(43), 1, + anon_sym_extend, + ACTIONS(47), 1, + anon_sym_repeated, + ACTIONS(49), 1, + anon_sym_group, + ACTIONS(51), 1, + anon_sym_oneof, + ACTIONS(53), 1, + anon_sym_map, + ACTIONS(57), 1, + anon_sym_reserved, + ACTIONS(59), 1, + anon_sym_extensions, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(117), 1, + anon_sym_RBRACE, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(299), 1, + sym_type, + ACTIONS(35), 2, + anon_sym_export, + anon_sym_local, + ACTIONS(45), 2, + anon_sym_optional, + anon_sym_required, + STATE(5), 12, + sym_empty_statement, + sym_option, + sym_enum, + sym_message, + sym_extend, + sym_group, + sym_field, + sym_oneof, + sym_map_field, + sym_reserved, + sym_extensions, + aux_sym_message_body_repeat1, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [470] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(119), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(121), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [511] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(123), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(125), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [552] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(127), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(129), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [593] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(131), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(133), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [634] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(135), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(137), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [675] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(139), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(141), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [716] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(143), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(145), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [757] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(147), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(149), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [798] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(151), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(153), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [839] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(155), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(157), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [880] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(159), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(161), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [921] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(163), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(165), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [962] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(167), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(169), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1003] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(171), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(173), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1044] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(175), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(177), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1085] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(179), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(181), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1126] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(183), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(185), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1167] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(187), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(189), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1208] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(191), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(193), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1249] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(195), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(197), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1290] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(199), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(201), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1331] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(203), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(205), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1372] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(207), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(209), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1413] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(211), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(213), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1454] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(215), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(217), 30, + anon_sym_option, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_optional, + anon_sym_required, + anon_sym_repeated, + anon_sym_group, + anon_sym_oneof, + anon_sym_map, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + anon_sym_reserved, + anon_sym_extensions, + sym_identifier, + [1495] = 11, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(219), 1, + anon_sym_SEMI, + ACTIONS(221), 1, + anon_sym_option, + ACTIONS(223), 1, + anon_sym_RBRACE, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(342), 1, + sym_type, + STATE(34), 4, + sym_empty_statement, + sym_option, + sym_oneof_field, + aux_sym_oneof_repeat1, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [1546] = 11, + ACTIONS(3), 1, + sym_comment, + ACTIONS(225), 1, + anon_sym_SEMI, + ACTIONS(228), 1, + anon_sym_option, + ACTIONS(231), 1, + anon_sym_DOT, + ACTIONS(234), 1, + anon_sym_RBRACE, + ACTIONS(239), 1, + sym_identifier, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(342), 1, + sym_type, + STATE(33), 4, + sym_empty_statement, + sym_option, + sym_oneof_field, + aux_sym_oneof_repeat1, + ACTIONS(236), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [1597] = 11, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(219), 1, + anon_sym_SEMI, + ACTIONS(221), 1, + anon_sym_option, + ACTIONS(242), 1, + anon_sym_RBRACE, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(342), 1, + sym_type, + STATE(33), 4, + sym_empty_statement, + sym_option, + sym_oneof_field, + aux_sym_oneof_repeat1, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [1648] = 9, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(244), 1, + anon_sym_repeated, + ACTIONS(246), 1, + anon_sym_group, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(276), 1, + sym_type, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [1690] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(248), 5, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_LBRACE, + anon_sym_RBRACE, + anon_sym_LBRACK, + ACTIONS(250), 17, + anon_sym_option, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + sym_identifier, + [1720] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(256), 1, + anon_sym_LBRACK, + ACTIONS(252), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(254), 17, + anon_sym_option, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + sym_identifier, + [1751] = 8, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(246), 1, + anon_sym_group, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(276), 1, + sym_type, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [1790] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(258), 4, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + anon_sym_LBRACK, + ACTIONS(260), 17, + anon_sym_option, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + sym_identifier, + [1819] = 15, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(266), 1, + anon_sym_LBRACK, + ACTIONS(268), 1, + anon_sym_COLON, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(276), 1, + sym_hex_lit, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(127), 1, + sym_constant, + ACTIONS(264), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(274), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [1872] = 15, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(276), 1, + sym_hex_lit, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(284), 1, + anon_sym_LBRACK, + ACTIONS(286), 1, + anon_sym_COLON, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(142), 1, + sym_constant, + ACTIONS(264), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(274), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [1925] = 12, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(13), 1, + anon_sym_import, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(17), 1, + anon_sym_package, + ACTIONS(21), 1, + anon_sym_enum, + ACTIONS(23), 1, + anon_sym_message, + ACTIONS(25), 1, + anon_sym_extend, + ACTIONS(27), 1, + anon_sym_service, + ACTIONS(288), 1, + ts_builtin_sym_end, + ACTIONS(19), 2, + anon_sym_export, + anon_sym_local, + STATE(45), 9, + sym_empty_statement, + sym_import, + sym_package, + sym_option, + sym_enum, + sym_message, + sym_extend, + sym_service, + aux_sym_source_file_repeat1, + [1971] = 7, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(61), 1, + sym_identifier, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(320), 1, + sym_type, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [2007] = 12, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(13), 1, + anon_sym_import, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(17), 1, + anon_sym_package, + ACTIONS(21), 1, + anon_sym_enum, + ACTIONS(23), 1, + anon_sym_message, + ACTIONS(25), 1, + anon_sym_extend, + ACTIONS(27), 1, + anon_sym_service, + ACTIONS(290), 1, + ts_builtin_sym_end, + ACTIONS(19), 2, + anon_sym_export, + anon_sym_local, + STATE(42), 9, + sym_empty_statement, + sym_import, + sym_package, + sym_option, + sym_enum, + sym_message, + sym_extend, + sym_service, + aux_sym_source_file_repeat1, + [2053] = 12, + ACTIONS(3), 1, + sym_comment, + ACTIONS(292), 1, + ts_builtin_sym_end, + ACTIONS(294), 1, + anon_sym_SEMI, + ACTIONS(297), 1, + anon_sym_import, + ACTIONS(300), 1, + anon_sym_option, + ACTIONS(303), 1, + anon_sym_package, + ACTIONS(309), 1, + anon_sym_enum, + ACTIONS(312), 1, + anon_sym_message, + ACTIONS(315), 1, + anon_sym_extend, + ACTIONS(318), 1, + anon_sym_service, + ACTIONS(306), 2, + anon_sym_export, + anon_sym_local, + STATE(45), 9, + sym_empty_statement, + sym_import, + sym_package, + sym_option, + sym_enum, + sym_message, + sym_extend, + sym_service, + aux_sym_source_file_repeat1, + [2099] = 14, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(323), 1, + anon_sym_RBRACK, + ACTIONS(327), 1, + sym_hex_lit, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(202), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2149] = 14, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + ACTIONS(329), 1, + anon_sym_RBRACK, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(183), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2199] = 14, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(276), 1, + sym_hex_lit, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(331), 1, + anon_sym_LBRACK, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(141), 1, + sym_constant, + ACTIONS(264), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(274), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2249] = 7, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(61), 1, + sym_identifier, + STATE(241), 1, + sym_message_or_enum_type, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(274), 1, + sym_type, + ACTIONS(55), 15, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + [2285] = 14, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + ACTIONS(333), 1, + anon_sym_RBRACK, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(197), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2335] = 14, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(276), 1, + sym_hex_lit, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(335), 1, + anon_sym_LBRACK, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(139), 1, + sym_constant, + ACTIONS(264), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(274), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2385] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(337), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(339), 17, + anon_sym_option, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + sym_identifier, + [2413] = 12, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(13), 1, + anon_sym_import, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(17), 1, + anon_sym_package, + ACTIONS(21), 1, + anon_sym_enum, + ACTIONS(23), 1, + anon_sym_message, + ACTIONS(25), 1, + anon_sym_extend, + ACTIONS(27), 1, + anon_sym_service, + ACTIONS(290), 1, + ts_builtin_sym_end, + ACTIONS(19), 2, + anon_sym_export, + anon_sym_local, + STATE(45), 9, + sym_empty_statement, + sym_import, + sym_package, + sym_option, + sym_enum, + sym_message, + sym_extend, + sym_service, + aux_sym_source_file_repeat1, + [2459] = 14, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + ACTIONS(341), 1, + anon_sym_RBRACK, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(195), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2509] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(211), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(213), 17, + anon_sym_option, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + sym_identifier, + [2537] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(123), 3, + anon_sym_SEMI, + anon_sym_DOT, + anon_sym_RBRACE, + ACTIONS(125), 17, + anon_sym_option, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + anon_sym_double, + anon_sym_float, + anon_sym_bytes, + sym_identifier, + [2565] = 13, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(327), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2612] = 13, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(308), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2659] = 13, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(232), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2706] = 13, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(222), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2753] = 13, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(251), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2800] = 13, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(322), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2847] = 13, + ACTIONS(3), 1, + sym_comment, + ACTIONS(262), 1, + anon_sym_LBRACE, + ACTIONS(270), 1, + sym_identifier, + ACTIONS(278), 1, + sym_float_lit, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + ACTIONS(327), 1, + sym_hex_lit, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(326), 1, + sym_constant, + ACTIONS(272), 2, + sym_true, + sym_false, + ACTIONS(321), 2, + anon_sym_DASH, + anon_sym_PLUS, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + STATE(112), 5, + sym_block_lit, + sym_full_ident, + sym_bool, + sym_int_lit, + sym_string, + [2894] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(123), 13, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_RBRACE, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + anon_sym_rpc, + [2913] = 3, + ACTIONS(3), 1, + sym_comment, + STATE(335), 1, + sym_key_type, + ACTIONS(343), 12, + anon_sym_int32, + anon_sym_int64, + anon_sym_uint32, + anon_sym_uint64, + anon_sym_sint32, + anon_sym_sint64, + anon_sym_fixed32, + anon_sym_fixed64, + anon_sym_sfixed32, + anon_sym_sfixed64, + anon_sym_bool, + anon_sym_string, + [2934] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(211), 13, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_RBRACE, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + anon_sym_rpc, + [2953] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(345), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [2970] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(347), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [2987] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(203), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3004] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(179), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3021] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(349), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3038] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(119), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3055] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(351), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3072] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(175), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3089] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(183), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3106] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(353), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3123] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(355), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3140] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(359), 1, + anon_sym_DOT, + STATE(78), 1, + aux_sym__option_name_repeat1, + ACTIONS(357), 9, + anon_sym_SEMI, + anon_sym_EQ, + anon_sym_RPAREN, + anon_sym_LBRACE, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3161] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(187), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3178] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(191), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3195] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(195), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3212] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(199), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3229] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(362), 11, + ts_builtin_sym_end, + anon_sym_SEMI, + anon_sym_import, + anon_sym_option, + anon_sym_package, + anon_sym_export, + anon_sym_local, + anon_sym_enum, + anon_sym_message, + anon_sym_extend, + anon_sym_service, + [3246] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(366), 1, + anon_sym_DOT, + STATE(78), 1, + aux_sym__option_name_repeat1, + ACTIONS(364), 8, + anon_sym_SEMI, + anon_sym_RPAREN, + anon_sym_LBRACE, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3266] = 7, + ACTIONS(3), 1, + sym_comment, + ACTIONS(368), 1, + anon_sym_SEMI, + ACTIONS(371), 1, + anon_sym_option, + ACTIONS(374), 1, + anon_sym_RBRACE, + ACTIONS(376), 1, + anon_sym_reserved, + ACTIONS(379), 1, + sym_identifier, + STATE(85), 5, + sym_empty_statement, + sym_option, + sym_enum_field, + sym_reserved, + aux_sym_enum_body_repeat1, + [3292] = 7, + ACTIONS(3), 1, + sym_comment, + ACTIONS(382), 1, + anon_sym_SEMI, + ACTIONS(384), 1, + anon_sym_option, + ACTIONS(386), 1, + anon_sym_RBRACE, + ACTIONS(388), 1, + anon_sym_reserved, + ACTIONS(390), 1, + sym_identifier, + STATE(87), 5, + sym_empty_statement, + sym_option, + sym_enum_field, + sym_reserved, + aux_sym_enum_body_repeat1, + [3318] = 7, + ACTIONS(3), 1, + sym_comment, + ACTIONS(382), 1, + anon_sym_SEMI, + ACTIONS(384), 1, + anon_sym_option, + ACTIONS(388), 1, + anon_sym_reserved, + ACTIONS(390), 1, + sym_identifier, + ACTIONS(392), 1, + anon_sym_RBRACE, + STATE(85), 5, + sym_empty_statement, + sym_option, + sym_enum_field, + sym_reserved, + aux_sym_enum_body_repeat1, + [3344] = 7, + ACTIONS(3), 1, + sym_comment, + ACTIONS(382), 1, + anon_sym_SEMI, + ACTIONS(384), 1, + anon_sym_option, + ACTIONS(388), 1, + anon_sym_reserved, + ACTIONS(390), 1, + sym_identifier, + ACTIONS(394), 1, + anon_sym_RBRACE, + STATE(91), 5, + sym_empty_statement, + sym_option, + sym_enum_field, + sym_reserved, + aux_sym_enum_body_repeat1, + [3370] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(366), 1, + anon_sym_DOT, + STATE(84), 1, + aux_sym__option_name_repeat1, + ACTIONS(396), 8, + anon_sym_SEMI, + anon_sym_RPAREN, + anon_sym_LBRACE, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3390] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(357), 10, + anon_sym_SEMI, + anon_sym_EQ, + anon_sym_RPAREN, + anon_sym_DOT, + anon_sym_LBRACE, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3406] = 7, + ACTIONS(3), 1, + sym_comment, + ACTIONS(382), 1, + anon_sym_SEMI, + ACTIONS(384), 1, + anon_sym_option, + ACTIONS(388), 1, + anon_sym_reserved, + ACTIONS(390), 1, + sym_identifier, + ACTIONS(398), 1, + anon_sym_RBRACE, + STATE(85), 5, + sym_empty_statement, + sym_option, + sym_enum_field, + sym_reserved, + aux_sym_enum_body_repeat1, + [3432] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(402), 1, + anon_sym_DQUOTE, + ACTIONS(405), 1, + anon_sym_SQUOTE, + STATE(92), 1, + aux_sym_string_repeat3, + ACTIONS(400), 6, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3453] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + STATE(92), 1, + aux_sym_string_repeat3, + ACTIONS(408), 6, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3474] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(400), 8, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + anon_sym_DQUOTE, + anon_sym_SQUOTE, + [3488] = 7, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + ACTIONS(410), 1, + sym_reserved_identifier, + STATE(179), 1, + sym_range, + STATE(189), 1, + sym_int_lit, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + STATE(325), 2, + sym_ranges, + sym_reserved_field_names, + [3512] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(412), 1, + anon_sym_SEMI, + ACTIONS(415), 1, + anon_sym_option, + ACTIONS(418), 1, + anon_sym_RBRACE, + ACTIONS(420), 1, + anon_sym_rpc, + STATE(96), 4, + sym_empty_statement, + sym_option, + sym_rpc, + aux_sym_service_repeat1, + [3534] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(423), 8, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + anon_sym_DQUOTE, + anon_sym_SQUOTE, + [3548] = 7, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + ACTIONS(410), 1, + sym_reserved_identifier, + STATE(179), 1, + sym_range, + STATE(189), 1, + sym_int_lit, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + STATE(303), 2, + sym_ranges, + sym_reserved_field_names, + [3572] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(425), 1, + anon_sym_RBRACE, + ACTIONS(427), 1, + anon_sym_rpc, + STATE(96), 4, + sym_empty_statement, + sym_option, + sym_rpc, + aux_sym_service_repeat1, + [3594] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(427), 1, + anon_sym_rpc, + ACTIONS(429), 1, + anon_sym_RBRACE, + STATE(99), 4, + sym_empty_statement, + sym_option, + sym_rpc, + aux_sym_service_repeat1, + [3616] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(302), 1, + sym_string, + ACTIONS(431), 3, + anon_sym_weak, + anon_sym_public, + anon_sym_option, + [3637] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(433), 1, + anon_sym_RBRACE, + STATE(104), 3, + sym_empty_statement, + sym_option, + aux_sym_rpc_repeat1, + [3655] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(435), 1, + anon_sym_RBRACE, + STATE(106), 3, + sym_empty_statement, + sym_option, + aux_sym_rpc_repeat1, + [3673] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(435), 1, + anon_sym_RBRACE, + STATE(107), 3, + sym_empty_statement, + sym_option, + aux_sym_rpc_repeat1, + [3691] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(437), 1, + anon_sym_RBRACE, + STATE(108), 3, + sym_empty_statement, + sym_option, + aux_sym_rpc_repeat1, + [3709] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(437), 1, + anon_sym_RBRACE, + STATE(107), 3, + sym_empty_statement, + sym_option, + aux_sym_rpc_repeat1, + [3727] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(439), 1, + anon_sym_SEMI, + ACTIONS(442), 1, + anon_sym_option, + ACTIONS(445), 1, + anon_sym_RBRACE, + STATE(107), 3, + sym_empty_statement, + sym_option, + aux_sym_rpc_repeat1, + [3745] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(7), 1, + anon_sym_SEMI, + ACTIONS(15), 1, + anon_sym_option, + ACTIONS(447), 1, + anon_sym_RBRACE, + STATE(107), 3, + sym_empty_statement, + sym_option, + aux_sym_rpc_repeat1, + [3763] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(258), 6, + anon_sym_SEMI, + anon_sym_LBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + anon_sym_to, + [3775] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(449), 6, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3787] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(451), 6, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3799] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(453), 6, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3811] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + STATE(179), 1, + sym_range, + STATE(189), 1, + sym_int_lit, + STATE(286), 1, + sym_ranges, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [3831] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(455), 6, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3843] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(457), 6, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + anon_sym_RBRACK, + sym_identifier, + [3855] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(327), 1, + sym_hex_lit, + ACTIONS(459), 1, + sym_float_lit, + STATE(111), 1, + sym_int_lit, + ACTIONS(325), 2, + sym_decimal_lit, + sym_octal_lit, + [3872] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(210), 1, + sym_field_option, + STATE(305), 1, + sym_field_options, + STATE(317), 1, + sym__option_name, + [3891] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(465), 2, + anon_sym_SEMI, + anon_sym_COMMA, + ACTIONS(467), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [3904] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + STATE(189), 1, + sym_int_lit, + STATE(229), 1, + sym_range, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [3921] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(469), 1, + anon_sym_stream, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(318), 1, + sym_message_or_enum_type, + [3940] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(471), 2, + anon_sym_SEMI, + anon_sym_COMMA, + ACTIONS(473), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [3953] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(210), 1, + sym_field_option, + STATE(317), 1, + sym__option_name, + STATE(331), 1, + sym_field_options, + [3972] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(475), 2, + anon_sym_SEMI, + anon_sym_RBRACE, + ACTIONS(477), 3, + anon_sym_option, + anon_sym_reserved, + sym_identifier, + [3985] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(211), 2, + anon_sym_SEMI, + anon_sym_RBRACE, + ACTIONS(213), 3, + anon_sym_option, + anon_sym_reserved, + sym_identifier, + [3998] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(210), 1, + sym_field_option, + STATE(317), 1, + sym__option_name, + STATE(332), 1, + sym_field_options, + [4017] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + STATE(36), 1, + sym_int_lit, + STATE(218), 1, + sym_field_number, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [4034] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(479), 2, + anon_sym_SEMI, + anon_sym_COMMA, + ACTIONS(481), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [4047] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(258), 5, + anon_sym_SEMI, + anon_sym_RBRACE, + anon_sym_LBRACK, + anon_sym_COMMA, + sym_identifier, + [4058] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + STATE(36), 1, + sym_int_lit, + STATE(269), 1, + sym_field_number, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [4075] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + ACTIONS(483), 1, + anon_sym_DASH, + STATE(256), 1, + sym_int_lit, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [4092] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(485), 2, + anon_sym_SEMI, + anon_sym_RBRACE, + ACTIONS(487), 3, + anon_sym_option, + anon_sym_reserved, + sym_identifier, + [4105] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(491), 1, + sym_octal_lit, + STATE(36), 1, + sym_int_lit, + STATE(37), 1, + sym_field_number, + ACTIONS(489), 2, + sym_decimal_lit, + sym_hex_lit, + [4122] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(123), 2, + anon_sym_SEMI, + anon_sym_RBRACE, + ACTIONS(125), 3, + anon_sym_option, + anon_sym_reserved, + sym_identifier, + [4135] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(493), 2, + anon_sym_SEMI, + anon_sym_COMMA, + ACTIONS(495), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [4148] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + STATE(36), 1, + sym_int_lit, + STATE(234), 1, + sym_field_number, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [4165] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(497), 2, + anon_sym_SEMI, + anon_sym_RBRACE, + ACTIONS(499), 3, + anon_sym_option, + anon_sym_reserved, + sym_identifier, + [4178] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(210), 1, + sym_field_option, + STATE(298), 1, + sym_field_options, + STATE(317), 1, + sym__option_name, + [4197] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(501), 2, + anon_sym_SEMI, + anon_sym_RBRACE, + ACTIONS(503), 3, + anon_sym_option, + anon_sym_reserved, + sym_identifier, + [4210] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(505), 2, + anon_sym_SEMI, + anon_sym_COMMA, + ACTIONS(507), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [4223] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + STATE(36), 1, + sym_int_lit, + STATE(253), 1, + sym_field_number, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [4240] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(509), 2, + anon_sym_SEMI, + anon_sym_COMMA, + ACTIONS(511), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [4253] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(513), 2, + anon_sym_SEMI, + anon_sym_COMMA, + ACTIONS(515), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [4266] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + ACTIONS(517), 1, + anon_sym_max, + STATE(230), 1, + sym_int_lit, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [4283] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(276), 1, + sym_hex_lit, + ACTIONS(459), 1, + sym_float_lit, + STATE(111), 1, + sym_int_lit, + ACTIONS(274), 2, + sym_decimal_lit, + sym_octal_lit, + [4300] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + STATE(36), 1, + sym_int_lit, + STATE(235), 1, + sym_field_number, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [4317] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(519), 2, + anon_sym_SEMI, + anon_sym_RBRACE, + ACTIONS(521), 3, + anon_sym_option, + anon_sym_reserved, + sym_identifier, + [4330] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(210), 1, + sym_field_option, + STATE(273), 1, + sym_field_options, + STATE(317), 1, + sym__option_name, + [4349] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(207), 2, + anon_sym_SEMI, + anon_sym_RBRACE, + ACTIONS(209), 3, + anon_sym_option, + anon_sym_reserved, + sym_identifier, + [4362] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + STATE(36), 1, + sym_int_lit, + STATE(219), 1, + sym_field_number, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [4379] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(523), 1, + anon_sym_stream, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(341), 1, + sym_message_or_enum_type, + [4398] = 6, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(61), 1, + sym_identifier, + ACTIONS(525), 1, + anon_sym_stream, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(337), 1, + sym_message_or_enum_type, + [4417] = 4, + ACTIONS(527), 1, + anon_sym_SQUOTE, + ACTIONS(531), 1, + sym_comment, + STATE(160), 1, + aux_sym_string_repeat2, + ACTIONS(529), 2, + aux_sym_string_token2, + sym_escape_sequence, + [4431] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(533), 1, + anon_sym_RBRACE, + ACTIONS(535), 1, + anon_sym_LBRACK, + ACTIONS(537), 1, + sym_identifier, + STATE(161), 1, + aux_sym_block_lit_repeat2, + [4447] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(194), 1, + sym_enum_value_option, + STATE(310), 1, + sym__option_name, + [4463] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(539), 1, + sym_identifier, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(288), 1, + sym_message_or_enum_type, + [4479] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(201), 1, + sym_enum_value_option, + STATE(310), 1, + sym__option_name, + [4495] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(543), 1, + anon_sym_DOT, + ACTIONS(541), 3, + anon_sym_RPAREN, + anon_sym_GT, + sym_identifier, + [4507] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(223), 1, + sym_enum_value_option, + STATE(310), 1, + sym__option_name, + [4523] = 4, + ACTIONS(531), 1, + sym_comment, + ACTIONS(545), 1, + anon_sym_DQUOTE, + STATE(177), 1, + aux_sym_string_repeat1, + ACTIONS(547), 2, + aux_sym_string_token1, + sym_escape_sequence, + [4537] = 4, + ACTIONS(531), 1, + sym_comment, + ACTIONS(545), 1, + anon_sym_SQUOTE, + STATE(178), 1, + aux_sym_string_repeat2, + ACTIONS(549), 2, + aux_sym_string_token2, + sym_escape_sequence, + [4551] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(535), 1, + anon_sym_LBRACK, + ACTIONS(537), 1, + sym_identifier, + ACTIONS(551), 1, + anon_sym_RBRACE, + STATE(174), 1, + aux_sym_block_lit_repeat2, + [4567] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(233), 1, + sym_field_option, + STATE(317), 1, + sym__option_name, + [4583] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(539), 1, + sym_identifier, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(337), 1, + sym_message_or_enum_type, + [4599] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(543), 1, + anon_sym_DOT, + ACTIONS(553), 3, + anon_sym_RPAREN, + anon_sym_GT, + sym_identifier, + [4611] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(33), 1, + anon_sym_DOT, + ACTIONS(539), 1, + sym_identifier, + STATE(271), 1, + aux_sym_message_or_enum_type_repeat1, + STATE(278), 1, + sym_message_or_enum_type, + [4627] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(295), 1, + sym_string, + [4643] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(555), 4, + anon_sym_SEMI, + anon_sym_option, + anon_sym_RBRACE, + anon_sym_rpc, + [4653] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(557), 4, + anon_sym_SEMI, + anon_sym_option, + anon_sym_RBRACE, + anon_sym_rpc, + [4663] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(559), 4, + anon_sym_SEMI, + anon_sym_option, + anon_sym_RBRACE, + anon_sym_rpc, + [4673] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(561), 4, + anon_sym_SEMI, + anon_sym_option, + anon_sym_RBRACE, + anon_sym_rpc, + [4683] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(563), 4, + anon_sym_SEMI, + anon_sym_option, + anon_sym_RBRACE, + anon_sym_rpc, + [4693] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(280), 1, + anon_sym_DQUOTE, + ACTIONS(282), 1, + anon_sym_SQUOTE, + STATE(93), 1, + aux_sym_string_repeat3, + STATE(284), 1, + sym_string, + [4709] = 4, + ACTIONS(527), 1, + anon_sym_DQUOTE, + ACTIONS(531), 1, + sym_comment, + STATE(159), 1, + aux_sym_string_repeat1, + ACTIONS(565), 2, + aux_sym_string_token1, + sym_escape_sequence, + [4723] = 5, + ACTIONS(3), 1, + sym_comment, + ACTIONS(515), 1, + anon_sym_RBRACE, + ACTIONS(567), 1, + anon_sym_LBRACK, + ACTIONS(570), 1, + sym_identifier, + STATE(174), 1, + aux_sym_block_lit_repeat2, + [4739] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(325), 1, + sym_octal_lit, + STATE(231), 1, + sym_int_lit, + ACTIONS(327), 2, + sym_decimal_lit, + sym_hex_lit, + [4753] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(543), 1, + anon_sym_DOT, + ACTIONS(573), 3, + anon_sym_RPAREN, + anon_sym_GT, + sym_identifier, + [4765] = 4, + ACTIONS(531), 1, + sym_comment, + ACTIONS(575), 1, + anon_sym_DQUOTE, + STATE(177), 1, + aux_sym_string_repeat1, + ACTIONS(577), 2, + aux_sym_string_token1, + sym_escape_sequence, + [4779] = 4, + ACTIONS(531), 1, + sym_comment, + ACTIONS(580), 1, + anon_sym_SQUOTE, + STATE(178), 1, + aux_sym_string_repeat2, + ACTIONS(582), 2, + aux_sym_string_token2, + sym_escape_sequence, + [4793] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(585), 1, + anon_sym_SEMI, + ACTIONS(587), 1, + anon_sym_COMMA, + STATE(209), 1, + aux_sym_ranges_repeat1, + [4806] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(589), 1, + anon_sym_COMMA, + ACTIONS(591), 1, + anon_sym_RBRACK, + STATE(181), 1, + aux_sym_enum_field_repeat1, + [4819] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(593), 1, + anon_sym_COMMA, + ACTIONS(596), 1, + anon_sym_RBRACK, + STATE(181), 1, + aux_sym_enum_field_repeat1, + [4832] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(333), 1, + anon_sym_RBRACK, + ACTIONS(598), 1, + anon_sym_COMMA, + STATE(199), 1, + aux_sym_block_lit_repeat1, + [4845] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(333), 1, + anon_sym_RBRACK, + ACTIONS(598), 1, + anon_sym_COMMA, + STATE(200), 1, + aux_sym_block_lit_repeat1, + [4858] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(600), 1, + anon_sym_COMMA, + ACTIONS(602), 1, + anon_sym_RBRACK, + STATE(191), 1, + aux_sym_field_options_repeat1, + [4871] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(473), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [4880] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(604), 1, + anon_sym_SEMI, + ACTIONS(606), 1, + anon_sym_COMMA, + STATE(207), 1, + aux_sym_reserved_field_names_repeat1, + [4893] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(598), 1, + anon_sym_COMMA, + ACTIONS(608), 1, + anon_sym_RBRACK, + STATE(199), 1, + aux_sym_block_lit_repeat1, + [4906] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(340), 1, + sym__option_name, + [4919] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(612), 1, + anon_sym_to, + ACTIONS(610), 2, + anon_sym_SEMI, + anon_sym_COMMA, + [4930] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(366), 1, + anon_sym_DOT, + ACTIONS(614), 1, + anon_sym_EQ, + STATE(78), 1, + aux_sym__option_name_repeat1, + [4943] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(616), 1, + anon_sym_COMMA, + ACTIONS(619), 1, + anon_sym_RBRACK, + STATE(191), 1, + aux_sym_field_options_repeat1, + [4956] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(621), 1, + anon_sym_SEMI, + ACTIONS(623), 1, + anon_sym_COMMA, + STATE(192), 1, + aux_sym_reserved_field_names_repeat1, + [4969] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(495), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [4978] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(589), 1, + anon_sym_COMMA, + ACTIONS(626), 1, + anon_sym_RBRACK, + STATE(203), 1, + aux_sym_enum_field_repeat1, + [4991] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(598), 1, + anon_sym_COMMA, + ACTIONS(628), 1, + anon_sym_RBRACK, + STATE(187), 1, + aux_sym_block_lit_repeat1, + [5004] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(630), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [5013] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(341), 1, + anon_sym_RBRACK, + ACTIONS(598), 1, + anon_sym_COMMA, + STATE(213), 1, + aux_sym_block_lit_repeat1, + [5026] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(507), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [5035] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(632), 1, + anon_sym_COMMA, + ACTIONS(635), 1, + anon_sym_RBRACK, + STATE(199), 1, + aux_sym_block_lit_repeat1, + [5048] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(341), 1, + anon_sym_RBRACK, + ACTIONS(598), 1, + anon_sym_COMMA, + STATE(199), 1, + aux_sym_block_lit_repeat1, + [5061] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(589), 1, + anon_sym_COMMA, + ACTIONS(637), 1, + anon_sym_RBRACK, + STATE(180), 1, + aux_sym_enum_field_repeat1, + [5074] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(329), 1, + anon_sym_RBRACK, + ACTIONS(598), 1, + anon_sym_COMMA, + STATE(182), 1, + aux_sym_block_lit_repeat1, + [5087] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(589), 1, + anon_sym_COMMA, + ACTIONS(637), 1, + anon_sym_RBRACK, + STATE(181), 1, + aux_sym_enum_field_repeat1, + [5100] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(511), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [5109] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(344), 1, + sym__option_name, + [5122] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(481), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [5131] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(606), 1, + anon_sym_COMMA, + ACTIONS(639), 1, + anon_sym_SEMI, + STATE(192), 1, + aux_sym_reserved_field_names_repeat1, + [5144] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(366), 1, + anon_sym_DOT, + ACTIONS(641), 1, + anon_sym_EQ, + STATE(190), 1, + aux_sym__option_name_repeat1, + [5157] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(587), 1, + anon_sym_COMMA, + ACTIONS(643), 1, + anon_sym_SEMI, + STATE(217), 1, + aux_sym_ranges_repeat1, + [5170] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(600), 1, + anon_sym_COMMA, + ACTIONS(645), 1, + anon_sym_RBRACK, + STATE(184), 1, + aux_sym_field_options_repeat1, + [5183] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(467), 3, + anon_sym_RBRACE, + anon_sym_LBRACK, + sym_identifier, + [5192] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(366), 1, + anon_sym_DOT, + ACTIONS(647), 1, + anon_sym_EQ, + STATE(78), 1, + aux_sym__option_name_repeat1, + [5205] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(598), 1, + anon_sym_COMMA, + ACTIONS(628), 1, + anon_sym_RBRACK, + STATE(199), 1, + aux_sym_block_lit_repeat1, + [5218] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(366), 1, + anon_sym_DOT, + ACTIONS(649), 1, + anon_sym_EQ, + STATE(212), 1, + aux_sym__option_name_repeat1, + [5231] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(338), 1, + sym__option_name, + [5244] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(461), 1, + anon_sym_LPAREN, + ACTIONS(463), 1, + sym_identifier, + STATE(339), 1, + sym__option_name, + [5257] = 4, + ACTIONS(3), 1, + sym_comment, + ACTIONS(651), 1, + anon_sym_SEMI, + ACTIONS(653), 1, + anon_sym_COMMA, + STATE(217), 1, + aux_sym_ranges_repeat1, + [5270] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(656), 1, + anon_sym_SEMI, + ACTIONS(658), 1, + anon_sym_LBRACK, + [5280] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(660), 1, + anon_sym_SEMI, + ACTIONS(662), 1, + anon_sym_LBRACK, + [5290] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(621), 2, + anon_sym_SEMI, + anon_sym_COMMA, + [5298] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(664), 1, + sym_identifier, + STATE(247), 1, + sym_enum_name, + [5308] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(666), 2, + anon_sym_COMMA, + anon_sym_RBRACK, + [5316] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(596), 2, + anon_sym_COMMA, + anon_sym_RBRACK, + [5324] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(668), 1, + sym_identifier, + STATE(311), 1, + sym_service_name, + [5334] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(670), 1, + sym_identifier, + STATE(237), 1, + aux_sym_message_or_enum_type_repeat1, + [5344] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(672), 1, + sym_identifier, + STATE(300), 1, + sym_full_ident, + [5354] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(664), 1, + sym_identifier, + STATE(243), 1, + sym_enum_name, + [5364] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(674), 1, + sym_identifier, + STATE(262), 1, + sym_message_name, + [5374] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(651), 2, + anon_sym_SEMI, + anon_sym_COMMA, + [5382] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(676), 2, + anon_sym_SEMI, + anon_sym_COMMA, + [5390] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(678), 1, + anon_sym_SEMI, + ACTIONS(680), 1, + anon_sym_LBRACK, + [5400] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(682), 2, + anon_sym_COMMA, + anon_sym_RBRACK, + [5408] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(619), 2, + anon_sym_COMMA, + anon_sym_RBRACK, + [5416] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(684), 1, + anon_sym_LBRACE, + STATE(14), 1, + sym_message_body, + [5426] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(686), 1, + anon_sym_SEMI, + ACTIONS(688), 1, + anon_sym_LBRACK, + [5436] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(674), 1, + sym_identifier, + STATE(277), 1, + sym_message_name, + [5446] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(690), 1, + sym_identifier, + STATE(246), 1, + aux_sym_message_or_enum_type_repeat1, + [5456] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(692), 1, + anon_sym_SEMI, + ACTIONS(694), 1, + anon_sym_LBRACE, + [5466] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(674), 1, + sym_identifier, + STATE(321), 1, + sym_message_name, + [5476] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(433), 1, + anon_sym_SEMI, + ACTIONS(696), 1, + anon_sym_LBRACE, + [5486] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(698), 2, + anon_sym_GT, + sym_identifier, + [5494] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(700), 1, + sym_identifier, + STATE(297), 1, + sym_rpc_name, + [5504] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(702), 1, + anon_sym_LBRACE, + STATE(79), 1, + sym_enum_body, + [5514] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(704), 1, + anon_sym_LBRACE, + STATE(80), 1, + sym_message_body, + [5524] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(706), 2, + anon_sym_DQUOTEproto3_DQUOTE, + anon_sym_DQUOTEproto2_DQUOTE, + [5532] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(708), 1, + sym_identifier, + STATE(246), 1, + aux_sym_message_or_enum_type_repeat1, + [5542] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(702), 1, + anon_sym_LBRACE, + STATE(74), 1, + sym_enum_body, + [5552] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(674), 1, + sym_identifier, + STATE(244), 1, + sym_message_name, + [5562] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(672), 1, + sym_identifier, + STATE(333), 1, + sym_full_ident, + [5572] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(672), 1, + sym_identifier, + STATE(279), 1, + sym_full_ident, + [5582] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(635), 2, + anon_sym_COMMA, + anon_sym_RBRACK, + [5590] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(711), 2, + anon_sym_EQ, + anon_sym_LBRACE, + [5598] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(684), 1, + anon_sym_LBRACE, + STATE(11), 1, + sym_message_body, + [5608] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(713), 1, + anon_sym_enum, + ACTIONS(715), 1, + anon_sym_message, + [5618] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(672), 1, + sym_identifier, + STATE(268), 1, + sym_full_ident, + [5628] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(717), 1, + anon_sym_SEMI, + ACTIONS(719), 1, + anon_sym_LBRACK, + [5638] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(721), 1, + anon_sym_LBRACE, + STATE(21), 1, + sym_enum_body, + [5648] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(684), 1, + anon_sym_LBRACE, + STATE(22), 1, + sym_message_body, + [5658] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(684), 1, + anon_sym_LBRACE, + STATE(23), 1, + sym_message_body, + [5668] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(721), 1, + anon_sym_LBRACE, + STATE(24), 1, + sym_enum_body, + [5678] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(684), 1, + anon_sym_LBRACE, + STATE(25), 1, + sym_message_body, + [5688] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(704), 1, + anon_sym_LBRACE, + STATE(70), 1, + sym_message_body, + [5698] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(664), 1, + sym_identifier, + STATE(257), 1, + sym_enum_name, + [5708] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(674), 1, + sym_identifier, + STATE(258), 1, + sym_message_name, + [5718] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(672), 1, + sym_identifier, + STATE(259), 1, + sym_full_ident, + [5728] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(664), 1, + sym_identifier, + STATE(260), 1, + sym_enum_name, + [5738] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(674), 1, + sym_identifier, + STATE(261), 1, + sym_message_name, + [5748] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(704), 1, + anon_sym_LBRACE, + STATE(75), 1, + sym_message_body, + [5758] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(723), 1, + anon_sym_SEMI, + ACTIONS(725), 1, + anon_sym_LBRACK, + [5768] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(727), 1, + anon_sym_enum, + ACTIONS(729), 1, + anon_sym_message, + [5778] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(670), 1, + sym_identifier, + STATE(246), 1, + aux_sym_message_or_enum_type_repeat1, + [5788] = 3, + ACTIONS(3), 1, + sym_comment, + ACTIONS(435), 1, + anon_sym_SEMI, + ACTIONS(731), 1, + anon_sym_LBRACE, + [5798] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(733), 1, + anon_sym_RBRACK, + [5805] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(735), 1, + anon_sym_GT, + [5812] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(737), 1, + anon_sym_EQ, + [5819] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(739), 1, + sym_identifier, + [5826] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(741), 1, + anon_sym_EQ, + [5833] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(743), 1, + anon_sym_RPAREN, + [5840] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(745), 1, + anon_sym_RPAREN, + [5847] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(747), 1, + anon_sym_SEMI, + [5854] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(749), 1, + anon_sym_LBRACE, + [5861] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(543), 1, + anon_sym_DOT, + [5868] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(751), 1, + anon_sym_EQ, + [5875] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(753), 1, + anon_sym_SEMI, + [5882] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(755), 1, + anon_sym_SEMI, + [5889] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(757), 1, + anon_sym_SEMI, + [5896] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(759), 1, + anon_sym_SEMI, + [5903] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(761), 1, + anon_sym_RPAREN, + [5910] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(763), 1, + sym_identifier, + [5917] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(765), 1, + anon_sym_EQ, + [5924] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(767), 1, + anon_sym_LPAREN, + [5931] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(769), 1, + sym_identifier, + [5938] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(771), 1, + anon_sym_returns, + [5945] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(773), 1, + anon_sym_LBRACE, + [5952] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(775), 1, + anon_sym_SEMI, + [5959] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(777), 1, + anon_sym_LPAREN, + [5966] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(779), 1, + anon_sym_LPAREN, + [5973] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(781), 1, + anon_sym_RBRACK, + [5980] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(783), 1, + sym_identifier, + [5987] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(785), 1, + anon_sym_RBRACK, + [5994] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(787), 1, + anon_sym_EQ, + [6001] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(789), 1, + anon_sym_SEMI, + [6008] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(791), 1, + anon_sym_SEMI, + [6015] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(793), 1, + anon_sym_SEMI, + [6022] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(795), 1, + anon_sym_RBRACK, + [6029] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(797), 1, + sym_reserved_identifier, + [6036] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(799), 1, + anon_sym_SEMI, + [6043] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(801), 1, + anon_sym_SEMI, + [6050] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(803), 1, + anon_sym_LBRACE, + [6057] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(805), 1, + anon_sym_EQ, + [6064] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(807), 1, + anon_sym_LBRACE, + [6071] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(809), 1, + anon_sym_returns, + [6078] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(811), 1, + anon_sym_SEMI, + [6085] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(813), 1, + anon_sym_LPAREN, + [6092] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(815), 1, + sym_identifier, + [6099] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(817), 1, + anon_sym_EQ, + [6106] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(819), 1, + anon_sym_EQ, + [6113] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(821), 1, + anon_sym_RPAREN, + [6120] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(823), 1, + ts_builtin_sym_end, + [6127] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(825), 1, + sym_identifier, + [6134] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(827), 1, + anon_sym_EQ, + [6141] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(829), 1, + anon_sym_SEMI, + [6148] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(831), 1, + anon_sym_EQ, + [6155] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(833), 1, + anon_sym_SEMI, + [6162] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(835), 1, + anon_sym_SEMI, + [6169] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(837), 1, + anon_sym_SEMI, + [6176] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(839), 1, + anon_sym_SEMI, + [6183] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(841), 1, + anon_sym_EQ, + [6190] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(843), 1, + anon_sym_EQ, + [6197] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(845), 1, + anon_sym_SEMI, + [6204] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(847), 1, + anon_sym_RBRACK, + [6211] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(849), 1, + anon_sym_RBRACK, + [6218] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(851), 1, + anon_sym_SEMI, + [6225] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(853), 1, + anon_sym_COMMA, + [6232] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(855), 1, + anon_sym_COMMA, + [6239] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(857), 1, + sym_identifier, + [6246] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(859), 1, + anon_sym_RPAREN, + [6253] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(861), 1, + anon_sym_EQ, + [6260] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(863), 1, + anon_sym_EQ, + [6267] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(865), 1, + anon_sym_EQ, + [6274] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(867), 1, + anon_sym_RPAREN, + [6281] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(869), 1, + sym_identifier, + [6288] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(871), 1, + anon_sym_LT, + [6295] = 2, + ACTIONS(3), 1, + sym_comment, + ACTIONS(873), 1, + anon_sym_EQ, +}; + +static const uint32_t ts_small_parse_table_map[] = { + [SMALL_STATE(2)] = 0, + [SMALL_STATE(3)] = 94, + [SMALL_STATE(4)] = 188, + [SMALL_STATE(5)] = 282, + [SMALL_STATE(6)] = 376, + [SMALL_STATE(7)] = 470, + [SMALL_STATE(8)] = 511, + [SMALL_STATE(9)] = 552, + [SMALL_STATE(10)] = 593, + [SMALL_STATE(11)] = 634, + [SMALL_STATE(12)] = 675, + [SMALL_STATE(13)] = 716, + [SMALL_STATE(14)] = 757, + [SMALL_STATE(15)] = 798, + [SMALL_STATE(16)] = 839, + [SMALL_STATE(17)] = 880, + [SMALL_STATE(18)] = 921, + [SMALL_STATE(19)] = 962, + [SMALL_STATE(20)] = 1003, + [SMALL_STATE(21)] = 1044, + [SMALL_STATE(22)] = 1085, + [SMALL_STATE(23)] = 1126, + [SMALL_STATE(24)] = 1167, + [SMALL_STATE(25)] = 1208, + [SMALL_STATE(26)] = 1249, + [SMALL_STATE(27)] = 1290, + [SMALL_STATE(28)] = 1331, + [SMALL_STATE(29)] = 1372, + [SMALL_STATE(30)] = 1413, + [SMALL_STATE(31)] = 1454, + [SMALL_STATE(32)] = 1495, + [SMALL_STATE(33)] = 1546, + [SMALL_STATE(34)] = 1597, + [SMALL_STATE(35)] = 1648, + [SMALL_STATE(36)] = 1690, + [SMALL_STATE(37)] = 1720, + [SMALL_STATE(38)] = 1751, + [SMALL_STATE(39)] = 1790, + [SMALL_STATE(40)] = 1819, + [SMALL_STATE(41)] = 1872, + [SMALL_STATE(42)] = 1925, + [SMALL_STATE(43)] = 1971, + [SMALL_STATE(44)] = 2007, + [SMALL_STATE(45)] = 2053, + [SMALL_STATE(46)] = 2099, + [SMALL_STATE(47)] = 2149, + [SMALL_STATE(48)] = 2199, + [SMALL_STATE(49)] = 2249, + [SMALL_STATE(50)] = 2285, + [SMALL_STATE(51)] = 2335, + [SMALL_STATE(52)] = 2385, + [SMALL_STATE(53)] = 2413, + [SMALL_STATE(54)] = 2459, + [SMALL_STATE(55)] = 2509, + [SMALL_STATE(56)] = 2537, + [SMALL_STATE(57)] = 2565, + [SMALL_STATE(58)] = 2612, + [SMALL_STATE(59)] = 2659, + [SMALL_STATE(60)] = 2706, + [SMALL_STATE(61)] = 2753, + [SMALL_STATE(62)] = 2800, + [SMALL_STATE(63)] = 2847, + [SMALL_STATE(64)] = 2894, + [SMALL_STATE(65)] = 2913, + [SMALL_STATE(66)] = 2934, + [SMALL_STATE(67)] = 2953, + [SMALL_STATE(68)] = 2970, + [SMALL_STATE(69)] = 2987, + [SMALL_STATE(70)] = 3004, + [SMALL_STATE(71)] = 3021, + [SMALL_STATE(72)] = 3038, + [SMALL_STATE(73)] = 3055, + [SMALL_STATE(74)] = 3072, + [SMALL_STATE(75)] = 3089, + [SMALL_STATE(76)] = 3106, + [SMALL_STATE(77)] = 3123, + [SMALL_STATE(78)] = 3140, + [SMALL_STATE(79)] = 3161, + [SMALL_STATE(80)] = 3178, + [SMALL_STATE(81)] = 3195, + [SMALL_STATE(82)] = 3212, + [SMALL_STATE(83)] = 3229, + [SMALL_STATE(84)] = 3246, + [SMALL_STATE(85)] = 3266, + [SMALL_STATE(86)] = 3292, + [SMALL_STATE(87)] = 3318, + [SMALL_STATE(88)] = 3344, + [SMALL_STATE(89)] = 3370, + [SMALL_STATE(90)] = 3390, + [SMALL_STATE(91)] = 3406, + [SMALL_STATE(92)] = 3432, + [SMALL_STATE(93)] = 3453, + [SMALL_STATE(94)] = 3474, + [SMALL_STATE(95)] = 3488, + [SMALL_STATE(96)] = 3512, + [SMALL_STATE(97)] = 3534, + [SMALL_STATE(98)] = 3548, + [SMALL_STATE(99)] = 3572, + [SMALL_STATE(100)] = 3594, + [SMALL_STATE(101)] = 3616, + [SMALL_STATE(102)] = 3637, + [SMALL_STATE(103)] = 3655, + [SMALL_STATE(104)] = 3673, + [SMALL_STATE(105)] = 3691, + [SMALL_STATE(106)] = 3709, + [SMALL_STATE(107)] = 3727, + [SMALL_STATE(108)] = 3745, + [SMALL_STATE(109)] = 3763, + [SMALL_STATE(110)] = 3775, + [SMALL_STATE(111)] = 3787, + [SMALL_STATE(112)] = 3799, + [SMALL_STATE(113)] = 3811, + [SMALL_STATE(114)] = 3831, + [SMALL_STATE(115)] = 3843, + [SMALL_STATE(116)] = 3855, + [SMALL_STATE(117)] = 3872, + [SMALL_STATE(118)] = 3891, + [SMALL_STATE(119)] = 3904, + [SMALL_STATE(120)] = 3921, + [SMALL_STATE(121)] = 3940, + [SMALL_STATE(122)] = 3953, + [SMALL_STATE(123)] = 3972, + [SMALL_STATE(124)] = 3985, + [SMALL_STATE(125)] = 3998, + [SMALL_STATE(126)] = 4017, + [SMALL_STATE(127)] = 4034, + [SMALL_STATE(128)] = 4047, + [SMALL_STATE(129)] = 4058, + [SMALL_STATE(130)] = 4075, + [SMALL_STATE(131)] = 4092, + [SMALL_STATE(132)] = 4105, + [SMALL_STATE(133)] = 4122, + [SMALL_STATE(134)] = 4135, + [SMALL_STATE(135)] = 4148, + [SMALL_STATE(136)] = 4165, + [SMALL_STATE(137)] = 4178, + [SMALL_STATE(138)] = 4197, + [SMALL_STATE(139)] = 4210, + [SMALL_STATE(140)] = 4223, + [SMALL_STATE(141)] = 4240, + [SMALL_STATE(142)] = 4253, + [SMALL_STATE(143)] = 4266, + [SMALL_STATE(144)] = 4283, + [SMALL_STATE(145)] = 4300, + [SMALL_STATE(146)] = 4317, + [SMALL_STATE(147)] = 4330, + [SMALL_STATE(148)] = 4349, + [SMALL_STATE(149)] = 4362, + [SMALL_STATE(150)] = 4379, + [SMALL_STATE(151)] = 4398, + [SMALL_STATE(152)] = 4417, + [SMALL_STATE(153)] = 4431, + [SMALL_STATE(154)] = 4447, + [SMALL_STATE(155)] = 4463, + [SMALL_STATE(156)] = 4479, + [SMALL_STATE(157)] = 4495, + [SMALL_STATE(158)] = 4507, + [SMALL_STATE(159)] = 4523, + [SMALL_STATE(160)] = 4537, + [SMALL_STATE(161)] = 4551, + [SMALL_STATE(162)] = 4567, + [SMALL_STATE(163)] = 4583, + [SMALL_STATE(164)] = 4599, + [SMALL_STATE(165)] = 4611, + [SMALL_STATE(166)] = 4627, + [SMALL_STATE(167)] = 4643, + [SMALL_STATE(168)] = 4653, + [SMALL_STATE(169)] = 4663, + [SMALL_STATE(170)] = 4673, + [SMALL_STATE(171)] = 4683, + [SMALL_STATE(172)] = 4693, + [SMALL_STATE(173)] = 4709, + [SMALL_STATE(174)] = 4723, + [SMALL_STATE(175)] = 4739, + [SMALL_STATE(176)] = 4753, + [SMALL_STATE(177)] = 4765, + [SMALL_STATE(178)] = 4779, + [SMALL_STATE(179)] = 4793, + [SMALL_STATE(180)] = 4806, + [SMALL_STATE(181)] = 4819, + [SMALL_STATE(182)] = 4832, + [SMALL_STATE(183)] = 4845, + [SMALL_STATE(184)] = 4858, + [SMALL_STATE(185)] = 4871, + [SMALL_STATE(186)] = 4880, + [SMALL_STATE(187)] = 4893, + [SMALL_STATE(188)] = 4906, + [SMALL_STATE(189)] = 4919, + [SMALL_STATE(190)] = 4930, + [SMALL_STATE(191)] = 4943, + [SMALL_STATE(192)] = 4956, + [SMALL_STATE(193)] = 4969, + [SMALL_STATE(194)] = 4978, + [SMALL_STATE(195)] = 4991, + [SMALL_STATE(196)] = 5004, + [SMALL_STATE(197)] = 5013, + [SMALL_STATE(198)] = 5026, + [SMALL_STATE(199)] = 5035, + [SMALL_STATE(200)] = 5048, + [SMALL_STATE(201)] = 5061, + [SMALL_STATE(202)] = 5074, + [SMALL_STATE(203)] = 5087, + [SMALL_STATE(204)] = 5100, + [SMALL_STATE(205)] = 5109, + [SMALL_STATE(206)] = 5122, + [SMALL_STATE(207)] = 5131, + [SMALL_STATE(208)] = 5144, + [SMALL_STATE(209)] = 5157, + [SMALL_STATE(210)] = 5170, + [SMALL_STATE(211)] = 5183, + [SMALL_STATE(212)] = 5192, + [SMALL_STATE(213)] = 5205, + [SMALL_STATE(214)] = 5218, + [SMALL_STATE(215)] = 5231, + [SMALL_STATE(216)] = 5244, + [SMALL_STATE(217)] = 5257, + [SMALL_STATE(218)] = 5270, + [SMALL_STATE(219)] = 5280, + [SMALL_STATE(220)] = 5290, + [SMALL_STATE(221)] = 5298, + [SMALL_STATE(222)] = 5308, + [SMALL_STATE(223)] = 5316, + [SMALL_STATE(224)] = 5324, + [SMALL_STATE(225)] = 5334, + [SMALL_STATE(226)] = 5344, + [SMALL_STATE(227)] = 5354, + [SMALL_STATE(228)] = 5364, + [SMALL_STATE(229)] = 5374, + [SMALL_STATE(230)] = 5382, + [SMALL_STATE(231)] = 5390, + [SMALL_STATE(232)] = 5400, + [SMALL_STATE(233)] = 5408, + [SMALL_STATE(234)] = 5416, + [SMALL_STATE(235)] = 5426, + [SMALL_STATE(236)] = 5436, + [SMALL_STATE(237)] = 5446, + [SMALL_STATE(238)] = 5456, + [SMALL_STATE(239)] = 5466, + [SMALL_STATE(240)] = 5476, + [SMALL_STATE(241)] = 5486, + [SMALL_STATE(242)] = 5494, + [SMALL_STATE(243)] = 5504, + [SMALL_STATE(244)] = 5514, + [SMALL_STATE(245)] = 5524, + [SMALL_STATE(246)] = 5532, + [SMALL_STATE(247)] = 5542, + [SMALL_STATE(248)] = 5552, + [SMALL_STATE(249)] = 5562, + [SMALL_STATE(250)] = 5572, + [SMALL_STATE(251)] = 5582, + [SMALL_STATE(252)] = 5590, + [SMALL_STATE(253)] = 5598, + [SMALL_STATE(254)] = 5608, + [SMALL_STATE(255)] = 5618, + [SMALL_STATE(256)] = 5628, + [SMALL_STATE(257)] = 5638, + [SMALL_STATE(258)] = 5648, + [SMALL_STATE(259)] = 5658, + [SMALL_STATE(260)] = 5668, + [SMALL_STATE(261)] = 5678, + [SMALL_STATE(262)] = 5688, + [SMALL_STATE(263)] = 5698, + [SMALL_STATE(264)] = 5708, + [SMALL_STATE(265)] = 5718, + [SMALL_STATE(266)] = 5728, + [SMALL_STATE(267)] = 5738, + [SMALL_STATE(268)] = 5748, + [SMALL_STATE(269)] = 5758, + [SMALL_STATE(270)] = 5768, + [SMALL_STATE(271)] = 5778, + [SMALL_STATE(272)] = 5788, + [SMALL_STATE(273)] = 5798, + [SMALL_STATE(274)] = 5805, + [SMALL_STATE(275)] = 5812, + [SMALL_STATE(276)] = 5819, + [SMALL_STATE(277)] = 5826, + [SMALL_STATE(278)] = 5833, + [SMALL_STATE(279)] = 5840, + [SMALL_STATE(280)] = 5847, + [SMALL_STATE(281)] = 5854, + [SMALL_STATE(282)] = 5861, + [SMALL_STATE(283)] = 5868, + [SMALL_STATE(284)] = 5875, + [SMALL_STATE(285)] = 5882, + [SMALL_STATE(286)] = 5889, + [SMALL_STATE(287)] = 5896, + [SMALL_STATE(288)] = 5903, + [SMALL_STATE(289)] = 5910, + [SMALL_STATE(290)] = 5917, + [SMALL_STATE(291)] = 5924, + [SMALL_STATE(292)] = 5931, + [SMALL_STATE(293)] = 5938, + [SMALL_STATE(294)] = 5945, + [SMALL_STATE(295)] = 5952, + [SMALL_STATE(296)] = 5959, + [SMALL_STATE(297)] = 5966, + [SMALL_STATE(298)] = 5973, + [SMALL_STATE(299)] = 5980, + [SMALL_STATE(300)] = 5987, + [SMALL_STATE(301)] = 5994, + [SMALL_STATE(302)] = 6001, + [SMALL_STATE(303)] = 6008, + [SMALL_STATE(304)] = 6015, + [SMALL_STATE(305)] = 6022, + [SMALL_STATE(306)] = 6029, + [SMALL_STATE(307)] = 6036, + [SMALL_STATE(308)] = 6043, + [SMALL_STATE(309)] = 6050, + [SMALL_STATE(310)] = 6057, + [SMALL_STATE(311)] = 6064, + [SMALL_STATE(312)] = 6071, + [SMALL_STATE(313)] = 6078, + [SMALL_STATE(314)] = 6085, + [SMALL_STATE(315)] = 6092, + [SMALL_STATE(316)] = 6099, + [SMALL_STATE(317)] = 6106, + [SMALL_STATE(318)] = 6113, + [SMALL_STATE(319)] = 6120, + [SMALL_STATE(320)] = 6127, + [SMALL_STATE(321)] = 6134, + [SMALL_STATE(322)] = 6141, + [SMALL_STATE(323)] = 6148, + [SMALL_STATE(324)] = 6155, + [SMALL_STATE(325)] = 6162, + [SMALL_STATE(326)] = 6169, + [SMALL_STATE(327)] = 6176, + [SMALL_STATE(328)] = 6183, + [SMALL_STATE(329)] = 6190, + [SMALL_STATE(330)] = 6197, + [SMALL_STATE(331)] = 6204, + [SMALL_STATE(332)] = 6211, + [SMALL_STATE(333)] = 6218, + [SMALL_STATE(334)] = 6225, + [SMALL_STATE(335)] = 6232, + [SMALL_STATE(336)] = 6239, + [SMALL_STATE(337)] = 6246, + [SMALL_STATE(338)] = 6253, + [SMALL_STATE(339)] = 6260, + [SMALL_STATE(340)] = 6267, + [SMALL_STATE(341)] = 6274, + [SMALL_STATE(342)] = 6281, + [SMALL_STATE(343)] = 6288, + [SMALL_STATE(344)] = 6295, +}; + +static const TSParseActionEntry ts_parse_actions[] = { + [0] = {.entry = {.count = 0, .reusable = false}}, + [1] = {.entry = {.count = 1, .reusable = false}}, RECOVER(), + [3] = {.entry = {.count = 1, .reusable = true}}, SHIFT_EXTRA(), + [5] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_source_file, 0, 0, 0), + [7] = {.entry = {.count = 1, .reusable = true}}, SHIFT(66), + [9] = {.entry = {.count = 1, .reusable = true}}, SHIFT(328), + [11] = {.entry = {.count = 1, .reusable = true}}, SHIFT(316), + [13] = {.entry = {.count = 1, .reusable = true}}, SHIFT(101), + [15] = {.entry = {.count = 1, .reusable = true}}, SHIFT(205), + [17] = {.entry = {.count = 1, .reusable = true}}, SHIFT(249), + [19] = {.entry = {.count = 1, .reusable = true}}, SHIFT(254), + [21] = {.entry = {.count = 1, .reusable = true}}, SHIFT(221), + [23] = {.entry = {.count = 1, .reusable = true}}, SHIFT(228), + [25] = {.entry = {.count = 1, .reusable = true}}, SHIFT(255), + [27] = {.entry = {.count = 1, .reusable = true}}, SHIFT(224), + [29] = {.entry = {.count = 1, .reusable = true}}, SHIFT(30), + [31] = {.entry = {.count = 1, .reusable = false}}, SHIFT(216), + [33] = {.entry = {.count = 1, .reusable = true}}, SHIFT(225), + [35] = {.entry = {.count = 1, .reusable = false}}, SHIFT(270), + [37] = {.entry = {.count = 1, .reusable = false}}, SHIFT(263), + [39] = {.entry = {.count = 1, .reusable = true}}, SHIFT(82), + [41] = {.entry = {.count = 1, .reusable = false}}, SHIFT(264), + [43] = {.entry = {.count = 1, .reusable = false}}, SHIFT(265), + [45] = {.entry = {.count = 1, .reusable = false}}, SHIFT(35), + [47] = {.entry = {.count = 1, .reusable = false}}, SHIFT(38), + [49] = {.entry = {.count = 1, .reusable = false}}, SHIFT(236), + [51] = {.entry = {.count = 1, .reusable = false}}, SHIFT(336), + [53] = {.entry = {.count = 1, .reusable = false}}, SHIFT(343), + [55] = {.entry = {.count = 1, .reusable = false}}, SHIFT(241), + [57] = {.entry = {.count = 1, .reusable = false}}, SHIFT(95), + [59] = {.entry = {.count = 1, .reusable = false}}, SHIFT(113), + [61] = {.entry = {.count = 1, .reusable = false}}, SHIFT(157), + [63] = {.entry = {.count = 1, .reusable = true}}, SHIFT(69), + [65] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(30), + [68] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(216), + [71] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(225), + [74] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(270), + [77] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(263), + [80] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), + [82] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(264), + [85] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(265), + [88] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(35), + [91] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(38), + [94] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(236), + [97] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(336), + [100] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(343), + [103] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(241), + [106] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(95), + [109] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(113), + [112] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_message_body_repeat1, 2, 0, 0), SHIFT_REPEAT(157), + [115] = {.entry = {.count = 1, .reusable = true}}, SHIFT(28), + [117] = {.entry = {.count = 1, .reusable = true}}, SHIFT(27), + [119] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum_body, 3, 0, 0), + [121] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_enum_body, 3, 0, 0), + [123] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_option, 5, 0, 0), + [125] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_option, 5, 0, 0), + [127] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_extensions, 3, 0, 0), + [129] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_extensions, 3, 0, 0), + [131] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_oneof, 4, 0, 0), + [133] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_oneof, 4, 0, 0), + [135] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_group, 5, 0, 0), + [137] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_group, 5, 0, 0), + [139] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_oneof, 5, 0, 0), + [141] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_oneof, 5, 0, 0), + [143] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field, 5, 0, 0), + [145] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_field, 5, 0, 0), + [147] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_group, 6, 0, 0), + [149] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_group, 6, 0, 0), + [151] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field, 6, 0, 0), + [153] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_field, 6, 0, 0), + [155] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field, 7, 0, 0), + [157] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_field, 7, 0, 0), + [159] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field, 8, 0, 0), + [161] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_field, 8, 0, 0), + [163] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field, 10, 0, 0), + [165] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_field, 10, 0, 0), + [167] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_map_field, 10, 0, 0), + [169] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_map_field, 10, 0, 0), + [171] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_map_field, 13, 0, 0), + [173] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_map_field, 13, 0, 0), + [175] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum, 3, 0, 0), + [177] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_enum, 3, 0, 0), + [179] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_message, 3, 0, 0), + [181] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_message, 3, 0, 0), + [183] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_extend, 3, 0, 0), + [185] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_extend, 3, 0, 0), + [187] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum, 4, 0, 0), + [189] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_enum, 4, 0, 0), + [191] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_message, 4, 0, 0), + [193] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_message, 4, 0, 0), + [195] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum_body, 2, 0, 0), + [197] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_enum_body, 2, 0, 0), + [199] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_message_body, 2, 0, 0), + [201] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_message_body, 2, 0, 0), + [203] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_message_body, 3, 0, 0), + [205] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_message_body, 3, 0, 0), + [207] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_reserved, 3, 0, 0), + [209] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_reserved, 3, 0, 0), + [211] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_empty_statement, 1, 0, 0), + [213] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_empty_statement, 1, 0, 0), + [215] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field, 9, 0, 0), + [217] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_field, 9, 0, 0), + [219] = {.entry = {.count = 1, .reusable = true}}, SHIFT(55), + [221] = {.entry = {.count = 1, .reusable = false}}, SHIFT(188), + [223] = {.entry = {.count = 1, .reusable = true}}, SHIFT(10), + [225] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_oneof_repeat1, 2, 0, 0), SHIFT_REPEAT(55), + [228] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_oneof_repeat1, 2, 0, 0), SHIFT_REPEAT(188), + [231] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_oneof_repeat1, 2, 0, 0), SHIFT_REPEAT(225), + [234] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_oneof_repeat1, 2, 0, 0), + [236] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_oneof_repeat1, 2, 0, 0), SHIFT_REPEAT(241), + [239] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_oneof_repeat1, 2, 0, 0), SHIFT_REPEAT(157), + [242] = {.entry = {.count = 1, .reusable = true}}, SHIFT(12), + [244] = {.entry = {.count = 1, .reusable = false}}, SHIFT(43), + [246] = {.entry = {.count = 1, .reusable = false}}, SHIFT(239), + [248] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field_number, 1, 0, 0), + [250] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_field_number, 1, 0, 0), + [252] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_oneof_field, 4, 0, 0), + [254] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_oneof_field, 4, 0, 0), + [256] = {.entry = {.count = 1, .reusable = true}}, SHIFT(125), + [258] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_int_lit, 1, 0, 0), + [260] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_int_lit, 1, 0, 0), + [262] = {.entry = {.count = 1, .reusable = true}}, SHIFT(153), + [264] = {.entry = {.count = 1, .reusable = true}}, SHIFT(144), + [266] = {.entry = {.count = 1, .reusable = true}}, SHIFT(50), + [268] = {.entry = {.count = 1, .reusable = true}}, SHIFT(51), + [270] = {.entry = {.count = 1, .reusable = false}}, SHIFT(89), + [272] = {.entry = {.count = 1, .reusable = false}}, SHIFT(110), + [274] = {.entry = {.count = 1, .reusable = false}}, SHIFT(128), + [276] = {.entry = {.count = 1, .reusable = true}}, SHIFT(128), + [278] = {.entry = {.count = 1, .reusable = false}}, SHIFT(112), + [280] = {.entry = {.count = 1, .reusable = true}}, SHIFT(173), + [282] = {.entry = {.count = 1, .reusable = true}}, SHIFT(152), + [284] = {.entry = {.count = 1, .reusable = true}}, SHIFT(46), + [286] = {.entry = {.count = 1, .reusable = true}}, SHIFT(48), + [288] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_source_file, 2, 0, 0), + [290] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_source_file, 1, 0, 0), + [292] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), + [294] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), SHIFT_REPEAT(66), + [297] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), SHIFT_REPEAT(101), + [300] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), SHIFT_REPEAT(205), + [303] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), SHIFT_REPEAT(249), + [306] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), SHIFT_REPEAT(254), + [309] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), SHIFT_REPEAT(221), + [312] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), SHIFT_REPEAT(228), + [315] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), SHIFT_REPEAT(255), + [318] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_source_file_repeat1, 2, 0, 0), SHIFT_REPEAT(224), + [321] = {.entry = {.count = 1, .reusable = true}}, SHIFT(116), + [323] = {.entry = {.count = 1, .reusable = true}}, SHIFT(141), + [325] = {.entry = {.count = 1, .reusable = false}}, SHIFT(109), + [327] = {.entry = {.count = 1, .reusable = true}}, SHIFT(109), + [329] = {.entry = {.count = 1, .reusable = true}}, SHIFT(127), + [331] = {.entry = {.count = 1, .reusable = true}}, SHIFT(47), + [333] = {.entry = {.count = 1, .reusable = true}}, SHIFT(139), + [335] = {.entry = {.count = 1, .reusable = true}}, SHIFT(54), + [337] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_oneof_field, 7, 0, 0), + [339] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_oneof_field, 7, 0, 0), + [341] = {.entry = {.count = 1, .reusable = true}}, SHIFT(118), + [343] = {.entry = {.count = 1, .reusable = true}}, SHIFT(334), + [345] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_edition, 4, 0, 2), + [347] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_import, 4, 0, 3), + [349] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_package, 3, 0, 0), + [351] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_service, 5, 0, 0), + [353] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_syntax, 4, 0, 0), + [355] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_import, 3, 0, 1), + [357] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym__option_name_repeat1, 2, 0, 0), + [359] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym__option_name_repeat1, 2, 0, 0), SHIFT_REPEAT(292), + [362] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_service, 4, 0, 0), + [364] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_full_ident, 2, 0, 0), + [366] = {.entry = {.count = 1, .reusable = true}}, SHIFT(292), + [368] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_enum_body_repeat1, 2, 0, 0), SHIFT_REPEAT(124), + [371] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_enum_body_repeat1, 2, 0, 0), SHIFT_REPEAT(215), + [374] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_enum_body_repeat1, 2, 0, 0), + [376] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_enum_body_repeat1, 2, 0, 0), SHIFT_REPEAT(98), + [379] = {.entry = {.count = 2, .reusable = false}}, REDUCE(aux_sym_enum_body_repeat1, 2, 0, 0), SHIFT_REPEAT(283), + [382] = {.entry = {.count = 1, .reusable = true}}, SHIFT(124), + [384] = {.entry = {.count = 1, .reusable = false}}, SHIFT(215), + [386] = {.entry = {.count = 1, .reusable = true}}, SHIFT(81), + [388] = {.entry = {.count = 1, .reusable = false}}, SHIFT(98), + [390] = {.entry = {.count = 1, .reusable = false}}, SHIFT(283), + [392] = {.entry = {.count = 1, .reusable = true}}, SHIFT(72), + [394] = {.entry = {.count = 1, .reusable = true}}, SHIFT(26), + [396] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_full_ident, 1, 0, 0), + [398] = {.entry = {.count = 1, .reusable = true}}, SHIFT(7), + [400] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_string_repeat3, 2, 0, 0), + [402] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_string_repeat3, 2, 0, 0), SHIFT_REPEAT(173), + [405] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_string_repeat3, 2, 0, 0), SHIFT_REPEAT(152), + [408] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_string, 1, 0, 0), + [410] = {.entry = {.count = 1, .reusable = true}}, SHIFT(186), + [412] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_service_repeat1, 2, 0, 0), SHIFT_REPEAT(66), + [415] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_service_repeat1, 2, 0, 0), SHIFT_REPEAT(205), + [418] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_service_repeat1, 2, 0, 0), + [420] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_service_repeat1, 2, 0, 0), SHIFT_REPEAT(242), + [423] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_string_repeat3, 3, 0, 0), + [425] = {.entry = {.count = 1, .reusable = true}}, SHIFT(73), + [427] = {.entry = {.count = 1, .reusable = true}}, SHIFT(242), + [429] = {.entry = {.count = 1, .reusable = true}}, SHIFT(83), + [431] = {.entry = {.count = 1, .reusable = true}}, SHIFT(172), + [433] = {.entry = {.count = 1, .reusable = true}}, SHIFT(168), + [435] = {.entry = {.count = 1, .reusable = true}}, SHIFT(169), + [437] = {.entry = {.count = 1, .reusable = true}}, SHIFT(170), + [439] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_rpc_repeat1, 2, 0, 0), SHIFT_REPEAT(66), + [442] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_rpc_repeat1, 2, 0, 0), SHIFT_REPEAT(205), + [445] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_rpc_repeat1, 2, 0, 0), + [447] = {.entry = {.count = 1, .reusable = true}}, SHIFT(171), + [449] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_bool, 1, 0, 0), + [451] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_constant, 2, 0, 0), + [453] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_constant, 1, 0, 0), + [455] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_block_lit, 2, 0, 0), + [457] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_block_lit, 3, 0, 0), + [459] = {.entry = {.count = 1, .reusable = true}}, SHIFT(111), + [461] = {.entry = {.count = 1, .reusable = true}}, SHIFT(250), + [463] = {.entry = {.count = 1, .reusable = true}}, SHIFT(208), + [465] = {.entry = {.count = 1, .reusable = true}}, SHIFT(185), + [467] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 6, 0, 0), + [469] = {.entry = {.count = 1, .reusable = false}}, SHIFT(163), + [471] = {.entry = {.count = 1, .reusable = true}}, SHIFT(193), + [473] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 7, 0, 0), + [475] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum_field, 8, 0, 0), + [477] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_enum_field, 8, 0, 0), + [479] = {.entry = {.count = 1, .reusable = true}}, SHIFT(198), + [481] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 4, 0, 0), + [483] = {.entry = {.count = 1, .reusable = true}}, SHIFT(175), + [485] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum_field, 4, 0, 0), + [487] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_enum_field, 4, 0, 0), + [489] = {.entry = {.count = 1, .reusable = true}}, SHIFT(39), + [491] = {.entry = {.count = 1, .reusable = false}}, SHIFT(39), + [493] = {.entry = {.count = 1, .reusable = true}}, SHIFT(196), + [495] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 8, 0, 0), + [497] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum_field, 9, 0, 0), + [499] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_enum_field, 9, 0, 0), + [501] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum_field, 7, 0, 0), + [503] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_enum_field, 7, 0, 0), + [505] = {.entry = {.count = 1, .reusable = true}}, SHIFT(211), + [507] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 5, 0, 0), + [509] = {.entry = {.count = 1, .reusable = true}}, SHIFT(206), + [511] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 3, 0, 0), + [513] = {.entry = {.count = 1, .reusable = true}}, SHIFT(204), + [515] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 2, 0, 0), + [517] = {.entry = {.count = 1, .reusable = true}}, SHIFT(230), + [519] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum_field, 5, 0, 0), + [521] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_enum_field, 5, 0, 0), + [523] = {.entry = {.count = 1, .reusable = false}}, SHIFT(155), + [525] = {.entry = {.count = 1, .reusable = false}}, SHIFT(165), + [527] = {.entry = {.count = 1, .reusable = false}}, SHIFT(94), + [529] = {.entry = {.count = 1, .reusable = true}}, SHIFT(160), + [531] = {.entry = {.count = 1, .reusable = false}}, SHIFT_EXTRA(), + [533] = {.entry = {.count = 1, .reusable = true}}, SHIFT(114), + [535] = {.entry = {.count = 1, .reusable = true}}, SHIFT(226), + [537] = {.entry = {.count = 1, .reusable = true}}, SHIFT(41), + [539] = {.entry = {.count = 1, .reusable = true}}, SHIFT(157), + [541] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_message_or_enum_type, 1, 0, 0), + [543] = {.entry = {.count = 1, .reusable = true}}, SHIFT(289), + [545] = {.entry = {.count = 1, .reusable = false}}, SHIFT(97), + [547] = {.entry = {.count = 1, .reusable = true}}, SHIFT(177), + [549] = {.entry = {.count = 1, .reusable = true}}, SHIFT(178), + [551] = {.entry = {.count = 1, .reusable = true}}, SHIFT(115), + [553] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_message_or_enum_type, 2, 0, 0), + [555] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_rpc, 10, 0, 0), + [557] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_rpc, 11, 0, 0), + [559] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_rpc, 12, 0, 0), + [561] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_rpc, 13, 0, 0), + [563] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_rpc, 14, 0, 0), + [565] = {.entry = {.count = 1, .reusable = true}}, SHIFT(159), + [567] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 2, 0, 0), SHIFT_REPEAT(226), + [570] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 2, 0, 0), SHIFT_REPEAT(41), + [573] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_message_or_enum_type, 3, 0, 0), + [575] = {.entry = {.count = 1, .reusable = false}}, REDUCE(aux_sym_string_repeat1, 2, 0, 0), + [577] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_string_repeat1, 2, 0, 0), SHIFT_REPEAT(177), + [580] = {.entry = {.count = 1, .reusable = false}}, REDUCE(aux_sym_string_repeat2, 2, 0, 0), + [582] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_string_repeat2, 2, 0, 0), SHIFT_REPEAT(178), + [585] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_ranges, 1, 0, 0), + [587] = {.entry = {.count = 1, .reusable = true}}, SHIFT(119), + [589] = {.entry = {.count = 1, .reusable = true}}, SHIFT(158), + [591] = {.entry = {.count = 1, .reusable = true}}, SHIFT(304), + [593] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_enum_field_repeat1, 2, 0, 0), SHIFT_REPEAT(158), + [596] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_enum_field_repeat1, 2, 0, 0), + [598] = {.entry = {.count = 1, .reusable = true}}, SHIFT(61), + [600] = {.entry = {.count = 1, .reusable = true}}, SHIFT(162), + [602] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field_options, 2, 0, 0), + [604] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_reserved_field_names, 1, 0, 0), + [606] = {.entry = {.count = 1, .reusable = true}}, SHIFT(306), + [608] = {.entry = {.count = 1, .reusable = true}}, SHIFT(134), + [610] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_range, 1, 0, 0), + [612] = {.entry = {.count = 1, .reusable = true}}, SHIFT(143), + [614] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym__option_name, 2, 0, 0), + [616] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_field_options_repeat1, 2, 0, 0), SHIFT_REPEAT(162), + [619] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_field_options_repeat1, 2, 0, 0), + [621] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_reserved_field_names_repeat1, 2, 0, 0), + [623] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_reserved_field_names_repeat1, 2, 0, 0), SHIFT_REPEAT(306), + [626] = {.entry = {.count = 1, .reusable = true}}, SHIFT(324), + [628] = {.entry = {.count = 1, .reusable = true}}, SHIFT(121), + [630] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat2, 9, 0, 0), + [632] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat1, 2, 0, 0), SHIFT_REPEAT(61), + [635] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_block_lit_repeat1, 2, 0, 0), + [637] = {.entry = {.count = 1, .reusable = true}}, SHIFT(285), + [639] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_reserved_field_names, 2, 0, 0), + [641] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym__option_name, 1, 0, 0), + [643] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_ranges, 2, 0, 0), + [645] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field_options, 1, 0, 0), + [647] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym__option_name, 4, 0, 0), + [649] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym__option_name, 3, 0, 0), + [651] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_ranges_repeat1, 2, 0, 0), + [653] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_ranges_repeat1, 2, 0, 0), SHIFT_REPEAT(119), + [656] = {.entry = {.count = 1, .reusable = true}}, SHIFT(19), + [658] = {.entry = {.count = 1, .reusable = true}}, SHIFT(147), + [660] = {.entry = {.count = 1, .reusable = true}}, SHIFT(13), + [662] = {.entry = {.count = 1, .reusable = true}}, SHIFT(137), + [664] = {.entry = {.count = 1, .reusable = true}}, SHIFT(294), + [666] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum_value_option, 3, 0, 0), + [668] = {.entry = {.count = 1, .reusable = true}}, SHIFT(309), + [670] = {.entry = {.count = 1, .reusable = true}}, SHIFT(164), + [672] = {.entry = {.count = 1, .reusable = true}}, SHIFT(89), + [674] = {.entry = {.count = 1, .reusable = true}}, SHIFT(252), + [676] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_range, 3, 0, 0), + [678] = {.entry = {.count = 1, .reusable = true}}, SHIFT(146), + [680] = {.entry = {.count = 1, .reusable = true}}, SHIFT(156), + [682] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_field_option, 3, 0, 0), + [684] = {.entry = {.count = 1, .reusable = true}}, SHIFT(6), + [686] = {.entry = {.count = 1, .reusable = true}}, SHIFT(15), + [688] = {.entry = {.count = 1, .reusable = true}}, SHIFT(122), + [690] = {.entry = {.count = 1, .reusable = true}}, SHIFT(176), + [692] = {.entry = {.count = 1, .reusable = true}}, SHIFT(167), + [694] = {.entry = {.count = 1, .reusable = true}}, SHIFT(102), + [696] = {.entry = {.count = 1, .reusable = true}}, SHIFT(103), + [698] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_type, 1, 0, 0), + [700] = {.entry = {.count = 1, .reusable = true}}, SHIFT(296), + [702] = {.entry = {.count = 1, .reusable = true}}, SHIFT(86), + [704] = {.entry = {.count = 1, .reusable = true}}, SHIFT(2), + [706] = {.entry = {.count = 1, .reusable = true}}, SHIFT(313), + [708] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_message_or_enum_type_repeat1, 2, 0, 0), SHIFT_REPEAT(282), + [711] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_message_name, 1, 0, 0), + [713] = {.entry = {.count = 1, .reusable = true}}, SHIFT(227), + [715] = {.entry = {.count = 1, .reusable = true}}, SHIFT(248), + [717] = {.entry = {.count = 1, .reusable = true}}, SHIFT(131), + [719] = {.entry = {.count = 1, .reusable = true}}, SHIFT(154), + [721] = {.entry = {.count = 1, .reusable = true}}, SHIFT(88), + [723] = {.entry = {.count = 1, .reusable = true}}, SHIFT(16), + [725] = {.entry = {.count = 1, .reusable = true}}, SHIFT(117), + [727] = {.entry = {.count = 1, .reusable = true}}, SHIFT(266), + [729] = {.entry = {.count = 1, .reusable = true}}, SHIFT(267), + [731] = {.entry = {.count = 1, .reusable = true}}, SHIFT(105), + [733] = {.entry = {.count = 1, .reusable = true}}, SHIFT(287), + [735] = {.entry = {.count = 1, .reusable = true}}, SHIFT(315), + [737] = {.entry = {.count = 1, .reusable = true}}, SHIFT(126), + [739] = {.entry = {.count = 1, .reusable = true}}, SHIFT(323), + [741] = {.entry = {.count = 1, .reusable = true}}, SHIFT(140), + [743] = {.entry = {.count = 1, .reusable = true}}, SHIFT(272), + [745] = {.entry = {.count = 1, .reusable = true}}, SHIFT(214), + [747] = {.entry = {.count = 1, .reusable = true}}, SHIFT(17), + [749] = {.entry = {.count = 1, .reusable = true}}, SHIFT(32), + [751] = {.entry = {.count = 1, .reusable = true}}, SHIFT(130), + [753] = {.entry = {.count = 1, .reusable = true}}, SHIFT(68), + [755] = {.entry = {.count = 1, .reusable = true}}, SHIFT(123), + [757] = {.entry = {.count = 1, .reusable = true}}, SHIFT(9), + [759] = {.entry = {.count = 1, .reusable = true}}, SHIFT(20), + [761] = {.entry = {.count = 1, .reusable = true}}, SHIFT(312), + [763] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_message_or_enum_type_repeat1, 2, 0, 0), + [765] = {.entry = {.count = 1, .reusable = true}}, SHIFT(149), + [767] = {.entry = {.count = 1, .reusable = true}}, SHIFT(151), + [769] = {.entry = {.count = 1, .reusable = true}}, SHIFT(90), + [771] = {.entry = {.count = 1, .reusable = true}}, SHIFT(314), + [773] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_enum_name, 1, 0, 0), + [775] = {.entry = {.count = 1, .reusable = true}}, SHIFT(67), + [777] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_rpc_name, 1, 0, 0), + [779] = {.entry = {.count = 1, .reusable = true}}, SHIFT(150), + [781] = {.entry = {.count = 1, .reusable = true}}, SHIFT(280), + [783] = {.entry = {.count = 1, .reusable = true}}, SHIFT(290), + [785] = {.entry = {.count = 1, .reusable = true}}, SHIFT(40), + [787] = {.entry = {.count = 1, .reusable = true}}, SHIFT(129), + [789] = {.entry = {.count = 1, .reusable = true}}, SHIFT(77), + [791] = {.entry = {.count = 1, .reusable = true}}, SHIFT(148), + [793] = {.entry = {.count = 1, .reusable = true}}, SHIFT(136), + [795] = {.entry = {.count = 1, .reusable = true}}, SHIFT(330), + [797] = {.entry = {.count = 1, .reusable = true}}, SHIFT(220), + [799] = {.entry = {.count = 1, .reusable = true}}, SHIFT(31), + [801] = {.entry = {.count = 1, .reusable = true}}, SHIFT(64), + [803] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_service_name, 1, 0, 0), + [805] = {.entry = {.count = 1, .reusable = true}}, SHIFT(60), + [807] = {.entry = {.count = 1, .reusable = true}}, SHIFT(100), + [809] = {.entry = {.count = 1, .reusable = true}}, SHIFT(291), + [811] = {.entry = {.count = 1, .reusable = true}}, SHIFT(76), + [813] = {.entry = {.count = 1, .reusable = true}}, SHIFT(120), + [815] = {.entry = {.count = 1, .reusable = true}}, SHIFT(275), + [817] = {.entry = {.count = 1, .reusable = true}}, SHIFT(245), + [819] = {.entry = {.count = 1, .reusable = true}}, SHIFT(59), + [821] = {.entry = {.count = 1, .reusable = true}}, SHIFT(238), + [823] = {.entry = {.count = 1, .reusable = true}}, ACCEPT_INPUT(), + [825] = {.entry = {.count = 1, .reusable = true}}, SHIFT(301), + [827] = {.entry = {.count = 1, .reusable = true}}, SHIFT(135), + [829] = {.entry = {.count = 1, .reusable = true}}, SHIFT(133), + [831] = {.entry = {.count = 1, .reusable = true}}, SHIFT(145), + [833] = {.entry = {.count = 1, .reusable = true}}, SHIFT(138), + [835] = {.entry = {.count = 1, .reusable = true}}, SHIFT(29), + [837] = {.entry = {.count = 1, .reusable = true}}, SHIFT(8), + [839] = {.entry = {.count = 1, .reusable = true}}, SHIFT(56), + [841] = {.entry = {.count = 1, .reusable = true}}, SHIFT(166), + [843] = {.entry = {.count = 1, .reusable = true}}, SHIFT(132), + [845] = {.entry = {.count = 1, .reusable = true}}, SHIFT(18), + [847] = {.entry = {.count = 1, .reusable = true}}, SHIFT(307), + [849] = {.entry = {.count = 1, .reusable = true}}, SHIFT(52), + [851] = {.entry = {.count = 1, .reusable = true}}, SHIFT(71), + [853] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_key_type, 1, 0, 0), + [855] = {.entry = {.count = 1, .reusable = true}}, SHIFT(49), + [857] = {.entry = {.count = 1, .reusable = true}}, SHIFT(281), + [859] = {.entry = {.count = 1, .reusable = true}}, SHIFT(240), + [861] = {.entry = {.count = 1, .reusable = true}}, SHIFT(62), + [863] = {.entry = {.count = 1, .reusable = true}}, SHIFT(63), + [865] = {.entry = {.count = 1, .reusable = true}}, SHIFT(57), + [867] = {.entry = {.count = 1, .reusable = true}}, SHIFT(293), + [869] = {.entry = {.count = 1, .reusable = true}}, SHIFT(329), + [871] = {.entry = {.count = 1, .reusable = true}}, SHIFT(65), + [873] = {.entry = {.count = 1, .reusable = true}}, SHIFT(58), +}; + +#ifdef __cplusplus +extern "C" { +#endif +#ifdef TREE_SITTER_HIDE_SYMBOLS +#define TS_PUBLIC +#elif defined(_WIN32) +#define TS_PUBLIC __declspec(dllexport) +#else +#define TS_PUBLIC __attribute__((visibility("default"))) +#endif + +TS_PUBLIC const TSLanguage *tree_sitter_proto(void) { + static const TSLanguage language = { + .version = LANGUAGE_VERSION, + .symbol_count = SYMBOL_COUNT, + .alias_count = ALIAS_COUNT, + .token_count = TOKEN_COUNT, + .external_token_count = EXTERNAL_TOKEN_COUNT, + .state_count = STATE_COUNT, + .large_state_count = LARGE_STATE_COUNT, + .production_id_count = PRODUCTION_ID_COUNT, + .field_count = FIELD_COUNT, + .max_alias_sequence_length = MAX_ALIAS_SEQUENCE_LENGTH, + .parse_table = &ts_parse_table[0][0], + .small_parse_table = ts_small_parse_table, + .small_parse_table_map = ts_small_parse_table_map, + .parse_actions = ts_parse_actions, + .symbol_names = ts_symbol_names, + .field_names = ts_field_names, + .field_map_slices = ts_field_map_slices, + .field_map_entries = ts_field_map_entries, + .symbol_metadata = ts_symbol_metadata, + .public_symbol_map = ts_symbol_map, + .alias_map = ts_non_terminal_alias_map, + .alias_sequences = &ts_alias_sequences[0][0], + .lex_modes = ts_lex_modes, + .lex_fn = ts_lex, + .primary_state_ids = ts_primary_state_ids, + }; + return &language; +} +#ifdef __cplusplus +} +#endif diff --git a/gitnexus/vendor/tree-sitter-proto/src/tree_sitter/alloc.h b/gitnexus/vendor/tree-sitter-proto/src/tree_sitter/alloc.h new file mode 100644 index 0000000000..1abdd12015 --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/src/tree_sitter/alloc.h @@ -0,0 +1,54 @@ +#ifndef TREE_SITTER_ALLOC_H_ +#define TREE_SITTER_ALLOC_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include + +// Allow clients to override allocation functions +#ifdef TREE_SITTER_REUSE_ALLOCATOR + +extern void *(*ts_current_malloc)(size_t size); +extern void *(*ts_current_calloc)(size_t count, size_t size); +extern void *(*ts_current_realloc)(void *ptr, size_t size); +extern void (*ts_current_free)(void *ptr); + +#ifndef ts_malloc +#define ts_malloc ts_current_malloc +#endif +#ifndef ts_calloc +#define ts_calloc ts_current_calloc +#endif +#ifndef ts_realloc +#define ts_realloc ts_current_realloc +#endif +#ifndef ts_free +#define ts_free ts_current_free +#endif + +#else + +#ifndef ts_malloc +#define ts_malloc malloc +#endif +#ifndef ts_calloc +#define ts_calloc calloc +#endif +#ifndef ts_realloc +#define ts_realloc realloc +#endif +#ifndef ts_free +#define ts_free free +#endif + +#endif + +#ifdef __cplusplus +} +#endif + +#endif // TREE_SITTER_ALLOC_H_ diff --git a/gitnexus/vendor/tree-sitter-proto/src/tree_sitter/array.h b/gitnexus/vendor/tree-sitter-proto/src/tree_sitter/array.h new file mode 100644 index 0000000000..a17a574f04 --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/src/tree_sitter/array.h @@ -0,0 +1,291 @@ +#ifndef TREE_SITTER_ARRAY_H_ +#define TREE_SITTER_ARRAY_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include "./alloc.h" + +#include +#include +#include +#include +#include + +#ifdef _MSC_VER +#pragma warning(push) +#pragma warning(disable : 4101) +#elif defined(__GNUC__) || defined(__clang__) +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wunused-variable" +#endif + +#define Array(T) \ + struct { \ + T *contents; \ + uint32_t size; \ + uint32_t capacity; \ + } + +/// Initialize an array. +#define array_init(self) \ + ((self)->size = 0, (self)->capacity = 0, (self)->contents = NULL) + +/// Create an empty array. +#define array_new() \ + { NULL, 0, 0 } + +/// Get a pointer to the element at a given `index` in the array. +#define array_get(self, _index) \ + (assert((uint32_t)(_index) < (self)->size), &(self)->contents[_index]) + +/// Get a pointer to the first element in the array. +#define array_front(self) array_get(self, 0) + +/// Get a pointer to the last element in the array. +#define array_back(self) array_get(self, (self)->size - 1) + +/// Clear the array, setting its size to zero. Note that this does not free any +/// memory allocated for the array's contents. +#define array_clear(self) ((self)->size = 0) + +/// Reserve `new_capacity` elements of space in the array. If `new_capacity` is +/// less than the array's current capacity, this function has no effect. +#define array_reserve(self, new_capacity) \ + _array__reserve((Array *)(self), array_elem_size(self), new_capacity) + +/// Free any memory allocated for this array. Note that this does not free any +/// memory allocated for the array's contents. +#define array_delete(self) _array__delete((Array *)(self)) + +/// Push a new `element` onto the end of the array. +#define array_push(self, element) \ + (_array__grow((Array *)(self), 1, array_elem_size(self)), \ + (self)->contents[(self)->size++] = (element)) + +/// Increase the array's size by `count` elements. +/// New elements are zero-initialized. +#define array_grow_by(self, count) \ + do { \ + if ((count) == 0) break; \ + _array__grow((Array *)(self), count, array_elem_size(self)); \ + memset((self)->contents + (self)->size, 0, (count) * array_elem_size(self)); \ + (self)->size += (count); \ + } while (0) + +/// Append all elements from one array to the end of another. +#define array_push_all(self, other) \ + array_extend((self), (other)->size, (other)->contents) + +/// Append `count` elements to the end of the array, reading their values from the +/// `contents` pointer. +#define array_extend(self, count, contents) \ + _array__splice( \ + (Array *)(self), array_elem_size(self), (self)->size, \ + 0, count, contents \ + ) + +/// Remove `old_count` elements from the array starting at the given `index`. At +/// the same index, insert `new_count` new elements, reading their values from the +/// `new_contents` pointer. +#define array_splice(self, _index, old_count, new_count, new_contents) \ + _array__splice( \ + (Array *)(self), array_elem_size(self), _index, \ + old_count, new_count, new_contents \ + ) + +/// Insert one `element` into the array at the given `index`. +#define array_insert(self, _index, element) \ + _array__splice((Array *)(self), array_elem_size(self), _index, 0, 1, &(element)) + +/// Remove one element from the array at the given `index`. +#define array_erase(self, _index) \ + _array__erase((Array *)(self), array_elem_size(self), _index) + +/// Pop the last element off the array, returning the element by value. +#define array_pop(self) ((self)->contents[--(self)->size]) + +/// Assign the contents of one array to another, reallocating if necessary. +#define array_assign(self, other) \ + _array__assign((Array *)(self), (const Array *)(other), array_elem_size(self)) + +/// Swap one array with another +#define array_swap(self, other) \ + _array__swap((Array *)(self), (Array *)(other)) + +/// Get the size of the array contents +#define array_elem_size(self) (sizeof *(self)->contents) + +/// Search a sorted array for a given `needle` value, using the given `compare` +/// callback to determine the order. +/// +/// If an existing element is found to be equal to `needle`, then the `index` +/// out-parameter is set to the existing value's index, and the `exists` +/// out-parameter is set to true. Otherwise, `index` is set to an index where +/// `needle` should be inserted in order to preserve the sorting, and `exists` +/// is set to false. +#define array_search_sorted_with(self, compare, needle, _index, _exists) \ + _array__search_sorted(self, 0, compare, , needle, _index, _exists) + +/// Search a sorted array for a given `needle` value, using integer comparisons +/// of a given struct field (specified with a leading dot) to determine the order. +/// +/// See also `array_search_sorted_with`. +#define array_search_sorted_by(self, field, needle, _index, _exists) \ + _array__search_sorted(self, 0, _compare_int, field, needle, _index, _exists) + +/// Insert a given `value` into a sorted array, using the given `compare` +/// callback to determine the order. +#define array_insert_sorted_with(self, compare, value) \ + do { \ + unsigned _index, _exists; \ + array_search_sorted_with(self, compare, &(value), &_index, &_exists); \ + if (!_exists) array_insert(self, _index, value); \ + } while (0) + +/// Insert a given `value` into a sorted array, using integer comparisons of +/// a given struct field (specified with a leading dot) to determine the order. +/// +/// See also `array_search_sorted_by`. +#define array_insert_sorted_by(self, field, value) \ + do { \ + unsigned _index, _exists; \ + array_search_sorted_by(self, field, (value) field, &_index, &_exists); \ + if (!_exists) array_insert(self, _index, value); \ + } while (0) + +// Private + +typedef Array(void) Array; + +/// This is not what you're looking for, see `array_delete`. +static inline void _array__delete(Array *self) { + if (self->contents) { + ts_free(self->contents); + self->contents = NULL; + self->size = 0; + self->capacity = 0; + } +} + +/// This is not what you're looking for, see `array_erase`. +static inline void _array__erase(Array *self, size_t element_size, + uint32_t index) { + assert(index < self->size); + char *contents = (char *)self->contents; + memmove(contents + index * element_size, contents + (index + 1) * element_size, + (self->size - index - 1) * element_size); + self->size--; +} + +/// This is not what you're looking for, see `array_reserve`. +static inline void _array__reserve(Array *self, size_t element_size, uint32_t new_capacity) { + if (new_capacity > self->capacity) { + if (self->contents) { + self->contents = ts_realloc(self->contents, new_capacity * element_size); + } else { + self->contents = ts_malloc(new_capacity * element_size); + } + self->capacity = new_capacity; + } +} + +/// This is not what you're looking for, see `array_assign`. +static inline void _array__assign(Array *self, const Array *other, size_t element_size) { + _array__reserve(self, element_size, other->size); + self->size = other->size; + memcpy(self->contents, other->contents, self->size * element_size); +} + +/// This is not what you're looking for, see `array_swap`. +static inline void _array__swap(Array *self, Array *other) { + Array swap = *other; + *other = *self; + *self = swap; +} + +/// This is not what you're looking for, see `array_push` or `array_grow_by`. +static inline void _array__grow(Array *self, uint32_t count, size_t element_size) { + uint32_t new_size = self->size + count; + if (new_size > self->capacity) { + uint32_t new_capacity = self->capacity * 2; + if (new_capacity < 8) new_capacity = 8; + if (new_capacity < new_size) new_capacity = new_size; + _array__reserve(self, element_size, new_capacity); + } +} + +/// This is not what you're looking for, see `array_splice`. +static inline void _array__splice(Array *self, size_t element_size, + uint32_t index, uint32_t old_count, + uint32_t new_count, const void *elements) { + uint32_t new_size = self->size + new_count - old_count; + uint32_t old_end = index + old_count; + uint32_t new_end = index + new_count; + assert(old_end <= self->size); + + _array__reserve(self, element_size, new_size); + + char *contents = (char *)self->contents; + if (self->size > old_end) { + memmove( + contents + new_end * element_size, + contents + old_end * element_size, + (self->size - old_end) * element_size + ); + } + if (new_count > 0) { + if (elements) { + memcpy( + (contents + index * element_size), + elements, + new_count * element_size + ); + } else { + memset( + (contents + index * element_size), + 0, + new_count * element_size + ); + } + } + self->size += new_count - old_count; +} + +/// A binary search routine, based on Rust's `std::slice::binary_search_by`. +/// This is not what you're looking for, see `array_search_sorted_with` or `array_search_sorted_by`. +#define _array__search_sorted(self, start, compare, suffix, needle, _index, _exists) \ + do { \ + *(_index) = start; \ + *(_exists) = false; \ + uint32_t size = (self)->size - *(_index); \ + if (size == 0) break; \ + int comparison; \ + while (size > 1) { \ + uint32_t half_size = size / 2; \ + uint32_t mid_index = *(_index) + half_size; \ + comparison = compare(&((self)->contents[mid_index] suffix), (needle)); \ + if (comparison <= 0) *(_index) = mid_index; \ + size -= half_size; \ + } \ + comparison = compare(&((self)->contents[*(_index)] suffix), (needle)); \ + if (comparison == 0) *(_exists) = true; \ + else if (comparison < 0) *(_index) += 1; \ + } while (0) + +/// Helper macro for the `_sorted_by` routines below. This takes the left (existing) +/// parameter by reference in order to work with the generic sorting function above. +#define _compare_int(a, b) ((int)*(a) - (int)(b)) + +#ifdef _MSC_VER +#pragma warning(pop) +#elif defined(__GNUC__) || defined(__clang__) +#pragma GCC diagnostic pop +#endif + +#ifdef __cplusplus +} +#endif + +#endif // TREE_SITTER_ARRAY_H_ diff --git a/gitnexus/vendor/tree-sitter-proto/src/tree_sitter/parser.h b/gitnexus/vendor/tree-sitter-proto/src/tree_sitter/parser.h new file mode 100644 index 0000000000..799f599bd4 --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/src/tree_sitter/parser.h @@ -0,0 +1,266 @@ +#ifndef TREE_SITTER_PARSER_H_ +#define TREE_SITTER_PARSER_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include + +#define ts_builtin_sym_error ((TSSymbol)-1) +#define ts_builtin_sym_end 0 +#define TREE_SITTER_SERIALIZATION_BUFFER_SIZE 1024 + +#ifndef TREE_SITTER_API_H_ +typedef uint16_t TSStateId; +typedef uint16_t TSSymbol; +typedef uint16_t TSFieldId; +typedef struct TSLanguage TSLanguage; +#endif + +typedef struct { + TSFieldId field_id; + uint8_t child_index; + bool inherited; +} TSFieldMapEntry; + +typedef struct { + uint16_t index; + uint16_t length; +} TSFieldMapSlice; + +typedef struct { + bool visible; + bool named; + bool supertype; +} TSSymbolMetadata; + +typedef struct TSLexer TSLexer; + +struct TSLexer { + int32_t lookahead; + TSSymbol result_symbol; + void (*advance)(TSLexer *, bool); + void (*mark_end)(TSLexer *); + uint32_t (*get_column)(TSLexer *); + bool (*is_at_included_range_start)(const TSLexer *); + bool (*eof)(const TSLexer *); + void (*log)(const TSLexer *, const char *, ...); +}; + +typedef enum { + TSParseActionTypeShift, + TSParseActionTypeReduce, + TSParseActionTypeAccept, + TSParseActionTypeRecover, +} TSParseActionType; + +typedef union { + struct { + uint8_t type; + TSStateId state; + bool extra; + bool repetition; + } shift; + struct { + uint8_t type; + uint8_t child_count; + TSSymbol symbol; + int16_t dynamic_precedence; + uint16_t production_id; + } reduce; + uint8_t type; +} TSParseAction; + +typedef struct { + uint16_t lex_state; + uint16_t external_lex_state; +} TSLexMode; + +typedef union { + TSParseAction action; + struct { + uint8_t count; + bool reusable; + } entry; +} TSParseActionEntry; + +typedef struct { + int32_t start; + int32_t end; +} TSCharacterRange; + +struct TSLanguage { + uint32_t version; + uint32_t symbol_count; + uint32_t alias_count; + uint32_t token_count; + uint32_t external_token_count; + uint32_t state_count; + uint32_t large_state_count; + uint32_t production_id_count; + uint32_t field_count; + uint16_t max_alias_sequence_length; + const uint16_t *parse_table; + const uint16_t *small_parse_table; + const uint32_t *small_parse_table_map; + const TSParseActionEntry *parse_actions; + const char * const *symbol_names; + const char * const *field_names; + const TSFieldMapSlice *field_map_slices; + const TSFieldMapEntry *field_map_entries; + const TSSymbolMetadata *symbol_metadata; + const TSSymbol *public_symbol_map; + const uint16_t *alias_map; + const TSSymbol *alias_sequences; + const TSLexMode *lex_modes; + bool (*lex_fn)(TSLexer *, TSStateId); + bool (*keyword_lex_fn)(TSLexer *, TSStateId); + TSSymbol keyword_capture_token; + struct { + const bool *states; + const TSSymbol *symbol_map; + void *(*create)(void); + void (*destroy)(void *); + bool (*scan)(void *, TSLexer *, const bool *symbol_whitelist); + unsigned (*serialize)(void *, char *); + void (*deserialize)(void *, const char *, unsigned); + } external_scanner; + const TSStateId *primary_state_ids; +}; + +static inline bool set_contains(TSCharacterRange *ranges, uint32_t len, int32_t lookahead) { + uint32_t index = 0; + uint32_t size = len - index; + while (size > 1) { + uint32_t half_size = size / 2; + uint32_t mid_index = index + half_size; + TSCharacterRange *range = &ranges[mid_index]; + if (lookahead >= range->start && lookahead <= range->end) { + return true; + } else if (lookahead > range->end) { + index = mid_index; + } + size -= half_size; + } + TSCharacterRange *range = &ranges[index]; + return (lookahead >= range->start && lookahead <= range->end); +} + +/* + * Lexer Macros + */ + +#ifdef _MSC_VER +#define UNUSED __pragma(warning(suppress : 4101)) +#else +#define UNUSED __attribute__((unused)) +#endif + +#define START_LEXER() \ + bool result = false; \ + bool skip = false; \ + UNUSED \ + bool eof = false; \ + int32_t lookahead; \ + goto start; \ + next_state: \ + lexer->advance(lexer, skip); \ + start: \ + skip = false; \ + lookahead = lexer->lookahead; + +#define ADVANCE(state_value) \ + { \ + state = state_value; \ + goto next_state; \ + } + +#define ADVANCE_MAP(...) \ + { \ + static const uint16_t map[] = { __VA_ARGS__ }; \ + for (uint32_t i = 0; i < sizeof(map) / sizeof(map[0]); i += 2) { \ + if (map[i] == lookahead) { \ + state = map[i + 1]; \ + goto next_state; \ + } \ + } \ + } + +#define SKIP(state_value) \ + { \ + skip = true; \ + state = state_value; \ + goto next_state; \ + } + +#define ACCEPT_TOKEN(symbol_value) \ + result = true; \ + lexer->result_symbol = symbol_value; \ + lexer->mark_end(lexer); + +#define END_STATE() return result; + +/* + * Parse Table Macros + */ + +#define SMALL_STATE(id) ((id) - LARGE_STATE_COUNT) + +#define STATE(id) id + +#define ACTIONS(id) id + +#define SHIFT(state_value) \ + {{ \ + .shift = { \ + .type = TSParseActionTypeShift, \ + .state = (state_value) \ + } \ + }} + +#define SHIFT_REPEAT(state_value) \ + {{ \ + .shift = { \ + .type = TSParseActionTypeShift, \ + .state = (state_value), \ + .repetition = true \ + } \ + }} + +#define SHIFT_EXTRA() \ + {{ \ + .shift = { \ + .type = TSParseActionTypeShift, \ + .extra = true \ + } \ + }} + +#define REDUCE(symbol_name, children, precedence, prod_id) \ + {{ \ + .reduce = { \ + .type = TSParseActionTypeReduce, \ + .symbol = symbol_name, \ + .child_count = children, \ + .dynamic_precedence = precedence, \ + .production_id = prod_id \ + }, \ + }} + +#define RECOVER() \ + {{ \ + .type = TSParseActionTypeRecover \ + }} + +#define ACCEPT_INPUT() \ + {{ \ + .type = TSParseActionTypeAccept \ + }} + +#ifdef __cplusplus +} +#endif + +#endif // TREE_SITTER_PARSER_H_ From 802b6281b6d59a4acebcbf53f8199be0d81d9f4f Mon Sep 17 00:00:00 2001 From: ivkond Date: Sun, 12 Apr 2026 12:40:38 +0000 Subject: [PATCH 06/10] chore: add .gitignore for vendored tree-sitter-proto build artifacts https://claude.ai/code/session_01SFUCxgKMMQ8EgRHYw91xPU --- gitnexus/vendor/tree-sitter-proto/.gitignore | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 gitnexus/vendor/tree-sitter-proto/.gitignore diff --git a/gitnexus/vendor/tree-sitter-proto/.gitignore b/gitnexus/vendor/tree-sitter-proto/.gitignore new file mode 100644 index 0000000000..d073176a63 --- /dev/null +++ b/gitnexus/vendor/tree-sitter-proto/.gitignore @@ -0,0 +1,3 @@ +vendor/tree-sitter-proto/build/ +vendor/tree-sitter-proto/node_modules/ +vendor/tree-sitter-proto/package-lock.json From 935f3f5ad3a9c6e2c28479e9d87abaf5b4ce1547 Mon Sep 17 00:00:00 2001 From: ivkond Date: Sun, 12 Apr 2026 13:06:51 +0000 Subject: [PATCH 07/10] fix: correct .gitignore paths for vendored tree-sitter-proto Patterns should be relative to the .gitignore file's directory. https://claude.ai/code/session_01SFUCxgKMMQ8EgRHYw91xPU --- gitnexus/vendor/tree-sitter-proto/.gitignore | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/gitnexus/vendor/tree-sitter-proto/.gitignore b/gitnexus/vendor/tree-sitter-proto/.gitignore index d073176a63..009351ab89 100644 --- a/gitnexus/vendor/tree-sitter-proto/.gitignore +++ b/gitnexus/vendor/tree-sitter-proto/.gitignore @@ -1,3 +1,3 @@ -vendor/tree-sitter-proto/build/ -vendor/tree-sitter-proto/node_modules/ -vendor/tree-sitter-proto/package-lock.json +build/ +node_modules/ +package-lock.json From 2f28bfc3b9ec3876b054f6bf328c132893561b9f Mon Sep 17 00:00:00 2001 From: ivkond Date: Mon, 13 Apr 2026 07:16:44 +0000 Subject: [PATCH 08/10] refactor(group): address Copilot review feedback on #796 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six fixes suggested by the Copilot AI review: 1. **`normalizeHttpPath` root-path edge case** — stripping trailing slashes on the input `/` produced an empty string, yielding malformed contract ids like `http::GET::`. Now preserves `/` for the root handler/fetch case. 2. **Dedupe `scanFiles` call** — `extract()` was globbing the source-scan file list twice (once for the provider fallback, once for the consumer fallback). Moved to a single lazy call that memoizes the result for the rest of the method. 3. **HTTP `scanFiles` now ignores `**/vendor/**`** — every other extractor's glob already ignored vendored sources; the HTTP one didn't. Fixed for consistency. 4. **`loadPackageDefinition` check is now structural** — was calling `tree.rootNode.text.includes('loadPackageDefinition')` which forces materialization of the entire file text from the parse tree (expensive on large files). Replaced with a dedicated compiled query on `(call_expression function: [(identifier) | (member_expression)])` so the check stays in the AST domain. 5. **`grpc-extractor.ts` header docstring updated** — still claimed ".proto parsing is not tree-sitter-based because no grammar is installed". Now describes the actual behaviour: tree-sitter when `tree-sitter-proto` is available (optionalDependency), manual fallback otherwise. 6. **Eliminated the double proto file parse on the fallback path** — `buildProtoContext` already globs + parses every `.proto` file to build `servicesByName`. On the `!hasProtoPlugin` branch the extractor was globbing + parsing again via the now-removed `parseProtoFile` helper. The fallback branch now iterates the map that `buildProtoContext` already produced to emit provider contracts directly — single pass per proto file. ## Tests - `topic-extractor.test.ts` — 30/30 pass - `http-route-extractor.test.ts` — 18/18 pass - `grpc-extractor.test.ts` — 43/43 pass - `manifest-extractor.test.ts` — 8/8 pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude --- .../core/group/extractors/grpc-extractor.ts | 78 ++++++++----------- .../group/extractors/grpc-patterns/node.ts | 23 +++++- .../group/extractors/http-route-extractor.ts | 20 ++++- 3 files changed, 68 insertions(+), 53 deletions(-) diff --git a/gitnexus/src/core/group/extractors/grpc-extractor.ts b/gitnexus/src/core/group/extractors/grpc-extractor.ts index 584b1e49ef..0c914b782b 100644 --- a/gitnexus/src/core/group/extractors/grpc-extractor.ts +++ b/gitnexus/src/core/group/extractors/grpc-extractor.ts @@ -17,21 +17,22 @@ import { * * Two parts: * - * 1. **`.proto` parsing** — done in-process by a small string-sanitizing - * parser (see `stripProtoCommentsAndStrings` + `extractServiceBlocks` - * below). NOT tree-sitter-based because no `tree-sitter-proto` - * grammar is installed in the repo. The parser preserves offsets so + * 1. **`.proto` parsing** — tree-sitter when `tree-sitter-proto` is + * installed (optionalDependency vendored in `vendor/tree-sitter-proto/`), + * via the `.proto` entry in `grpc-patterns/` and `hasProtoPlugin`. + * When the grammar isn't available (platform incompatibility, native + * build failure) the orchestrator falls back to the in-process + * string-sanitizing parser defined below (`stripProtoCommentsAndStrings` + * + `extractServiceBlocks`). The fallback preserves offsets so any * downstream regex scans run against a sanitized copy without - * affecting the line numbers of the original. Kept as a pragmatic - * exception to the "no regex in extractors" rule until / unless - * maintainers want to add a proto grammar. + * affecting line numbers of the original. * * 2. **Source-scan providers / consumers** — delegated to per-language * plugins in `./grpc-patterns/`. The orchestrator imports NO * tree-sitter grammars or query strings — each plugin owns its own. */ -// ─── .proto parsing (not tree-sitter) ──────────────────────────────── +// ─── .proto fallback parser (used only when tree-sitter-proto is absent) ─── function readSafe(repoPath: string, rel: string): string | null { const abs = path.resolve(repoPath, rel); @@ -372,23 +373,29 @@ export class GrpcExtractor implements ContractExtractor { // ─── Proto files — definitive provider source ───────────────── // When tree-sitter-proto is available, .proto files are handled by // the plugin loop below (they're in GRPC_SCAN_GLOB). Otherwise - // fall back to the manual string-sanitizing parser. + // emit provider contracts directly from the proto map that + // `buildProtoContext` already built — no second glob / parse pass. if (!hasProtoPlugin) { - const protoFiles = await glob('**/*.proto', { - cwd: repoPath, - ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**'], - nodir: true, - }); - for (const rel of protoFiles) { - const content = readSafe(repoPath, rel); - if (content) { - out.push( - ...this.parseProtoFile( - content, - rel, - protoContext.packagesByProto.get(normalizeProtoPath(rel)) ?? '', - ), - ); + for (const infos of protoMap.values()) { + for (const info of infos) { + for (const methodName of info.methods) { + const cid = contractId(info.package, info.serviceName, methodName); + out.push( + makeContract( + cid, + 'provider', + info.protoPath, + `${info.serviceName}.${methodName}`, + 0.85, + { + package: info.package, + service: info.serviceName, + method: methodName, + source: 'proto', + }, + ), + ); + } } } } @@ -422,29 +429,6 @@ export class GrpcExtractor implements ContractExtractor { return this.dedupe(out); } - private parseProtoFile(content: string, filePath: string, pkg: string): ExtractedContract[] { - const out: ExtractedContract[] = []; - - for (const { name: serviceName, body } of extractServiceBlocks(content)) { - const rpcRe = /rpc\s+(\w+)\s*\(/g; - let rpcMatch: RegExpExecArray | null; - while ((rpcMatch = rpcRe.exec(body)) !== null) { - const methodName = rpcMatch[1]; - const cid = contractId(pkg, serviceName, methodName); - out.push( - makeContract(cid, 'provider', filePath, `${serviceName}.${methodName}`, 0.85, { - package: pkg, - service: serviceName, - method: methodName, - source: 'proto', - }), - ); - } - } - - return out; - } - /** * Convert a plugin `GrpcDetection` into a concrete `ExtractedContract` * by resolving the short service name against the proto map, building diff --git a/gitnexus/src/core/group/extractors/grpc-patterns/node.ts b/gitnexus/src/core/group/extractors/grpc-patterns/node.ts index 2abe33ea32..033962206e 100644 --- a/gitnexus/src/core/group/extractors/grpc-patterns/node.ts +++ b/gitnexus/src/core/group/extractors/grpc-patterns/node.ts @@ -82,12 +82,28 @@ const NEW_QUALIFIED_CTOR_SPEC: PatternSpec> = { `, }; +// Detect whether the file uses `loadPackageDefinition` (gRPC dynamic +// proto loader). Matches either a bare call or an `obj.loadPackageDefinition(...)` +// call. Plugin gates the qualified-constructor consumer on this — +// structural check avoids materializing `tree.rootNode.text` for every file. +const LOAD_PACKAGE_DEFINITION_SPEC: PatternSpec> = { + meta: {}, + query: ` + (call_expression + function: [ + (identifier) @fn (#eq? @fn "loadPackageDefinition") + (member_expression property: (property_identifier) @fn (#eq? @fn "loadPackageDefinition")) + ]) + `, +}; + interface NodeGrpcPatternBundle { grpcMethod: CompiledPatterns>; grpcClient: CompiledPatterns>; getService: CompiledPatterns>; newSimpleCtor: CompiledPatterns>; newQualifiedCtor: CompiledPatterns>; + loadPackageDefinition: CompiledPatterns>; } function compileBundle(language: unknown, name: string): NodeGrpcPatternBundle { @@ -103,6 +119,7 @@ function compileBundle(language: unknown, name: string): NodeGrpcPatternBundle { getService: mk(GET_SERVICE_SPEC, 'get-service'), newSimpleCtor: mk(NEW_SIMPLE_CTOR_SPEC, 'new-simple-ctor'), newQualifiedCtor: mk(NEW_QUALIFIED_CTOR_SPEC, 'new-qualified-ctor'), + loadPackageDefinition: mk(LOAD_PACKAGE_DEFINITION_SPEC, 'load-package-definition'), }; } @@ -255,8 +272,10 @@ function scanBundle(bundle: NodeGrpcPatternBundle, tree: Parser.Tree): GrpcDetec // ─── Consumer: loadPackageDefinition dynamic proto loader ──────── // Only emit when the file uses loadPackageDefinition, otherwise a // generic `new foo.bar.Something()` in unrelated code would falsely - // register as a gRPC consumer. - const usesLoadPackage = tree.rootNode.text.includes('loadPackageDefinition'); + // register as a gRPC consumer. Check structurally via a dedicated + // query — avoids materializing `tree.rootNode.text` for the whole + // file (expensive on large files). + const usesLoadPackage = runCompiledPatterns(bundle.loadPackageDefinition, tree).length > 0; if (usesLoadPackage) { for (const match of runCompiledPatterns(bundle.newQualifiedCtor, tree)) { const ctorNode = match.captures.ctor; diff --git a/gitnexus/src/core/group/extractors/http-route-extractor.ts b/gitnexus/src/core/group/extractors/http-route-extractor.ts index 96a7ffe200..52ad5e3902 100644 --- a/gitnexus/src/core/group/extractors/http-route-extractor.ts +++ b/gitnexus/src/core/group/extractors/http-route-extractor.ts @@ -63,7 +63,10 @@ export function normalizeHttpPath(p: string): string { s = s.replace(/:\w+/g, '{param}'); s = s.replace(/\{[^}]+\}/g, '{param}'); s = s.replace(/\[[^\]]+\]/g, '{param}'); - return s; + // Preserve root: after stripping trailing slashes, the root "/" + // collapses to "" which would produce malformed contract ids like + // `http::GET::`. Restore a single slash for the root case. + return s === '' ? '/' : s; } /** @@ -192,19 +195,28 @@ export class HttpRouteExtractor implements ContractExtractor { } }; + // Glob the source-scan file list at most once per extract() — + // both provider and consumer fallback paths share the same list. + let scannedFiles: string[] | null = null; + const getScannedFiles = async (): Promise => { + if (scannedFiles) return scannedFiles; + scannedFiles = await this.scanFiles(repoPath); + return scannedFiles; + }; + const graphProviders = dbExecutor != null ? await this.extractProvidersGraph(dbExecutor, getDetections) : []; const providers = graphProviders.length > 0 ? graphProviders - : this.extractProvidersSourceScan(await this.scanFiles(repoPath), getDetections); + : this.extractProvidersSourceScan(await getScannedFiles(), getDetections); const graphConsumers = dbExecutor != null ? await this.extractConsumersGraph(dbExecutor, getDetections) : []; const consumers = graphConsumers.length > 0 ? graphConsumers - : this.extractConsumersSourceScan(await this.scanFiles(repoPath), getDetections); + : this.extractConsumersSourceScan(await getScannedFiles(), getDetections); return [...providers, ...consumers]; } @@ -212,7 +224,7 @@ export class HttpRouteExtractor implements ContractExtractor { private async scanFiles(repoPath: string): Promise { return glob(HTTP_SCAN_GLOB, { cwd: repoPath, - ignore: ['**/node_modules/**', '**/.git/**', '**/dist/**', '**/build/**'], + ignore: ['**/node_modules/**', '**/.git/**', '**/dist/**', '**/build/**', '**/vendor/**'], nodir: true, }); } From 7b22776560636bef9df007ad2b83cb146c7cac1d Mon Sep 17 00:00:00 2001 From: ivkond Date: Mon, 13 Apr 2026 07:24:03 +0000 Subject: [PATCH 09/10] refactor(group): address Claude review feedback (bugs + dedup + hygiene) on #796 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follows up `2f28bfc` with the remaining items from the Claude AI review: ## Bugs **Bug 2 — Label-unaware Cypher queries in `resolveSymbol`.** The manifest-extractor's lookup queries were `MATCH (n) WHERE n.name = $x` with no label filter, so a topic/service/package name could silently match any node type (File, Variable, Import, Folder, …). Added label filters: - `topic` → `(n:Function|Method|Class|Interface)` (topics are best-effort symbol-name matches against listener/publisher symbols) - `grpc` method → `(n:Function|Method)` - `grpc` service → `(n:Class|Interface)` - `lib` → `(n:Package|Module)` All 8 manifest-extractor tests still pass (mock executor is label-agnostic, but the production LadybugDB graph now gets correctly scoped queries). **Bug 8 — Tautological `!handlerName` condition.** `http-route-extractor.ts:extractProvidersGraph` had `let handlerName = null; if (!method || !handlerName) { ... }` — the `!handlerName` clause was always true since there was no intervening assignment. Simplified to always run the plugin-scan lookup (we need the handler name even when `methodFromRouteReason` already resolved the method). ## Clean code / dedup **Design 7 — `readSafe` was copy-pasted in all three orchestrators.** Extracted to `extractors/fs-utils.ts` as the single source of truth for the path-traversal guard. Dropped the three local copies and the now-unused `fs`/`path` imports from topic-extractor. **Style 10 — Language-specific `_test.go` skip in the topic orchestrator.** Was `if (rel.endsWith('_test.go')) continue;` inside the language- agnostic extraction loop. Pushed into the glob's ignore list (`'**/*_test.go'`) alongside the existing `node_modules`, `vendor`, `dist`, `build` entries, with a comment explaining that other languages' test file conventions either live in separate directories (Python `tests/`, Java `src/test/`) or are already covered by the existing ignores. ## Already addressed in `2f28bfc` (mentioned again in Claude review) - Bug 3: `normalizeHttpPath('/')` returns `''` — fixed - Bug 4: double glob + double parse of `.proto` — fixed - Bug 5: `scanFiles` called twice in HTTP — fixed - Bug 6: missing `**/vendor/**` in HTTP glob — fixed - Design 9 partially: `tree.rootNode.text.includes('loadPackageDefinition')` replaced with a dedicated structural query ## Deferred - Bug 1 (`http::*::path` vs `http::GET::path` matching) — out of scope; sync.ts matching logic lands in #793, manifest extractor already emits correct synthetic uids for unresolved HTTP contracts. - Design 9 full (change plugin `scan(tree)` → `scan(tree, source)`) — the only real use case (`loadPackageDefinition` gate) is already fixed via a structural query, so the interface change would be cosmetic churn without a concrete consumer. ## Tests - `topic-extractor.test.ts` — 30/30 pass - `http-route-extractor.test.ts` — 18/18 pass - `grpc-extractor.test.ts` — 43/43 pass - `manifest-extractor.test.ts` — 8/8 pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude --- .../src/core/group/extractors/fs-utils.ts | 23 +++++++++++ .../core/group/extractors/grpc-extractor.ts | 14 +------ .../group/extractors/http-route-extractor.ts | 38 ++++++------------- .../group/extractors/manifest-extractor.ts | 20 +++++++--- .../core/group/extractors/topic-extractor.ts | 32 ++++++++-------- 5 files changed, 66 insertions(+), 61 deletions(-) create mode 100644 gitnexus/src/core/group/extractors/fs-utils.ts diff --git a/gitnexus/src/core/group/extractors/fs-utils.ts b/gitnexus/src/core/group/extractors/fs-utils.ts new file mode 100644 index 0000000000..384f632031 --- /dev/null +++ b/gitnexus/src/core/group/extractors/fs-utils.ts @@ -0,0 +1,23 @@ +import * as fs from 'node:fs'; +import * as path from 'node:path'; + +/** + * Safely read a file inside a repo, rejecting any path that escapes + * `repoPath` via `..` traversal or absolute segments. Returns `null` if + * the path is outside the repo or the file can't be read. + * + * Used by every source-scan extractor under this directory. Kept as a + * single shared implementation so the path-traversal guard (security- + * sensitive) lives in exactly one place. + */ +export function readSafe(repoPath: string, rel: string): string | null { + const abs = path.resolve(repoPath, rel); + const base = path.resolve(repoPath); + const relToBase = path.relative(base, abs); + if (relToBase.startsWith('..') || path.isAbsolute(relToBase)) return null; + try { + return fs.readFileSync(abs, 'utf-8'); + } catch { + return null; + } +} diff --git a/gitnexus/src/core/group/extractors/grpc-extractor.ts b/gitnexus/src/core/group/extractors/grpc-extractor.ts index 0c914b782b..c6af9138a1 100644 --- a/gitnexus/src/core/group/extractors/grpc-extractor.ts +++ b/gitnexus/src/core/group/extractors/grpc-extractor.ts @@ -1,9 +1,9 @@ -import * as fs from 'node:fs'; import * as path from 'node:path'; import { glob } from 'glob'; import Parser from 'tree-sitter'; import type { ContractExtractor, CypherExecutor } from '../contract-extractor.js'; import type { ExtractedContract, RepoHandle } from '../types.js'; +import { readSafe } from './fs-utils.js'; import { GRPC_SCAN_GLOB, getPluginForFile, @@ -34,18 +34,6 @@ import { // ─── .proto fallback parser (used only when tree-sitter-proto is absent) ─── -function readSafe(repoPath: string, rel: string): string | null { - const abs = path.resolve(repoPath, rel); - const base = path.resolve(repoPath); - const relToBase = path.relative(base, abs); - if (relToBase.startsWith('..') || path.isAbsolute(relToBase)) return null; - try { - return fs.readFileSync(abs, 'utf-8'); - } catch { - return null; - } -} - function contractId(pkg: string, service: string, method: string): string { const prefix = pkg ? `${pkg}.${service}` : service; return `grpc::${prefix}/${method}`; diff --git a/gitnexus/src/core/group/extractors/http-route-extractor.ts b/gitnexus/src/core/group/extractors/http-route-extractor.ts index 52ad5e3902..0b07090e15 100644 --- a/gitnexus/src/core/group/extractors/http-route-extractor.ts +++ b/gitnexus/src/core/group/extractors/http-route-extractor.ts @@ -1,9 +1,9 @@ -import * as fs from 'node:fs'; import * as path from 'node:path'; import { glob } from 'glob'; import Parser from 'tree-sitter'; import type { ContractExtractor, CypherExecutor } from '../contract-extractor.js'; import type { ExtractedContract, RepoHandle } from '../types.js'; +import { readSafe } from './fs-utils.js'; import { getPluginForFile, HTTP_SCAN_GLOB, type HttpDetection } from './http-patterns/index.js'; /** @@ -97,20 +97,6 @@ function contractIdFor(method: string, pathNorm: string): string { return `http::${method.toUpperCase()}::${pathNorm}`; } -// ─── File read helper (path-traversal safe) ────────────────────────── - -function readSafe(repoPath: string, rel: string): string | null { - const abs = path.resolve(repoPath, rel); - const base = path.resolve(repoPath); - const relToBase = path.relative(base, abs); - if (relToBase.startsWith('..') || path.isAbsolute(relToBase)) return null; - try { - return fs.readFileSync(abs, 'utf-8'); - } catch { - return null; - } -} - // ─── Graph row helpers ─────────────────────────────────────────────── function methodFromRouteReason(reason: string): string | null { @@ -249,20 +235,20 @@ export class HttpRouteExtractor implements ContractExtractor { const routeSource = String(row.routeSource ?? row.routeReason ?? ''); let method = methodFromRouteReason(routeSource); - // Fallback: look up method + handler name from the plugin's scan - // of the handler file. This replaces the old regex-based - // `inferMethodFromFileScan` and `pickJavaHandlerName` helpers — - // tree-sitter gives both pieces of information structurally. + // Look up handler name (and backfill method if missing) from the + // plugin's scan of the handler file. This replaces the old + // regex-based `inferMethodFromFileScan` and `pickJavaHandlerName` + // helpers — tree-sitter gives both pieces of information + // structurally. Always run the lookup: even when method is set by + // `methodFromRouteReason`, we still need the handler name. const detections = filePath ? getDetections(filePath) : []; const providerDetections = detections.filter((d) => d.role === 'provider'); let handlerName: string | null = null; - if (!method || !handlerName) { - const normalizedRoute = normalizeHttpPath(routePath); - const match = providerDetections.find((d) => normalizeHttpPath(d.path) === normalizedRoute); - if (match) { - if (!method) method = match.method; - handlerName = match.name; - } + const normalizedRoute = normalizeHttpPath(routePath); + const match = providerDetections.find((d) => normalizeHttpPath(d.path) === normalizedRoute); + if (match) { + if (!method) method = match.method; + handlerName = match.name; } if (!method) method = 'GET'; diff --git a/gitnexus/src/core/group/extractors/manifest-extractor.ts b/gitnexus/src/core/group/extractors/manifest-extractor.ts index 09817b356c..4cfcd54788 100644 --- a/gitnexus/src/core/group/extractors/manifest-extractor.ts +++ b/gitnexus/src/core/group/extractors/manifest-extractor.ts @@ -141,8 +141,13 @@ export class ManifestExtractor { { normalized }, ); } else if (link.type === 'topic') { + // Topic names aren't a first-class NodeLabel in the graph — + // topics are referenced by function/method symbols (Kafka + // listeners, publishers). Restrict to symbol-like labels to + // avoid cross-matching Files/Variables/Imports that happen to + // share the topic name. rows = await executor( - `MATCH (n) WHERE n.name = $contract + `MATCH (n:Function|Method|Class|Interface) WHERE n.name = $contract RETURN n.id AS uid, n.name AS name, n.filePath AS filePath ORDER BY n.filePath ASC LIMIT 1`, @@ -153,12 +158,15 @@ export class ManifestExtractor { // variants). Prefer matching by method name when present, otherwise // by service name. NO .proto path fallback — that's guaranteed to // return a wrong symbol in any repo with more than one proto file. + // Label filters scope lookups: methods → Function|Method, services + // → Class|Interface (no label match = no silent wrong hits on + // File/Variable nodes that happen to share the name). const parts = link.contract.split('/'); const serviceName = parts[0]?.trim() ?? ''; const methodName = parts[1]?.trim() ?? ''; if (methodName) { rows = await executor( - `MATCH (n) WHERE n.name = $methodName + `MATCH (n:Function|Method) WHERE n.name = $methodName RETURN n.id AS uid, n.name AS name, n.filePath AS filePath ORDER BY n.filePath ASC LIMIT 1`, @@ -166,7 +174,7 @@ export class ManifestExtractor { ); } else if (serviceName) { rows = await executor( - `MATCH (n) WHERE n.name = $serviceName + `MATCH (n:Class|Interface) WHERE n.name = $serviceName RETURN n.id AS uid, n.name AS name, n.filePath AS filePath ORDER BY n.filePath ASC LIMIT 1`, @@ -178,9 +186,11 @@ export class ManifestExtractor { } else if (link.type === 'lib') { // Only exact match on the symbol's name. Previous fallback to // CONTAINS on n.filePath would promote "react" to "react-native" - // or "@types/react" — silent wrong attribution. + // or "@types/react" — silent wrong attribution. Restrict to + // package-level labels so we don't return arbitrary symbols + // named after a library. rows = await executor( - `MATCH (n) WHERE n.name = $contract + `MATCH (n:Package|Module) WHERE n.name = $contract RETURN n.id AS uid, n.name AS name, n.filePath AS filePath ORDER BY n.filePath ASC LIMIT 1`, diff --git a/gitnexus/src/core/group/extractors/topic-extractor.ts b/gitnexus/src/core/group/extractors/topic-extractor.ts index faab19cf30..1fbccac8a1 100644 --- a/gitnexus/src/core/group/extractors/topic-extractor.ts +++ b/gitnexus/src/core/group/extractors/topic-extractor.ts @@ -1,9 +1,8 @@ -import * as fs from 'node:fs'; -import * as path from 'node:path'; import { glob } from 'glob'; import Parser from 'tree-sitter'; import type { ContractExtractor, CypherExecutor } from '../contract-extractor.js'; import type { ExtractedContract, RepoHandle } from '../types.js'; +import { readSafe } from './fs-utils.js'; import { scanFile, unquoteLiteral } from './tree-sitter-scanner.js'; import { TOPIC_SCAN_GLOB, @@ -28,18 +27,6 @@ import { * Adding a new language is a one-file edit in `topic-patterns/index.ts`. */ -function readSafe(repoPath: string, rel: string): string | null { - const abs = path.resolve(repoPath, rel); - const base = path.resolve(repoPath); - const relToBase = path.relative(base, abs); - if (relToBase.startsWith('..') || path.isAbsolute(relToBase)) return null; - try { - return fs.readFileSync(abs, 'utf-8'); - } catch { - return null; - } -} - function makeContract(topicName: string, meta: TopicMeta, filePath: string): ExtractedContract { return { contractId: `topic::${topicName}`, @@ -71,7 +58,20 @@ export class TopicExtractor implements ContractExtractor { ): Promise { const files = await glob(TOPIC_SCAN_GLOB, { cwd: repoPath, - ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**', '**/dist/**', '**/build/**'], + ignore: [ + '**/node_modules/**', + '**/.git/**', + '**/vendor/**', + '**/dist/**', + '**/build/**', + // Language-level test file conventions. Go test files + // `*_test.go` live next to source; other languages either use + // separate test directories (Python's `tests/`, Java's + // `src/test/`) or are already covered by the dist/build ignores. + // Pushed to the glob level so the orchestrator stays + // language-agnostic. + '**/*_test.go', + ], nodir: true, }); @@ -81,8 +81,6 @@ export class TopicExtractor implements ContractExtractor { const out: ExtractedContract[] = []; for (const rel of files) { - if (rel.endsWith('_test.go')) continue; - const provider = getProviderForFile(rel); if (!provider) continue; From 7016b32af2dd70337baab7414297ae8b2d7e0fe0 Mon Sep 17 00:00:00 2001 From: ivkond Date: Mon, 13 Apr 2026 07:36:15 +0000 Subject: [PATCH 10/10] docs+fix(group): address remaining Claude review items + add pipeline flow chart MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Fixes **Remaining 🔴 — HTTP contract id wildcard format.** Documented the `http::*::` format as an intentional wildcard for manifest links that omit the HTTP method, alongside the explicit-method form (`GET::/path` → `http::GET::/path`). The docblock on `buildContractId` now states both forms, notes that wildcard-aware matching is the responsibility of the sync / cross-impact layer (#793), and recommends the explicit-method form whenever the author knows the method (it round-trips through exact equality without needing wildcard logic downstream). Tests unchanged — the wildcard format is what they've always asserted. **Minor 1 — stale comment at `manifest-extractor.ts:124-126`.** The comment claimed "creates a contract with an empty symbolUid/ref" but the code switched to `manifestSymbolUid(repo, contractId)` a few commits back. Updated to describe the actual synthetic-uid fallback semantics and the cross-impact path that relies on both sides of the join deriving the same uid. **Minor 2 — exhaustiveness guard on `buildContractId`.** The `switch(type)` covered all five current `ContractType` variants but silently returned `undefined` if a new variant was added. Added a `default: const _exhaustive: never = type; throw new Error(...)` clause so the build fails loudly on an unhandled variant. **Minor 3 — `tree.rootNode.text` in `grpc-patterns/node.ts`.** Already fixed in `2f28bfc` via a dedicated structural query (`LOAD_PACKAGE_DEFINITION_SPEC`). No action needed. ## New: pipeline flow chart (per @magyargergo's request) Added `src/core/group/PIPELINE.md` with four Mermaid diagrams: 1. **High-level overview** — `group.yaml` → extractors + manifest → contract matching → `bridge.lbug` → `runGroupImpact`. 2. **Per-repo extractor two-strategy shape** — graph-assisted Strategy A vs. source-scan Strategy B. 3. **Plugin architecture** — orchestrator → registry → per-language `*-patterns/.ts` → `tree-sitter-scanner.ts` → `ExtractedContract`. 4. **Manifest extraction** — label-scoped `resolveSymbol` with the synthetic-uid fallback. 5. **Cross-impact query (#606)** — local impact → bridge join → cross-repo fan-out. Each diagram is annotated with which PRs own which stage (this PR: extractors + manifest; #795: bridge storage; #606: cross-impact runtime) and points at the concrete files/functions involved. ## Tests - 99/99 extractor tests pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude --- gitnexus/src/core/group/PIPELINE.md | 139 ++++++++++++++++++ .../group/extractors/manifest-extractor.ts | 36 ++++- 2 files changed, 172 insertions(+), 3 deletions(-) create mode 100644 gitnexus/src/core/group/PIPELINE.md diff --git a/gitnexus/src/core/group/PIPELINE.md b/gitnexus/src/core/group/PIPELINE.md new file mode 100644 index 0000000000..7730b48e7c --- /dev/null +++ b/gitnexus/src/core/group/PIPELINE.md @@ -0,0 +1,139 @@ +# Group Analysis Pipeline + +Flow chart of the cross-repo contract extraction + matching pipeline. +This covers what runs **inside this PR** (extractors + manifest) and +the downstream handoff to the bridge storage (PR #795) and +cross-impact query (PR #606). + +## High-level overview + +```mermaid +flowchart TD + A[group.yaml] --> B[GroupConfig parser] + B --> C{For each repo
in group} + C --> D[Per-repo LadybugDB
indexed by main pipeline] + + D --> E1[TopicExtractor] + D --> E2[HttpRouteExtractor] + D --> E3[GrpcExtractor] + + E1 --> F[ExtractedContract array
per repo] + E2 --> F + E3 --> F + + B --> M[ManifestExtractor] + M --> G[Manifest contracts
+ cross-links] + + F --> H[Contract matching
exact + wildcard] + G --> H + + H --> I[(bridge.lbug
#795)] + + I --> J[runGroupImpact
#606] + J --> K[CrossRepoImpact] +``` + +## Per-repo extractor pipeline + +Each extractor under `src/core/group/extractors/` follows the same +two-strategy shape: + +```mermaid +flowchart TD + R[RepoHandle + CypherExecutor
for this repo] --> S{Graph-assisted
Strategy A
available?} + + S -->|yes| A1[Cypher query against
per-repo LadybugDB] + A1 --> A2{non-empty
result?} + A2 -->|yes| OUT[ExtractedContract array] + A2 -->|no| B1 + + S -->|no| B1[Source-scan Strategy B] + B1 --> B2[glob repo source files] + B2 --> B3{ext in registry?} + B3 -->|yes| B4[Per-language plugin
scan parsed tree] + B3 -->|no| SKIP[skip file] + B4 --> OUT + + SKIP --> B2 +``` + +**Strategy A** (graph-assisted) uses Cypher over edges already produced +by the main ingestion pipeline: +- HTTP: `HANDLES_ROUTE` / `FETCHES` edges from `(File)-[]->(Route)` +- topic: none (pipeline doesn't yet produce topic nodes — Strategy B only) +- gRPC: none (Strategy B + proto map only) + +**Strategy B** (source-scan) is 100% tree-sitter based after this PR. +Each `*-patterns/.ts` plugin owns its grammar + S-expression +queries; the top-level orchestrator imports neither. + +## Plugin architecture + +```mermaid +flowchart LR + O[Orchestrator
topic|http|grpc-extractor.ts] --> REG[REGISTRY
*-patterns/index.ts] + REG --> P1[java.ts
tree-sitter-java] + REG --> P2[go.ts
tree-sitter-go] + REG --> P3[python.ts
tree-sitter-python] + REG --> P4[node.ts
JS + TS + TSX] + REG --> P5[php.ts
tree-sitter-php
HTTP only] + REG --> P6[proto.ts
tree-sitter-proto
gRPC only, optional] + + P1 --> SCAN[tree-sitter-scanner.ts
compilePatterns + runCompiledPatterns] + P2 --> SCAN + P3 --> SCAN + P4 --> SCAN + P5 --> SCAN + P6 --> SCAN + + SCAN --> DET[Detection objects
TopicMeta / HttpDetection / GrpcDetection] + DET --> O + O --> CT[ExtractedContract array] +``` + +The orchestrator never imports a grammar. Adding a new language / +framework = drop one file in `*-patterns/`, register it in +`index.ts`. No orchestrator edits required. + +## Manifest extraction + +```mermaid +flowchart TD + Y[group.yaml links] --> ME[ManifestExtractor] + ME --> LOOP{for each link} + LOOP --> RES[resolveSymbol
label-scoped Cypher] + RES --> OK{found?} + OK -->|yes| REF[real symbol uid + ref] + OK -->|no| SYN[synthetic uid
manifest::repo::cid] + + REF --> EMIT[emit provider + consumer
Contract objects
+ CrossLink] + SYN --> EMIT + + EMIT --> BRIDGE[(bridge.lbug
#795)] +``` + +Label-scoped queries in `resolveSymbol` keep accidental cross-matches +out: +- `topic` → `(n:Function|Method|Class|Interface)` +- `grpc` method → `(n:Function|Method)`, service → `(n:Class|Interface)` +- `lib` → `(n:Package|Module)` + +## Cross-impact query (PR #606) + +```mermaid +flowchart TD + U[User changes symbol S
in repo R] --> LI[Local impact engine
per-repo uid expansion] + LI --> IDS[Affected uid set] + + IDS --> BR[Bridge query
MATCH Contract WHERE uid IN ids] + BR --> CL[CrossLink traversal] + CL --> OTHER[Matching contract in
other repo] + + OTHER --> FE[Fan-out impact
to consuming repo] + FE --> OUT[CrossRepoImpact
per affected repo] +``` + +The bridge stores every extracted contract keyed by `symbolUid`. +Manifest-sourced contracts use the synthetic uid form so both sides +of the `(local impact) ↔ (bridge query)` join derive the same uid +without coordinating through any shared state. diff --git a/gitnexus/src/core/group/extractors/manifest-extractor.ts b/gitnexus/src/core/group/extractors/manifest-extractor.ts index 4cfcd54788..29c8f9b21d 100644 --- a/gitnexus/src/core/group/extractors/manifest-extractor.ts +++ b/gitnexus/src/core/group/extractors/manifest-extractor.ts @@ -121,9 +121,12 @@ export class ManifestExtractor { // match "/suborders", and a gRPC manifest entry in a repo with any // .proto file would attach to a random proto symbol. // - // If resolveSymbol returns null, the extractor creates a contract with - // an empty symbolUid/ref — cross-impact still works via name-based - // matching through the `hint` path in runGroupImpact. + // If resolveSymbol returns null, the extractor falls back to a + // deterministic synthetic uid via `manifestSymbolUid(repo, contractId)` + // (see the function's docstring for why synthetic rather than empty). + // Cross-impact still works: the bridge query joins on the synthetic + // uid, and the local impact engine derives the same uid for the + // unresolved symbol — name-based hints are the additional safety net. try { let rows: Record[]; if (link.type === 'http') { @@ -219,6 +222,29 @@ export class ManifestExtractor { return null; } + /** + * Build a canonical contract id for a manifest link. + * + * HTTP is the only type with two valid forms: + * - Explicit method: `"GET::/api/orders"` → `"http::GET::/api/orders"` + * (matches exactly against `HttpRouteExtractor` provider/consumer + * contracts, which are also keyed by `http::::`). + * - Method-agnostic: `"/api/orders"` → `"http::*::/api/orders"` + * — the `*` is a wildcard and is intended to match any concrete + * HTTP method on that path. Wildcard-aware matching is the + * responsibility of the sync / cross-impact layer (see #793); + * downstream code should treat `http::*::` as matching + * every `http::::` for the same path. + * + * Recommend the explicit-method form in group.yaml whenever the + * manifest author knows the method — it round-trips through exact + * equality matching without requiring wildcard logic downstream. + * + * NOTE on exhaustiveness: the switch covers every current + * `ContractType` variant and falls through to a `never` assertion so + * TypeScript fails the build if a new variant is added without a + * corresponding case. + */ private buildContractId(type: ContractType, contract: string): string { switch (type) { case 'http': { @@ -233,6 +259,10 @@ export class ManifestExtractor { return `lib::${contract}`; case 'custom': return `custom::${contract}`; + default: { + const _exhaustive: never = type; + throw new Error(`Unhandled ContractType: ${String(_exhaustive)}`); + } } } }