perf: content-addressed parse and embedding caches for faster reindexing

[davidbeesley]

Summary

Cache deterministic per-file operations to reduce reindexing time.

Motivation / context

When under active development, code changes quickly and the graph gets stale. Reducing reindex time improves DX.

Areas touched

• gitnexus/ (CLI / core / MCP server)
• gitnexus-web/ (Vite / React UI)
• .github/ (workflows, actions)
• eval/ or other tooling
• Docs / agent config only (AGENTS.md, CLAUDE.md, .cursor/, llms.txt, etc.)

Scope & constraints

In scope

• Parse cache: content-addressed per-file Tree-sitter results (SHA-256 of file content).
• Embedding cache: content-addressed per-node embedding vectors (SHA-256 of embedding input text).
• Both caches survive --force (content-addressed, always safe to reuse).
• Replaces old LadybugDB-based embedding cache (lost on --force, keyed by node ID not content).

Explicitly out of scope / not done here

• Incremental LadybugDB load

Implementation notes

• Shared cache-io.ts for atomic JSON read/write/delete with version validation
• Parse cache stored in .gitnexus/parse-cache.json, embedding cache in .gitnexus/embedding-cache.json
• Both cleaned up by gitnexus clean (lives inside .gitnexus/)
• Embedding cache invalidates on model ID or dimension change
• Parse cache version bump invalidates on ParseWorkerResult schema change
• mergeParseResult extracted from parsing-processor.ts for shared use by worker merge and cache replay
• Embedding pipeline defers ONNX model load until after cache check — skips entirely when 100% cached

Testing & verification

• cd gitnexus && npm test — 33 new tests across parse-cache.test.ts and embedding-cache.test.ts
• cd gitnexus && npx tsc --noEmit
• Manual: analyze --force on GitNexus verified cache hit counts and timing improvements

Risk & rollout

• Cache files are new — no migration needed, absent cache triggers full rebuild
• --force no longer clears parse cache (intentional — content-addressed means always correct)
• Run npx gitnexus analyze after upgrading to populate caches on first run

Checklist

• PR body meets repo minimum length
• If AGENTS.md / overlays changed: N/A
• No secrets, tokens, or machine-specific paths committed

[vercel]

@davidbeesley is attempting to deploy a commit to the NexusCore Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions]

CI Report

✅ All checks passed

Pipeline Status

Stage	Status	Details
✅ Typecheck	success	tsc --noEmit
✅ Tests	success	unit tests, 3 platforms
✅ E2E	success	gitnexus-web changes only

Test Results

Tests	Passed	Failed	Skipped	Duration
4708	4707	0	1	187s

✅ All 4707 tests passed

1 test(s) skipped — expand for details

• buildTypeEnv > known limitations (documented skip tests) > Ruby block parameter: users.each { |user| } — closure param inference, different feature

Code Coverage

Tests

Metric	Coverage	Covered	Base	Delta	Status
Statements	71.93%	12841/17851	72.24%	📉 -0.3	🔴 ██████████████░░░░░░
Branches	61.65%	8495/13779	62.16%	📉 -0.5	🔴 ████████████░░░░░░░░
Functions	76.36%	1131/1481	75.86%	📈 +0.5	🟢 ███████████████░░░░░
Lines	74.41%	11503/15458	74.72%	📉 -0.3	🔴 ██████████████░░░░░░

---

_{📋 View full run · Generated by CI}

[magyargergo]

@claude ultra-think while reviewing this pr and check the performance characteristics and try to find gaps and potential bugs. We need a way to benchmark this very soon.

[github-actions]

Claude finished @magyargergo's task in 3m 19s —— View job

---

PR Review: Content-Addressed Parse & Embedding Caches

Tasks

• Read and understand the PR context
• Read all changed files in depth
• Analyze performance characteristics
• Identify bugs and correctness gaps
• Assess benchmarking readiness
• Post full review

---

Overview

The design is clean and the core abstractions are sound. Content-addressing both caches is the right call — it makes --force not destroying the cache safe and simplifies invalidation logic. The atomic write pattern (tmp → rename) is correct. Tests cover the happy paths and basic error handling well.

That said, deep-reading the integration across pipeline.ts, embedding-pipeline.ts, and analyze.ts surfaced several real bugs and performance issues worth addressing before merging.

---

Bugs

1. pruneCache is imported but never called — cache grows forever

analyze.ts:18 imports pruneCache. pipeline.ts:6 also imports it. Neither file actually calls it.

Impact: Every renamed or deleted file accumulates a dead entry in parse-cache.json permanently. On an actively-developed large repo over weeks, the cache file balloons with dead entries.

Fix: Call pruneCache(parseCache, new Set(allPaths)) in analyze.ts after pipelineResult returns and before saveParseCache.

Fix this →

---

2. filterRawResultsByFile is O(n × m) — quadratic for large chunks

pipeline.ts:537-572 — for each newly-parsed file, this function iterates over all raw worker results and scans every nodes/calls/imports/etc. array to extract per-file data.

For a 20MB chunk with 800 files producing 100,000 call records: filterRawResultsByFile is called 800 times, each scanning 100,000 calls = 80 million comparisons just for calls. On a mid-size repo this can add several seconds of post-parse overhead.

Better approach: During processParsingWithWorkers, the worker already has per-file results; emit a Map<filePath, ParseWorkerResult> alongside the merged result, so the cache-write loop is O(1) per file.

Fix this →

---

3. Sequential fallback never writes to the parse cache

pipeline.ts:777-783:

if (parseCache && chunkWorkerData?.rawResults) {
  // ← only runs when rawResults is set (worker path only)
}

When a worker pool cannot be created (small repo under 15 files, or fallback triggered), processParsing returns null and the guard chunkWorkerData?.rawResults is falsy. The sequential path also never checks the cache — it goes straight to processParsingSequential. Small repos and fallback scenarios never benefit from caching at all, and if a repo shrinks below the worker threshold after previously being cached, the cache entries go stale.

---

4. Double SHA-256 computation in embedding pipeline

embedding-pipeline.ts:184-193 hashes every text to check allCached, then embedding-pipeline.ts:231-241 hashes them again during the main cache-lookup pass:

// Pass 1 — just to check allCached:
for (let i = 0; i < allTexts.length; i++) {
  if (!embeddingCache.entries[embeddingTextHash(allTexts[i])]) { ...}
}

// Pass 2 — same hashes recomputed:
for (let i = 0; i < nodes.length; i++) {
  const hash = embeddingTextHash(allTexts[i]);  // ← duplicate
  ...
}

For 50,000 nodes with 100-char average text, SHA-256 over all texts twice wastes a measurable amount of time. Combine the two passes — compute the hash once in Pass 2, and remove the separate allCached check.

Fix this →

---

5. Potential false-positive in belongsToFile path matching

pipeline.ts:550-551:

const filePathSuffix = `:${filePath}`;
const belongsToFile = (id: string) =>
  id === fileNodeId || id.includes(filePathWithColon) || id.endsWith(filePathSuffix);

The endsWith(filePathSuffix) arm can match an ID belonging to a different file that shares a path suffix. Example: if filePath = "src/foo.ts", then filePathWithColon = ":src/foo.ts:" — the includes check would also match an ID like "Class:vendor/lib/src/foo.ts:Bar" since it contains :src/foo.ts:. This would cause cache entries to include relationships from different files, corrupting the per-file cached result.

Fix this →

---

Performance Concerns

6. Single JSON blob — scalability wall

Both caches are serialized as one JSON object. For a repo at the 50,000-node limit:

• Embedding cache: 50,000 × 384 floats ≈ 75MB JSON on disk and in memory per run
• Parse cache on a 3,000-file TS repo: each ParseWorkerResult entry carries nodes, relationships, symbols, calls, imports — could easily reach 100–200MB

JSON.parse on a 100MB string is synchronous and blocks the event loop for ~500ms. JSON.stringify on the same data is similarly expensive. On every analyze run — even when nothing changed and 100% hits — the full cache is loaded, parsed, and re-serialized.

Recommendation for a follow-up: Sharding the cache by filename prefix (e.g., first 2 hex chars of file hash → separate shard file) would reduce per-run I/O to only the shards touched by changed files.

7. Embedding cache loaded into memory before model init check

analyze.ts:275 loads the full embedding cache (potentially 75MB) unconditionally when --embeddings is passed. If the cache dimensions/modelId don't match, it's immediately discarded and re-created empty. This is wasteful — check the metadata fields first before deserializing the full entries.

---

Benchmarking Gap

The PR explicitly mentions --force on GitNexus was tested manually and the need for benchmarks is acknowledged. Here's a concrete proposal for the required benchmark infrastructure:

Suggested additions in a follow-up:

gitnexus/bench/
  run-analyze-bench.ts      # Runs analyze twice, reports cold vs warm timing + cache stats
  fixtures/                 # Small synthetic repo with ~500 known files

Key metrics to capture:

Metric	How
Cold parse time (no cache)	First run, capture total time
Warm parse time (100% cached)	Second run, assert cacheHits == totalFiles
Cache file size	stat .gitnexus/parse-cache.json
Embedding cold vs warm	With --embeddings, run twice
Memory peak (RSS)	process.memoryUsage().rss before/after each phase

The CI report shows 4701 tests passing and full typecheck success — the functional correctness is solid. The benchmarking gap is the main outstanding need before this change can be properly validated at scale.

---

Minor Notes

• EmbeddingCache validation lives in caller, not in loadEmbeddingCache: The dimensions/modelId staleness check is in analyze.ts lines 276-280, not in the cache module itself. If another caller uses loadEmbeddingCache without re-implementing this check, stale vectors silently poison results. Consider adding a validateEmbeddingCache(cache, dimensions, modelId): boolean helper to embedding-cache.ts.

• Parse cache keys are absolute paths: If the same cache is used across machines (e.g., checked into a shared volume) or after git worktree, all entries will miss. Not a current use case but worth documenting.

• cacheReplayAccumulators draining: The drainAndMerge pattern (pipeline.ts line 788) is clever for cross-chunk accumulation but the comment says "drain" while it actually copies and clears. The name cacheReplayAccumulators for a variable that also accumulates fresh-parse data from cache hits is somewhat confusing.

---

Summary

Issue	Severity
pruneCache never called — unbounded cache growth	High (correctness)
filterRawResultsByFile O(n×m) post-parse overhead	High (performance)
Sequential path never cached	Medium
Double SHA-256 in embedding pipeline	Low (perf)
belongsToFile false positives	Medium (correctness)
JSON blob scalability	Medium (future)
Missing benchmark harness	High (validation)

The core design is right and the tests are good. Fixing issue #1 (prune) and #2 (O(n×m)) before merge would be my recommendation — both have clear, bounded fixes. The benchmarking infrastructure can land in a follow-up once the gaps above are addressed.

[magyargergo]

@davidbeesley could you please look at it? 🙏

[davidbeesley]

Took a crack at the list:

1. pruneCache never called — Addressed.
2. filterRawResultsByFile O(n×m) — Addressed.
3. Sequential fallback never cached — Not addressed. The worker and sequential paths are architecturally different as workers return per-file rawResults that feed into cache storage, while the sequential path writes directly into the graph without producing per-file results. Only parse cache writes are skipped, the embedding cache is unaffected since it operates on nodes already in the graph regardless of how they were parsed.
4. Double SHA-256 in embedding pipeline — Addressed.
5. belongsToFile false positives — Addressed.
6. Single JSON blob / sharding — Not addressed. Cli tool has to have the entire cached equivalent content in memory to do its work anyway. Sharding would add complexity and overhead without a clear performance benefit. Could revisit later if profiling shows io or memory bottlenecks due to .json file size.
7. Embedding cache loaded before model check — Addressed.
8. Missing benchmark harness — Not addressed.

[magyargergo]

Thank you for your contribution! This is now blocked on #536.

[magyargergo]

⚠️ Upcoming Prettier formatting — rebase instructions

PR #563 adds Prettier as the code formatter for the repo. When it merges, the bulk format commit will touch ~350 files (style-only: whitespace, quotes, trailing commas). Your branch will likely conflict.

After #563 merges, rebase your branch:

git fetch origin
git checkout <your-branch>
git rebase origin/main

# Conflicts will be formatting-only — accept your version:
git checkout --theirs .
git add .
git rebase --continue

# Then re-format your branch to match the new style:
npx prettier --write .
git add -A
git commit -m "style: apply prettier formatting"
git push --force-with-lease

New setup step: Run npm install at the repo root (not just in gitnexus/) to get prettier + activate the pre-commit hook. The hook auto-formats staged files on every commit going forward.

[magyargergo]

Please submit a new PR if this is still relevant