feat: add LOD graph rendering and performance optimizations

[naicud]

Summary

Adds Level-of-Detail (LOD) graph rendering to handle large codebases (10K+ nodes) that would overwhelm the browser, plus performance optimizations and a major UI restructuring.

LOD Rendering

Three rendering levels that progressively reveal detail:

1. Summary view — cluster-level super-nodes showing functional areas
2. Hierarchy view — expandable cluster groups with child counts
3. Full view — complete node graph (with hard cap protection)

Users click to expand clusters, drill into neighborhoods, and navigate large graphs fluidly.

Performance

• NODE_HARD_CAP — server-side limit prevents sending too many nodes to browser
• Graph truncation — buildGraph() with limit/truncation and truncated/totalAvailable response metadata
• buildStructuralSummaryFromDb() — server-side summary generation from graph DB
• fastStripNullable() — type resolution optimization
• skipGraphPhases — pipeline option to skip expensive graph phases

New API Endpoints

/api/graph/{info, summary, expand, neighbors, hierarchy, ancestors, neighbor-counts, schema}

UI Restructuring

• useAppState refactored into 6 per-concern hooks: useGraphState, useFilterState, useChatState, useWorkerState, useKeyboardShortcuts, useStyleConfig
• 6 new components: DataExplorer, ContextMenu, NeighborPanel, SchemaGraph, SearchPanel, StylingPanel
• Graph adapters: summary-graph-adapter.ts, hierarchy-graph-adapter.ts for LOD data transformation

Changes (37 files)

Area	Key files	What
Backend	api.ts, graph-summary.ts, pipeline.ts	LOD endpoints, summary builder
Web services	graph-lod.ts, summary-graph-adapter.ts, hierarchy-graph-adapter.ts	LOD data layer
Components	6 new + GraphCanvas.tsx, App.tsx	LOD UI, exploration tools
Hooks	6 new + useAppState.tsx, useSigma.ts	State management refactor
Adapters	graph-adapter.ts, constants.ts	Batch import, ClusterGroup type

Test plan

• Build passes: cd gitnexus && npm run build && cd ../gitnexus-web && npm run build
• Load a large repo (1000+ nodes) — should show summary view by default
• Click a cluster group to expand into hierarchy view
• Click "Show all" to see full graph (with truncation warning if applicable)
• Right-click a node for context menu with neighbor exploration
• DataExplorer panel shows node details and neighbor counts
• Schema graph shows type-level overview

🤖 Generated with Claude Code

[magyargergo]

@claude ultra-think while reviewing this pull request and check if performance improvements have been achieved

[github-actions]

Claude finished @magyargergo's task in 3m 35s —— View job

---

PR Performance Review — feat: add LOD graph rendering and performance optimizations

Tasks

• Understand the request
• Read backend API changes (api.ts, graph-summary.ts, pipeline.ts)
• Read LOD service layer (graph-lod.ts, adapters)
• Read frontend state/hook refactor (useAppState.tsx, new hooks)
• Read GraphCanvas changes and worker changes
• Read performance-critical utilities (fastStripNullable, buildGraph with limit)
• Analyze whether claimed performance improvements are real and effective
• Post comprehensive review

---

Summary

The core LOD (Level-of-Detail) architecture is genuinely effective for its stated goal of preventing browser overload on large repos. Most claimed optimizations check out, though several real issues deserve attention.

---

✅ Verified Performance Improvements

1. NODE_HARD_CAP + Graph Truncation — Real, Effective

gitnexus/src/server/api.ts:26–108

const NODE_HARD_CAP = 20000;
const buildGraph = async (limit?: number) => { ... }

The hard cap prevents sending > 20K nodes to the browser in one shot. The multi-table loop early-exits when the cap is hit, and the truncated/totalAvailable response metadata is correct. Edge filtering is also pushed into Cypher rather than JavaScript:

// Push edge filtering to Cypher (avoid fetching all edges for large repos)
WHERE a.id IN $ids AND b.id IN $ids

This is the most impactful change in the PR. ✅

2. Graph Summary Precomputed and Cached — Real

gitnexus/src/core/ingestion/pipeline.ts:463–475

During pipeline, generateGraphSummary() writes graph-summary.json to disk. The /api/graph/summary endpoint reads it with a single fs.readFile() — O(1) vs O(E) query. ✅

3. Batch Graph Import — Real

gitnexus-web/src/lib/graph-adapter.ts:344–345 and summary-graph-adapter.ts:114–115

graph.import({ nodes: batchNodes, edges: batchEdges });

Both adapters use batch import, eliminating per-node event emissions that would trigger Sigma.js re-renders during the add loop. ✅

4. fastStripNullable — Elegant, Measurable

gitnexus/src/core/ingestion/type-env.ts:89–94

const fastStripNullable = (typeName: string): string | undefined => {
  if (FAST_NULLABLE_KEYWORDS.has(typeName)) return undefined;
  return (typeName.indexOf('|') === -1 && typeName.indexOf('?') === -1)
    ? typeName
    : stripNullable(typeName);
};

Called on every type resolution lookup. For 90%+ of identifiers (plain User, Order, etc.), it skips the full nullable parse and returns the string directly. A genuine hot-path optimization that compounds across large repos with many type annotations. ✅

5. skipGraphPhases — Correct

gitnexus/src/core/ingestion/pipeline.ts:39, 342

Bypasses MRO, community detection, and process extraction. Makes test runs faster and is non-breaking. ✅

6. Lazy Neighbor Count Query

gitnexus/src/server/api.ts:742–757

// Skip expensive count query if we got fewer than the limit
if (nodes.length >= limit) { ... count query ... }

Smart: if the result is less than the limit it's obviously the full count. Saves a round trip in the common case. ✅

7. buildStructuralSummaryFromDb Timeout Guard

api.ts:200–204

await Promise.race([
  sampleEdges(),
  new Promise((_, reject) => setTimeout(() => reject(...), 15000)),
]);

Prevents the structural summary builder from blocking indefinitely on large repos. ✅

---

⚠️ Real Issues Found

1. Cypher Injection in Multiple Endpoints — Medium Risk

gitnexus/src/server/api.ts:551–578

WHERE c.heuristicLabel = '${groupLabel.replace(/'/g, "\\'")}'

groupLabel from the query string is interpolated directly into Cypher with only single-quote escaping. Similar patterns appear at lines 577, 593, 599, 864, 869, 897. executeParameterizedQuery already exists and is used correctly in the neighbors endpoint — it should be used here too. While the server only binds to 127.0.0.1 by default, the pattern is dangerous and inconsistent.

Fix this →

2. LOD Mode Threshold Mismatch

gitnexus/src/server/api.ts:335

const mode = (totalNodes > 100000) ? 'hierarchy' : (totalNodes > 50000) ? 'summary' : 'full';

NODE_HARD_CAP is 20,000, but the mode doesn't switch to 'summary' until 50,000 nodes. Repos with 20K–50K nodes get mode: 'full' from the info endpoint, but then buildGraph silently truncates them. Users see an incomplete graph without a clear explanation. The threshold should be lowered to match the cap, or the full mode should be blocked when truncation is active.

Fix this →

3. buildGraph Fires N Extra Count Queries

gitnexus/src/server/api.ts:74–80

// Count total available for this table
const countRows = await executeQuery(`MATCH (n:${table}) RETURN count(n) AS cnt`);

After fetching nodes from each table, another query runs just to count them. For ~12 node table types, that's 12 extra round trips every time the full graph is loaded. These counts could be included in the first query (e.g., WITH count(n) OVER () AS total), or queried once upfront per table using a single combined count query.

4. O(n) getVisibleNodeCount Before Every Expansion

gitnexus-web/src/lib/hierarchy-graph-adapter.ts:272–278

export function getVisibleNodeCount(graph) {
  let count = 0;
  graph.forEachNode((_, attrs) => { if (!attrs.hidden) count++; });
  return count;
}

Called via wouldExceedBudget() before every hierarchy expansion. Graphology's graph.order already returns total nodes — and in this use case nearly all nodes are visible, so graph.order would be sufficient and O(1). For a hierarchy tree with NODE_BUDGET = 5000 this won't be a bottleneck, but it's semantically wrong and can be simplified.

5. useMemo Composition Doesn't Prevent Consumer Re-renders

gitnexus-web/src/hooks/useAppState.tsx:208–309

const value = useMemo(() => ({
  ...graphState,
  ...filterState,
  ...workerState,
  ...chatState,
}), [graphState, filterState, workerState, chatState]);

Any state change in any of the 4 hooks (e.g., a single chat message arrives) invalidates the memo and creates a new value object — causing every component subscribed to useAppState() to re-render. Splitting into 6 hooks is a great maintainability improvement, but the re-render behavior is unchanged from the monolith. To actually improve render performance, components would need to subscribe to specific sub-contexts (e.g., useChatState directly) rather than the composed useAppState. This is a known trade-off with the pattern but worth documenting.

6. File Content Included in Full Graph Load

gitnexus/src/server/api.ts:39, 47

query = `MATCH (n:File) RETURN n.id AS id, n.name AS name, n.filePath AS filePath, n.content AS content LIMIT ${remaining}`;
// ...
query = `MATCH (n:${table}) RETURN ... n.content AS content LIMIT ${remaining}`;

n.content is fetched for every File node and every symbol node in the full graph load. For repos with hundreds of files, content alone could add megabytes to the JSON response. The graph visualization never uses content — it's only needed by the chat/search tools. Stripping content from the graph endpoint would significantly reduce payload size and deserialization time.

Fix this →

---

🔍 Minor Notes

• Non-deterministic positions: expandGroupInGraph and expandNodeInHierarchy use Math.random() for jitter/curvature (summary-graph-adapter.ts:193, hierarchy-graph-adapter.ts:110). Collapsing and re-expanding a group produces a different layout each time.
• collapseGroupInGraph is O(E): Iterates all summary.interGroupEdges to reconnect edges on collapse (summary-graph-adapter.ts:278). Fine for small summaries but will slow down as the number of inter-group edges grows.
• Duplicate edge aggregation logic: generateGraphSummary and generateStructuralSummary in graph-summary.ts share nearly identical edge counting loops (lines 112–145 and 256–291). Could be extracted to a shared helper.
• Coverage regression on new code: The CI report shows unit test coverage dropped 3–4% across all metrics on this branch. The new LOD endpoints, adapters, and hooks have no dedicated unit tests. The full graph endpoint test coverage depends on integration tests.

---

Verdict

The core LOD rendering architecture is sound and the performance gains are real for the primary use case (preventing browser OOM on 10K+ node repos). The fastStripNullable optimization is a clean hot-path improvement. The precomputed summary cache is an effective O(1) serving strategy.

The most important issue to address before merge is the LOD mode threshold mismatch (#2), which causes silent data loss for mid-size repos. The Cypher injection patterns (#1) should also be fixed for consistency even on localhost. The file content in graph payload (#6) is a potentially significant bandwidth optimization that's straightforward to fix.

[zander-raycraft]

@naicud Can you review the comment above and address that in another commit to this branch, good work so far!

[magyargergo]

Please resolve the merge conflicts too 🙏

[github-actions]

CI Report

✅ All checks passed   ab600c6

Pipeline

Stage	Status	Ubuntu	Windows	macOS
Typecheck	✅ success	—	—	—
Tests	✅ success	✅	✅	✅

Tests

Metric	Value
Total	3655
Passed	3635
Skipped	20
Files	1046
Duration	1m 48s

✅ All 3635 tests passed across 1046 files

20 test(s) skipped

• buildTypeEnv > known limitations (documented skip tests) > Ruby block parameter: users.each { |user| } — closure param inference, different feature
• Python match/case as-pattern type binding > resolves u.save() to User#save via match/case as-pattern binding
• Python match/case as-pattern type binding > does NOT resolve u.save() to Repo#save (negative disambiguation)
• Swift constructor-inferred type resolution > detects User and Repo classes, both with save methods
• Swift constructor-inferred type resolution > resolves user.save() to Models/User.swift via constructor-inferred type
• Swift constructor-inferred type resolution > resolves repo.save() to Models/Repo.swift via constructor-inferred type
• Swift constructor-inferred type resolution > emits exactly 2 save() CALLS edges (one per receiver type)
• Swift self resolution > detects User and Repo classes, each with a save function
• Swift self resolution > resolves self.save() inside User.process to User.save, not Repo.save
• Swift parent resolution > detects BaseModel and User classes plus Serializable protocol
• Swift parent resolution > emits EXTENDS edge: User → BaseModel
• Swift parent resolution > emits IMPLEMENTS edge: User → Serializable (protocol conformance)
• Swift cross-file User.init() inference > resolves user.save() via User.init(name:) inference
• Swift cross-file User.init() inference > resolves user.greet() via User.init(name:) inference
• Swift return type inference > detects User class and getUser function
• Swift return type inference > detects save function on User (Swift class methods are Function nodes)
• Swift return type inference > resolves user.save() to User#save via return type of getUser() -> User
• Swift return-type inference via function return type > resolves user.save() to User#save via return type of getUser()
• Swift return-type inference via function return type > user.save() does NOT resolve to Repo#save
• Swift return-type inference via function return type > resolves repo.save() to Repo#save via return type of getRepo()

Coverage

Metric	Coverage	Covered	Base (main)	Delta
Statements	57.07%	9471/16594	68.96%	📉 -11.9%
Branches	50.71%	6346/12513	60.09%	📉 -9.4%
Functions	58.36%	858/1470	71.47%	📉 -13.1%
Lines	58.75%	8460/14398	71.22%	📉 -12.5%

---

_{📋 Full run · Coverage from Ubuntu · Generated by CI}

[naicud]

Working on :)

[naicud]

@zander-raycraft Check now plz and tell me if you need other changes sorry for the delay :)

[zander-raycraft]

@magyargergo This looks good to me how does it look to you?

[zander-raycraft]

@naicud Thank you!

[magyargergo]

@magyargergo This looks good to me how does it look to you?

I think there a lot of unrelated changes. I'll take a closer look.

[magyargergo]

@naicud may i ask you to lean this pr down to just strictly to the level of detail performance improvement changes? 🙏

[magyargergo]

@claude can you identify the group of changes in this PR? I feel like there's more happening here than just the LOD? What other categories are there? Can they be split up?

[github-actions]

Claude finished @magyargergo's task in 3m 7s —— View job

---

PR Change Category Analysis

• Understand the request
• Analyze changed files and categorize by concern
• Assess splitability for each category
• Post findings

---

Categories of Changes

You're right — this PR contains at least 6 distinct feature groups bundled together. Here's the breakdown across the 100 changed files and 65 commits:

---

1. LOD Graph Rendering (the intended scope) — ~36 files

The core feature. Entirely within the squashed feat: add LOD graph rendering commit (236ee06) plus follow-up review-fix commits.

Files	What
gitnexus/src/server/api.ts	LOD endpoints, NODE_HARD_CAP, buildGraph
gitnexus/src/core/ingestion/graph-summary.ts	Summary builder
gitnexus/src/core/ingestion/pipeline.ts	skipGraphPhases
gitnexus-web/src/lib/hierarchy-graph-adapter.ts	LOD adapter
gitnexus-web/src/lib/summary-graph-adapter.ts	LOD adapter
gitnexus-web/src/services/graph-lod.ts	LOD service layer
gitnexus-web/src/components/ContextMenu.tsx, NeighborPanel.tsx	LOD UI (tightly coupled)
gitnexus-web/src/components/GraphCanvas.tsx	LOD canvas rendering
gitnexus-web/src/lib/graph-adapter.ts	Batch import refactor

---

2. useAppState Refactor — ~8 files

useAppState.tsx was split into 6 per-concern hooks. This is a useful refactor but has no functional dependency on LOD — it could stand alone as a separate PR.

Files	What
useAppState.tsx (-1086 lines)	Refactored to delegate
useChatState.ts, useFilterState.ts, useGraphState.ts, useKeyboardShortcuts.ts, useStyleConfig.ts, useWorkerState.ts	6 new hooks extracted
useSigma.ts	LOD-related Sigma updates

---

3. AWS Bedrock Integration — ~4 files

Commits like 9ba906a feat: implement AWS Bedrock proxy, 10433c5 Chat with AWS. These belong in PR #358 (already approved as a separate PR per naicud's comment).

Files	What
gitnexus-web/src/core/llm/bedrock-browser.ts (506 lines, ADDED)	Bedrock client
gitnexus-web/src/core/llm/agent.ts, settings-service.ts, types.ts	Bedrock integration
gitnexus-web/src/components/SettingsPanel.tsx (+474 lines)	Bedrock settings UI

---

4. COBOL Language Support — ~8 files

Commits c54869d, 0029eae, 855c2bb, 354fb66. These belong in PR #359.

Files	What
gitnexus/scripts/patch-tree-sitter-cobol.cjs (ADDED, 188 lines)	Tree-sitter COBOL patch
gitnexus-web/src/core/ingestion/tree-sitter-queries.ts	COBOL queries
gitnexus-web/src/core/tree-sitter/parser-loader.ts	COBOL parser
docs/code-indexing/cobol/ (7 files)	COBOL docs
gitnexus/package.json, package-lock.json	COBOL deps

---

5. TUI Wizards + Embed CLI — ~12 files

Commits b887777 feat(tui): add interactive wizards, c583f49 feat: add embed command. These belong in PR #361.

Files	What
gitnexus/src/cli/tui/ (3 files ADDED)	Interactive TUI
gitnexus/src/cli/embed.ts, embed-config.ts, assess.ts (ADDED)	Embed/assess CLI
gitnexus/src/cli/analyze.ts (+297/-145)	Analyze wizard
gitnexus/src/cli/index.ts, clean.ts, list.ts, serve.ts, setup.ts, status.ts	CLI refactors
analyze.sh, build.sh	Shell scripts

---

6. Documentation — ~22 files

All docs/ additions (pipeline, worker, languages, COBOL, Neptune, TUI, embeddings), plus gitnexus/README.md and gitnexus-web/README.md. These were added across multiple feature commits and are coupled to their respective features.

---

7. Skills / Config Files — ~8 files

.claude/skills/, AGENTS.md, CLAUDE.md, CHANGELOG.md, gitnexus-claude-plugin/skills/, gitnexus/skills/. Minor metadata updates across multiple features.

---

Why the Extra Features Appear in This PR

PRs #358–#361 were authored on a shared working branch that eventually became this PR (#362). Those PRs were reviewed and approved but haven't been merged into main yet — so their changes still show up in the diff of this PR against main.

Can They Be Split?

Split	Feasibility	Notes
Merge #358–#361 into main first, then rebase this branch	✅ Easiest path	Most of the "extra" diff vanishes automatically after those merges land
Extract useAppState refactor as its own PR	✅ Feasible	No LOD dependency; clean 8-file scope
Keep ContextMenu + NeighborPanel with LOD	Recommended	They are LOD-specific UI — separating them adds no value
New standalone UI panels (SchemaGraph, SearchPanel, CypherConsole)	⚠️ Possible but coupled	These are wired into App.tsx alongside LOD rendering

Recommended Path

The cleanest approach for @naicud:

1. Merge PRs feat: add AWS Bedrock integration for LLM chat #358, feat: add COBOL language support with regex extraction pipeline #359, feat: add AWS Neptune as alternative graph database backend #360, feat: add TUI wizards, interactive modes, and embed CLI #361 into main (they're already approved)
2. Rebase pr/performance-lod-rendering onto the new main — the COBOL, Bedrock, Neptune, TUI changes will disappear from the diff automatically
3. What remains will be the true LOD scope: ~36 files instead of 100

The useAppState refactor is the only in-scope question — it's bundled inside the LOD squash commit so separating it would require splitting that commit. Given it's closely related to making useGraphState work for LOD, keeping it together is a reasonable call.

[magyargergo]

@naicud can you please have a look at this? We need to split this change to reduce any risks in the future and it helps us with reviewing your changes more easier.

[magyargergo]

⚠️ Upcoming Prettier formatting — rebase instructions

PR #563 adds Prettier as the code formatter for the repo. When it merges, the bulk format commit will touch ~350 files (style-only: whitespace, quotes, trailing commas). Your branch will likely conflict.

After #563 merges, rebase your branch:

git fetch origin
git checkout <your-branch>
git rebase origin/main

# Conflicts will be formatting-only — accept your version:
git checkout --theirs .
git add .
git rebase --continue

# Then re-format your branch to match the new style:
npx prettier --write .
git add -A
git commit -m "style: apply prettier formatting"
git push --force-with-lease

New setup step: Run npm install at the repo root (not just in gitnexus/) to get prettier + activate the pre-commit hook. The hook auto-formats staged files on every commit going forward.

[magyargergo]

Please submit a new PR if this is still relevant