Skip to content

fix(conversations): bound reconnect walk + memoize turn hashes — 10-130s pre-routing stall on long agent histories - #10800

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
adevwithpurpose:fix/7847-prerouting-hot-path
Aug 20, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
adevwithpurpose:fix/7847-prerouting-hot-path

Conversation

@adevwithpurpose

Copy link
Copy Markdown
Contributor

Summary

resolveConversationId (the agentic conversation tracker, open-sse/services/conversationTracker.ts) blocks the chat request path for 10–130 s on long coding-agent histories. On a production deployment routing a real 1500-message / 92-tool agent session, measured POST → ROUTING latency was:

Request class (same instance, same window) n p50 max >10 s
≥ 200 messages 132 12.6 s 130.2 s 128
< 200 messages 70 0.2 s 0.6 s 0

The stall grows as the conversation grows (early turns 0.2 s → late turns 130 s), and its synchronous work also starves unrelated endpoints — a trivial GET /healthz on the same instance spiked to 11.1 s during large-request processing. This is the request-path face of the long-context cost family tracked in #7847.

Root cause

resolveConversationId runs once per chat request in the validate phase, before routing (src/sse/handlers/chat.ts, resolveConversationId call). To detect continuation it must find where the incoming turn list reconnects to a recorded chain. findReconnectMatch:

  • iterates every start turn × every duplicate anchor sharing that turn's content hash (real agentic traffic is full of byte-identical tool outputs — the module's own docs note 28 duplicates of a single turn in one real conversation), and
  • walks the chain forward, computing chainNodeId per step — two createHmac("sha256") calls per step, with the turn's full (often multi-KB) text re-hashed at every step (no memoization).

That is O(starts × anchors × walkLength) full-content HMACs, synchronous on the request path. A local repro of the exact duplicate-heavy shape (800 turns, alternating identical "ok" tool outputs and 40 KB file-content turns, second request editing every large turn) spent 17.2 s inside resolveConversationId on otherwise-idle hardware.

Fix (two surgical changes in conversationTracker.ts)

  1. Memoize turn content hashes per request. Each turn is hashed exactly once (turnHashes); every walk step then hashes ~130 fixed-size bytes (parentId + turnHash) instead of re-hashing the turn's full text twice. buildNewNodes reuses the same memoized hashes.
  2. Bound the walk with a shared step budget (DEFAULT_RECONNECT_MAX_STEPS = 150_000), charged one step per anchor claim and per forward step, shared across all fingerprint candidates. Budget exhaustion degrades exactly like a no-match — the request mints a new conversation — never an unverified attachment and never unbounded latency.

findReconnectMatch is now exported with { turnHashes, maxSteps, budget } options so both properties are directly unit-testable.

Result on the repro above: 17.2 s → 0.3 s (suite runs in ~1.2 s including DB setup). Selection semantics are unchanged: longest verified run still wins, ties still break toward the childless anchor, edits still mint a new conversation, compaction-window reconnection still works.

Tests

tests/unit/conversationTracker-reconnect-7847.test.ts (new):

  • budget cap: adversarial duplicate-anchor shape with maxSteps: 50 reports stepsUsed ≤ 50;
  • zero-budget degradation: a genuine 3-turn continuation matches with the default budget, and yields no match (not an unverified attach) with maxSteps: 0;
  • wall-clock regression guard: the duplicate-heavy 800-turn end-to-end resolve must complete under 2 s (fails at 17.2 s pre-fix);
  • DB-backed tests use an isolated temp DATA_DIR and close the DB handle in test.after.

All 17 existing conversationTracker.test.ts semantics tests pass unchanged, plus conversationTurnContent.test.ts (5). npm run typecheck:core clean; scoped eslint + prettier clean; full husky pre-commit gates passed.

Notes

…souzapw#7847)

resolveConversationId runs once per chat request in the validate phase,
before routing (src/sse/handlers/chat.ts). Its reconnect walk evaluates
every (start turn x duplicate anchor) pair and walks the chain forward,
computing two HMACs per step with the turn FULL text re-hashed at every
step. On long coding-agent histories (1000+ turns, heavily duplicated
tool outputs) that is O(starts x anchors x walkLength) re-hashing of
multi-KB content, synchronous on the request path.

Measured on a production deployment routing a real 1500-message /
92-tool agent session: POST -> ROUTING p50 12.6s (n=132, max 130s) for
requests >= 200 messages, while <200-message requests on the same
instance routed in 0.2s p50. A local repro of the exact duplicate-heavy
shape spent 17.2s inside resolveConversationId before this fix.

- Memoize hashTurnContent per request: each turn is hashed once; every
  walk step then hashes ~130 fixed-size bytes (parentId + turnHash).
- Bound the walk with a shared step budget (DEFAULT_RECONNECT_MAX_STEPS
  = 150000) across all fingerprint candidates, starts and anchors.
  Budget exhaustion degrades exactly like a no-match (a new conversation
  is minted downstream), never an unverified attachment.
- Export findReconnectMatch with { turnHashes, maxSteps, budget } so the
  bound and memoization are directly unit-testable.

Tests: tests/unit/conversationTracker-reconnect-7847.test.ts pins the
budget cap, zero-budget no-match degradation, and a wall-clock bound on
a duplicate-heavy 800-turn resolve. All existing conversationTracker
semantics tests pass unchanged.
@diegosouzapw
diegosouzapw merged commit 12d0acb into diegosouzapw:release/v3.8.50 Aug 20, 2026
5 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…souzapw#7847) (diegosouzapw#10800)

Obrigado — bug de produção real e muito bem raiz-causado: resolveConversationId travava a request path por 10-130s em históricos longos de agente (medido em produção: p50 12.6s / max 130.2s em requests com ≥200 mensagens), por re-hashear o texto completo de cada turno a cada passo do walk de reconexão (O(starts × anchors × walkLength) HMACs síncronos).

Fix cirúrgico: memoiza o hash de cada turno por request + budget de passos compartilhado entre candidatos (degrada como no-match, nunca como attach não verificado ou latência ilimitada). Resultado medido: 17.2s → 0.3s no repro.

Validação (worktree combinado a partir de origin/release/v3.8.50, 0 conflitos):
- typecheck:core limpo, complexity/cognitive-complexity dentro do baseline
- tests/unit/conversationTracker-reconnect-7847.test.ts (novo) — cobre o cap de budget, degradação com budget zero, e guarda de regressão de wall-clock (falha em 17.2s pré-fix)
- 17 testes de semântica pré-existentes + conversationTurnContent (5) — passando sem alteração
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants