fix(cli): route openai-compatible providers through streamText for memory consolidation

[rusak47]

Issue

Fixes # — (no reported issue; related to #12674)

Context

OpenAI-compatible providers (api.npm === "@ai-sdk/openai-compatible"), including the 9router -> opencode.ai/zen -> DeepSeek chain used during memory consolidation, return SSE bodies even on non-streaming generateText requests. The SDK cannot parse them, raising Invalid JSON response, which surfaces as the recurring DeepSeek 400s and failed memory consolidation.

Implementation

Extends the existing openai branch condition in memoryText (packages/opencode/src/kilocode/memory/ports.ts) to also match openai-compatible, so those providers are routed through streamText, which already consumes SSE parts correctly. Additive single-line condition (+3/-1); no behavior change for the non-openai path.

This commit carries only the routing fix

Screenshots / Video

N/A — no UI or end-user-visible behavioral change.

How to Test

Manual/local verification

• bun test test/kilocode/memory/memory-ports.test.ts on fix/memory-streamtext: 3 pass / 11 expect calls (base memory-ports suite).
• bun run typecheck (@kilocode/cli + @kilocode/kilo-memory): clean, on this branch and on the ort-merged combination with memory-output-cap.

Reviewer test steps

1. Configure an OpenAI-compatible provider in opencode.json (e.g. a 9router/DeepSeek entry exposed via api.npm: "@ai-sdk/openai-compatible").
2. Start a session using that provider and let it reach the idle-consolidation trigger (or force a digest).
3. Observe memory consolidation completes (the memory.updated bus event fires) with no Invalid JSON response in the log. (Without this change, the same flow raised the parse error.)

Blocked checks and substitute verification

• bun test / typecheck against live DeepSeek traffic could not run (no DeepSeek/9router credentials in CI/local). Substituted: the ort merge of routing-only + config-cap passed all tests on the combined tree, and the routing target (streamText) is the existing, test-covered branch.
• A targeted unit test exercising api.npm === "@ai-sdk/openai-compatible" is left for follow-up to avoid coupling this fix to mock-LLM harness work.

Checklist

• Issue linked above, or exception explained (exception noted: no issue; local cleanup).
• Tests/verification described (see How to Test; agent: test runner + tsgo. Human: reviewer steps 1-3 require credentials.)
• Screenshots/video — N/A (non-visual change).
• Changeset considered for user-facing changes (added .changeset/memory-streamtext-openai.md, patch).
• I personally reviewed the diff, tested locally and can explain the changes, including any AI-assisted work.

Get in Touch

realcash-4u@inbox.lv

[kilo-code-bot]

Code Review Summary

Status: 1 Issue Found | Recommendation: Suggestion only — safe to merge once considered

Overview

Severity	Count
CRITICAL	0
WARNING	0
SUGGESTION	1

Issue Details (click to expand)

SUGGESTION

File	Line	Issue
packages/opencode/src/kilocode/memory/ports.ts	221	The first consolidation per provider/model on an affected backend still pays a failed generateText plus a full streamText retry, and both attempts share the single timeoutMs budget (timer starts before work() is raced), so a slow first attempt can surface as memory model timed out instead of a result

Files Reviewed (2 files)

• packages/opencode/src/kilocode/memory/ports.ts - 1 issue
• packages/opencode/test/kilocode/memory/memory-ports.test.ts - 0 issues

Resolved in the latest commits: streamNeeded.add(key) now happens only after consumeStream actually succeeds, so a failed fallback no longer pins the model; resetStreamNeeded() plus beforeEach makes the memoization observable and the compat tests order-independent (they no longer need invented model ids); and the dead/shadowing mid parameter on compatLanguage / compatProvider is gone.

Residual note on the memo (not re-raised inline): the Set still has no expiry, so a single transient parse error keeps a model on streamText for the rest of the process. Acceptable given the reset hook, but Instance.state scoping would bound it more naturally.

No memory-leak concerns: the timeout timer is cleared and the controller aborted in finally, the fallback reuses the same abort signal, and streamNeeded is bounded by the number of configured provider/model pairs.

Notes/assumptions: incremental review of d815e31..0ef24bc only. Tests were not executed and dependencies are not installed here, so the exact APICallError -> JSONParseError cause chain in @ai-sdk/provider-utils is still inferred rather than confirmed from source. Indentation drift in the new stay on generateText when it succeeds test is left to CI formatting.

Fix these issues in Kilo Cloud

Previous Review Summaries (3 snapshots, latest commit d815e31)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit d815e31)

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	0
SUGGESTION	3

Issue Details (click to expand)

SUGGESTION

File	Line	Issue
packages/opencode/src/kilocode/memory/ports.ts	182	streamNeeded is a process-global, never-reset cache: one transient JSONParseError permanently pins a model to streamText, and the entry is added before the fallback is known to succeed. It also makes this file's tests order-dependent (the new cases use distinct model ids only to dodge pollution)
packages/opencode/src/kilocode/memory/ports.ts	217	The first consolidation per model on an affected provider still pays a failed generateText plus a full streamText retry, and both share the single timeoutMs budget
packages/opencode/test/kilocode/memory/memory-ports.test.ts	91	The new mid parameter on compatLanguage is never forwarded from compatProvider (line 145), so the mock's modelId stays "compat"; it also shadows the module-level mid

Files Reviewed (2 files)

• packages/opencode/src/kilocode/memory/ports.ts - 2 issues
• packages/opencode/test/kilocode/memory/memory-ports.test.ts - 1 issue

Resolved since the last review: the guard now also unwraps APICallError.cause, which is the shape the SDK's JSON response handler actually throws; a test covers both the wrapped and bare JSONParseError forms; native / compat replace the snake_case locals; and the unrelated recalledMemory / Effect.catch reformatting is gone, so the diff against base is now confined to the fix.

No memory-leak concerns: the timeout timer is cleared and the controller aborted in finally, the fallback reuses the same abort signal, and the new Set is bounded by the number of configured provider/model pairs.

Notes/assumptions: dependencies are not installed in this environment, so the exact APICallError -> JSONParseError cause chain in @ai-sdk/provider-utils could not be confirmed from source; it is inferred from the Invalid JSON response message. Tests were not executed (read-only mode).

Fix these issues in Kilo Cloud

Previous review (commit d3451d0)

Status: 5 Issues Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	2
SUGGESTION	3

Issue Details (click to expand)

WARNING

File	Line	Issue
packages/opencode/src/kilocode/memory/ports.ts	217	JSONParseError.isInstance(err) likely never matches: the SDK's createJsonResponseHandler rethrows the parse failure as APICallError("Invalid JSON response", { cause: JSONParseError }), so the streamText fallback may never run
packages/opencode/test/kilocode/memory/memory-ports.test.ts	390	The mock throws a bare JSONParseError, an error shape the real provider path never produces, so the test passes even if the production condition misses

SUGGESTION

File	Line	Issue
packages/opencode/src/kilocode/memory/ports.ts	209	native_openai / compatible_openai use snake_case; prefer short camelCase (native / compat)
packages/opencode/src/kilocode/memory/ports.ts	215	Providers that always emit SSE pay a failed generateText plus a full streamText retry on every consolidation, and the retry shares the original timeout budget; consider memoizing the outcome per provider/model
packages/opencode/src/kilocode/memory/ports.ts	138	Unrelated reformatting (recalledMemory, the Effect.catch pipe block) widens the diff on a shared-fork file

Files Reviewed (3 files)

• packages/opencode/src/kilocode/memory/ports.ts - 4 issues
• packages/opencode/test/kilocode/memory/memory-ports.test.ts - 1 issue
• .changeset/memory-streamtext-openai.md - 0 issues

Previously reported issues are resolved: the changeset now targets @kilocode/cli with a user-facing description, and the broad openai-compatible routing is now a narrow error-triggered fallback rather than an unconditional streamText path.

Notes/assumptions: dependencies are not installed in this environment and network fetches were unavailable, so the APICallError-wraps-JSONParseError behavior of @ai-sdk/provider-utils@4.0.40 could not be confirmed directly; it is inferred from the Invalid JSON response message quoted in the PR description, which is APICallError's message. No memory-leak concerns: the timer is cleared and the controller aborted in finally, and the fallback reuses the same abort signal.

Fix these issues in Kilo Cloud

Previous review (commit 039ebfe)

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	2

Issue Details (click to expand)

WARNING

File	Line	Issue
packages/opencode/src/kilocode/memory/ports.ts	186	@ai-sdk/openai-compatible is the default npm fallback for undeclared providers and covers all local/self-hosted entries, so this routes far more providers through streamText than the one that is broken, with no generateText fallback and no test pinning the branch choice

SUGGESTION

File	Line	Issue
packages/opencode/src/kilocode/memory/ports.ts	184	openai const now means "stream the response"; rename (e.g. stream) to avoid two different predicates named openai in the same file
.changeset/memory-streamtext-openai.md	2	CLI-only change declares the kilo-code package; prior CLI changesets use @kilocode/cli, and the description is implementation-flavoured for release notes

Files Reviewed (2 files)

• packages/opencode/src/kilocode/memory/ports.ts - 2 issues
• .changeset/memory-streamtext-openai.md - 1 issue

Notes/assumptions: the underlying claim that these providers emit SSE on non-streaming requests could not be verified from the code (nothing in provider.ts forces streaming), and no credentialed provider run was possible in this environment, so the risk assessment is based on how broadly the new condition matches. No memory-leak concerns found — the timer is cleared and the controller aborted in finally.

Fix these issues in Kilo Cloud

---

_{Reviewed by claude-opus-5 · Input: 28 · Output: 8.8K · Cached: 714.2K}

_{Review guidance: REVIEW.md from base branch main}

[rusak47]

Follow-up: streamText fix — captured Invalid JSON response evidence (2026-08-01)

For the pending PR that routes openai-compatible memory capture calls through streamText
(fix/memory-streamtext, #12748). This note documents the exact failure the fix targets, with
trimmed evidence; full data in tmp/ of this repo.

Received issue

Both digest and typed consolidation fall back with Invalid JSON response statusCode=200.
Debug run, 17:46 UTC — see trimmed lines in tmp/drafts/attachments/streaming-pr-followup-9router-evidence/devlog-memory-error-2026-08-01.txt:

summary=consolidate error=Invalid JSON response statusCode=200 body={"id":"22fb5253-...","model":"deepseek-v4-f...
kind=digest result=fallback ... reason=Invalid JSON response statusCode=200 body={"id":"gen-...","model":"poolside/laguna-s-2.1:free",...
kind=typed  result=fallback ... reason=Invalid JSON response statusCode=200 body={"id":"22fb5253-...","model":"deepseek-v4-flash",...

Root cause (9router provider path)

The provider returns a valid single JSON object (5_res_provider.txt, 7700 B, ends "cost":"0"})
but the client receives that JSON plus a trailing SSE terminator (7_res_client.txt, 7714 B):

...,"cost":"0"}data: [DONE]\n\n

A non-streaming JSON.parse throws on the appended data: [DONE] → SDK InvalidJSONResponseError
→ capture falls back. Byte-level diff in tmp/drafts/attachments/streaming-pr-followup-9router-evidence/9router-evidence-data-done-suffix.txt.
Reproduced across 12 request/response dirs (big-pickle + kilo-auto-free) in the same window.

Why this validates the streamText fix

streamText consumes the SSE data: frames properly (and never concatenates [DONE] onto a JSON
body), so this class of transport failure disappears at the root for both consolidate and digest.

AFTER — fix validated in live logs (2026-08-01 19:02–19:20 UTC)

Post-fix kilo binary (built on fix/memory-streamtext): Invalid JSON response count drops
6 → 0 while the data: [DONE] transport artifact is still produced by 9router — proving the
consumer, not the provider, changed. Both previously-failing models now consolidate with
parsed=true, fallback=false:

• deepseek-v4-flash (target big-pickle): capture big-pickle_20260801_191315_681 (provider
  1025009 B vs client 1025060 B, +51 B artifact) → digest saved tokens=5371, typed saved tokens=10059 ops=3 (dev.log L96/L100/L101).
• poolside/laguna-s-2.1:free (target kilo-auto/free): capture kilo-auto-free_20260801_190223_842
  (provider 33123 B vs client 33147 B, +24 B artifact) → typed saved tokens=7276 and tokens=7200
  (dev.log L63/L64, L75/L76).

Trimmed lines: devlog-memory-after-fix-2026-08-01.txt in the attachments below.

Attachments (relevant parts only)

• 9router-evidence-data-done-suffix.txt — the provider-vs-client byte diff.
• devlog-memory-error-2026-08-01.txt — trimmed DEBUG lines showing both fallbacks.
• devlog-memory-after-fix-2026-08-01.txt — trimmed DEBUG lines showing both models succeeding (parsed=true, saved).
  devlog-memory-error-2026-08-01.txt
  9router-evidence-data-done-suffix.txt
  devlog-memory-after-fix-2026-08-01.txt