Skip to content

feat(chat): stream AI replies over SSE (#159) - #160

Merged
thomasluizon merged 1 commit into
mainfrom
feature/159-chat-sse-streaming
Jun 10, 2026
Merged

feat(chat): stream AI replies over SSE (#159)#160
thomasluizon merged 1 commit into
mainfrom
feature/159-chat-sse-streaming

Conversation

@thomasluizon

Copy link
Copy Markdown
Owner

Closes #159. Paired backend PR: thomasluizon/orbit-api#199 (merge that one first — the SSE endpoint must be live before these clients consume it).

Also carries the fix for the reported "liste meus hábitos de hoje is missing habits" bug — that fix is backend prompt rules in the paired PR (no client change needed); details there.

What streams now

Send a message → tokens render into the AI bubble as they arrive, on web and mobile identically. Typing dots show until the first token, then the bubble grows; tool-calling turns keep the dots during tool work (the backend emits round keepalives) and stream only the final answer.

  • shared owns the protocol: chatStreamEventSchema (mirrors the backend record — contract-aligner verified 0 drift), an incremental SSE parser (createChatSseParser) + stream consumer (consumeChatSseStream), new chat-store actions appendToMessageContent/updateMessage, and CHAT_STREAM_IDLE_TIMEOUT_MS = 60s replacing the whole-response CHAT_SEND_TIMEOUT_MS.
  • web: new dedicated BFF route app/api/chat/stream/route.ts that pipes the upstream body straight through (the catch-all proxy buffers, which would defeat streaming) with the same cookie-auth + single 401-refresh as the catch-all. The composer fetches it directly; the sendChatMessage Server Action is deleted (Server Actions can't stream).
  • mobile: new lib/chat-stream.ts on expo/fetch (RN's built-in fetch has no readable response body) with SecureStore bearer + single refresh rotation. TextDecoder comes from Expo's winter runtime — no polyfill needed.

Failure semantics (the #156 affordances survive)

  • The 60s bound is now an idle watchdog — reset on every received chunk, with round events covering long tool work. Firing aborts → 408 → the existing timeout copy + Retry button.
  • SSE error events and pre-stream HTTP failures feed the same classifySendFailure flow: timeout/generic arm Retry, limit/upgrade don't, premium denials still route to /upgrade.
  • The final event ALWAYS replaces the streamed draft content (covers the backend's StripJsonWrapper edge) and attaches actions/correlationId/relatedSurfaces; a reset clears the draft when a round turns out to be a tool round.
  • Quota counting, optimistic aiMessagesUsed bump, query invalidations: unchanged.

Tests

  • shared: parser (fragmentation, CRLF-split frames, comments, malformed-JSON resilience, forward-compatible unknown events) + consumer (terminal outcomes, incomplete-stream) + store actions. Suite: 875.
  • web composer suite rewritten against mocked fetch + real ReadableStream SSE bytes: incremental render, reset-no-duplicate, idle-watchdog timeout + retry, limit no-retry, retry-no-duplicate-user-bubble, pre-stream classification. Suite: 1,460.
  • mobile composer suite mirrors web on the openChatStream seam + offline gate + quota + pending-ops. Suite: 415.
  • parity-checker: all PAIRED. Lint + type-check clean across workspaces.

Manual verification checklist (auth-walled, needs your eyes)

  1. Web + mobile: ask a read-only question → text streams in; Render logs show First content token after Nms with N ≤ ~2000 after the final round.
  2. "liste meus hábitos de hoje" → EVERY due/overdue habit listed (Before Bed group included), habits already done today absent; single AI API responded round in Render logs.
  3. Tool turn ("create a habit to stretch daily") → answer streams after the tool work, action chips render, habit exists.
  4. Kill connectivity mid-stream on mobile → banner + Retry within ≤60s; Retry resends without duplicating your bubble.
  5. Free account at the message limit → limit copy, no Retry button; premium-gated reply still routes to /upgrade.
  6. Support request via chat still carries [trace: …] in the email.

🤖 Generated with Claude Code

…e-watchdog retry

- shared: chatStreamEventSchema, incremental SSE parser + stream consumer, CHAT_STREAM_IDLE_TIMEOUT_MS (replaces CHAT_SEND_TIMEOUT_MS), chat-store updateMessage/appendToMessageContent
- web: streaming BFF route (the catch-all proxy buffers), composer streams into a draft bubble with a 60s idle watchdog; sendChatMessage server action removed
- mobile: expo/fetch transport with SecureStore auth + single 401 refresh; composer mirrors web

Closes #159

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
orbit-ui-mobile-web Ignored Ignored Jun 10, 2026 6:22pm

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid end-to-end streaming implementation. The shared SSE parser and consumer are well-factored: CRLF normalization happens at concatenation time (so cross-chunk splits are handled correctly), unknown event types are silently dropped for forward compat, and the incomplete outcome cleanly covers the half-open case. Both platform hooks are structurally identical — same idle watchdog, same 401 refresh path, same final-always-wins semantics — which satisfies the cross-platform parity rule. The streaming BFF route correctly pipes the upstream body without buffering, mirrors the catch-all's auth pattern, and the x-accel-buffering: no header is the right touch for Nginx deployments. CHAT_SEND_TIMEOUT_MS and sendChatMessage are fully removed with zero remaining callers. Test coverage is thorough: fragmented frames, CRLF splits, comments/unknown events, watchdog fire, limit-no-retry, retry-no-duplicate-user-bubble, and pre-stream HTTP classification are all exercised.

@sonarqubecloud

Copy link
Copy Markdown

@thomasluizon
thomasluizon merged commit 48b4da7 into main Jun 10, 2026
8 checks passed
@thomasluizon
thomasluizon deleted the feature/159-chat-sse-streaming branch June 10, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AI chat: stream the final completion (SSE) so first tokens render within ~2s

1 participant