Support TanStack Start server functions in stream() adapter - #2
Closed
tombeckenham wants to merge 44 commits into
Closed
Support TanStack Start server functions in stream() adapter#2tombeckenham wants to merge 44 commits into
tombeckenham wants to merge 44 commits into
Conversation
…tructured output support, and refresh models (TanStack#312) * Update openrouter package, models, and scripts to generate openrouter models Fixes TanStack#310 * ci: apply automated fixes * Added compare script and fixed key name issue in fetch models Fixes TanStack#310 * resolved coderabbit issues * ci: apply automated fixes * Added support for openrouter structured output * Address PR TanStack#312 review feedback: improve error handling and cleanup - Add explicit guard for empty content in structuredOutput before JSON.parse - Remove redundant Sets in compare script (Map.has() is already O(1)) - Suppress stderr leaks from execSync in compare script - Update example to use valid model name Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update model metadata and pricing in OpenRouter, add new models, and refactor parameter handling - Introduced new models: AI21 Jamba Large 1.7, AionLabs Aion-1.0, AionLabs Aion-1.0 Mini, AionLabs Aion-2.0, AionLabs Aion-RP Llama 3.1 8B, AlfredPros CodeLLaMa 7B Instruct Solidity, and Tongyi DeepResearch 30B A3B. - Updated existing model parameters, including context windows and max output tokens. - Refactored parameter handling in scripts to improve consistency and readability, including the introduction of a mapping function for API parameters. - Adjusted pricing structures for several models to reflect updated costs. - Ensured all model entries are sorted for better organization. * ci: apply automated fixes * Update openrouter package models and added prettier to fetch script Fixes TanStack#310 * Updated models and script Fixes TanStack#310 * ci: apply automated fixes * Update openrouter package to 0.9.11 Fixes TanStack#310 * Refactor OpenRouter options passthrough and bump fal client Spread modelOptions first in request construction so provider-specific options pass through correctly, only override with explicit options when defined. Remove unused InternalTextProviderOptions import. Bump @fal-ai/client to ^1.9.4. Fixes TanStack#310 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor text-provider-options to derive types from SDK's ChatGenerationParams Replace hand-written interfaces with type aliases derived from @openrouter/sdk's ChatGenerationParams, eliminating type drift and keeping provider options aligned with the SDK automatically. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: apply automated fixes * Changeset updated Fixes TanStack#310 * ci: apply automated fixes * chore(ai-openrouter): upgrade @openrouter/sdk to 0.12.13 The SDK renamed every chat-related exported type (ChatGenerationParams → ChatRequest, ChatResponse → ChatResult, etc.) and renamed the request wrapper key on chat.send from chatGenerationParams to chatRequest. Migrate adapters and tests to the new names. The SDK also narrowed ChatRequest to OpenAI-compatible fields, so Zod strips topK/topA/minP/repetitionPenalty/includeReasoning/verbosity/ webSearchOptions from outbound requests. Drop these keys from OpenRouterBaseOptions and the model catalog so callers get a TS error instead of silent no-op behavior, and add them to excludedParams in the catalog generator so future syncs stay honest. Also restore the SDK-derived shape of text-provider-options that was lost to an upstream merge, re-keyed off ChatRequest. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * chore(ai-openrouter): expand changeset, wire format into generate:models Rewrite the ai-openrouter changeset to reflect the full scope of the PR: the 0.12.13 SDK bump (not 0.9.11), the ChatGenerationParams -> ChatRequest rename, and the deliberate removal of topK/topA/minP/repetitionPenalty/ includeReasoning/verbosity/webSearchOptions from OpenRouterBaseOptions so callers get a TS error instead of silent stripping. Append pnpm format to generate:models so regenerated model-meta and provider files land formatted in a single command. Remove the now-unused compare-openrouter-models.ts script — the models file is sorted alphabetically so a plain git diff is sufficient. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Corrected openrouter params in example Fixes TanStack#310 * chore(ai-openrouter): bump @openrouter/sdk to 0.12.14 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(ai-openrouter): clarify changeset, tidy image error check, document common options Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(ai-openrouter): explain modelOptions-first spread ordering in text adapter Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ai-openrouter): apply OpenAI-strict transformation to structuredOutput schemas OpenRouter forwards `json_schema` requests with `strict: true` to upstream providers (notably OpenAI), which reject schemas that don't mark every property required and set `additionalProperties: false`. Run the schema through `convertSchemaToJsonSchema(..., { forStructuredOutput: true })` before sending so Zod / ArkType / Valibot schemas work out of the box. Adds adapter-level regression tests covering the transformation (nested objects, arrays, optional-to-nullable widening) and a structured-output example page in `ts-react-chat` to exercise the path end-to-end. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add a new getting-started/agent-skills.md guide explaining how to use @tanstack/intent install to wire the bundled Agent Skills from @tanstack/ai and @tanstack/ai-code-mode into Claude Code, Cursor, GitHub Copilot, and other AI coding assistants, register it in docs/config.json, and add a disambiguation callout on code-mode-with-skills.md so readers searching for "skills" land on the correct page. Add description and keywords frontmatter to all 64 hand-authored docs for search-engine discoverability. Auto-generated TypeDoc reference pages under docs/reference/ are left untouched since they would be overwritten on the next regeneration. A follow-up PR on tanstack.com will wire these fields into the document head.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
The `mapCommonOptionsToOllama()` method was silently dropping the `systemPrompts` field from chat options. System prompts passed via `chat({ systemPrompts: [...] })` now correctly reach the Ollama API as the `system` parameter on the chat request. This is how other adapters (e.g. Anthropic) handle it.
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* feat(ai): add @ag-ui/core as dependency for spec-compliant event types
* feat(ai): redefine AG-UI event types extending @ag-ui/core
Replace custom AG-UI event types with interfaces that extend @ag-ui/core
types for spec compliance. This is the foundational type change for the
AG-UI protocol alignment.
- Import all event types from @ag-ui/core with AGUI* aliases
- Replace BaseAGUIEvent to extend @ag-ui/core BaseEvent
- Replace each event interface to extend its @ag-ui/core equivalent
- Add TanStack-internal extension fields (model, deprecated aliases)
- Add new event types: ToolCallResultEvent, Reasoning* events
- Deprecate AGUIEventType in favor of EventType enum
- Re-export EventType enum from @ag-ui/core
- Add threadId/runId to TextOptions interface
- Update AGUIEvent union and StreamChunk type alias
* feat(ai): add stripToSpec middleware to strip non-spec fields from stream events
Creates a middleware that removes TanStack-internal extension fields
(model, rawEvent, deprecated aliases) from StreamChunk events so the
yielded stream is @ag-ui/core spec-compliant. Registered as the last
middleware in the chat activity chain so devtools and user middleware
still see the full extended events.
* feat(ai): plumb threadId and runId through chat() to adapters
Add threadId/runId to TextActivityOptions interface and TextEngine class
so they flow from user-facing chat() options through to adapter.chatStream().
ThreadId is auto-generated if not provided. Adapters will consume these
in subsequent tasks to include them in RUN_STARTED/RUN_FINISHED events.
* feat(ai-openai): update text adapter for AG-UI spec compliance
Add threadId to RUN_STARTED/RUN_FINISHED events, toolCallName to
TOOL_CALL_START/TOOL_CALL_END, stepName to STEP_STARTED/STEP_FINISHED,
flatten RUN_ERROR with top-level message/code fields, and emit
REASONING_START/MESSAGE_START/CONTENT/MESSAGE_END/END events alongside
legacy STEP events for reasoning content.
* fix: update tests and fix type errors for AG-UI spec compliance
Update test utilities and tests to use AG-UI spec field names:
- Add threadId to RUN_STARTED/RUN_FINISHED events
- Add toolCallName alongside deprecated toolName on tool events
- Add stepName alongside deprecated stepId on step events
- Use flat message field on RUN_ERROR (with deprecated error nested form)
Fix critical bugs discovered during testing:
- StreamProcessor: prefer chunk.message over chunk.error?.message for RUN_ERROR
- TextEngine: process original chunks for internal state before middleware strips fields
- Remove auto-applied stripToSpecMiddleware from chat() (breaks internal state since
it strips finishReason, delta, content needed by TextEngine and StreamProcessor)
- Fix type compatibility issues with @ag-ui/core EventType enum vs string literals
Also fix type errors in:
- stream-generation-result.ts: use EventType enum and add threadId
- generateVideo/index.ts: add StreamChunk casts and threadId
- tool-calls.ts: cast TOOL_CALL_END yield to ToolCallEndEvent
- devtools-middleware.ts: handle toolCallName fallback and RUN_ERROR message field
- processor.ts: handle developer role, Messages snapshot type cast, finishReason undefined
* fix(ai): re-add stripToSpec middleware, process raw chunks internally, fix test assertions
* fix(ai): pipe tool-phase events through middleware, strip toolCallName, fix type errors
- Fix EventType enum vs string literal type errors in test files by
relaxing chunk helper type params and adding cast helpers
- Pipe tool-phase events (TOOL_CALL_END, TOOL_CALL_RESULT, CUSTOM)
through the middleware pipeline so strip-to-spec and devtools
middleware observe all events, not just model-stream events
- Add toolCallName to TOOL_CALL_END strip set in strip-to-spec
middleware since AG-UI spec ToolCallEndEvent only has toolCallId
- Update test assertions to use TOOL_CALL_RESULT (spec event) instead
of checking stripped fields on TOOL_CALL_END
* style: format test files
* test(ai): add tests for REASONING events, TOOL_CALL_RESULT, threadId, and strip compliance
* fix: resolve eslint, type, and test failures across all packages
- Fix 5 ESLint errors in @tanstack/ai (array-type, no-unnecessary-condition,
no-unnecessary-type-assertion, sort-imports)
- Fix ESLint error in @tanstack/ai-event-client (no-unnecessary-condition)
- Fix string literal vs EventType enum type errors across all 7 adapter
packages by adding asChunk helper that casts event objects to StreamChunk
- Fix @tanstack/ai-client source type errors (chunk.error possibly undefined,
runId access on RUN_ERROR events, connection-adapters push calls)
- Fix @tanstack/ai-client and @tanstack/ai-openrouter test type errors
- Fix tool-call-manager tests to use toolCallName instead of deprecated toolName
* fix: resolve eslint and test failures from strip-to-spec middleware
Remove assertions for fields (content, finishReason, usage) that the
stripToSpec middleware now strips from emitted events. Fix unnecessary
nullish coalescing in ai-openai and add type casts in ai-vue tests.
* fix: honor caller runId, prevent duplicate thinking, add error IDs, fix reasoning ordering
- Honor caller-provided runId/threadId in all 7 adapters using ?? fallback
- Prevent duplicate thinking content from dual STEP_FINISHED/REASONING_MESSAGE_CONTENT events
- Assert exact threadId value in chat test instead of just toBeDefined
- Add runId/threadId to RUN_ERROR in generateVideo and stream-generation-result
- Move reasoning processing before content processing in OpenRouter adapter
* fix(smoke-tests): update harness to read spec-compliant event fields
TOOL_CALL_START now uses toolCallName (spec) instead of toolName (deprecated).
TOOL_CALL_END fields (toolName, input, result) are stripped by spec middleware;
harness now falls back to data captured during START/ARGS phases.
Added TOOL_CALL_RESULT handler for spec-compliant tool result delivery.
RUN_FINISHED finishReason/usage are optional extensions.
* ci: apply automated fixes
* fix(smoke-tests): remove unnecessary as-any casts, use proper type narrowing
* ci: apply automated fixes
* fix(ai-ollama): emit TOOL_CALL_ARGS before TOOL_CALL_END for spec compliance
Ollama doesn't stream tool args incrementally — it delivers them all at once
in TOOL_CALL_END.input. Since the strip middleware removes input from
TOOL_CALL_END, consumers had no way to get the args. Now emits a
TOOL_CALL_ARGS event with the full args as delta before TOOL_CALL_END.
* fix(ai): hide strip-to-spec middleware from devtools instrumentation
* fix(ai): handle TOOL_CALL_RESULT in StreamProcessor to create tool-result parts
Root cause: The strip middleware removes 'result' from TOOL_CALL_END events.
The StreamProcessor's TOOL_CALL_END handler only creates tool-result parts
when chunk.result is present. With it stripped, no tool-result parts were
created on the client side.
TOOL_CALL_RESULT events (spec-compliant tool result delivery) were received
but ignored (no-op). Without tool-result parts, areAllToolsComplete() behaved
incorrectly, and the client could not detect server tool completion.
Fix: Handle TOOL_CALL_RESULT by creating tool-result parts and updating
tool-call output, mirroring TOOL_CALL_END's result handling logic.
* fix(ai): stop stripping finishReason from RUN_FINISHED events
finishReason is essential for client-side continuation logic. Without it,
the chat-client cannot distinguish 'stop' (no continuation needed) from
'tool_calls' (client tools need execution), causing infinite request loops
when server-side tool results leave tool-call parts as the last message part.
* fix(ai): only strip deprecated aliases and rawEvent, keep all extras
@ag-ui/core BaseEventSchema uses .passthrough(), so extra fields are allowed
and won't break spec validation. Only strip:
- Deprecated aliases: toolName, stepId, state (nudge toward spec names)
- Deprecated nested error object on RUN_ERROR
- rawEvent (debug payload, potentially large)
Keep everything else: model, content, args, usage, finishReason, input,
result, index, providerMetadata, stepType, delta, etc.
* ci: apply automated fixes
* fix(ai): stop stripping fields — passthrough allows all extras
@ag-ui/core BaseEventSchema uses .passthrough() so extra fields are allowed.
Only strip the deprecated nested error object from RUN_ERROR (conflicts with
spec's flat message/code). Everything else passes through: model, content,
toolName, stepId, usage, finishReason, result, input, args, etc.
* fix: resolve type errors from @ag-ui/core Zod passthrough types
Zod passthrough adds `& { [k: string]: unknown }` to inferred types,
preventing TypeScript from narrowing the `type` discriminant in switch
statements. Add explicit casts where needed. Also fix toolCallName ->
toolName rename in realtime types to match consumer code.
* ci: apply automated fixes
* chore(ai): bump @ag-ui/core from 0.0.48 to 0.0.49
Removes rxjs from the transitive dependency tree. All exported types
and EventType enum values are identical between versions.
* fix: CR fixes for AG-UI core interop
- Use this.threadId in createSyntheticFinishedEvent instead of
regenerating a new ID on each call
- Add defensive delta guard in handleReasoningMessageContentEvent
matching sibling handler patterns
- Prefer spec chunk.message over deprecated chunk.error in devtools
middleware, generation client, and video generation client
- Add flat message field to synthesized RUN_ERROR in connection adapters
- Fix processChunk JSDoc listing RUN_STARTED as ignored (it has a handler)
- Fix comment referencing toolName when code uses toolCallName
- Document RUN_ERROR in stream-generation-result @returns
- Add meaningful assertions to TOOL_CALL_RESULT processor test
- Clarify threadId test describes adapter passthrough behavior
* ci: apply automated fixes
* fix(ai-client): use cast for RUN_ERROR message to satisfy eslint
chunk.message is typed as required string by @ag-ui/core but may be
absent at runtime from events constructed via as-unknown casts.
Cast to string|undefined to allow the || fallback chain while keeping
the no-unnecessary-condition rule happy.
* fix(ai): add StreamChunk casts for TOOL_CALL_START/ARGS in continuation re-executions
The merge from main brought in TanStack#372 which emits these events but used
string literals instead of the AGUIEvent enum, breaking the build.
* fix(ai-openrouter): prevent duplicate TEXT_MESSAGE_END and RUN_FINISHED events
OpenAI-compatible APIs often send two chunks with finishReason — one for
the finish signal and a separate trailing chunk carrying usage data. The
adapter had no guard against this, causing TEXT_MESSAGE_END and
RUN_FINISHED to be emitted twice per run.
Root cause: processChoice emitted finish events on every finishReason
occurrence without tracking whether they had already been sent.
Fix:
- Add hasEmittedRunFinished / hasEmittedTextMessageEnd guards to AGUIState
- Accumulate usage from any finishReason chunk into deferredUsage
- Move RUN_FINISHED emission to after the stream loop so it always
carries the latest usage data (even when it arrives on a later chunk)
Adds tests for duplicate-finish-chunk scenarios, usage preservation, and
event ordering.
* fix(ai-openrouter, ai): emit single STEP_FINISHED per reasoning block, remove [DONE] sentinel
STEP_FINISHED was emitted on every reasoning delta (N events for N
deltas) but only one STEP_STARTED was emitted, causing verifiers to
report orphan STEP_FINISHED events. Move the single STEP_FINISHED to
the point where reasoning closes (before text starts or at stream end)
so every STEP_STARTED has exactly one matching STEP_FINISHED.
Remove the `data: [DONE]\n\n` sentinel from toServerSentEventsStream.
The AG-UI protocol already uses RUN_FINISHED as the terminal event, so
the [DONE] marker is redundant and forces every client to special-case
non-JSON data in the SSE stream. Client-side parsers still tolerate
[DONE] for backward compatibility with external servers.
* fix(ai-client): warn when receiving deprecated [DONE] sentinel
Old servers still emit `data: [DONE]\n\n` after the stream. The client
already skips it, but now logs a deprecation warning so users know to
upgrade their @tanstack/ai server package.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…erop) (TanStack#474) chore: add changesets for PR TanStack#411 (AG-UI core interop) The original PR was merged without changesets. This adds retroactive changesets so the release pipeline picks up the breaking event-shape work across core, adapters, client, and event-client. - @tanstack/ai: major (flat RunErrorEvent, REASONING_* events, threadId/runId, strip-to-spec middleware, AG-UI core EventType re-export, JSON-patch StateDelta) - All provider adapters (openai, anthropic, gemini, ollama, openrouter, grok, groq): patch — emit AG-UI-compliant shapes; no public API change - @tanstack/ai-client: patch — consumer side of the new event shapes - @tanstack/ai-event-client: patch — devtools middleware alignment
…k#428) * fix(ai): move @standard-schema/spec to dependencies Without this package installed, all types that depend on StandardJSONSchemaV1 silently degrade to any. Moving from devDependencies to dependencies ensures consumers get it transitively. Fixes TanStack#235 * changeset: fix standard-schema dependency * chore: regenerate pnpm-lock.yaml for @standard-schema/spec devDep * chore(knip): whitelist @standard-schema/spec in ai-client It's not imported directly in ai-client's source, but is needed at build time so TypeScript can resolve the forward references to `StandardJSONSchemaV1` that live inside `InferToolInput` / `InferToolOutput` (both imported from @tanstack/ai and re-used in ai-client's public .d.ts surface). Knip's static import graph can't see that, so whitelist the dep here.
…ack#476) chore: downgrade @tanstack/ai changeset from major to minor for AG-UI interop
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* docs: add migration guide from Vercel AI SDK to TanStack AI
Add comprehensive migration guide covering:
- Package installation differences
- Server-side API migration (streamText -> chat)
- Client-side useChat hook differences
- Isomorphic tool system migration
- Provider adapter changes (OpenAI, Anthropic, Gemini)
- Streaming response formats
- Multimodal content handling
- Type safety enhancements
- Complete before/after code examples
* docs: address review feedback on Vercel AI migration guide
- Installation: add @ai-sdk/react to v5+ deps and update quick-reference
table to show the v5 framework package names
- System prompts: use root-level systemPrompts: [...] instead of
prepending a system message to the messages array (verified against
packages/typescript/ai/src/types.ts)
- useChat API table: rewrite against current Vercel AI SDK v5+ API
(sendMessage, status, regenerate, DefaultChatTransport) so the
comparison is accurate rather than mixing v4/v5
- MessagePart: expand to full discriminated union with real field names
(arguments/input/approval on tool-call, content on tool-result) and
real ToolCallState values
- Fix nonexistent toStreamResponse references -> toServerSentEventsResponse
(and add toHttpResponse where appropriate)
- Fix AbortController section heading (h4 -> h3, resolves MD001)
- Update tool schema section to note parameters -> inputSchema rename
in AI SDK v5
- Tighten tool approval example with optional chaining and a note on
arguments vs parsed input
* docs: expand Vercel AI SDK migration guide with full option + v6 coverage
- Add exhaustive streamText -> chat() option mapping table covering
every AI SDK v6 parameter (tools, toolChoice, activeTools, stopWhen,
prepareStep, experimental_transform/context/telemetry/repairToolCall,
all sampling controls, abort, headers, providerOptions -> modelOptions)
- Add streamText result -> TanStack equivalent table (textStream,
fullStream, text, usage, finishReason, steps, toUIMessageStreamResponse,
pipeTextStreamToResponse, consumeStream, etc.)
- Expand Generation Options with topK/presence/frequency/seed/stop under
modelOptions, clarify flat typed modelOptions vs provider-keyed
providerOptions
- New section: Structured Output (generateObject / streamObject / v6
Output.object) -> outputSchema on chat(); notes on Standard Schema
libraries, provider strategies, and the current gap for partial
object streaming
- New section: Agent Loop Control — stopWhen / hasToolCall / stepCountIs
mapped to maxIterations / untilFinishReason / combineStrategies, and
prepareStep mapped to middleware onConfig/onIteration
- New section: Middleware — wrapLanguageModel + experimental_transform
mapped to a single ChatMiddleware array; full hook inventory;
toolCacheMiddleware usage; common-pattern mapping table
- New section: Observability — where to plug logging/metrics/tracing
- Update generateText coverage to chat({ stream: false }) returning a
real Promise<string> (not just streamToText)
- Update Tool Approval "Before" to show AI SDK v6's native needsApproval
+ sendAutomaticallyWhen flow; the two APIs are now symmetric
- Reframe "Removed Features" -> "Features Not Yet Covered" and scope
it to embeddings, partial-object streaming, built-in retries/timeouts
- Update frontmatter for the docs/migration/ location (order, description,
keywords); fix cross-links to the new directory layout
(../advanced/middleware, ../chat/structured-outputs, etc.)
* docs: round-1 CR fixes on Vercel AI migration guide
Factual corrections verified against source:
- Multimodal image source shape uses { type: 'url'|'data', value, mimeType }
not { url, base64, mediaType } (types.ts:142-183)
- toolCacheMiddleware is exported from @tanstack/ai/middlewares, not the
root (packages/typescript/ai/src/middlewares/index.ts)
- toolDefinition({ description }) is required; add it to the two doc
examples that were missing it (tool-definition.ts:31)
- stream() connection adapter factory is (messages, data?) with no
signal arg; rewrite custom-adapter example (connection-adapters.ts:441)
AI SDK v6 accuracy:
- addToolResult -> addToolOutput (v6 rename)
- experimental_output -> output (de-experimentalized)
- Soften "replaced" claim about generateObject/streamObject — they are
deprecated, not removed
- Vercel addToolApprovalResponse row: v6 has this; replace "N/A"
- First Basic Text Generation Before example now uses v5+ API
(convertToModelMessages + toUIMessageStreamResponse) with a v4
toDataStreamResponse callout
Consistency:
- Agent-loop tables reconciled: only one truly built-in strategy
(maxIterations / untilFinishReason / combineStrategies); hasToolCall
requires a custom AgentLoopStrategy. Both tables now agree.
- prepareStep Before/After actually demonstrates equivalent behavior:
Before shows step-level config tweak, After uses onConfig;
mid-loop model switching split into its own subsection with the
two-chat pattern the prose describes
- Message Structure section qualifies that ToolCallPart.input is the
ai-client projection (server-side reads arguments directly)
- toHttpStream/Response comment in client connection example clarified
- Complete Example clarifies why convertToModelMessages disappears in
the After (chat() accepts UI messages directly)
- clientTools() auto-execution comment expanded to state that no
onToolCall/addToolOutput call is needed
- Anchor slug for Structured Output simplified to #structured-output
Rot hygiene:
- "current releases" removed from v5/v6 note
- "Every option" softened to "Options accepted ... as of AI SDK v6"
- "now expose" / "AI SDK v6 offers" / "v6 consolidated" reworded to
avoid tense decay across future releases
* docs: use toServerSentEventsResponse/toHttpResponse init options
Both helpers accept ResponseInit & { abortController }, so custom headers,
status, and cancellation flow through the helpers directly. Drop the
hand-rolled `new Response(toServerSentEventsStream(...), { headers: {...} })`
example and keep the raw stream helpers only for the genuine "pipe elsewhere"
case.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alem Tuzlak <t.zlak@hotmail.com>
Co-authored-by: Jack Herrington <jherr@pobox.com>
…chema (TanStack#484) fix(ai): mark optional nested objects/arrays nullable under strict schema makeStructuredOutputCompatible adds every property to required[] under forStructuredOutput: true, but optional nested objects/arrays were taking the recursive branches and never reaching the 'null'-wrap — producing a schema that OpenAI-style strict json_schema providers reject. Wrap transformed composites as type: ['object', 'null'] / ['array', 'null'] when wasOptional. Extends the OpenRouter regression test with the previously-untested array case and a new nested-object case. Fixes TanStack#483 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…nStack#466) * feat(ai): add ProviderTool phantom-branded subtype * feat(ai): add toolCapabilities to TextAdapter type channel * feat(ai): gate TextActivityOptions.tools on model toolCapabilities * fix(ai): prevent ProviderTool subsumption in TextActivityOptions.tools union * test(ai): type tests for TextActivityOptions.tools gating * test(ai): fix import order and cover broad-TKind ProviderTool rejection * feat(ai-anthropic): add supports.tools and ToolCapabilitiesByName map * refactor(ai-anthropic): use Array for supports.tools to match supports.input * feat(ai-anthropic): brand provider tool factory return types * refactor(ai-anthropic): replace tools barrel with named re-exports * fix(ai-anthropic): build AnthropicTool union from branded types * Revert "fix(ai-anthropic): build AnthropicTool union from branded types" This reverts commit 801255b. * feat(ai-anthropic): thread toolCapabilities through text adapter * feat(ai-anthropic): export AnthropicChatModelToolCapabilitiesByName * feat(ai-anthropic): add /tools subpath export * test(ai-anthropic): per-model type tests for provider tools * test(ai-anthropic): cover all unsupported tools in haiku-3-5 negative case * feat(ai-openai): add ToolCapabilitiesByName and expanded supports.tools * feat(ai-openai): brand provider tool factories and replace barrel export * feat(ai-openai): add /tools subpath export * test(ai-openai): per-model type tests for provider tools * feat(ai-gemini): split capabilities into capabilities + tools Separates tool-type entries (code_execution, file_search, search_grounding, grounding_with_gmaps, url_context, image_generation) from general capability flags in ModelMeta.supports. Adds a new tools field to the supports shape, renames grounding_with_gmaps → google_maps and search_grounding → google_search, drops image_generation (no tool factory yet), and introduces GeminiChatModelToolCapabilitiesByName. Threads a fifth TToolCapabilities generic through GeminiTextAdapter and exports the new type map from the root. * feat(ai-gemini): brand provider tool factories and replace barrel export * feat(ai-gemini): add /tools subpath export and type tests * feat(ai-openrouter)!: move and rename createWebSearchTool to webSearchTool on /tools subpath Renames createWebSearchTool → webSearchTool, brands its return type as OpenRouterWebSearchTool (ProviderTool<'openrouter', 'web_search'>), moves web_search exports to the new ./tools subpath, adds OpenRouterChatModelToolCapabilitiesByName mapped type (all chat models support web_search via the gateway), threads TToolCapabilities through the text adapter, and adds per-model type tests. * feat(ai-grok, ai-groq): add supports.tools, /tools subpath, and thread toolCapabilities - Add tools?: ReadonlyArray<never> to ModelMeta.supports interface in both packages - Add tools: [] as const to every chat model constant - Export GrokChatModelToolCapabilitiesByName / GroqChatModelToolCapabilitiesByName type maps - Add 5th TToolCapabilities generic to GrokTextAdapter / GroqTextAdapter via ResolveToolCapabilities - Add ./tools subpath to package.json exports and vite.config.ts entry for both packages - Re-export new ToolCapabilitiesByName types from root index.ts in both packages * chore(scripts): update sync-models templates to include supports.tools * test(ai-anthropic): runtime smoke test for provider tool factories * docs: add provider tools concept page * docs(adapters): add Provider Tools sections for every adapter * docs: cross-link provider tools concept page and migration section 6 * chore: changesets for provider-tools surface * fix(ai-groq): reorder imports to satisfy import/first ESLint rule * fix(changesets): bump adapters to minor for new /tools subpath + openrouter breaking * fix(ai-openrouter): match @deprecated tag wording across adapters * fix(changesets): confirm gemini relocation claim is accurate for all active models Investigated file_search and search_grounding entries in model-meta.ts: - All active (exported) models: correctly have these entries in tools: array - Commented-out preview models (GEMINI_2_5_FLASH_LIVE, GEMINI_2_FLASH_LIVE): still have them in capabilities: but are not part of the released API - Changeset claim is accurate: relocation is complete for all models that matter * fix(ai-gemini): export convertToolsToProviderFormat from /tools * test(ai-openai): cover all 10 unsupported tools in gpt-3.5-turbo * test(ai-gemini): add negative cases for gemini-3.1-pro-preview * fix(ai-anthropic): debrand customTool — return plain Tool (universal) * fix(ai-openai): debrand customTool/functionTool — remove unused brand types * docs: remove customTool from branded-factories matrix * test(ai-anthropic): cover debranded customTool acceptance on any model * ci: apply automated fixes * review: address CodeRabbit PR feedback - Tighten TProviderOptions constraint from 'extends object' to 'extends Record<string, any>' across all five text adapters to match BaseTextAdapter (openai, anthropic, gemini, grok, groq). - OpenAI & Anthropic: wire chatStream / structuredOutput to the TProviderOptions generic so per-model provider options are enforced at the call site, matching the openrouter/gemini pattern. - OpenRouter webSearchTool: brand via a stable __kind marker and gate the converter on that; reject malformed metadata explicitly instead of trusting tool.name === 'web_search' (users can collide on name). - Docs: fix fileSearchTool example to use the real fileSearchStoreNames shape; switch webSearchTool example back to type: 'web_search' (preview variant is a separate factory); update local_shell/shell/apply_patch supported-models copy to match the actual supports.tools gating; drop 'todays' typo. - Tests: apply ESLint import/order + sort-imports to the five tools-per-model type-safety specs. * review: address CodeRabbit feedback (round 2) - ai-openrouter: tighten web_search metadata validation against null/arrays (typeof 'object' alone accepted both) - ai-gemini: include googleSearchRetrievalTool in the "rejects all provider tools" case so no gemini provider-tool kind can regress silently * review: rename generic `A` to `TAdapter` in typedTools helpers Satisfies @typescript-eslint/naming-convention rule (type params must match /^(T|T[A-Z][A-Za-z]+)$/). Applies the fix to all four per-adapter tools-per-model-type-safety test files (anthropic, gemini, openai, openrouter). * fix(ai-openrouter): type metadata.web_search as unknown for runtime null/array guard Prior typing as `WebSearchToolConfig['web_search']` was non-nullable at the type level, so `metadata.web_search === null` and `Array.isArray(...)` tripped `@typescript-eslint/no-unnecessary-condition` and failed CI lint. Widen to `unknown` so the defensive null/array runtime checks are type-meaningful, then narrow on return. * chore: remove accidentally-committed terminalOutput artifact --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…nStack#467) * feat(ai): add Logger, DebugCategories, DebugConfig, DebugOption types * fix(ai): relocate logger type tests to tests/ for vitest discovery * docs(ai): add JSDoc to logger types and tighten type tests * feat(ai): add ConsoleLogger default logger implementation * feat(ai): add InternalLogger with per-category filtering and prefix * feat(ai): add resolveDebugOption normalizing DebugOption to InternalLogger * feat(ai): export Logger types publicly and InternalLogger via /adapter-internals subpath * feat(ai): thread InternalLogger through TextEngine, MiddlewareRunner, and TextOptions * feat(ai-openai): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-anthropic): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-gemini): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-grok): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-groq): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-ollama): emit request/provider/errors logs via InternalLogger in text adapter * feat(ai-openrouter): emit request/provider/errors logs via InternalLogger in text adapter * fix(ai): remove redundant top-level logger from StructuredOutputOptions and normalize provider log key * feat(ai-openai): emit logging events in summarize adapter * feat(ai-anthropic): emit logging events in summarize adapter * feat(ai-gemini): emit logging events in summarize adapter * feat(ai-grok): emit logging events in summarize adapter * feat(ai-ollama): emit logging events in summarize adapter * feat(ai-openrouter): emit logging events in summarize adapter * feat(ai): resolve debug option and thread InternalLogger through summarize() * feat(ai-openai): emit logging events in image adapter * feat(ai-gemini): emit logging events in image adapter * feat(ai-grok): emit logging events in image adapter * feat(ai-openrouter): emit logging events in image adapter * feat(ai-fal): emit logging events in image adapter * feat(ai): resolve debug option and thread InternalLogger through generateImage() * feat(ai-openai): emit logging events in video adapter * feat(ai-fal): emit logging events in video adapter * feat(ai): resolve debug option and thread InternalLogger through generateVideo() * feat(ai-openai): emit logging events in tts adapter * feat(ai-gemini): emit logging events in tts adapter * feat(ai): resolve debug option and thread InternalLogger through generateSpeech() * feat(ai-openai): emit logging events in transcription adapter * feat(ai): resolve debug option and thread InternalLogger through generateTranscription() * feat(ai-openai): emit logging events in realtime adapter * feat(ai-elevenlabs): emit logging events in realtime adapter * test(ai): add integration tests for debug logging across activities * fix(ai): remove redundant nullish coalescing on non-nullable result fields * test(e2e): debug logging emits expected prefixes for chat() * docs(advanced): add debug logging guide * docs(observability): cross-link to debug logging guide * docs(middleware): cross-link to debug logging guide * docs(nav): add Debug Logging entry to Advanced section * fix(ai): migrate remaining console.warn calls in realtime adapters to logger.errors * docs(advanced): fix stray comment syntax in debug-logging guide * ci: apply automated fixes * fix: resolve eslint and knip failures from debug logging PR - Remove unnecessary optional chains and nullish coalescing on required messages array across adapter log lines - Remove unnecessary fallback on non-nullable chunk/event types - Reorder imports in elevenlabs realtime types to satisfy import/first - Delete unused packages/typescript/ai/src/logger/index.ts barrel (public surface is re-exported from src/index.ts) * ci: apply automated fixes * fix(ai): suppress debug logs for internal devtools middleware The devtools middleware is injected automatically by chat() and is already excluded from aiEventClient instrumentation via shouldSkipInstrumentation. Its per-hook logger.middleware / logger.config calls were still firing though, flooding the [tanstack-ai:middleware] category with internal plumbing. Move those calls inside the same skip gate so debug output only reflects user-provided middleware. * feat(ai): pretty-print deeply nested meta in ConsoleLogger on Node Debug logs surface raw provider chunks whose nested structures (usage, output, reasoning, tools, response payloads) were being truncated to [Object] / [Array] because Node's default console formatting stops at depth 2. ConsoleLogger now lazily loads node:util and runs meta through inspect({ depth: null }) on Node so the entire structure renders. Browsers still get the raw object for interactive DevTools inspection. * ci: apply automated fixes * refactor(ai): use console.dir with depth:null instead of util.inspect console.dir is the purpose-built native API for depth-unlimited object inspection. It takes the same {depth, colors} options natively on Node and is a no-op/interactive-tree in browsers, so we get the expanded output in both environments without any dynamic import dance around node:util. * feat(ai): prefix each debug category with an emoji marker Makes it trivial to visually scan dense streaming logs — each category tag is now bracketed by its own emoji on both sides, e.g. '📨 [tanstack-ai:output] 📨 ...'. Mapping: request=📤, provider=📥, output=📨, middleware=🧩, tools=🔧, agentLoop=🔁, config=⚙️, errors=❌. Tests that asserted on the raw tag via startsWith were switched to includes so they remain robust to prefix changes. * ci: apply automated fixes * test: remove debug logging e2e spec Library-level unit tests in the @tanstack/ai test suite already cover the debug logging behaviour (logger wiring, category resolution, console.dir formatting, emoji prefixing). An e2e round-trip added no independent coverage, so drop the spec, its API route, its fixture, and the now-stale routeTree entry. * chore: add changesets for debug logging - @tanstack/ai: minor — new debug option on every activity, Logger / ConsoleLogger / DebugOption public surface, @tanstack/ai/adapter-internals subpath, emoji-prefixed category tags, console.dir-based meta formatting - All provider adapters: patch — wire adapters through the InternalLogger so request/provider/errors flow through the structured logger; drop leftover console.* calls in adapter catch blocks * ci: apply automated fixes * fix(ai): swallow user-logger exceptions and ship debug-logging skill Wrap the user-supplied Logger calls inside InternalLogger.emit in a try/catch so an exception from the injected logger never masks the real error that triggered the log call (e.g. a provider SDK failure inside the chat stream). Also ship a new ai-core/debug-logging skill under packages/typescript/ai/skills/ so agents can discover how to toggle debug logging on/off, narrow it per category, and pipe it into a custom logger. * docs(ai): document Logger try/catch guarantee in debug-logging guide * docs(ai): fix DebugOption type and add realtime note in debug-logging skill - Show actual `DebugOption = boolean | DebugConfig` type; describe the omitted-field behavior as a resolution rule rather than a type-arm. - Replace the misleading "when any flag is set" comment; flags default to true whenever a DebugConfig is passed. - Acknowledge that provider realtime session adapters (openaiRealtime, elevenlabsRealtime) also accept the same debug option. * test(ai-openrouter): add missing logger field to structuredOutput test PR TanStack#484 on main added a new structuredOutput test that didn't pass a logger in chatOptions. After merging main into this branch, the new required logger field on TextOptions (from the debug-logging PR) tripped the typecheck. Pass the silent testLogger already defined at the top of the file to match the pattern used by every other test in this file. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ia + 3.1 Flash TTS, streaming generateAudio + hooks (TanStack#463) * feat: add fal audio, speech, and transcription adapters Adds falSpeech, falTranscription, and falAudio adapters to @tanstack/ai-fal, completing fal's media coverage alongside image and video. Introduces a new generateAudio activity in @tanstack/ai for music and sound-effect generation, with matching devtools events and types. Closes TanStack#328 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add ElevenLabs TTS/music/SFX/transcription adapters and Gemini Lyria + 3.1 Flash TTS Extends @tanstack/ai-elevenlabs (which already covers realtime voice) with Speech, Music, Sound Effects, and Transcription adapters, each tree-shakeable under its own import. Adds Gemini Lyria 3 Pro / Clip music generation via a new generateAudio adapter, plus the new Gemini 3.1 Flash TTS Preview model with multi-speaker dialogue support. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: document fal audio, speech, and transcription adapters Adds a new Audio Generation page, expands the fal adapter reference with sections for text-to-speech, transcription, and audio/music, and adds fal sections to the Text-to-Speech and Transcription guides. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add example pages and tests for audio/tts providers Expand the ts-react-chat example with provider tabs for OpenAI, ElevenLabs, Gemini, and Fal on the TTS and transcription pages, plus a new /generations/audio page covering ElevenLabs Music, ElevenLabs SFX, Gemini Lyria, and Fal audio generation. Add a Gemini TTS unit test and wire an audio-gen feature into the E2E harness (adapter factory, API route, UI, fixture, and Playwright spec). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * docs: lead audio generation guide with Gemini and ElevenLabs Reorder the Audio Generation page so the direct Gemini (Lyria) and ElevenLabs (music/sfx) adapters appear before fal.ai, and update the environment variables + result-shape notes to cover all three providers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(ts-react-chat): add audio home tile, sample prompts, and fal model selector Expose an Audio tile on the welcome grid, offer one-click sample prompts for every audio provider, and let the Fal provider pick between current text-to-music models (default MiniMax v2.6). Threads a model override through the audio API and server fn. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * chore: split ElevenLabs audio adapters out to separate PR (TanStack#485) Moves the new ElevenLabs TTS / Music / SFX / Transcription REST adapters out of this PR into their own issue (TanStack#485) and branch (`elevenlabs-audio-adapters`) so the fal + Gemini audio work can ship independently. The follow-up PR will rebuild these adapters on top of the official `@elevenlabs/elevenlabs-js` SDK rather than hand-rolled fetch calls. Removed from this branch: - `packages/typescript/ai-elevenlabs/src/{adapters,utils,model-meta.ts}` and their tests (realtime voice code untouched) - ElevenLabs sections in `docs/media/audio-generation.md` - ElevenLabs entries in `examples/ts-react-chat` audio-providers catalog, server adapter factories, zod schemas, and default provider wiring - `@tanstack/ai-elevenlabs` bump from the audio changeset Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * fix(ai-fal, ai-gemini): audio adapter bug fixes - ai-fal: replace `btoa(String.fromCharCode(...bytes))` with a chunked helper; the spread form throws RangeError on any realistic TTS clip (V8 arg limit ~65k). - ai-gemini: honor `TTSOptions.voice` as a fallback for the prebuilt voice name, move `systemInstruction` inside `config` per the @google/genai contract, and wrap raw `audio/L16;codec=pcm` output in a RIFF/WAV container so the result is actually playable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(ts-react-chat): warn on rejected audio model overrides Log a warning instead of silently swapping to the default when a client sends a model id outside the provider's allowlist, so stale clients or typo'd config ids are debuggable. Also correct the AudioProviderConfig JSDoc to describe the models[] ordering as a non-binding UI convention. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: split generateAudio into generateMusic and generateSoundEffects Replaces the unreleased generateAudio activity with two distinct activities so music and sound-effects each have their own types, adapter kinds, provider factories, and devtools events. This lets providers advertise only the capabilities they support (Gemini Lyria is music-only; fal has distinct music and SFX catalogs) and leaves room for kind-specific options without a breaking change. - Core: generateMusic/generateSoundEffects activities and MusicAdapter/ SoundEffectsAdapter interfaces + bases; GeneratedAudio shared between MusicGenerationResult and SoundEffectsGenerationResult - Events: music:request:* and soundEffects:request:* replace audio:* - fal: falMusic + falSoundEffects factories sharing internal request/response helpers; FalMusic/FalSoundEffectsProviderOptions in model-meta - Gemini: geminiMusic/createGeminiMusic/GeminiMusicAdapter (Lyria is music-only so no SFX counterpart) - ts-react-chat: /generations/music and /generations/sound-effects routes backed by a shared AudioGenerationForm; split server fns and API routes - E2E: music-gen + sound-effects-gen features, parameterized MediaAudioGenUI, split fixtures and specs (both feature support sets are empty since aimock 1.14 cannot mock Gemini's Lyria AUDIO modality) - Docs: music-generation.md + sound-effects-generation.md; fal adapter docs split; changesets rewritten in place Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fixed type issue * Delete terminal output * revert: restore single generateAudio activity Supersedes 1010e9b. The split into generateMusic + generateSoundEffects doesn't hold up against fal's audio catalog: dozens of models span audio-to-audio, voice-change/clone, enhancement, separation, isolation, merge, and understanding, and individual models (e.g. stable-audio-25) generate music AND sound effects. A single broader generateAudio activity fits that reality. Keeps the aimock Gemini-Lyria gap: audio-gen feature-support stays empty because aimock 1.14 has no AUDIO-modality mock for generateContent — the E2E is green by skipping rather than by hitting a mock that doesn't exist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: enforce exactly one of url or b64Json on GeneratedImage and GeneratedAudio Model GeneratedImage and GeneratedAudio on a shared mutually-exclusive GeneratedMediaSource union so the type rejects empty objects and objects that set both fields. Update the openai, gemini, grok, openrouter, and fal image adapters to construct results by branching on which field is present; openrouter and fal no longer synthesize a data URI on url when returning base64. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * chore(e2e): drop audio-gen scaffolding pending aimock support The audio-gen feature set was empty because aimock cannot currently mock audio generation, so the Playwright spec ran against zero providers. Remove the dead scaffolding; the wiring can return once aimock audio support lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add useGenerateAudio hook and streaming support for generateAudio Closes the parity gap with the other media activities — audio generation now has the same client-hook UX (connection + fetcher transports) as image, speech, video, transcription, and summarize. Adds streaming to generateAudio so it can ride the SSE transport, a matching AudioGenerateInput type in ai-client, framework hooks in ai-react / ai-solid / ai-vue / ai-svelte, unit tests, an updated ts-react-chat example, and docs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ai-fal): translate duration per audio model Fal audio models use different input field names for length: ElevenLabs Music takes `music_length_ms` in milliseconds, Stable Audio 2.5 takes `seconds_total`, and most others accept `duration`. The adapter was passing a generic `duration` unconditionally, so the slider in the example was silently ignored for ElevenLabs and Stable Audio. Also: align the Gemini Lyria adapter with the API's MP3 default (only send responseMimeType when the caller asks for WAV), expand the example to include Lyria 3 Pro and a dedicated Fal SFX provider, and rename the example's "Direct" mode to "Hooks" to better reflect what it demos. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(ai-gemini): rename GEMINI_LYRIA_MODELS to GEMINI_AUDIO_MODELS Align the audio model constant and its re-export with the `generateAudio` activity naming used across providers, and drop the unused duplicate `GeminiLyriaModel` type — `GeminiAudioModel` is the single canonical type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ai-gemini): address CR findings — constructor config, TTS model name, PCM channels, voice validation, image error surfacing * fix(ai-fal): address CR findings — generateId entropy, fetch.ok guards, response-shape validation, size params, proxy+apiKey, content types * fix(ai-fal): throw on unknown image response shape instead of returning empty * fix(ai-image-adapters): fix double-wrapped errors, duplicate keys, signature mismatch, null guards * fix(ai-gemini): address CR findings — test import, image model output meta, option filtering * fix(example-ts-react-chat): blob URL revocation, route link, body validation, falsy duration render * fix(ai-core): emit adapter-error events, consistent async, reordered base adapter ctor, type sync * fix(ai-openrouter): drop redundant null guards that TS types already enforce The defensive nullish-coalescing on response.choices and img/img.imageUrl guards that the fix-loop added are impossible per the SDK type signatures; eslint's no-unnecessary-condition correctly rejects them. Keep only the typeof url !== 'string' check, which is a real runtime shape guard (imageUrl.url is typed as string but provider may send a non-string in rare degraded responses). * fix: address CodeRabbit review feedback — SSE types, mime normalization, voice validation, etc. Applies the reviewer-flagged changes that weren't load-bearing for the merge: - event-client: AudioRequestCompletedEvent.audio is now a mutually-exclusive {url; never b64Json} | {b64Json; never url} union so consumers can't read both fields simultaneously, mirroring the GeneratedAudio contract in core. - fal utils: extractUrlExtension now strips URL fragments and trailing slashes, parses via the URL API so a TLD like `.com` isn't mistaken for an extension, and only inspects the final path segment. - fal utils: deriveAudioContentType returns `audio/aac` for aac, separated from the `m4a`/`mp4` → `audio/mp4` case. - fal speech: prefer URL-derived extension when deriving `format`, and normalize `mpeg` → `mp3` so the field is a usable file extension. - gemini audio: drop `negativePrompt` (not accepted by GenerateContentConfig) and `responseMimeType` (Lyria Clip rejects it, Pro returns MP3 by default) from the public provider options surface, and document that the generic `duration` option is ignored by Lyria (Clip is fixed at 30s, Pro takes duration via the natural-language prompt). - gemini tts: multiSpeakerVoiceConfig.speakerVoiceConfigs length is now validated (1 or 2 speakers), partial user-supplied voiceConfig correctly falls back to the standard voice/'Kore' default, parsePcmMimeType tightens detection to exclude subtypes containing "wav" so containerized `audio/wav;codec=pcm` is no longer re-wrapped, and createGeminiSpeech / createGeminiAudio factory functions now spread config before the explicit apiKey argument so caller config can't silently override the API key. - ts-react-chat API routes: replace zod 4's removed `.flatten()` with `z.treeifyError()` for validation error details. - ts-react-chat audio route: `toAudioOutput` returns `null` per the `onResult` hook contract instead of throwing synchronously — failures are still surfaced via the hook's error state. - Updates the tests affected by the above behavior changes. * docs: document debug logging for new audio/speech/transcription activities - debug-logging.md: list generateAudio/generateTranscription in Non-chat activities section; clarify that the `provider` category now applies to streaming generateAudio/generateSpeech/generateTranscription calls too. - audio-generation.md, text-to-speech.md, transcription.md: add a single contextual callout at the moment a builder is most likely to need it (immediately before the Options table / next to Error Handling), pointing to the debug-logging guide. * docs(skill): add audio/speech CR gotchas + debug-logging to media-generation skill Agents hitting the new generateAudio/generateSpeech/generateTranscription activities will run into: - Gemini Lyria doesn't accept responseMimeType or negativePrompt via GenerateContentConfig — shape the prompt instead. - Lyria 3 Clip is fixed 30s; Lyria 3 Pro reads duration from natural-language in the prompt, not the duration option. fal audio maps duration per-model. - Gemini TTS multiSpeakerVoiceConfig is validated to 1 or 2 speakers. - debug: DebugOption is threaded through every generate*() activity — reach for it instead of writing logging middleware. Adds four Common Mistake entries, sources the debug-logging doc, and cross-references the ai-core/debug-logging sub-skill. * fix(ai-fal): decode data URL audio inputs to Blob for transcription fal-client auto-uploads Blob/File inputs via fal.storage.upload but passes strings through unchanged, so data URLs reached fal's API and got rejected with 422 "Unsupported data URL". Decode data URL strings to a Blob in buildInput so the auto-upload path handles them; plain http(s) URLs still pass through. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: regenerate API documentation (TanStack#494) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Alem Tuzlak <t.zlak@hotmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* feat(examples): add AI-powered search example - Introduces a comprehensive React example demonstrating natural language search capabilities - Users can query merchant data (orders, disputes, settlements) using conversational language like "show me orders from last week" - AI converts natural language prompts into structured search parameters with proper filtering and date ranges - Includes full UI with data tables, filters, and responsive design using Tailwind CSS - Leverages TanStack Start, TanStack Router, TanStack AI * feat(examples/ts-react-search): add navigation component to hero section - Added a new Navigation component with links to Home, Orders, Disputes, and Settlements pages - Integrated navigation into the hero section for improved user experience * refactor(navigation): simplify route references in Navigation component - Replace imported route objects with hardcoded string paths - Remove unused route imports * refactor(routes): restructure API search route into directory - Moved api.search.ts to api/search.ts for better organization - Updated route tree imports to reflect new file structure - Maintains existing functionality while improving code organization * feat(examples): integrate TanStack DB for client-side data management - Replace server functions with TanStack DB collections and live queries - Add @tanstack/react-db, @tanstack/query-db-collection, and related packages - Implement disputes, orders, and settlements collections with Zod validation - Create useLiveQuery hooks for reactive data filtering and searching - Update components to use client-side collections instead of server functions * feat(examples): update search API to use server-sent events streaming - Migrated from `toStreamResponse` to `toServerSentEventsResponse` for improved streaming - Updated OpenAI adapter to use `openaiText` with model specification - Updated multiple dependencies including React, TanStack Router, and TailwindCSS * refactor(search): migrate search API from streaming to synchronous with structured validation - Replaced Server-Sent Events with React Query mutation for search requests - Added Zod schema validation for structured output in search API - Updated search component to handle JSON responses instead of streaming - Improved error handling and type safety for search parameters * refactor(search): extract search mutation logic into reusable hook - Moved search API mutation logic from Search component into dedicated useSearchMutation hook - Improves code reusability and separation of concerns - Enables search functionality to be used across multiple components - Reduces code duplication and improves maintainability * build(ts-react-search): bump TanStack Router stack and refresh lockfile - Align the ts-react-search example with newer @tanstack/react-router, react-start, router-plugin, and devtools packages so it stays compatible with current TanStack releases - Regenerated route tree types so the layout route reports fullPath as '/' instead of an empty string, matching the updated router codegen - Updated the workspace lockfile so installs resolve consistently with the new dependency graph * feat(ts-react-search): switch AI search adapter from OpenAI to Groq - Example depends on @tanstack/ai-groq and uses groqText with openai/gpt-oss-20b - Server checks GROQ_API_KEY instead of OPENAI_API_KEY - Output schema is wrapped with toGroqCompatibleSchema so Groq accepts JSON Schema unions (additionalProperties on anyOf) * ci: apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…tion (TanStack#429) * fix(ai-client): prevent drainPostStreamActions re-entrancy stealing queued actions When multiple client tools complete in the same round, each addToolResult() queues a checkForContinuation action. The first drain executes one action which calls streamResponse(), whose finally block calls drainPostStreamActions() again (nested). The inner drain steals the remaining actions, permanently stalling the conversation. Add a draining flag to skip nested drain calls. The outer drain processes all actions sequentially, preventing action theft. Also fix shouldAutoSend() to require at least one tool call in the last assistant message. Previously it returned true for text-only responses (areAllToolsComplete() returns true when toolParts.length === 0), causing the second queued checkForContinuation action to incorrectly trigger an extra continuation round and produce duplicate content. Fixes TanStack#302 * ci: apply automated fixes * changeset: fix drain post-stream re-entrancy * fix: resolve type errors in drain re-entrancy test * test: add e2e regression test for drain re-entrancy stall (TanStack#302) Add a Playwright e2e test that verifies parallel client tools complete and the continuation fires with a follow-up text response. Without the drainPostStreamActions() re-entrancy guard, nested drain calls steal queued actions and permanently stall the conversation after both tools complete. The test asserts that the follow-up text "All displayed" arrives, which would time out without the fix. * ci: apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…ygiene (TanStack#465) * test(ai-code-mode-skills): add unit test coverage for skill library The package had 13 source files with zero unit tests. Added 116 tests across 9 files covering trust strategies, memory + file storage, skill management tools (including name-validation boundaries for register_skill), bindings, skills-to-tools execution with mocked isolate driver, type generation, the system-prompt renderer, and skill selection with a mocked chat adapter. * feat(ai-isolate-cloudflare): support production deployments and harden tool-name handling The Worker was documented, commented, and configured as if unsafe_eval only worked in wrangler dev. Updated src/worker/index.ts, wrangler.toml, and the README to describe the production path (Cloudflare accounts with the unsafe_eval binding enabled), and pointed users to auth / rate limiting as the real production gate. Also added assertSafeToolName in wrap-code.ts to reject tool names that would break out of the generated function identifier, e.g. "foo'); process.exit(1); (function bar() {". Added tests covering quotes, backticks, whitespace, semicolons, newlines, empty strings, leading digits, and the valid identifier shapes. Added a new escape-attempts.test.ts covering JSON.stringify escaping of adversarial tool-result values and verifying the result lands in a plain object-literal assignment (never a template literal). * refactor(ai-ollama): extract tool-converter with test coverage Tool handling was inlined inside the text adapter with raw type casts. Extracted into src/tools/function-tool.ts + tool-converter.ts matching the structure used by ai-openai, ai-anthropic, ai-grok, and ai-groq. Re-exported as convertFunctionToolToAdapterFormat and convertToolsToProviderFormat from the package index. Added 29 unit tests covering the converter, client utilities (createOllamaClient, getOllamaHostFromEnv, generateId, estimateTokens), and the text adapter's streaming behaviour: RUN/TEXT_MESSAGE/tool-call lifecycle events, id synthesis when Ollama omits a tool-call id, tool forwarding to the SDK in provider format, and structured-output JSON parsing with error wrapping. The package previously had 73 source files and zero unit tests. * fix(frameworks): propagate useChat callback changes after re-render onResponse, onChunk, and onCustomEvent were captured by reference at ChatClient creation time. When a parent component re-rendered with fresh closures, the client kept calling the originals. - ai-react / ai-preact: wrap the three callbacks the same way onFinish/onError already were, reading from optionsRef.current at call time. - ai-vue / ai-solid: wrap the callbacks to read options.xxx at call time. This also fixes a subtler bug where using client.updateOptions to swap callbacks could not clear them (the "!== undefined" guard silently skipped undefined values). - ai-svelte: documented the capture-at-creation behaviour — Svelte's createChat runs once per instance and there's no per-render hook, so callbacks are frozen unless the caller mutates the options object or calls client.updateOptions imperatively. Added a React regression test that rerenders with a new onChunk and verifies the new callback fires while the original does not. * refactor(ai, ai-openai): narrow error handling and stop logging raw errors The three catch blocks that convert thrown values into RUN_ERROR events (stream-to-response.ts, activities/stream-generation-result.ts, activities/generateVideo/index.ts) were using catch(error: any) and dereferencing .message / .code without checks. Added a shared toRunErrorPayload(error, fallback) helper under activities/ that accepts Error instances, plain objects with message/code fields, or bare strings, and funnels all three sites through it with a per-site fallback message. Removed four console.error calls in the OpenAI text adapter's chatStream that dumped the full error object to stdout. SDK errors can carry the original request (including auth headers), so the library no longer logs them; upstream callers should convert errors into structured events. Added 8 unit tests for toRunErrorPayload including a leaked-properties test confirming the helper does not expose extra fields. * test(isolates): add sandbox escape-attempt tests for Node and QuickJS drivers Covers the attack surface a malicious skill / code-mode snippet might probe: process/require/fetch should be unavailable, prototype pollution must not leak to the host or between contexts, synchronous CPU-spin loops must be interrupted by the timeout (not hang), and Function- constructor escape attempts must execute inside the isolate (never returning a real host process object). QuickJS also gets a test that globalThis mutations inside one context do not bleed into a sibling context. * ci: apply automated fixes * fix: address PR review feedback - ai-preact: forward onCustomEvent in useChat (changeset claimed the fix covered preact but it was silently dropped before reaching ChatClient). - ai-isolate-cloudflare: reject JS reserved keywords as tool names (return, class, function, if, await, ...) so the wrapper fails fast at generation time instead of with a cryptic SyntaxError at eval. - ai/src/activities/error-payload: apply typeof string check to the Error branch's code field, matching the plain-object branch. Some SDKs attach numeric or Symbol codes to Error instances. - ai-ollama text-adapter test: strengthen OLLAMA_HOST assertion by tracking the mocked Ollama constructor args, so the test fails if the env var is ignored. - ai-ollama utils test: rename 'when OLLAMA_HOST is unset' to 'empty' since the setup stubs an empty string. - ai-code-mode-skills file-storage test: use vi.useFakeTimers() for the createdAt/updatedAt round-trip instead of a 5ms real sleep. * ci: apply automated fixes * fix(ai, ai-ollama): merge-driven regressions from CR Address CR findings after merging main: - ai-ollama tests: inject testLogger (from resolveDebugOption(false)) into every adapter.chatStream and adapter.structuredOutput call — main's TanStack#467 made `logger` required on TextOptions, the PR's new tests were written against the pre-TanStack#467 contract and crashed at runtime on `logger.errors`/`logger.request` dereference. - generateVideo: narrow `error` via toRunErrorPayload before handing it to logger.errors. Previously passed the raw error object through the logger meta, which would surface SDK request state (headers, payloads) to any user-supplied logger — defeating the hardening the PR applies to the RUN_ERROR event. - error-narrowing changeset: update wording to match actual code. The OpenAI text adapter's chatStream still logs under the merge, but now through the narrowed `{message, code}` payload rather than raw errors. Changeset previously claimed "the library now re-throws without logging", which didn't match shipped behavior. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…anStack#506) * feat(ai-grok): add audio and speech adapters for xAI Add `grokSpeech` (TTS via /v1/tts), `grokTranscription` (STT via /v1/stt), and `grokRealtime` + `grokRealtimeToken` (Voice Agent via /v1/realtime) because xAI's standalone audio APIs were shipped publicly and the adapter previously exposed only text/image/summarize. The TTS/STT endpoints are not OpenAI-compatible so these adapters use direct fetch rather than the OpenAI SDK; the realtime API mirrors OpenAI's shape with URL/provider swaps. E2E coverage is wired via mock.mount('/v1/tts'...) on aimock. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Merge from upstream * feat(ai-grok): wire shared debug logger into audio and realtime adapters Adopt the @tanstack/ai/adapter-internals logger across grokSpeech, grokTranscription, grokRealtimeToken, and grokRealtime so users can toggle debug output the same way they do on other adapters — `debug: true` for full tracing, `debug: false` to silence, or a DebugConfig for per-category control and a custom Logger. Replaces the remaining console.error / console.warn calls in the realtime adapter with logger.errors so nothing is lost when debugging is off. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * fix(ai-grok): correct super() arg order in audio adapters The transcription and TTS adapters were calling super(config, model), but BaseTranscriptionAdapter/BaseTTSAdapter expect (model, config), causing TS2345 build errors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ai-grok): pass logger to audio adapter tests After the logger was wired into the audio adapters, the unit tests need to provide one when calling transcribe/generateSpeech directly (activities normally inject it via resolveDebugOption). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(ai-grok): route audio adapter tests through core functions Per project convention, tests should not invoke adapter methods directly — they call generateSpeech()/generateTranscription() with the adapter instance, so the core function injects logger, emits events, and exercises the real public surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * fix(ai-grok): address cr-loop round 1 findings ai-grok realtime adapter: - cleanup pc/localStream/audioContext/dataChannel on connect() failure - dataChannelReady rejects on error/close/ICE-failed/timeout - RTCErrorEvent extracted properly instead of [object Event] - onmessage parse errors emit to consumers - input_audio_transcription no longer overrides caller on every update - response.done preserves idle mode after stopAudioCapture - setupOutputAudioAnalysis disposes prior audioElement, surfaces autoplay blocks - audioContext.resume failures emit error instead of silent swallow - currentMessageId reset on response.created (tool-only turns) - pc.onconnectionstatechange / oniceconnectionstatechange emit status_change - sendImage uses object image_url for OpenAI-realtime compatibility - unknown server events logged via default branch ai-grok TTS/STT: - getContentType returns audio/L16 for pcm (valid IANA MIME) - toAudioFile requires explicit audio_format for bare base64 - transcription option renamed format -> inverse_text_normalization ai-grok realtime token: - expires_at unit-safety guard (seconds vs ms) ai-grok types: - single source of truth for GrokRealtimeModel (model-meta) ai-grok tests: - cover aac/flac in pickCodec test - normalize header assertions via Headers() - add realtime-token unit-safety tests examples/ts-react-chat: - resolveModel fails loud via InvalidModelOverrideError (no silent fallback) - audio/speech/transcribe routes return 400 with structured body testing/e2e: - media-providers uses valid grok-2-image-1212 model - test-matrix imports from feature-support (dedupe) * fix(ai-grok): address cr-loop round 2 confirmation findings ai-grok realtime adapter: - shared teardownConnection() helper runs on SDP and post-SDP failure paths, disposing input/output analysers, audio element, mic, data channel, pc, and audio context - pre-open dataChannelReady rejection on failed/closed/disconnected pc states - pc.onconnectionstatechange is sole source of status_change (ice handler only rejects) - sendImage detects data: prefix (no more double-wrap) ai-grok audio utils: - malformed data: URI MIME parse throws instead of silently defaulting to audio/mpeg - empty/missing base64 payload throws - explicit audioFormat argument wins over URI-embedded MIME ai-grok TTS: - audio/L16 content-type includes required rate= parameter from modelOptions.sample_rate ai-grok tests: - realtime-token afterEach restores original XAI_API_KEY - new coverage for malformed data URIs, audioFormat precedence, and rate= in audio/L16 examples/ts-react-chat: - new UnknownProviderError typed class, 400 mapping in audio/speech/transcribe routes - server-fns ServerFnError wraps typed adapter errors with stable code/details * fix(ai-grok): address cr-loop round 3 confirmation findings examples/ts-react-chat: - generateSpeechFn/transcribeFn/generateSpeechStreamFn/transcribeStreamFn now wrap adapter construction with rethrowAudioAdapterError for consistent typed-error responses - realtime image display guards against data:/http(s): double-wrap ai-grok realtime adapter: - teardownConnection drains pendingEvents; sendEvent logs and skips after teardown ai-grok TTS: - sample_rate always forwarded in output_format so body and contentType rate agree * fix(ai-grok): address cr-loop round 4 confirmation findings ai-grok realtime adapter: - teardownConnection on getUserMedia failure (mic/pc/dataChannel leak on mic denial) - response.function_call_arguments.done drops event if call_id absent (no item_id fallback) - isTornDown set at top of teardown to guard handlers firing during close() awaits - setupInputAudioAnalysis/setupOutputAudioAnalysis skip when torn down - onconnectionstatechange no longer double-emits status_change during disconnect() ai-grok audio utils: - toAudioFile Blob/File branch prefers explicit audioFormat over Blob.type ai-grok TTS: - sample_rate forwarded only when caller provides one or codec is pcm (don't override server defaults for container codecs) Tests updated to cover new audioFormat precedence paths and adjusted sample_rate assertions. * fix(ai-grok): address cr-loop round 5 confirmation findings ai-grok realtime adapter: - pc.connectionState=failed triggers automatic teardownConnection (mic/pc/audioContext no longer leak on spontaneous failure) - flushPendingEvents wraps send in try/catch; emits error on failure instead of hanging caller - handleServerEvent case 'error' validates shape of event.error; preserves code/type/param; safe against null/missing fields - autoplay and audioContext.resume failures log without emitting fatal error events (routine browser-policy outcomes) - dataChannel.onerror/onclose gated behind isTornDown to suppress post-disconnect error events examples/ts-react-chat: - realtime.tsx handleImageUpload validates FileReader result, file.type, and base64 extraction; surfaces errors visibly * fix(ai-grok): extensionFor maps mulaw/alaw MIME types to sensible filenames utils/audio.ts produced 'audio.basic' and 'audio.x-alaw-basic' for mulaw/alaw via the default-branch MIME split. Servers using filename as a format hint now see 'audio.mulaw' / 'audio.alaw', matching the reverse toMimeType mapping. * ci: apply automated fixes * refactor(ai-grok): extract form/body builders, adopt ModelMeta convention, fix xAI realtime event names Refactors from user review: adapters: - tts.ts: extract buildTTSRequestBody helper (codec/sample_rate/voice default resolution + body assembly). Export getContentType for consumer use. - transcription.ts: extract buildTranscriptionFormData helper (wire-field mapping including xAI's named 'format' boolean toggle for inverse text normalization). model-meta.ts: audio and realtime models now use the same `as const satisfies ModelMeta` convention as chat/image models (GROK_TTS, GROK_STT, GROK_VOICE_FAST_1, GROK_VOICE_THINK_FAST_1) with input/output modalities and tool_calling / reasoning capabilities. realtime adapter: - Replace drive-by 'as' casts on untyped server events with runtime-checked readers (readString, readObject, readObjectArray); malformed frames return undefined instead of throwing a TypeError. - Accept both legacy OpenAI-realtime event names and current xAI voice-agent names per docs.x.ai: response.output_audio.* / response.output_audio_transcript.* / response.text.* (plus existing response.audio.* / response.audio_transcript.* / response.output_text.* aliases for compatibility). - RealtimeServerError type replaces repeated 'as Error & { code?: string }' casts. realtime token: - Wrap request body with { session: { model } } per xAI /v1/realtime/client_secrets schema (was bare { model } before). * test(ai-grok): cover realtime token body { session: { model } } shape * ci: apply automated fixes * refactor(ai-grok): drop @tanstack/ai-client peer dep by inlining realtime contract The RealtimeAdapter / RealtimeConnection interfaces are duplicated locally in src/realtime/realtime-contract.ts. The adapter imports them from there instead of @tanstack/ai-client, so consumers of @tanstack/ai-grok no longer have to install @tanstack/ai-client unless they also want to construct a RealtimeClient from it (structural typing covers that use case). @tanstack/ai-client stays as a devDependency to run a type-level drift check (tests/realtime-contract.drift.test-d.ts) that asserts our inlined contract is bidirectionally assignable to the canonical one. If ai-client ever changes the interface, that file will fail to compile and we update both in lockstep. publint --strict: clean. * ci: apply automated fixes * fix(ai-grok): address CodeRabbit PR review - tts.ts / transcription.ts: spread `defaultHeaders` BEFORE Authorization / Content-Type so a caller-supplied header can't silently clobber the bearer token or auth content-type. - utils/audio.ts: new `arrayBufferToBase64` helper — Buffer fast path on Node, chunked btoa fallback everywhere else (browser, Workers, Bun). Replaces the Node-only `Buffer.from(arrayBuffer).toString('base64')` in tts.ts. - transcription.ts: new `GrokTranscriptionWord` interface extends the core `TranscriptionWord` with optional `confidence` and `speaker`. The adapter now preserves both fields when xAI returns them, so callers that narrow via `as Array<GrokTranscriptionWord>` get the diarization output they asked for. Test expectations updated. - tts.ts: mulaw/alaw `contentType` now includes a `;rate=…` parameter (as `audio/PCMU` / `audio/PCMA` per RFC 3551) when the caller requests a non-default sample rate, instead of the 8 kHz-implying `audio/basic` / `audio/x-alaw-basic`. - realtime/adapter.ts: `conversation.item.truncated` flips mode back to `listening` so the visualiser can't get stuck on `speaking` after an interrupt. `sendEvent` wraps `dataChannel.send` in try/catch consistent with `flushPendingEvents`. The shared `emptyFrequencyData` / `emptyTimeDomainData` buffers are gone — `getAudioVisualization` returns a fresh `Uint8Array` per call so consumers can't mutate a module-level instance. - realtime/token.ts: adds a 15s `AbortController` timeout on the client_secrets request so a dead endpoint can't hang the caller forever. Validates `client_secret.value` / `expires_at` shape at runtime before dereferencing so a malformed response throws a descriptive error. - realtime/realtime-contract.ts: JSDoc filename ref updated. - examples/ts-react-chat audio/speech/transcribe routes: unify the 400 unknown_provider payload under the `provider` key (was `providerId`) to match the invalid_model_override branch and the request body. * ci: apply automated fixes --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Alem Tuzlak <t.zlak@hotmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…adapter
The stream() factory now accepts Promise<AsyncIterable<StreamChunk>> and
Promise<Response> in addition to the existing AsyncIterable shape, so a
TanStack Start server function (which is just an async API endpoint) can
be wired directly into useChat without a route handler:
useChat({
connection: stream((messages) => chatFn({ data: { messages } })),
})
When the factory returns a Response (e.g. via toServerSentEventsResponse),
the adapter parses the SSE body into chunks. rpcStream() likewise accepts
a Promise-returning RPC call.
Adds unit tests for both new shapes and a docs section in
chat/connection-adapters.md.
Adds a working /server-fn-chat route that wires useChat to a TanStack
Start server function via the stream() connection adapter:
useChat({
connection: stream((messages) =>
chatFn({ data: { messages: messages as UIMessage[] } }),
),
})
The new chatFn handler in lib/server-fns.ts returns
toServerSentEventsResponse(chat({ ... })) — the stream() adapter awaits
the server function and parses the SSE response into chunks.
Sits alongside the existing index.tsx pattern (fetchServerSentEvents
to a route handler) so users can compare the two invocation styles.
🚀 Changeset Version Preview1 package(s) bumped directly, 14 bumped as dependents. 🟥 Major bumps
🟨 Minor bumps
🟩 Patch bumps
|
|
Hi! I'm the It looks like you correctly set up a CI job that uses the autofix.ci GitHub Action, but the autofix.ci GitHub App has not been installed for this repository. This means that autofix.ci unfortunately does not have the permissions to fix this pull request. If you are the repository owner, please install the app and then restart the CI workflow! 😃 |
autofix.ciLead with what stream() does (typed RPC into useChat), instead of calling a server function "just a fancy/async API endpoint." Same edits applied to the changeset and the stream() JSDoc. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…server-functions-pBsj5 # Conflicts: # examples/ts-react-chat/src/components/Header.tsx # examples/ts-react-chat/src/lib/server-fns.ts # packages/typescript/ai-client/src/connection-adapters.ts
The previous merge commit accidentally included stale references in @tanstack/ai-fal that this PR shouldn't have touched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Type-check was failing on CI because the new server-function and RPC async-iterable test fixtures yielded raw string literals for chunk type, which don't satisfy the EventType enum required by StreamChunk. Switch to the enum and add the required threadId to RUN_FINISHED. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…abortSignal Address review findings on the server-function stream() adapter PR: - Synthesized RUN_FINISHED/RUN_ERROR events in normalizeConnectionAdapter no longer use `as unknown as StreamChunk`. Track threadId/runId from upstream chunks during iteration and reuse them; fall back to synthesized IDs only when no upstream chunk carried them. Use the EventType enum and typed RunFinishedEvent/RunErrorEvent so missing required fields are caught by the compiler instead of papered over. - Stop swallowing JSON.parse failures in parseSSEChunks and fetchHttpStream. A malformed mid-stream chunk is a protocol error; let it throw so it surfaces as RUN_ERROR via the connect-wrapper's catch path instead of silently dropping data behind a console.warn the user never sees. - Widen stream() and rpcStream() factory signatures with an optional abortSignal third arg and pass it through. Backwards-compatible — callers that ignore the third parameter are unaffected. Lets long-running server functions cancel in-flight work when useChat aborts. Tests updated to assert SyntaxError propagation rather than silent dropping on malformed JSON, and to expect the new third call argument on factory mocks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…railing buffer Refine the SSE parser so the throw-on-parse-failure behavior doesn't regress on legitimate SSE traffic: - parseSSEChunks now skips SSE comment lines (`:`) and non-data fields (`event:`, `id:`, `retry:`) which proxies and CDNs commonly inject as keepalives. Previously these would have flowed into JSON.parse and thrown, killing otherwise-healthy streams behind any infrastructure that injects SSE control lines. - readStreamLines no longer yields the unterminated trailing buffer at stream end. A non-empty buffer means the connection was cut mid-line, so the content is partial by definition — yielding it would feed truncated JSON to the parser and surface a misleading RUN_ERROR for what is really a transport-layer issue. Warn and discard instead. Bare-line JSON (legacy/raw mode) is still accepted to preserve the existing public behavior covered by the `should handle SSE format without data: prefix` test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…t UIMessage[] in chat() Drops the `as any` / `as Array<UIMessage>` casts previously needed when wiring useChat through a TanStack Start server function into chat(). The stream() factory now declares Array<UIMessage> (with a runtime assert matching the ChatClient invariant), and chat()'s messages option accepts UIMessage[] directly alongside ConstrainedModelMessage[] — the runtime already normalised both via convertMessagesToModelMessages. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls the @tanstack/ai changes back out — the chat() messages-option widening to accept UIMessage[] is a separate concern from the stream() server-function feature this PR is about. Restores the example's `as any` cast with a comment, drops the @tanstack/ai minor bump from the changeset, and reverts chat/index.ts to its pre-PR state. Also bumps the example adapter to gpt-5.2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tombeckenham
added a commit
that referenced
this pull request
May 21, 2026
…TanStack#600) * feat(ai): structured-output middleware coverage (closes TanStack#390) Middleware now wraps the final structured-output provider call in `chat({ outputSchema })` for both Promise<T> and streaming variants. - Add `'structuredOutput'` to `ChatMiddlewarePhase` and set it on `ChatMiddlewareContext` for the duration of the final structured-output adapter call. - Add optional `ChatMiddleware.onStructuredOutputConfig` hook receiving a `StructuredOutputMiddlewareConfig` (with the JSON Schema) which may return a partial to transform the config before the final call. - Export new `StructuredOutputMiddlewareConfig` type extending `ChatMiddlewareConfig` with `outputSchema: JSONSchema`. - `onChunk` now observes chunks from the final structured-output call; `onFinish` fires once at the end of the whole `chat()` invocation after finalization completes. - Remove the previous `RUN_STARTED`/`RUN_FINISHED` suppression hack in `runStreamingStructuredOutput`; engine now emits exactly one outer pair around the whole run. * test(ai-e2e): add structured-output x middleware spec Adds a Playwright spec that exercises the structured-output finalization path end-to-end with middleware attached, plus a route + fixture for the mocked LLM call and a phase-capture helper for asserting middleware phase transitions. * docs(ai): document onStructuredOutputConfig hook and structuredOutput phase Updates middleware and structured-outputs skills, public docs, and regenerated TypeDoc reference for the new `structuredOutput` middleware phase and the `onStructuredOutputConfig` hook. * fix(ai): address CR findings on structured-output middleware coverage - Type: onConfig / onStructuredOutputConfig Promise return allows null - Error diagnostics: preserve cause + code on validation failures, with smarter message extraction for plain-object errors (Standard Schema) - Streaming consumers see RUN_ERROR on finalization failure (missing result or validation), guarded against double-emission - Synth structured-output.start carries threadId - Abort signal checked inside runStructuredFinalization for-await loop - runAgenticStructuredOutput rethrow preserves cause + code - Skill examples use (ctx, info) signature for onFinish/onError - Docs Mermaid diagram includes structuredOutput phase branch - Docs prose acknowledges 3 onConfig firings (init + beforeModel + structuredOutput boundary) - Docs add FinishInfo table marking info.usage explicitly optional - Changeset reflects suppression-hack relocation (not removal) - Test comment corrected (synthesis still happens) - E2E spec title honest about stream:true; docblock notes scope - kind=phase GET no longer gated behind OTEL_TEST_ENABLED Call-site enumeration (Procedure 2.8): - finalizationError gained cause?: unknown. Readers updated: * TextEngine terminal hook chooser — propagates cause via Error({ cause }) and surfaces code as a non-enumerable Object.defineProperty. * runAgenticStructuredOutput — same treatment when re-throwing. * getFinalizationError return type widened to include cause. - runStructuredFinalization gained a post-loop synthetic RUN_ERROR yield path, gated on yieldChunks and a new runErrorYielded flag. The streaming consumer in runStreamingStructuredOutputImpl iterates engine.run() and propagates the new chunk transparently — no consumer-side changes. - Synthesized structured-output.start gained threadId — passive readers, no behavioral impact. * fix(ai): address Round 2 CR findings on structured-output middleware - Mid-finalization abort routes through onAbort (not onError): skip missing-result attribution when isCancelled() - Finalization chunks no longer pollute agent-loop state: removed handleStreamChunk(chunk) call in runStructuredFinalization; targeted updates only for structured-output.complete + RUN_ERROR + RUN_FINISHED.usage - Synth RUN_ERROR for empty-stream case is preceded by a synth structured-output.start so client-side StructuredOutputPart routing works Call-site enumeration: - handleStreamChunk removal: accumulatedContent (now agent-loop only; info.content stays clean of JSON deltas), finishedEvent / lastFinishReason (finalization no longer overwrites the agent loop's real finish reason), currentMessageId (unchanged path), currentThinking* (no thinking pollution), earlyTermination (irrelevant — finalization is already terminating). Explicit branches still capture structured-output.complete, RUN_ERROR (finalizationError), and RUN_FINISHED.usage (runOnUsage). - isCancelled() early-return + chooser gating: run()'s finally block fires onAbort when !terminalHookCalled && isCancelled(). The terminal- hook chooser at the end of the try-block now additionally skips when isCancelled() so it can't pre-empt the finally with a stray onFinish. - Pre-synth-start before synth-RUN_ERROR: uses the same buildSynthesizedStart() + pipeThroughMiddleware path as the in-loop synth, gated on !startEmitted so we never double-emit. * fix(ai): align onFinish info docs with implementation - Docs/skill no longer claim onFinish.info.usage reflects the full run including finalization tokens. The Round 2 fix correctly segregated finalization state; info.* reflects the agent loop's terminal state only. - Add unit test pinning the documented semantics: tools-less structured-output run gives info.usage=undefined, finishReason=null, content='', while onUsage fires once for finalization tokens. * fix(ai): clarify tools on StructuredOutputMiddlewareConfig is not forwarded to the structured-output adapter call Round 4 CR finding: tools is structurally inherited from ChatMiddlewareConfig but the engine omits tools from structuredCallOptions.chatOptions. Document the caveat in the type JSDoc and the public middleware reference so middleware authors don't expect tools transformation at this boundary to take effect. * fix(ai): align runStreamingStructuredOutput JSDoc with implementation Round 5 CR finding: the JSDoc claimed "Validates the parsed object against the original Standard Schema" but the implementation explicitly defers validation to the consumer (via `void outputSchema`). Update the JSDoc to honestly describe the streaming-path validation policy and call out the deliberate asymmetry with `runAgenticStructuredOutput` (which does validate). * fix(docs): clarify server-side validation is path-dependent (streaming vs Promise<T>) Round 6 CR finding: docs/structured-outputs/overview.md claimed "Server-side validation against your schema is always authoritative" but the streaming path (chat({ outputSchema, stream: true })) deliberately defers validation to the consumer. Update the prose to reflect the actual path-dependent behavior — agentic Promise<T> validates server-side; streaming forwards the adapter event verbatim and consumers validate downstream. * fix(ai): apply Procedure 3 bucket-(c) audit promotions Bucket (c) Promotion Audit (cr-loop final step) flagged 4 items as load-bearing on the structured-output subject this PR makes authoritative: - PROMOTE_TO_A: runAgenticStructuredOutput was calling convertSchemaToJsonSchema without forStructuredOutput: true while runStreamingStructuredOutput did. Same Zod schema produced different JSON Schema depending on stream mode. Both paths now use the strict converter, eliminating the divergence. - PROMOTE_TO_B (3 trivial fixes on PR-adjacent surfaces): - fallbackStructuredOutputStream's IDs prefixed 'mock-' in production code; renamed to 'fallback-' to stop leaking test-style identifiers into user-visible run/thread/message IDs for Anthropic/Gemini/Ollama structured-output runs. - fallbackStructuredOutputStream's RUN_ERROR chunk was missing threadId while sibling RUN_STARTED and RUN_FINISHED carried it; added for consumer correlation. - chat() JSDoc example used chunk.type === 'content' (wrong); changed to 'TEXT_MESSAGE_CONTENT'. 26 other bucket-(c) items confirmed STAY_IN_C (pre-existing, not subject-load-bearing) and are reported to the loop-exit follow-up list. 1 item (gpt-5.2 model existence) REFUTED. * ci: apply automated fixes * fix(ai): address PR TanStack#600 review — critical + important findings - Skip agent loop when finalStructuredOutput is set and tools.length === 0 to avoid a wasted chatStream round-trip before finalization (Critical #1). - Omit `tools` structurally from StructuredOutputMiddlewareConfig — the field was inherited but silently discarded at the provider boundary (Important #2). - Preserve Standard Schema `issues[]` on validation failures via a new exported StandardSchemaValidationError carried as `error.cause` (Important #4). - Preserve the original adapter error (stack, cause, provider properties) on the fallbackStructuredOutputStream path via an onAdapterError callback (Important #5). - Add tests for messages-transform via onStructuredOutputConfig and for mid-finalization abort routing through onAbort (Important #6, #7). - Strip transitional source comments ("in Task 7", "closes issue TanStack#390"). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(examples): add /verify-pr600 page to ts-react-chat Adds a diagnostics page in the ts-react-chat example that exercises all four PR TanStack#600 review fixes against the real chat() engine using inline mock adapters. No API keys required — click "Run verification" and see pass/fail per scenario with observed values. - POST /api/verify-pr600 runs the four scenarios server-side. - /verify-pr600 page calls the endpoint and renders results. - Header nav gains a "Diagnostics" section linking to the page. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(examples): replace verify-pr600 with real-provider repros The mock-based verify-pr600 page only validated my own fixes; it didn't prove anything about real provider wire formats. Replace with two real-provider verifications: - New /issue-390-repro page runs the exact gist from the issue reporter (@imsherrill) against geminiText('gemini-2.5-flash') and surfaces the middleware logs + per-phase chunk counts. Fix is verified iff the middleware observed any chunks with ctx.phase === 'structuredOutput'. - Existing /generations/structured-output page now instruments every request with a counter middleware. Counts surface via a JSON field (non-streaming) or a trailing CUSTOM `phase-counts` event (streaming). Works across all configured providers (OpenAI/Anthropic via OpenRouter, Gemini via OpenRouter, Grok, Groq). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(examples): drop /issue-390-repro page The counter middleware on /generations/structured-output already demonstrates the PR TanStack#600 fix against real providers — the dedicated single-shot repro page is redundant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ai): drop unused ChatMiddlewarePhase import after merge Surfaced by tsc after merging origin/main (the eslint-config 0.4.0 bump in TanStack#607 strengthens unused-import detection). The type is re-exported elsewhere and not referenced inside this file. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * fix: address CI lint + openrouter test failures - Remove unnecessary `as object` cast in compose.ts; the upgraded typescript-eslint (via TanStack#607) flags it as unnecessary because `Object.keys` already accepts the original type. - Update the openrouter `chat() entrypoint with strict transformation` test: with Critical #1 (skip agent loop when tools.length === 0), the engine no longer consumes the streaming mock for an empty agent pass — the structured-output payload now arrives via the same streaming mock that previously held the placeholder 'ok' delta. Move the JSON payload into the streaming mock and assert via `responseFormat` presence instead of `stream === false`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Tom Beckenham <34339192+tombeckenham@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
Enhanced the
stream()connection adapter to support TanStack Start server functions that return either an async iterable directly or an SSEResponse. This enables seamless integration of server functions intouseChatwith end-to-end type safety.Key Changes
Refactored SSE parsing logic into reusable functions:
parseSSEChunks()— parses SSE-formatted lines intoStreamChunkobjectsresponseToSSEChunks()— converts aResponsebody to SSE chunksExtended
stream()adapter to handle three return shapes:AsyncIterable<StreamChunk>— direct in-process streams (existing behavior)Promise<AsyncIterable<StreamChunk>>— server function returning the chat stream directlyPromise<Response>— server function returningtoServerSentEventsResponse(stream)(recommended for network efficiency)Updated
rpcStream()adapter to acceptPromise<AsyncIterable<StreamChunk>>in addition to synchronous iterablesAdded comprehensive documentation and examples:
server-fn-chat.tsxroute demonstrating server function integrationserver-fns.tswithchatFnexampleExported
StreamFactoryResulttype for public API clarityExample Usage
✅ Checklist
pnpm run test:pr.🚀 Release Impact
https://claude.ai/code/session_01J7ZxWTciJhJEUob5kPHZcf