fix(cerebras): preserve thinking inline for models that reject reasoning_content - #10774
Conversation
|
@codex review |
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Thanks for the fast turnaround on the split! One required item before merge: the Azure AI Foundry provider (#10622) landed on main after your base, and it constructs ModelInfo (azure_foundry.rs, model_info_for_deployment) and OpenAiFormatOptions (openai_compatible.rs) without the new fields, so the branch merges cleanly textually but won't compile. Please rebase onto main and add the two new None fields / a ..Default::default() at those two sites- should be quick. LGTM after the rebase. |
…ing_content Cerebras rejects multi-turn requests that replay messages[].reasoning_content with 400 wrong_api_format, so any session with thinking enabled fails from the second turn onward. Models can now declare a thinking_preservation_format, replaying thinking inline in content (plain, or wrapped in <think> tags) instead of as a separate field. They can also declare request_params, per-model static request body params mirroring ModelConfig.request_params; Cerebras models use it to send reasoning_format: parsed so reasoning comes back structured. ModelInfo.reasoning stays bool. The generalized reasoning config is deferred. Fixes aaif-goose#10406
07511d5 to
0369490
Compare
addressed, thank you! |
* main: fix(desktop): allow removing images when editing user messages (#9979) Fetch cursor-agent models from CLI (#10367) fix(cerebras): preserve thinking inline for models that reject reasoning_content (#10774) perf: make CLI streaming render incremental instead of O(n^2) (#10487) Moving to issues as the new PRs (#10818) ci: defer privileged recipe scans for forks (#10825) docs: adopt issue-first contribution workflow (#10819) Add Azure AI Foundry multi-LLM provider (#10622) fix(together): parse model list response (#10576) docs: remove retired site resources (#10817) feat: streaming shell output while commands run (#10808)
* main: (224 commits) fix(desktop): allow removing images when editing user messages (aaif-goose#9979) Fetch cursor-agent models from CLI (aaif-goose#10367) fix(cerebras): preserve thinking inline for models that reject reasoning_content (aaif-goose#10774) perf: make CLI streaming render incremental instead of O(n^2) (aaif-goose#10487) Moving to issues as the new PRs (aaif-goose#10818) ci: defer privileged recipe scans for forks (aaif-goose#10825) docs: adopt issue-first contribution workflow (aaif-goose#10819) Add Azure AI Foundry multi-LLM provider (aaif-goose#10622) fix(together): parse model list response (aaif-goose#10576) docs: remove retired site resources (aaif-goose#10817) feat: streaming shell output while commands run (aaif-goose#10808) feat(acp): allow session/new to request a Hidden session via _meta.hidden (aaif-goose#10811) chore: removed stale text pnpm workspace (aaif-goose#10809) feat(otel): emit GenAI semantic convention attributes (aaif-goose#10700) fix: avoid double loading full conversation on session open (aaif-goose#10794) feat(providers): forward images and MCP embedded-resource blobs in Anthropic and Google formats (aaif-goose#10340) Deprecate and remove ui/text TUI (aaif-goose#10799) Upgrade to rmcp 3.0 (aaif-goose#10789) fix(release): signing environment (aaif-goose#10797) feat(tools): collapse const-union enums in tool schemas (aaif-goose#10577) ...
|
Confirmed on our side, tested with a Cerebras reasoning model (gpt-oss-120b), thanks for the implementation! |
fix(cerebras): preserve thinking inline for models that reject
reasoning_contentFixes #10406. Supersedes the Cerebras fix in #10417.
Problem
Cerebras rejects multi-turn requests that replay
messages[].reasoning_contentwith400 wrong_api_format. Once a session has one assistant turn with thinking, everysubsequent turn fails.
Before
cerebras-before.mov
After
after2.mp4
Fix
Two halves, both required.
Replay side — a declarative model can declare how its thinking is replayed:
thinking_preservation_formatcontent_prependcontent, plain textcontent_xmlcontent, wrapped in<think>tagsreasoning_contentreasoning_contentfield (OpenAI-compatible default)Request side —
ModelInfo.request_paramscarries per-model static request-bodyparams. Cerebras models send
{"reasoning_format": "parsed"}so reasoning comes back ina structured field.
request_paramsmirrors the existingModelConfig.request_params: same type, samesemantics, just sourced from the model declaration instead of user config. It is
unconditional and untyped — no
enabledflag, no coupling to effort levels, and not gatedon whether reasoning is on, since
reasoning_formatdescribes the shape of the responserather than whether the model should reason. It is declared per-model, so it only reaches
models that support it. Reserved keys (
model,messages,stream,stream_options)are skipped so a declaration cannot clobber the streaming setup.
Changes
goose-provider-types/src/base.rs—ThinkingPreservationFormatenum, and twooptional
ModelInfofields (thinking_preservation_format,request_params).DeclarativeProviderConfig.modelsis alreadyVec<ModelInfo>, so declarative JSON feedsboth without extra plumbing.
goose-provider-types/src/formats/openai.rs—OpenAiFormatOptions.thinking_preservation_formatandinline_reasoning_content. Itruns after
merge_split_tool_call_messages, which usesreasoning_contentto identifymessages split from the same assistant turn.
is_reserved_request_param_keyis nowpubso the provider reuses it rather than duplicating the list.
goose-providers/src/openai.rs—custom_modelsretains fullModelInfoinstead ofjust names (
fetch_supported_modelsmaps back to names).stream()looks the model uponce and uses both fields: it passes the format into
OpenAiFormatOptions, and mergesrequest_paramsinto the payload aftercreate_request_with_optionsreturns.apply_ollama_optionsis the precedent for provider-level payload shaping.goose-providers/src/declarative/definitions/cerebras.json— per-modelthinking_preservation_formatandrequest_params; addsgemma-4-31b.ModelInfoliterals gain the two new fields and twoOpenAiFormatOptionsliterals gain..Default::default(), acrossdatabricks.rs,declarative_providers.rs,provider_registry.rs, andtests/agent.rs.goose-self-test.yaml— adds Phase 3C, a manual multi-turn thinking-preservationprocedure (
AGENTS.md:71). It self-skips unlessCEREBRAS_API_KEYis set and thesession is on a Cerebras model, so the automated coverage is the unit tests below.
cerebras.jsondeliberately does not set"preserves_thinking": true:should_preserve_thinking_by_defaultalready defaults it totrueforProviderEngine::OpenAI, and cerebras is"engine": "openai". There is a test assertingthat.
Limitations
Cerebras models do not receive
reasoning_effort, so the thinking-effort setting has noeffect on them. This is not a regression —
is_openai_responses_modelmatches onlyo\d+and
gpt-5*, sogpt-oss-120b,zai-glm-4.7, andgemma-4-31bget no reasoning param onmaintoday either. Mapping thinking effort onto these models is follow-up work.Testing
Verified end to end against a live Cerebras key on all three models: multi-turn sessions
with thinking enabled now work, which is the scenario in #10406.
Unit tests added:
ModelInfodeserializes both new fields, and defaults them toNonewhen absent.contentand dropsreasoning_content.reunite into one assistant message with both tool calls, with the thinking inlined once.
cerebras.jsondeclares the expected format andreasoning_format: parsedper model,and still defaults
preserves_thinkingtotrue.request_paramscannot clobberstream,stream_options,model, ormessages.cargo fmt --all -- --check,cargo build --workspace --all-targets,cargo clippy --all-targets -- -D warnings, andcargo test -p goose-provider-types -p goose-providersall pass.