Skip to content

fix(cerebras): preserve thinking inline for models that reject reasoning_content - #10774

Merged
filipkujawa merged 1 commit into
aaif-goose:mainfrom
aaalaniz:cerebras-thinking-preservation
Jul 30, 2026
Merged

fix(cerebras): preserve thinking inline for models that reject reasoning_content#10774
filipkujawa merged 1 commit into
aaif-goose:mainfrom
aaalaniz:cerebras-thinking-preservation

Conversation

@aaalaniz

@aaalaniz aaalaniz commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

fix(cerebras): preserve thinking inline for models that reject reasoning_content

Fixes #10406. Supersedes the Cerebras fix in #10417.

Problem

Cerebras rejects multi-turn requests that replay messages[].reasoning_content with
400 wrong_api_format. Once a session has one assistant turn with thinking, every
subsequent turn fails.

Before

cerebras-before.mov

After

after2.mp4

Fix

Two halves, both required.

Replay side — a declarative model can declare how its thinking is replayed:

thinking_preservation_format Replayed as
content_prepend inline in content, plain text
content_xml inline in content, wrapped in <think> tags
reasoning_content the separate reasoning_content field (OpenAI-compatible default)

Request sideModelInfo.request_params carries per-model static request-body
params. Cerebras models send {"reasoning_format": "parsed"} so reasoning comes back in
a structured field.

request_params mirrors the existing ModelConfig.request_params: same type, same
semantics, just sourced from the model declaration instead of user config. It is
unconditional and untyped — no enabled flag, no coupling to effort levels, and not gated
on whether reasoning is on, since reasoning_format describes the shape of the response
rather than whether the model should reason. It is declared per-model, so it only reaches
models that support it. Reserved keys (model, messages, stream, stream_options)
are skipped so a declaration cannot clobber the streaming setup.

Changes

  • goose-provider-types/src/base.rsThinkingPreservationFormat enum, and two
    optional ModelInfo fields (thinking_preservation_format, request_params).
    DeclarativeProviderConfig.models is already Vec<ModelInfo>, so declarative JSON feeds
    both without extra plumbing.
  • goose-provider-types/src/formats/openai.rs
    OpenAiFormatOptions.thinking_preservation_format and inline_reasoning_content. It
    runs after merge_split_tool_call_messages, which uses reasoning_content to identify
    messages split from the same assistant turn. is_reserved_request_param_key is now pub
    so the provider reuses it rather than duplicating the list.
  • goose-providers/src/openai.rscustom_models retains full ModelInfo instead of
    just names (fetch_supported_models maps back to names). stream() looks the model up
    once and uses both fields: it passes the format into OpenAiFormatOptions, and merges
    request_params into the payload after create_request_with_options returns.
    apply_ollama_options is the precedent for provider-level payload shaping.
  • goose-providers/src/declarative/definitions/cerebras.json — per-model
    thinking_preservation_format and request_params; adds gemma-4-31b.
  • Mechanical (~8 lines)ModelInfo literals gain the two new fields and two
    OpenAiFormatOptions literals gain ..Default::default(), across databricks.rs,
    declarative_providers.rs, provider_registry.rs, and tests/agent.rs.
  • goose-self-test.yaml — adds Phase 3C, a manual multi-turn thinking-preservation
    procedure (AGENTS.md:71). It self-skips unless CEREBRAS_API_KEY is set and the
    session is on a Cerebras model, so the automated coverage is the unit tests below.

cerebras.json deliberately does not set "preserves_thinking": true:
should_preserve_thinking_by_default already defaults it to true for
ProviderEngine::OpenAI, and cerebras is "engine": "openai". There is a test asserting
that.

Limitations

Cerebras models do not receive reasoning_effort, so the thinking-effort setting has no
effect on them. This is not a regression — is_openai_responses_model matches only o\d+
and gpt-5*, so gpt-oss-120b, zai-glm-4.7, and gemma-4-31b get no reasoning param on
main today either. Mapping thinking effort onto these models is follow-up work.

Testing

Verified end to end against a live Cerebras key on all three models: multi-turn sessions
with thinking enabled now work, which is the scenario in #10406.

Unit tests added:

  • ModelInfo deserializes both new fields, and defaults them to None when absent.
  • Each of the three formats produces the expected content and drops reasoning_content.
  • Inlining runs after the split-tool-call merge: two split assistant/tool pairs still
    reunite into one assistant message with both tool calls, with the thinking inlined once.
  • cerebras.json declares the expected format and reasoning_format: parsed per model,
    and still defaults preserves_thinking to true.
  • request_params cannot clobber stream, stream_options, model, or messages.

cargo fmt --all -- --check, cargo build --workspace --all-targets,
cargo clippy --all-targets -- -D warnings, and
cargo test -p goose-provider-types -p goose-providers all pass.

@aaalaniz

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 07511d53eb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@aaalaniz
aaalaniz marked this pull request as ready for review July 28, 2026 20:13
@alexhancock alexhancock self-assigned this Jul 30, 2026
@filipkujawa filipkujawa self-assigned this Jul 30, 2026
@filipkujawa

Copy link
Copy Markdown
Collaborator

Thanks for the fast turnaround on the split!

One required item before merge: the Azure AI Foundry provider (#10622) landed on main after your base, and it constructs ModelInfo (azure_foundry.rs, model_info_for_deployment) and OpenAiFormatOptions (openai_compatible.rs) without the new fields, so the branch merges cleanly textually but won't compile. Please rebase onto main and add the two new None fields / a ..Default::default() at those two sites- should be quick.

LGTM after the rebase.

@filipkujawa filipkujawa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

…ing_content

Cerebras rejects multi-turn requests that replay messages[].reasoning_content
with 400 wrong_api_format, so any session with thinking enabled fails from the
second turn onward.

Models can now declare a thinking_preservation_format, replaying thinking
inline in content (plain, or wrapped in <think> tags) instead of as a separate
field. They can also declare request_params, per-model static request body
params mirroring ModelConfig.request_params; Cerebras models use it to send
reasoning_format: parsed so reasoning comes back structured.

ModelInfo.reasoning stays bool. The generalized reasoning config is deferred.

Fixes aaif-goose#10406
@aaalaniz
aaalaniz force-pushed the cerebras-thinking-preservation branch from 07511d5 to 0369490 Compare July 30, 2026 20:59
@aaalaniz

Copy link
Copy Markdown
Contributor Author

Thanks for the fast turnaround on the split!

One required item before merge: the Azure AI Foundry provider (#10622) landed on main after your base, and it constructs ModelInfo (azure_foundry.rs, model_info_for_deployment) and OpenAiFormatOptions (openai_compatible.rs) without the new fields, so the branch merges cleanly textually but won't compile. Please rebase onto main and add the two new None fields / a ..Default::default() at those two sites- should be quick.

LGTM after the rebase.

addressed, thank you!

@filipkujawa
filipkujawa added this pull request to the merge queue Jul 30, 2026
Merged via the queue into aaif-goose:main with commit ee5e5f1 Jul 30, 2026
25 checks passed
lifeizhou-ap added a commit that referenced this pull request Jul 31, 2026
* main:
  fix(desktop): allow removing images when editing user messages (#9979)
  Fetch cursor-agent models from CLI (#10367)
  fix(cerebras): preserve thinking inline for models that reject reasoning_content (#10774)
  perf: make CLI streaming render incremental instead of O(n^2) (#10487)
  Moving to issues as the new PRs (#10818)
  ci: defer privileged recipe scans for forks (#10825)
  docs: adopt issue-first contribution workflow (#10819)
  Add Azure AI Foundry multi-LLM provider (#10622)
  fix(together): parse model list response (#10576)
  docs: remove retired site resources (#10817)
  feat: streaming shell output while commands run (#10808)
lifeizhou-ap added a commit to vincenzopalazzo/goose that referenced this pull request Jul 31, 2026
* main: (224 commits)
  fix(desktop): allow removing images when editing user messages (aaif-goose#9979)
  Fetch cursor-agent models from CLI (aaif-goose#10367)
  fix(cerebras): preserve thinking inline for models that reject reasoning_content (aaif-goose#10774)
  perf: make CLI streaming render incremental instead of O(n^2) (aaif-goose#10487)
  Moving to issues as the new PRs (aaif-goose#10818)
  ci: defer privileged recipe scans for forks (aaif-goose#10825)
  docs: adopt issue-first contribution workflow (aaif-goose#10819)
  Add Azure AI Foundry multi-LLM provider (aaif-goose#10622)
  fix(together): parse model list response (aaif-goose#10576)
  docs: remove retired site resources (aaif-goose#10817)
  feat: streaming shell output while commands run (aaif-goose#10808)
  feat(acp): allow session/new to request a Hidden session via _meta.hidden (aaif-goose#10811)
  chore: removed stale text pnpm workspace (aaif-goose#10809)
  feat(otel): emit GenAI semantic convention attributes (aaif-goose#10700)
  fix: avoid double loading full conversation on session open (aaif-goose#10794)
  feat(providers): forward images and MCP embedded-resource blobs in Anthropic and Google formats (aaif-goose#10340)
  Deprecate and remove ui/text TUI (aaif-goose#10799)
  Upgrade to rmcp 3.0 (aaif-goose#10789)
  fix(release): signing environment (aaif-goose#10797)
  feat(tools): collapse const-union enums in tool schemas (aaif-goose#10577)
  ...
@aaalaniz
aaalaniz deleted the cerebras-thinking-preservation branch July 31, 2026 18:14
@jamesn-cerebras

Copy link
Copy Markdown

Confirmed on our side, tested with a Cerebras reasoning model (gpt-oss-120b), thanks for the implementation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cerebras Reasoning Support

4 participants