feat(opencode): respect provider/model streaming: false to disable response streaming

[sebdanielsson]

Issue for this PR

Closes #785

Type of change

• Bug fix
• New feature
• Refactor / code improvement
• Documentation

What does this PR do?

Some OpenAI-compatible backends either don't support streaming or return broken streamed output. In my case a self-hosted vLLM (Gemma) corrupts streamed tool-call arguments (duplicates characters), so every edit comes back garbled. The existing options.streaming config wasn't actually consumed, so there was no way to opt out.

This makes options.streaming: false (per-model or per-provider) actually work. When set, it adds the AI SDK's simulateStreamingMiddleware, which calls doGenerate (stream: false on the wire) and replays the result as a simulated stream — so the rest of the pipeline is unchanged. Defaults to streaming on, so existing behavior is untouched.

{ "provider": { "vllm": { "options": { "streaming": false } } } }

Only covers the default AI SDK path. The experimental experimentalNativeLlm runtime is a separate path and isn't handled here.

How did you verify your code works?

• Added a test in test/session/llm.test.ts that sets streaming: false and has the mock server return a non-streaming JSON completion. It only parses if the request was non-streaming (a streamed request expects SSE and fails), and asserts body.stream isn't true.
• bun test test/session/llm.test.ts -> 27 pass, plus typecheck and oxlint clean.

Screenshots / recordings

No UI changes.

Checklist

• I have tested my changes locally
• I have not included unrelated changes in this PR

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR adds a configuration-based way to disable true provider streaming while keeping the rest of the pipeline streaming-compatible by simulating a stream.

Changes:

• Add simulateStreamingMiddleware() to opt out of on-the-wire streaming when options.streaming === false.
• Add a test asserting that provider requests are not made with stream: true when streaming is disabled.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
packages/opencode/src/session/llm.ts	Adds streaming opt-out logic and conditionally injects simulated streaming middleware.
packages/opencode/test/session/llm.test.ts	Adds coverage to confirm provider requests are non-streaming when configured.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

Thanks for updating your PR! It now meets our contributing guidelines. 👍

[sebdanielsson]

Just this version in a GitHub Action workflow, and it worked without the stream-stripping proxy we're currently using to get around this limitation. 👍

[lmeyerov]

We hit the Bedrock side of this. Meta's Llama models on Bedrock reject tool use over /converse-stream (ValidationException: This model doesn't support tool use in streaming mode), so with tools they're unusable today. streaming: false fixes it: verified against real Llama 4 Maverick, which then made native tool calls and answered correctly, with streaming models unaffected (Claude on Bedrock still streamed, test/session/llm.test.ts green).

One Bedrock-specific gap I ran into: the prompt-transform middleware only runs for args.type === "stream", so under doGenerate the Bedrock message transform gets skipped. Widening that guard to also run when streaming is off makes the Bedrock path correct under streaming: false. Happy to send a patch against this branch.

[sebdanielsson]

We hit the Bedrock side of this. Meta's Llama models on Bedrock reject tool use over /converse-stream (ValidationException: This model doesn't support tool use in streaming mode), so with tools they're unusable today. streaming: false fixes it: verified against real Llama 4 Maverick, which then made native tool calls and answered correctly, with streaming models unaffected (Claude on Bedrock still streamed, test/session/llm.test.ts green).

One Bedrock-specific gap I ran into: the prompt-transform middleware only runs for args.type === "stream", so under doGenerate the Bedrock message transform gets skipped. Widening that guard to also run when streaming is off makes the Bedrock path correct under streaming: false. Happy to send a patch against this branch.

Feel free to send it, thanks for testing!👍

[michaeljguarino]

+1 this, we're also dealing with an internal gateway without stream support, and would be great to have an opencode level shim for that.

[sebdanielsson]

FYI: For those of you having problem using Gemma 4 31B on vLLM with OpenCode. Updating vLLM to 0.24.0 and using the vLLM example template solved all of our issues.

[TomLucidor]

@sebdanielsson so the same thing with E2B/E4B? Is it really just a bad template/parsing issue on Google's side, or (local) inference engine, or harness/scaffold side? NousResearch/hermes-agent#6626 (comment) NousResearch/hermes-agent#19887

[sebdanielsson]

@sebdanielsson so the same thing with E2B/E4B? Is it really just a bad template/parsing issue on Google's side, or (local) inference engine, or harness/scaffold side? NousResearch/hermes-agent#6626 (comment) NousResearch/hermes-agent#19887

Not my area of expertise but I think both. Many fixes for Gemma 4 landed in the latest 0.24.0 release. Have you tried it + using the vLLM provided template?

This is the arguments we start vLLM with:
vllm-project/vllm#39043 (comment)

Might be worth testing with the smaller MoW models.

[github-actions]

Automated PR Cleanup

Thank you for contributing to opencode.

Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions.

This PR was closed because it matched the following cleanup criteria:

• The PR was created more than 1 month ago
• The PR had fewer than 2 positive reactions
• Positive reactions are counted as thumbs-up, heart, celebration, or rocket reactions on the PR

PRs created within the last month are not affected by this cleanup.

If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate.

Thanks again for taking the time to contribute.

[jeremy-london]

This should be merged its blocking AWS Bedrock support for LLama models. Please consider