fix: forward reasoning content, Ollama max_tokens fallback, tool result images

[auriti]

Summary

Three significant fixes in the OpenAI shim addressing long-standing gaps in provider compatibility, reasoning model support, and tool result fidelity.

Changes

1. Forward reasoning_content from thinking models as Anthropic thinking blocks

Impact: Users of DeepSeek R1, o3, and other reasoning models can now see chain-of-thought output.

Previously, delta.reasoning_content was silently ignored — reasoning models appeared to "just think silently" with no visible output until the final answer.

Streaming path:

• Detects delta.reasoning_content (or delta.reasoning alias)
• Emits content_block_start with type: 'thinking'
• Streams thinking_delta events with the reasoning text
• Closes the thinking block when delta.content begins (model switches from reasoning to answering)
• Handles edge case: reasoning-only responses without text content

Non-streaming path:

• Detects message.reasoning_content (or reasoning alias)
• Prepends a { type: 'thinking', thinking: text, signature: '' } block before text content

2. Use max_tokens for local providers (Ollama/LM Studio compatibility)

Impact: Ollama < 0.5 users get their token budget respected.

Older Ollama versions silently ignore max_completion_tokens and use their internal default. Now mirrors the existing GitHub Models fallback: for local providers (detected via isLocalProviderUrl), max_completion_tokens is swapped to max_tokens.

3. Preserve images in tool_result blocks

Impact: Screenshot tools, browser tools, and any tool returning images now work with OpenAI providers.

Previously, image blocks in tool results were mapped to empty strings via c.text ?? ''. New buildToolResultContent() helper emits proper image_url content parts:

• Base64 images → data: URI
• URL images → direct URL passthrough
• Text + images → multipart content array (supported by OpenAI on tool role messages)

Relates to #30, #113, #114

Test plan

• bun test src/services/api/openaiShim.test.ts — pass
• bun test src/services/api/codexShim.test.ts — pass
• Verify DeepSeek R1 shows chain-of-thought in streaming
• Verify Ollama respects max_tokens with older versions
• Verify screenshot tool results include images for OpenAI providers

[Vasanthdev2004]

This is close, but I don't think it's safe to merge yet.

One blocker stood out in src/services/api/openaiShim.ts:560-590:

If a provider emits reasoning_content / reasoning and then goes straight into tool_calls without an intermediate text delta, hasEmittedThinkingStart is still true here, but we still start a tool_use block. The thinking block only gets stopped later at finish time.

That means the stream can have both blocks open at once, and the stop ordering becomes:

• content_block_start(thinking)
• content_block_start(tool_use)
• ...
• content_block_stop(thinking)
• content_block_stop(tool_use)

The existing Anthropic-style consumer logic is built around closing a block before the next one starts, so this can produce out-of-order content on reasoning models that think and then call tools.

I think this needs the same close-before-next-block handling for tool calls that the text path already has.

After that is fixed, I'd also want focused regression coverage for:

• reasoning -> tool_calls streaming
• tool_result image forwarding
• local-provider max_tokens fallback

[auriti]

Good catch @Vasanthdev2004 — fixed in the latest push.

The thinking block is now closed before any tool_use block starts, matching the same close-before-next pattern that already existed for text→tool transitions. The stream sequence for reasoning→tool_calls is now:

content_block_start(thinking, index 0)
thinking_delta...
content_block_stop(index 0)          ← closed before tool starts
content_block_start(tool_use, index 1)
input_json_delta...
content_block_stop(index 1)

Regarding regression tests — I'll add targeted tests for the three paths you mentioned (reasoning→tool_calls, tool_result images, local max_tokens fallback) in a follow-up commit.

[auriti]

Added the three regression tests @Vasanthdev2004 requested:

1. reasoning → tool_calls streaming — verifies the full thinking block lifecycle: content_block_start(thinking) → thinking_delta events → content_block_stop(thinking) → content_block_start(tool_use). Asserts thinking block index < tool_use index and thinking is closed BEFORE tool starts.

2. tool_result image forwarding — sends a tool_result with both text and base64 image, verifies the outgoing tool message contains an image_url content part with correct data:image/png;base64,... URI (not an empty string).

3. local-provider max_tokens fallback — sets OPENAI_BASE_URL=http://localhost:11434/v1, verifies the body contains max_tokens (not max_completion_tokens) and no stream_options.

All 7 tests pass.

[Vasanthdev2004]

Rechecked the follow-up changes. The thinking block now closes before tool_use starts, the three regression tests I asked for are in place, and the updated shim test plus build pass locally from my side.

[kevincodex1]

please fix conflicts

[kevincodex1]

@auriti please fix conflicts here. lets see if we can merge this

[auriti]

Fresh pass on the current branch from my side:

• the original reviewer blocker was fixed and explicitly approved by @Vasanthdev2004
• the focused shim suite still passes locally (bun test src/services/api/openaiShim.test.ts)
• the branch is now mainly blocked by merge conflicts / age, not by an unresolved technical objection in the thread

One note on scope versus current main: some of the reasoning-model coverage has since evolved further upstream, but the branch still carries a real, focused set of fixes around:

• tool_result image preservation
• local-provider max_tokens fallback
• the original reasoning -> tool_calls regression tests requested in review

So if this is still wanted, the next useful step is simply to refresh the branch onto current main and re-run the focused shim tests. I don’t see a new reviewer-reported blocker beyond the stale conflict state.

[kevincodex1]

@auriti kindly address conflicts please

[kevincodex1]

hello @auriti kindly fix the conflicts when you have time.

[jatmn]

Thanks for following up on the earlier review. The requested thinking-block close-before-tool behavior looks addressed now, and the three requested regression tests are also in place. I found one remaining issue below.

Findings

• [P1] Do not send image_url parts in tool-role messages
  src/services/api/openaiShim.ts:181
buildToolResultContent() now returns multipart content with { type: 'image_url' } for tool results, and convertMessages() places that array on a role: 'tool' Chat Completions message. The OpenAI Chat Completions schema allows array content for tool messages, but only text parts are supported for that role; image parts are accepted on user messages, not tool messages. So a screenshot/browser tool result that includes an image will be sent back as a tool message and the next OpenAI request can fail validation instead of preserving the prior text-only behavior. Please either keep tool-role content text-only for Chat Completions, move image data through a supported user-message representation, or gate this to providers/APIs that explicitly support image parts on tool messages.

[kevincodex1]

hello @auriti kindly rebase to main and fix conflicts.

[jatmn]

closing as abandoned