fix: enable haiku 4.5 json

[steebchen]

Summary

• mark Claude Haiku 4.5 and its dated mappings as supporting response_format: { type: "json_object" }
• auto-heal Anthropic and Bedrock json_object responses so fenced JSON is normalized before returning
• replace the old unsupported-model regression with a passing Haiku 4.5 JSON-mode e2e assertion

Verification

• pnpm exec vitest run packages/actions/src/prepare-request-body.spec.ts apps/gateway/src/chat/tools/heal-json-response.spec.ts --no-file-parallelism
• TEST_MODELS="anthropic/claude-haiku-4-5" pnpm test:e2e -- apps/gateway/src/chat-json.e2e.ts apps/gateway/src/api-individual.e2e.ts
• pnpm build

Summary by CodeRabbit

• New Features

  • JSON output format enabled for Anthropic models.

• Bug Fixes / Improvements

  • Improved JSON response handling and “healing” for Anthropic and AWS Bedrock providers to produce more reliable JSON outputs.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: e6e7e0ed-fc66-4565-a8c5-e2a54179c57e

📥 Commits

Reviewing files that changed from the base of the PR and between eb077cb and 2376030.

📒 Files selected for processing (1)

• apps/gateway/src/api-individual.e2e.ts

---

Walkthrough

Added jsonOutput: true to Anthropic (and AWS Bedrock) model entries, expanded streaming and non‑streaming response‑healing gating to treat response_format.type === "json_object" as eligible when provider is anthropic or aws-bedrock, and updated an E2E test to target OpenAI for an unsupported JSON output error case.

Changes

Cohort / File(s)	Summary
Model Capabilities packages/models/src/models/anthropic.ts	Added jsonOutput: true to multiple Anthropic provider entries (and corresponding AWS Bedrock entries) to declare JSON output capability.
Response Healing Logic apps/gateway/src/chat/chat.ts	Expanded gating in shouldBufferForHealing and shouldHealNonStreaming to allow JSON-object response healing when usedProvider is anthropic or aws-bedrock.
E2E Tests apps/gateway/src/api-individual.e2e.ts	Updated test "JSON output mode error for unsupported model" to use OpenAI provider (openai/gpt-4o-mini-search-preview) and corresponding env var check; assertions for 400 and error substring unchanged.

Sequence Diagram(s)

mermaid
sequenceDiagram
participant Client as Client
participant Gateway as Gateway
participant Provider as Anthropic/AWS/OpenAI
Client->>Gateway: POST /v1/chat/completions (response_format.type="json_object")
Gateway->>Gateway: evaluate gating (shouldBufferForHealing / shouldHealNonStreaming)
alt provider is anthropic/aws-bedrock OR provider supports jsonOutput
Gateway->>Provider: forward request (buffering/healing enabled)
Provider-->>Gateway: streaming or final JSON response
Gateway->>Client: 200 with JSON response
else provider doesn't support JSON output
Provider-->>Gateway: 400 unsupported-JSON error
Gateway->>Client: 400 with error body

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• feat(gateway): add streaming mode support for response healing plugin #1627: Modifies response‑healing gating in apps/gateway/src/chat/chat.ts (overlaps the same logic changes).
• test(json): add streaming tests for JSON output modes #1476: Changes JSON output handling and capability checks across chat and model capability code (related to jsonOutput flags and gating).
• fix(models): update anthropic models #1311: Edits packages/models/src/models/anthropic.ts model entries (overlaps model-definition edits).

Suggested labels

auto-merge

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: enable haiku 4.5 json' accurately summarizes the main change—enabling JSON output support for Claude Haiku 4.5 model.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch enable-haiku45-json

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Enables reliable OpenAI-compatible response_format: { type: "json_object" } usage for Claude Haiku 4.5 by declaring the capability in model metadata and normalizing (“healing”) fenced/mixed JSON responses from Anthropic and AWS Bedrock before returning them to clients.

Changes:

• Mark Claude Haiku 4.5 (and its dated mapping) as supporting json_object via jsonOutput: true.
• Auto-enable JSON response healing for Anthropic and AWS Bedrock when response_format.type === "json_object" (streaming + non-streaming).
• Replace the prior “unsupported JSON mode” e2e with a passing Haiku 4.5 JSON-mode assertion.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
packages/models/src/models/anthropic.ts	Advertises json_object support for Haiku 4.5 provider mappings via jsonOutput: true.
apps/gateway/src/chat/chat.ts	Ensures JSON healing runs (and buffering occurs for streaming) for Anthropic/Bedrock json_object responses even without the plugin enabled.
apps/gateway/src/api-individual.e2e.ts	Updates e2e coverage to assert Haiku 4.5 json_object returns parseable JSON content.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The test parses the JSON content twice (JSON.parse(content) in both assertions). Parse once into an object and assert on that to avoid duplicated work and make failures easier to debug (single parse + single object under test).

Suggested change

	expect(() => JSON.parse(content)).not.toThrow();
	expect(JSON.parse(content)).toHaveProperty("message");
	const parsedContent = JSON.parse(content);
	expect(parsedContent).toBeDefined();
	expect(parsedContent).toHaveProperty("message");

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b213374dba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Prevent early finish_reason in buffered JSON streams

Adding Anthropic and AWS Bedrock to shouldBufferForHealing routes response_format: {type:"json_object"} streams into the buffering path, but that path still forwards upstream chunks that contain finish_reason before the healed content is emitted at the end of the stream (see the existing send logic around choices[0].finish_reason and the later healed-content chunk write in chat.ts). For these providers, terminal events are normal, so clients that stop reading on the first non-null finish_reason can terminate before receiving any content, yielding empty/partial results specifically for streamed JSON-object requests.

Useful? React with 👍 / 👎.