harness fixes

[akshaydeo]

Summary

This PR delivers several new features and bug fixes across core, framework, and plugins, including partial-usage billing for cancelled/timed-out streams, airgapped model catalog sync, MCP header forwarding, and a fix for MCP tools failing on the Bedrock/Anthropic Responses path. It also migrates the Vertex provider's URL-inlining logic to cover image blocks in addition to documents, retires the Fable 5 / Mythos test suite in favour of Claude Opus 4.8, and extends the changelog-writer skill to collect and render closed GitHub issues.

Changes

• Failed-request billing — Core now emits BilledUsage on stream cancellation and timeout errors so tokens already processed by the provider are accounted for; framework and governance/logging plugins propagate this cost through to failed and cancelled requests.
• Airgapped local sync — Model catalog pricing and model parameters can now be loaded from local files, removing the requirement for outbound network access on airgapped deployments (closes [Bug]: Model parameters can't be configured locally #4305).
• MCP extra header forwarding — Allowlisted per-request extra headers are forwarded to MCP tool calls, including ping and list_tools operations.
• Bedrock MCP tool stripping — Provider-unsupported server tools (e.g. mcp) are now stripped on the Bedrock/Anthropic Responses path instead of failing the entire request, restoring pre-v1.5.0 behaviour (closes [Bug Report] #3795).
• Vertex image URL inlining — inlineDocumentURLs is renamed inlineRemoteURLSources and extended to also fetch and base64-encode remote image URL sources, since Anthropic-on-Vertex rejects source.type: "url" for both documents and images.
• Bedrock EventStream fix — Bedrock stream errors are encoded as EventStream exceptions, fixing ChecksumMismatch / corrupted EventStream on PostLLMHook errors (closes [Bug]: BifrostError returned from PostLLMHook during Bedrock streaming causes ChecksumMismatch / corrupted AWS EventStream #4545).
• Test harness updates — Cross-cut Round 30 is migrated from the discontinued Fable 5 / Mythos models to Claude Opus 4.8; Bedrock image generation tests switch from amazon.titan-image-generator-v2:0 to amazon.nova-canvas-v1:0; moonshotai/kimi-k2-thinking-maas and minimaxai/minimax-m2-maas are routed through Vertex Global; audio content-type responses are excluded from the shape-validation test script.
• Changelog skill — The changelog-writer SKILL.md now includes instructions for collecting every closed GitHub issue via GraphQL and commit-body grep, confirming each issue's state, and rendering them as a ## 🐙 Closed GitHub Issues section in transports/changelog.md.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

go test ./...

• Send a streaming request and cancel it mid-flight; verify BilledUsage is populated on the returned error and cost is recorded.
• Send a chat completion to a Vertex/Claude model with a message containing a remote image URL; confirm the request succeeds and the image is inlined as a base64 data URI.
• Send a request to a Bedrock/Anthropic Responses endpoint with an MCP tool in the tool list; confirm the request succeeds rather than returning an error.
• On an airgapped host, place model catalog files locally and confirm pricing/parameters are loaded without outbound calls.

Breaking changes

• Yes
• No

Related issues

Closes #3357
Closes #3795
Closes #4305
Closes #4545
Closes #4556
Related: #2887

Security considerations

Extra headers forwarded to MCP tool calls are restricted to an explicit allowlist; no new secrets or PII surfaces are introduced.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 92fe3ac1-ade0-4e30-be2b-8167f8960eac

📥 Commits

Reviewing files that changed from the base of the PR and between 1317e16 and 9fec659.

📒 Files selected for processing (15)

• .claude/skills/changelog-writer/SKILL.md
• core/changelog.md
• core/providers/bedrock/convert_tool_config_test.go
• core/providers/bedrock/rerank_test.go
• core/providers/openai/chat_test.go
• core/providers/vertex/vertex.go
• core/schemas/serialization_test.go
• core/schemas/utils.go
• framework/changelog.md
• plugins/governance/changelog.md
• plugins/logging/changelog.md
• tests/config.json
• tests/e2e/api/collections/provider-harness.json
• tests/integrations/python/config.json
• transports/changelog.md

✅ Files skipped from review due to trivial changes (5)

• plugins/logging/changelog.md
• framework/changelog.md
• plugins/governance/changelog.md
• transports/changelog.md
• core/changelog.md

🚧 Files skipped from review as they are similar to previous changes (9)

• tests/config.json
• core/providers/bedrock/rerank_test.go
• core/providers/openai/chat_test.go
• tests/integrations/python/config.json
• core/schemas/serialization_test.go
• core/schemas/utils.go
• core/providers/vertex/vertex.go
• core/providers/bedrock/convert_tool_config_test.go
• tests/e2e/api/collections/provider-harness.json

---

📝 Walkthrough

Summary by CodeRabbit

Release Notes

• New Features

  • Added billing for failed and cancelled (including partially used) streaming requests.
  • Added local model catalog syncing for airgapped deployments.
  • Allowlisted extra headers to be forwarded to MCP tool calls (including ping/list_tools).

• Bug Fixes

  • Improved Bedrock streaming for more consistent incremental delivery and better stream-error handling.
  • Inlined remote image (and remote file) inputs to base64 for Anthropic-on-Vertex.
  • Preserved OpenRouter server tools in the OpenAI Responses path and removed unsupported server tools instead of failing requests.

• Tests

  • Expanded server tool name preservation/deep-copy coverage and refreshed E2E scenarios/models.

• Closed GitHub Issues

  • Added links for the closed issues included in this release.

Walkthrough

Extends the Vertex Anthropic request pre-processor to inline remote image URL sources (in addition to documents) into base64 data URIs. Adds comprehensive server tool name preservation across inbound request conversions and deep-copy operations. Prepares a release by adding changelog entries for new features and fixes, documenting the closed-issues collection workflow in the changelog-writer skill, and refreshing the E2E test harness with updated model identifiers and an Opus 4.8 feature-gating suite.

Changes

Vertex Remote Image URL Inlining

Layer / File(s)	Summary
inlineRemoteURLSources helper and integration core/providers/vertex/vertex.go	Introduces inlineRemoteURLSources that fetches and base64-encodes remote FileURL sources (setting FileData/FileType) and remote ImageURLStruct sources (converting to data: URIs), replacing the prior document-only inliner in both ChatCompletion and ChatCompletionStream for Anthropic-on-Vertex with updated error messages.

Server Tool Preservation and Deep Copy Enhancement

Layer / File(s)	Summary
Server tool name preservation tests core/providers/bedrock/convert_tool_config_test.go, core/providers/openai/chat_test.go, core/schemas/serialization_test.go, core/providers/bedrock/rerank_test.go	Adds TestConvertChatParameters_ServerToolNameFromBody to verify Bedrock request conversion preserves server tool names; TestOpenAIInbound_ServerToolNameSurvives to validate OpenAI-compatible inbound parsing; TestChatTool_ServerToolNameRoundTrip to confirm serialization round-trips; and updates Bedrock rerank test provider resolution expectations.
DeepCopyChatTool expansion and validation core/schemas/utils.go, core/schemas/serialization_test.go	DeepCopyChatTool now copies server tool identity (Name, MCPServerName) and expands field coverage to include deferred-loading flags, caller lists, input examples, web-search/web-fetch config, computer/text-editor settings, and MCP toolset configs via new deepCopyChatMCPToolsetConfig helper. TestDeepCopyChatTool_PreservesServerToolFields validates all fields persist and deep copies remain independent.

Release Changelog, Skill Documentation, and Test Harness

Layer / File(s)	Summary
Release changelog entries across packages core/changelog.md, framework/changelog.md, plugins/governance/changelog.md, plugins/logging/changelog.md, transports/changelog.md	Adds feature entries for failed-request billing, airgapped local sync, and MCP extra header forwarding; adds fix entries for Bedrock stream error encoding, unsupported server tool stripping, Bedrock streaming identity encoding, OpenRouter tool preservation, and Vertex image URL inlining; introduces a Closed GitHub Issues section in transports listing issues #2887, #3357, #3795, #4305, #4530, #4542, #4545, #4556.
Changelog-writer skill: closed-issues collection workflow .claude/skills/changelog-writer/SKILL.md	Extends SKILL.md with a Collect Closed GitHub Issues section describing GraphQL discovery via closingIssuesReferences, commit-body keyword fallback, per-issue state/title verification, and markdown-link rendering; updates the transports/changelog.md template with a Closed GitHub Issues section and adds a formatting rule requiring the section whenever issues are closed.
Test configuration model list updates tests/config.json, tests/integrations/python/config.json	Expands both test config files' Vertex provider configuration by adding two model identifiers (moonshotai/kimi-k2-thinking-maas, minimaxai/minimax-m2-maas) to the Vertex key blacklist and Vertex Global Claude model lists.
E2E provider harness model and suite updates tests/e2e/api/collections/provider-harness.json	Swaps Anthropic model IDs to claude-opus-4-8 and claude-sonnet-4-5; replaces Bedrock image model from amazon.titan-image-generator-v2:0 to amazon.nova-canvas-v1:0 in three test locations; adds audio/ content-type exclusion to response validation gating; and replaces the Round 30 Fable5/Mythos feature-gating suite with an Opus 4.8 suite covering adaptive thinking, effort modes, structured outputs, fast-mode, task budgeting, computer-use tools, dynamic web search, and mid-conversation system message placement.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• maximhq/bifrost#4573: Main PR's changelog entries describing stripping provider-unsupported mcp server tools on the Bedrock/Anthropic Responses path directly overlap with this PR's server tool preservation logic and corresponding fix documentation.

Suggested reviewers

• danpiths
• Pratham-Mishra04

Poem

🐇 Hopping through images encoded in base64,
Tool names preserved through each format and door,
Deep copies flourish with fields newly found,
Changelogs gathered, closed issues abound,
Opus 4.8 tested—the changelog's crown! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The PR title 'harness fixes' is vague and generic, not clearly describing the main changes which span multiple systems including billing, image URL inlining, MCP tools, and changelog improvements.	Revise the title to specifically highlight the primary change or main theme—for example, 'Add failed-request billing and extend Vertex image URL inlining' or similar to better reflect the scope.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description check	✅ Passed	The PR description is comprehensive and well-structured, covering all major changes, affected areas, testing instructions, related issues, and security considerations according to the template.
Docstring Coverage	✅ Passed	Docstring coverage is 91.67% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 06-21-harness_fixes

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[akshaydeo]

• harness fixes #4580 👈 (View in Graphite)
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[greptile-apps]

Confidence Score: 5/5

Safe to merge — all changes are targeted fixes with corresponding diagnostic tests covering the full Unmarshal → convert → wire-marshal path.

The two main code changes are both correct: DeepCopyChatTool now exhaustively covers every field in the ChatTool struct (verified by reading the struct definition), and inlineRemoteURLSources correctly handles the absent Content-Type case by delegating to SanitizeImageURL, which byte-sniffs base64 data to produce a valid data URI. The diagnostic tests in three separate test files pin the fix across the transport inbound, schemas, and Bedrock converter layers. Test harness updates are model-name swaps with no logic change.

No files require special attention.

Important Files Changed

Filename	Overview
core/schemas/utils.go	DeepCopyChatTool extended to copy Name, MCPServerName, and all Anthropic server-tool variant fields; new deepCopyChatMCPToolsetConfig helper. All fields in ChatTool struct are now covered.
core/providers/vertex/vertex.go	inlineDocumentURLs renamed to inlineRemoteURLSources and extended to base64-encode remote image URL blocks; fallback to SanitizeImageURL when Content-Type header is absent. Both ChatCompletion and ChatCompletionStream paths updated consistently.
core/schemas/serialization_test.go	Two new diagnostic tests: ChatTool server-tool name round-trip through Unmarshal/MarshalSorted, and DeepCopyChatTool independence/completeness for all server-tool variant fields.
core/providers/bedrock/convert_tool_config_test.go	New diagnostic probe exercises full body→Unmarshal→ToBedrockChatCompletionRequest→MarshalSorted path to verify tool name survives for bash/computer/text_editor/memory tools.
tests/e2e/api/collections/provider-harness.json	Cross-Cut Round 30 migrated from Fable 5/Mythos to Opus 4.8; Bedrock image-gen switched to nova-canvas-v1:0; audio content-type excluded from shape-validation script guard.
tests/config.json	moonshotai/kimi-k2-thinking-maas and minimaxai/minimax-m2-maas added to Vertex Global models list and blacklisted from the standard Vertex provider, routing them via Vertex Global MaaS.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Vertex ChatCompletion / ChatCompletionStream] --> B{IsAnthropicModelFamily?}
    B -- No --> C[Gemini/Vertex native path]
    B -- Yes --> D[inlineRemoteURLSources]
    D --> E{For each ContentBlock}
    E --> F{File block with FileURL?}
    F -- Yes --> G[FetchAndEncodeURL]
    G --> H[Set FileData = base64 / Clear FileURL]
    H --> E
    F -- No --> I{ImageURLStruct with non-data: URL?}
    I -- Yes --> J[FetchAndEncodeURL]
    J --> K{mediaType present?}
    K -- Yes --> L[img.URL = data:mediaType;base64,encoded]
    K -- No --> M[SanitizeImageURL sniffs bytes for media type]
    M --> N[img.URL = sniffed data URI]
    L --> E
    N --> E
    I -- No --> E
    E -- all blocks done --> O[ToAnthropicChatRequest]
    O --> P[Vertex API call]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Vertex ChatCompletion / ChatCompletionStream] --> B{IsAnthropicModelFamily?}
    B -- No --> C[Gemini/Vertex native path]
    B -- Yes --> D[inlineRemoteURLSources]
    D --> E{For each ContentBlock}
    E --> F{File block with FileURL?}
    F -- Yes --> G[FetchAndEncodeURL]
    G --> H[Set FileData = base64 / Clear FileURL]
    H --> E
    F -- No --> I{ImageURLStruct with non-data: URL?}
    I -- Yes --> J[FetchAndEncodeURL]
    J --> K{mediaType present?}
    K -- Yes --> L[img.URL = data:mediaType;base64,encoded]
    K -- No --> M[SanitizeImageURL sniffs bytes for media type]
    M --> N[img.URL = sniffed data URI]
    L --> E
    N --> E
    I -- No --> E
    E -- all blocks done --> O[ToAnthropicChatRequest]
    O --> P[Vertex API call]

Loading

_{Reviews (4): Last reviewed commit: "harness fixes" | Re-trigger Greptile}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.claude/skills/changelog-writer/SKILL.md:
- Around line 197-228: The GraphQL query limiting closingIssuesReferences to
first:10 will miss issues if a PR closes more than 10, and the workflow lacks
explicit deduplication when combining results from both the GraphQL per-PR
queries and the commit-body grep fallback. Increase the first:10 limit to
first:100 in the GraphQL query for closingIssuesReferences, and add an explicit
deduplication step (using sort -u on issue numbers) before rendering the final
markdown list in ascending order to eliminate any duplicate entries from the
combined sources.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: bd42e42a-8b20-422c-9e4e-8f0acb110740

📥 Commits

Reviewing files that changed from the base of the PR and between 9027dab and a5e493c.

📒 Files selected for processing (9)

• .claude/skills/changelog-writer/SKILL.md
• core/changelog.md
• core/providers/vertex/vertex.go
• framework/changelog.md
• plugins/governance/changelog.md
• plugins/logging/changelog.md
• tests/config.json
• tests/e2e/api/collections/provider-harness.json
• transports/changelog.md

[akshaydeo]

Merge activity

• Jun 21, 11:42 AM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jun 21, 11:43 AM UTC: @akshaydeo merged this pull request with Graphite.